Splitting a monolith from a QA perspective: how we set up testing for a new microservice and made the most of it

Some people might ask: "How hard can it be to split a monolith? You just move chunks of code into a new service, update the endpoints in the tests and you are done." I used to think so too. Then one day our team was given a task to move part of the functionality into a new service. Almost immediately, the task turned into an epic that kept on growing. As we analysed the monolith and its functionality, we kept finding new uncovered cases, duplicated tests, logic errors in existing tests and business logic of the microservice that needed clarifying. So we decided to streamline both the migration of the automated tests and the testing of the new microservice. Along the way, we audited the tests. Think of it as a New Year clear-out: keep the best and leave the rest behind.

My name is Grigorii Sedelnikov, and I am an Automation QA engineer at EXANTE. In this article, I explain how our team moved the order risk assessment service out of the monolith and migrated around 200 automated tests to it. Spoiler: a regression run for this service now takes two minutes instead of an hour.

I will also talk about AI, which our product manager calls "imitation intelligence". We used it where it genuinely saved time, but did most of the migration by hand. I will explain why you cannot write a textbook-perfect prompt and go off to play padel while your corporate AI does the work, and why the manual effort became the foundation for future automation.

How the service worked inside the monolith

EXANTE is a fintech company and broker. Traders use our platform to make high-value trades, so it has to run without interruption, including while we rebuild parts of it. Stable services and fast incident resolution are the foundation for developing the platform. Every QA engineer knows that a bug that slips into production carries serious reputational risk. For us, it can also mean financial losses for clients, which is why we pay close attention to quality.

Our monolith is called Core. In essence, it is a giant calculator. It runs calculations for hundreds of thousands of clients across more than 3 million financial instruments, including shares, bonds, futures and options, on markets around the world. Core calculates orders (client requests to buy or sell), account positions and margin settings. There is no service in the system that does not send requests to Core.

Pre-margin, the service this article is about, calculates risk when an order is placed.

Here is a real-life example. A client opens the trading terminal to buy or sell an instrument. As soon as they hover over the Buy or Sell button, the terminal sends a request to check the account. The system assesses whether the client has enough funds to open a new position. Pre-margin does not need the vast amount of data stored in the monolith for this check. Account settings, instrument settings and the current state of the account are enough.

Why we moved Pre-margin out of the monolith

Splitting a microservice out of a monolith is not about following a trend. For a large system, it is essential for further growth. Here are the reasons as our team saw them.

Simpler, cheaper scaling. A monolith can only scale as a whole, even if load grows on just one of its functions. A separate service can scale independently, with resources allocated specifically to it. This matters especially for Pre-margin. It receives a request every time a client hovers over Buy or Sell, which is far more often than orders are actually placed.

A dedicated domain for the service and the team. One service handles one task, and it is owned by a team with deep knowledge of that domain. Such a service is easier to develop, test and debug, and changes to it do not affect neighbouring functionality.

Developers are not the only ones who benefit. QA gets its share of the perks too. For us, the main gains are faster incident investigation, lower cognitive load on duty engineers and faster regression runs. To explain why this mattered so much, let me describe how testing of the monolith was organised.

Why this matters for testing

We automated testing of Core a long time ago. Our framework now contains more than 3,500 automated tests written in Python. They are split into 16 jobs that run every night across several environments. Dedicated setup helpers create the required entities whenever a new environment is deployed. The tests have to be kept stable, and someone has to investigate those that fail overnight. Each week we choose a duty engineer from the monolith team for this, and we set aside four hours a day of their capacity just for analysing failures.

This set-up led to two major problems.

First, slow and noisy regression testing. Every change to Pre-margin went through regression testing. Because of the sheer number of tests, regression for even a small change took about an hour. Worse still, when accepting a new version, the engineer would see failures in tests covering functionality the change had not touched at all. They then had to work out whether it was a flaky test or whether we had actually broken the monolith's logic.

Service stability is a cornerstone for us. We do not release changes until we understand why tests failed on functionality we did not touch. As a result, releasing even a small feature or fix in Pre-margin could be delayed significantly.

Second, difficult incident investigation and high cognitive load. While the service lived inside the monolith, investigating incidents was hard. To get to the root cause, an engineer had to keep a huge amount of context in mind: how dependent modules worked, which account and instrument settings were involved in the calculation and which other parts of Core might have affected it. On top of that came a flood of logs, most of which had nothing to do with the problem at hand.

Now imagine the duty engineer trying to debug failing tests:

• across more than 10 different jobs,
• covering completely different functionality,
• with dependencies between different parts of the monolith's code.

By the end of the day, the duty engineer could look like this:

My colleague Vladimir Smirnov describes how our automated test duty works in his article The philosophy of automated tests. He covers how to analyse overnight failures, how to tell flaky tests from real problems and why tests do not have to be green.

To sum up, moving Pre-margin out of the monolith gave us the following benefits:

• A dedicated domain for the microservice. One microservice, one job.
• Faster regression testing.
• Faster bug localisation.
• Lower cognitive load on duty engineers and protection against burnout. As a bonus, other teams find the service easier to understand.

How we migrated the automated tests

While a team of dedicated developers carved the microservice out of the monolith, the testers took care of the automated tests. Here is everything we did to get the microservice ready for production.

We used AI to take stock of the tests

We had tests for the pre-order calculation, and we needed to find them across the monolith's various test suites. We gave Claude a two-part task:

• Find all tests that call and check the order evaluation endpoints.
• Compile the data into a table with the following fields: test name, location, what it checks and its preconditions.

It was important to give Claude detailed instructions:

• Find the tests in the monolith test folder that call the pre-order evaluation endpoint and check the result of that request with an assert in the test code, helpers or utilities.
• Compile a complete list of each test's preconditions. Take them from the pre-setup block, and also look at the parameters set in the entity block and configured in def setup_class and def setup_method.
• Put the data into a table with these mandatory fields: file, test class, test method, preconditions and what it checks, meaning which asserts it contains.

Within a couple of hours, we had a complete reference of the tests covering the new microservice's functionality.

We planned the architecture for the new microservice's test files

Next, we had to plan a new file and folder structure for the microservice's test suite. We grouped the tests by what they check. This makes them easier to work with. For example, it is quicker to find your way around when questions come up. Our tests also serve as a reference, showing what the service is responsible for.

We barely used AI to plan the architecture. Instead, we used our own heads and came up with solutions ourselves. AI mainly offered arguments to stress-test our ideas, and after a few iterations we could arrive at a cleaner solution.

If you rely on AI too heavily for architecture planning, it may build the structure in a way that suits it. That may not suit you, and you will only feel the full cost later, when it is too late to rework the structure because the team lacks the time and capacity.

We migrated the automated tests

This was the most labour-intensive step. The new microservice turned out to have around 200 tests covering its functionality. It might seem that you could write instructions for Claude and ask it to move the tests for the new microservice into a separate folder.

But when I tried this with one test file out of 19, I realised it would not work. We had to keep the tests for this functionality in the monolith and also bring them over to the new microservice, always with new, correctly created entities. In my experience, Claude kept forgetting to apply certain account and instrument settings. The problem was not a poor prompt. A broker's system is highly complex, with countless combinations of instrument, account and commission settings, among others.

So we made a decision that is unfashionable in the age of AI: to migrate the tests by hand. While manually reviewing and running the old tests, we found surprising things that no one would have thought to ask an agent to check. So if you are worried that AI will replace you, rest assured: there is still a role for us humans.

Here is how we migrated the tests in more detail.

1. First, we copied the test code and created new entities to remove dependencies between the monolith tests and the new service. There was nothing complicated about this. Doing it by hand was faster than writing prompts for AI.
2. Second, we ran every migrated test and compared it against a reference, which for us was the monolith. This was a crucial step. It was here that a large number of bugs in both the new service and the tests came to light.

The test framework was more than seven years old. Over that time, the company had introduced new standards and rules for writing tests. During the migration, we found:

• Missing setup helpers. Setup helpers are methods that create the settings we need in automated test code. Some settings had been configured manually, without them. This meant that when tests for the new microservice ran in a new environment, they could fail not because of a product bug but because of an error in the tests themselves, since the preconditions had been defined incorrectly. The team would have wasted time investigating non-existent bugs, and real ones could have been lost among false failures.
• Large test files that slowed down runs. We split these large files into several smaller ones so the runner could execute tests in parallel.
• Test gaps. This is how it happened. A tester would look at a test file, notice that a certain precondition was not being checked, dig deeper and realise that a test was missing, even though the scenario was very important. The tester would then either write the test straight away during the migration, if it was quick, or create a backlog ticket.
• Inaccuracies in the documentation. When closing a test gap, the tester worked through a scenario no one had checked before and compared the service's behaviour with the documentation. Sometimes the documentation described the case inaccurately or not at all. We updated it, of course, sometimes ourselves along the way and sometimes with help from technical writers.
• False positives. We found several tests that were effectively checking None == None and were always green. This was caused by errors in how helpers and utilities handled data. In reality, the test checked nothing, but until the migration no one had thought to question a test that always passed.
• Unnecessary lines of code that slowed down runs. In day-to-day work, we had no time to think about optimisation. During the migration, it became clear that we could speed up the tests by removing unnecessary code. For example, we cleaned up setup classes and methods that configured the same thing for every test class in a file. This brought a significant speed-up: run time fell from nine minutes to two and a half.

Once the core set of tests for the microservice was ready, we started adding new functionality. This work ran in parallel with development. After each commit, we ran the automated tests and then investigated any failures. It was interesting to see the Pareto principle in action. To paraphrase it slightly: 20% of the time spent at the daily stand-up solves 80% of the problems. We openly discussed the service's needs with the whole team, not only within QA but with the developers too. This significantly accelerated the service's development.

These open discussions brought other tangible benefits as well. We increased unit test coverage, shared knowledge within the team and deepened our product expertise.

What we achieved

Once our new creation was stable, it was time for the routine work: adding the automated test jobs to the nightly schedule, gradually switching client traffic to our service and seeing it settle into production.

Now for the most important part. Here is how the problems we set out to solve look today.

• Regression testing was slow. A Pre-margin regression run now takes two minutes. We moved the tests out of the monolith and optimised the run. As a result, we can check the service when rolling out a new feature 30 times faster.
• Incidents were hard to investigate. The narrow domain now sits in a separate microservice, so we no longer spread our attention across neighbouring functionality during checks. Bugs are easier and faster to localise.
• Duty engineers were overloaded. The work has become easier. To test Pre-margin, the duty engineer no longer needs to dig into the monolith's vast logic.

And this is roughly how the tester looks now: tired, but not broken.

Beyond the obvious problems we wanted to fix, we managed to solve several other important issues:

• Test quality improved. We fixed all the false positive tests, solved the flaky test problem and migrated the tests in line with the company's current coding patterns.
• Coverage became broader. Detailed analysis revealed test gaps, and we closed most of them during the migration. Only a small number moved to the backlog.
• Documentation became more accurate. The service's functionality is now described more fully, with more examples.
• The team deepened its product expertise. After diving deep into the service, everyone involved in developing Pre-margin now sees the full picture.
• We understood the real scale of our technical debt. Taking stock of the tests and migrating them showed it in full. I suspect few teams can say with confidence that they fully understand the technical debt in their own service.

Why AI played a modest role

A few more words about AI.

We gave AI the work it is genuinely good at: quickly analysing more than 3,500 tests and compiling a reference from them, and helping us stress-test our architectural ideas. This gave us a fast start. The migration itself, however, we did by hand, for two reasons.

The complexity of the domain. A broker's system relies on countless combinations of instrument, account and commission settings. In our very first experiment, Claude regularly missed some of them, and an error in the tests of a financial system is costly.

The economics. When you are moving a single service, writing, debugging and checking migration prompts takes longer and costs more than migrating the tests yourself. At this scale, prompt engineering simply does not pay off.

That said, AI remains a great tool when used wisely. Imagine you have a highly advanced automated surgical device. It can remove anything from a patient, but you do not know what needs to be removed, why or when. Everything will probably be removed perfectly, but the patient will die in production. Tests work the same way. You, not Claude's auto mode, should set the decisions, the strategy, clear boundaries and the instructions. Blind trust in AI would not have given us even 70% of the value we ended up with.

The manual migration was not a rejection of AI. On the contrary, it was preparation for automation. Having gone through the whole process by hand, we understood what steps the migration involves, where the pitfalls lie and which preconditions must not be missed. We now have a foundation to formalise the process and bring AI into it when the time comes to move the next services out of the monolith.

People are the architects and strategic minds behind large, complex decisions that change perspective. AI is the pair of hands that does what has been described and formalised in detail. And it can only work under human supervision.

Key takeaway

The main thing I want to say to fellow testers is this: if testing one big thing is causing you real pain, it can probably be split into smaller pieces. The process will become easier, and along the way you will learn a lot about your product.