Running Jest tests in parallel in Gitlab with multiple jobs
I've been working a lot lately with Jest tests. Some projects had 1k+ of Jest unit tests. Another project was using Jest for running integration and E2E tests. In both cases we wanted to get feedback on CI as fast as possible. Our initial solution was following:
- Split tests into different folderds
- Make sure that there's one Jest
spec
file per test - this way Jest will run tests in parallel better - Create a series of npm scripts in
package.json
file - Create a list of similar jobs in
.gitlab-ci.yml
that trigger different npm scripts - Gather together artifacts from this tests and aggregate a one Test Report.
The big problem with this approach is that it's not really scalable. Also, with unit tests it's not always convinient to group them by folders and always check that you have not missed or forgot some folder to inlucde into the test run.
Luckily, one of my colleguas showed me a better approach to this. It appears that Jest has test-sequencer which can be extended. It allows to split tests into separate groups and assign a list of tests per node. Gitlab has a feature that allows you to run the same job in multiple instances. It's called parallelization
This two factors enabled us to write an easily code that would scale in the future with no manual input fromt he developers. I have prepared a small Gist that gathers all the changes that are required to add to your Jest framework and things should just work faster :)