So you have written your Playwright tests and they are chugging along fine. Over a while, the number of test cases increased. The time it takes to run the tests starts to increase steadily. What do you do !! How do you get it under control ??
Let's assume for argument's sake you have 500 playwright tests for the app.
First of all, I would break down the 500 tests into 4 or 5 separate categories depending on how the features are tested. So, 5 blocks of 100 tests running in parallel. Any failures will ensure that you will have to retry one block of 100 tests instead of 500 tests.
If there are authentication flows involved, reuse the auth via fixtures to bypass the login every time. This should save a significant amount of time per UI test.
Dockerise the environment
- One can use Github actions to parallelise tests
Secondly, you can dockerize the environment and run the tests on a c5.4x large machine (or any memory-optimised EC2 instance) and run 12 parallel tests. One can improve the costs by using a mix of on-demand & spot instances for the workload at hand.
FROM mcr.microsoft.com/playwright:v1.37.0 RUN mkdir -p /opt/google/chrome RUN ln -s /ms-playwright/chromium-1048/chrome-linux/chrome /opt/google/chrome/chrome RUN npm uninstall -g yarn && \ apt-get remove -y nodejs python3 --purge && \ apt-get update && \ apt-get install -y openjdk-11-jre uuid-runtime && \ apt-get autoremove -y && \ apt-get clean -y && \ rm -rf /var/lib/apt/lists/* # Remove unused browser RUN rm -rf /ms-playwright/firefox-* /ms-playwright/webkit-* # Install nvm with node and npm RUN mkdir -p /usr/local/nvm ENV NVM_DIR /usr/local/nvm ENV NODE_VERSION v16.19.1 RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.2/install.sh | bash RUN /bin/bash -c "source $NVM_DIR/nvm.sh && nvm install $NODE_VERSION && nvm use --delete-prefix $NODE_VERSION" # Add node and npm to the PATH ENV NODE_PATH $NVM_DIR/versions/node/$NODE_VERSION/bin ENV PATH $NODE_PATH:$PATH WORKDIR /app RUN npm install -g yarn --loglevel silent RUN wget https://repo.maven.apache.org/maven2/io/qameta/allure/allure-commandline/2.17.2/allure-commandline-2.17.2.tgz RUN tar -xvzf allure-commandline-2.17.2.tgz && rm -rf allure-commandline-2.17.2.tgz ENV PATH /app/allure-2.17.2/bin:$PATH RUN yarn install --cache-folder ~/.cache/yarn --frozen-lockfile --ignore-scripts
Once you are done with these changes, you can look into mocking the API responses for 3rd party integrations. This will help with getting rid of one of the factors of flakiness
- Need to ensure the 3rd party mocks are still valid. You can use ZOD to ensure the response structure remains the same.
Improving and optimising the pipeline execution time is a continuous process.
Did you find this article valuable?
Support Jags by becoming a sponsor. Any amount is appreciated!