Website & API Testing for Digital Asset Management Company

Challenge

In this project, we have faced numerous environments that needed to be checked during the day, two versions of the application, each of which required at least three versions of auto tests. We provided this project with three automation and two manual QAs, who had to work with the QA manager and two manual QAs from the client’s side. The overall number of developers on this project reached 20+, which guaranteed a non-stop flow of updates, new features, and releases that required quality assurance services. Here are some significant tasks we had to tend to initially:

• Establishing QA workflow and developing needed test-related documentation from scratch;
• Support and continuous QA for new features introduced by developers;
• Optimizing workflow with the client’s ticket management system to determine automation candidates;
• Improving internal workflows with multiple simultaneous runs on the Jenkins server;
• Ensuring rational use of the allocated space on the Jenkins server and pre-paid Testrail plan;
• Ensuring fast automation deployment on separate client’s environments.

Solutions

Teaming up with the client’s QA specialists, we have created and implemented product-specific API, performance, regressions, and smoke tests. Here is more about what we have achieved working on this project:

• Implemented integrations between the following pairs: Liquidplanner & Testrail; Jira & Testrail. It allowed better issues’ detecting and tracking the latest status of the tickets.
• Using the mentioned integrations, generated a custom QA daily report for better visibility of QA team activities and depiction of issues that were found during the day;
• Implemented performance testing using k6.io that allowed coverage of the main workflows using API endpoints for four environments;
• Provided analysis of the result/metrics, establishing the maximum number of users a particular environment can support;
• Organized and provided daily support for four environments, using more than 2400 tests in total; 
• Created and implemented tools for fast analysis. Our QA created a custom tool, which collects test results from each environment and groups them by error hash. It allowed to bind all errors during one action easily;
• Optimized tests’ architecture, internal workflows, and custom tools/reports that allowed making analysis much faster and determining where the bug or test needs to be re-run;
• Improved test speed by rewriting tests using API endpoints for the generation of data required by tests. This approach was tried on a bunch of tests, showing a significant reduction in execution time. Now, it is being implemented for all the environments.
• Improved tests’ execution, introducing a new, custom tool for quick start of tests which should be re-run during the workday; 
• Implemented Readable screenshots feature that allowed adding the test’s output to screenshots, reducing time for locating the issue;
• Developed interactive visualization of test coverage—a unique solution that allows us in real-time to see where we don’t have test coverage.

Automation

• Achieved fast automation deployment on separate client’s environments;
• Used Jenkins CI to ensure all runs are running on schedule. After each rebuild of the environment, smoke and the main regression tests are launched;
• To ensure seamless UI automation, used a custom combination of ProtractorJS, Selenium, Jasmine programming languages

Business processes

To enhance internal business processes, we created a tracker for all important indicators:

    – how many users / roles were created 

    – how many files were uploaded

    – how many messages / tags were created, etc.

Now, this data comes from each test during a run and shows QA team, DevOps, Backend developers how many tests can be run in parallel and how many resources should be added to each server, where massive requests trigger server errors (MySQL, Redis, RabbitMQ, etc.).