Challenge

The client was experiencing rapid growth, conducting over 200,000 industrial inspection flights, but their QA processes weren’t scaling effectively with their expansion. Key challenges included:

• Fragmented testing processes across different projects
• Development team handling test automation independently
• Disconnected automation efforts
• Lack of standardized documentation
• Insufficient QA resources to support growth

Solutions

Our team conducted a comprehensive QA audit, identifying bottlenecks and gaps in their processes. We introduced a detailed test strategy, standardized workflows, and improved onboarding plans. 

• Analyzed existing QA documentation and processes
• Identified gaps and bottlenecks in current workflows
• Created detailed improvement recommendations
• Assessed team scaling requirements
• Developed detailed onboarding documentation
• Established clear roles and responsibilities

Our team recommended aligning automation testing with manual efforts and offered a metrics framework to track progress and ensure scalability.

Solutions

We have assembled a dedicated manual and automated testing team led by a Project Manager. Our team expanded from 3 to 20+ members, reflecting the increasing scope and complexity of the project. To ensure fruitful collaboration, we have:

• Seamlessly integrated with the client’s in-house and remote development teams.
• Participated in daily stand-ups and weekly Scrum meetings, aligning with the client’s 2-week sprint schedule.

We have created and over years enhanced Quality Assurance strategy at the intersection of Manual and Automated QA, Cloud infrastructure and new AI features and solutions.

Comprehensive Manual Testing

Surely, we cover all the standard bases, all the classics of manual testing, but with new features popping up and millions of orders pushing the system to its limits, every day brought fresh challenges. Our manual testing team had to be part detective, part daredevil. Here are some of the testing types we covered to keep manual QA bar high:

• Functional. Ensuring all features worked as intended across the platform.
• Ad-hoc. Identifying unexpected issues through exploratory testing.
• Acceptance. Verifying that the software met all acceptance criteria.
• Smoke. Quickly assessing the stability of new builds.
• Regression. Ensuring new changes didn’t negatively impact existing functionality.
• Accessibility. Verifying the platform’s usability for users with disabilities.
• UI/UX. Assessing the user interface for consistency and user-friendliness.
• Cross-platform. Verifying consistent performance across iOS, Android, and web platforms.
• Confirmation. Validating that identified issues were properly resolved.
• End-to-end. Assessing the entire user journey from product selection to order completion.

But we didn’t stop there. Our manual QA team became the architects of testing, creating unified test plans that brought together both manual and automated efforts. The goal was to create a testing ecosystem that would evolve with the platform. We set up TestRail to showcase both manual and automated test results, complete with screenshots and comments when needed. 

Advanced Automated Testing

With millions of customizable products and a constant stream of new features, our testing needs grew exponentially. Manual testing alone couldn’t cover everything without slowing down releases. We needed a way to test more thoroughly, more frequently, and across more devices – all without ballooning our team size. Automation was the clear solution. Here’s how we built our automation strategy to meet these challenges:

• Developed scripts for simultaneous testing on multiple mobile devices.
• Implemented automated consolidation of test results into comprehensive reports.
• Introduced automatic video recordings of failed tests to aid in debugging low-reproducibility issues.
• Continuously expanded the scope of automated tests to cover an increasing portion of the application.
• Сreated a mobile framework from the ground up, integrating CI/CD via Jenkins and implemented a custom device farm.
• Engineered 80 tests for iOS applications with Swift and XCUITest; optimized the CI/CD pipeline, allowing early error detection.
• Cataloged a Pytest API framework with 100+ automated backend tests.
• Set up the mobile web (mobile browsers Safari/Chrome) Appium framework from scratch (with BrowserStack, and lately, transferred to LambdaTest) and achieved 180 e2e test coverage.

By ramping up our automation efforts, we dramatically cut down on regression testing time. This was a game-changer for our manual testers. Freed from repetitive tasks, they could focus on exploratory testing of new features, edge cases, and complex scenarios that machines can’t handle.

Cross-Platform Device Testing

Some clients have the latest iPhones; others use older Android tablets. We needed to ensure our client’s app worked great on all of them. It’s not just about different screen sizes — each device has its own quirks. To address this issue, we have:

• Used over 40 iOS and Android devices, including both flagship and older models.
• Ensured consistent performance across a wide range of hardware configurations.

AI and Machine Learning Integration

Our client took personalization to new heights with AI. They built a smart system that does more than just sort products. This AI helps customers design items, suggesting themes, colors, and layouts. It even recommends how to place photos and add special touches. It’s like having a personal designer built into the app. To ensure this AI-powered personalization worked and delivered real benefits, our team:

• Conducted comprehensive testing of new AI-driven personalization features.
• Provided regular feedback to the development team, contributing to iterative improvements.
• Ensured successful rollout of the AI-enhanced version to 100% of users.

Cloud Migration Support

For a platform handling millions of orders yearly, moving to the cloud is inevitable necessity. But migrating such a vast system, with its mountain of data and complex operations, is like performing heart surgery while the patient runs a marathon. Every piece had to move seamlessly, without disrupting the constant flow of orders.​​

• Assisted in the transition of servers and resources to cloud infrastructure.
• Maintained regular testing duties while supporting the migration process.

Scalability and Performance Testing

• Implemented load testing scenarios to simulate high-volume transaction periods.
• Verified system stability and responsiveness under peak usage conditions.

Continuous Improvement and Adaptation

• Regularly reviewed and updated testing strategies to align with the evolving product.
• Expanded our testing scope to cover newly acquired software products and integrations.

Challenge

• Complex API structure. The API system was extensive, with numerous endpoints and dynamic data, requiring comprehensive understanding and efficient testing within a tight timeframe.
• Real-time data updates. Ensuring that real-time data updates were accurately reflected on the client side was a significant challenge.
• Billing and user flow. Verifying the functionality of billing plans, including activation, special conditions, payments, deactivation, and overdue payments, added another layer of complexity.
• New features to test. Just before the release, the client was finalizing new features and pop-ups for premium accounts, necessitating thorough UX/UI testing.

Solutions

We assembled a dedicated team for the job: 5 professionals, including 4 manual QA and a part-time Team Lead, all with a deep understanding of API testing and dynamic data environments.

Our collaboration with the development team was crucial. More than just running tests, we actively reviewed requirements, scrutinized test cases, and identified defects.

Our comprehensive testing strategy included:

• Functional testing. We verified that all data was displayed correctly across the client’s sites. We manipulated the database to simulate dynamic data changes and checked real-time updates.
• API testing. We conducted specific API testing, checking the availability and functionality of each endpoint.
• Billing and user flow testing. We tested billing plans and various user scenarios using test accounts.
• UI/UX testing. We focused on the new features and pop-ups for premium accounts, ensuring a smooth and user-friendly experience.

With a solid team and a clear strategy, we managed to cover over 500 test cases. We used tools like Postman for API testing, which helped us identify several serious bugs.

Challenge

In the process of revolutionizing software testing through AI, we encountered a series of testing roadblocks that demanded innovative solutions. These ranged from dealing with duplicated bug issues to enhancing test coverage and refining bug report creation, all the way to reimagining requirement testing and efficient report generation.

Overcoming Key Testing Roadblocks

Bug triage. Duplicated issues were a recurring problem, resulting from multiple authors logging defects into the bug tracking system.

Test coverage. Due to high-priority test execution activities, time for comprehensive documentation creation was often compromised, predominantly covering only positive scenarios.

Bug reports creation. The issues our client faced were being discussed across various forums and thematic groups, yet the simplicity of the textual explanations from customers necessitated a conversion into a more formal bug format.

Requirements testing. There was a need for structured user stories, conforming to a common user story creation approach.

Report creation. Report creation during regression testing linked data from disparate resources, a process that was significantly time-consuming.

Facing the Challenges of AI Integration in Testing

While tackling the current software testing processes, we’ve encountered additional challenges that extended beyond their initial concerns.

Confidentiality and data security. Given the sensitive nature of the software we were working with and the proprietary technology involved, maintaining the confidentiality of their data and ensuring the security of their software was paramount. Strict protocols had to be enforced during the testing process to prevent data leaks or any potential breaches.

False positives and negatives. When utilizing AI tools for software testing, there’s always a risk of encountering false positives—issues flagged as errors that are actually correct—and false negatives—actual errors that are overlooked. Mitigating these false responses, which could lead to wasted resources or overlooked problems, was a significant challenge.

Tool integration and compatibility. Another challenge was to integrate and make compatible the different AI tools—DeepTriage, ChatGPT, Grammarly, and Microsoft Power BI—with our client’s existing system. A smooth and seamless integration was crucial for the efficient operation of the AI tools and to prevent any disruptions to the current workflows.

Continuous monitoring and adjustment. AI tools, while highly effective, need continuous monitoring and adjustments to provide optimal results. The challenge was to ensure consistent and regular oversight, tweaking the AI models when needed and ensuring they evolve with the changing software dynamics and requirements.

Addressing these additional challenges required careful planning, strategic decision-making, and effective communication with our client to ensure a successful overhaul of their software testing process.

Solutions

To navigate the mentioned challenges, QArea suggested the adoption of several AI tools:

Bug triage. We recommended the use of DeepTriage, an AI-powered tool developed by Microsoft researchers. This tool leverages machine learning to streamline the bug triage process and assign bugs efficiently.

Test coverage. We deployed ChatGPT to work with the text format, typically represented by requirements, and retrieve test cases from it.

Bug reports creation. We utilized ChatGPT to analyze, structure, and rewrite customer experience reviews into a format suitable for enhancement and issue reports.

Requirements testing. ChatGPT and Grammarly were jointly used to restructure, analyze, and improve software requirements, ensuring a consistent style and correcting spelling errors.

Report creation. For creating cohesive and efficient reports, we utilized Microsoft Power BI. This tool supports data integration from various sources, offers data modeling capabilities, and provides AI-driven insights.

Practical Approaches: Enhancing Software Testing with Focused Strategies

Our team implemented several strategic solutions to ensure a seamless transition and an efficient software testing process.

To safeguard data confidentiality, we enforced security protocols and implemented encryption techniques. Also, we have never presented AI tools with 100% true data, changing numbers, names and places, as well as any information that may have been considered sensitive. Later, we have used an algorithm to use the true data at our side and receive precise results. 

To mitigate the issue of false positives and negatives, we employed validation techniques and cross-checks in our testing process. This approach ensured that any errors flagged by AI tools were verified, and any overlooked errors were caught in subsequent testing stages.

For the smooth integration of AI tools, we performed careful compatibility testing and gradually introduced the tools, ensuring there were no disruptions to existing workflows. Additionally, we provided technical support during and after the integration process to address any emerging issues promptly.

We organized a few training sessions for the client’s team to understand and effectively use the new AI tools. Additionally, we provided detailed documentation and ongoing support to help the team adapt to the new tools and processes.

Finally, as requested by our client, we established a dedicated team to monitor the performance of the AI tools continuously, adjusting parameters and refining the AI models as needed. By maintaining a close watch on the tools’ performance and making timely adjustments, we ensured that they continued to deliver optimal results, keeping pace with the evolving dynamics and requirements of the software.

Challenge

In this project, we have faced numerous environments that needed to be checked during the day, two versions of the application, each of which required at least three versions of auto tests. We provided this project with three automation and two manual QAs, who had to work with the QA manager and two manual QAs from the client’s side. The overall number of developers on this project reached 20+, which guaranteed a non-stop flow of updates, new features, and releases that required quality assurance services. Here are some significant tasks we had to tend to initially:

• Establishing QA workflow and developing needed test-related documentation from scratch;
• Support and continuous QA for new features introduced by developers;
• Optimizing workflow with the client’s ticket management system to determine automation candidates;
• Improving internal workflows with multiple simultaneous runs on the Jenkins server;
• Ensuring rational use of the allocated space on the Jenkins server and pre-paid Testrail plan;
• Ensuring fast automation deployment on separate client’s environments.

Solutions

Teaming up with the client’s QA specialists, we have created and implemented product-specific API, performance, regressions, and smoke tests. Here is more about what we have achieved working on this project:

• Implemented integrations between the following pairs: Liquidplanner & Testrail; Jira & Testrail. It allowed better issues’ detecting and tracking the latest status of the tickets.
• Using the mentioned integrations, generated a custom QA daily report for better visibility of QA team activities and depiction of issues that were found during the day;
• Implemented performance testing using k6.io that allowed coverage of the main workflows using API endpoints for four environments;
• Provided analysis of the result/metrics, establishing the maximum number of users a particular environment can support;
• Organized and provided daily support for four environments, using more than 2400 tests in total; 
• Created and implemented tools for fast analysis. Our QA created a custom tool, which collects test results from each environment and groups them by error hash. It allowed to bind all errors during one action easily;
• Optimized tests’ architecture, internal workflows, and custom tools/reports that allowed making analysis much faster and determining where the bug or test needs to be re-run;
• Improved test speed by rewriting tests using API endpoints for the generation of data required by tests. This approach was tried on a bunch of tests, showing a significant reduction in execution time. Now, it is being implemented for all the environments.
• Improved tests’ execution, introducing a new, custom tool for quick start of tests which should be re-run during the workday; 
• Implemented Readable screenshots feature that allowed adding the test’s output to screenshots, reducing time for locating the issue;
• Developed interactive visualization of test coverage—a unique solution that allows us in real-time to see where we don’t have test coverage.

Automation

• Achieved fast automation deployment on separate client’s environments;
• Used Jenkins CI to ensure all runs are running on schedule. After each rebuild of the environment, smoke and the main regression tests are launched;
• To ensure seamless UI automation, used a custom combination of ProtractorJS, Selenium, Jasmine programming languages

Business processes

To enhance internal business processes, we created a tracker for all important indicators:

    – how many users / roles were created 

    – how many files were uploaded

    – how many messages / tags were created, etc.

Now, this data comes from each test during a run and shows QA team, DevOps, Backend developers how many tests can be run in parallel and how many resources should be added to each server, where massive requests trigger server errors (MySQL, Redis, RabbitMQ, etc.).

Challenge

The client approached us to perform quality assurance on the product over six years ago, when the solution had a bare minimum of functionality and was in its early stages. Our team was tasked with automating the tests from the ground up, which is a challenge of its own, as there were barely any contributions from the manual QA team to base our work upon. Other key objectives and challenges for this project included:

• Developing a comprehensive testing strategy from scratch, given that there was little to no documentation at the start of the project;
• Creating in-depth integration tests to check how the core solution interacts with additional products it’s paired with;
• Finding an effective way to work with large databases that are constantly getting bigger;
• Checking how accurately the application calculates the time and money spent on cloud services within one event;
• Ensuring smooth communication and collaboration between our team, the manual QA team, and the development team, which are all located in different parts of the world;
• Implementing various complex technologies, mainly cloud-related, to fulfill the requirements of the stakeholders;
• Setting up an effective CI/CD process despite the ever-growing number of tests;
• Verifying the correct localization of the project, which is available in 5+ languages;
• Achieving the most practical use of the client’s resources while maintaining the highest quality of testing.

Solutions

After analyzing the existing state of the application and setting the groundwork for the collaboration with the manual QA team, we created a test plan that we have since updated and enhanced countless times to meet the changing requirements of the product. At different stages of the project, our team consisted of:

• An Automation QA Lead
• 2 to 3 Automation QAs

From the very beginning, our team focused on end-to-end testing to deliver the most consistent and reliable results. Besides that, we used:

• Smoke testing
• Regression testing
• UI testing
• Functional testing
• Localization testing
• Performance and Load testing
• Security testing

Over the course of the project, we have covered over 77% of functionality tests, up to 92% of API tests, and a large chunk of performance, load, and security tests. Overall, we have automated over 1,100 tests, and that number continues to grow. Through daily meetings and effective communication with the development and manual QA teams, we have ensured seamless collaboration across all departments.

In the early stages of the product’s development, the team on the client’s side was dedicating most of their time and effort to the functionality of the product, while the performance of the app was often treated as an afterthought. After our team discovered significant performance issues, we updated the testing strategy to include this aspect as well. As a result of our input, the development team has been successfully optimizing the application’s performance for over a year now.

As soon as the application’s user base started growing, we began testing not only the core product, but also the features and modules of the product tailored to the needs of the end customers. To date, we have tested over 20 of those solutions and continue providing ongoing support.

Challenge

By the time we received the request from the client, there already was an existing solution with some functionality. However, it was created using outdated technologies, many of which are no longer supported by the developers. So, our team was tasked with developing a web app solution from scratch. The objectives of the project included:

• Get rid of the old unproductive way of retrieving specifications from spreadsheets;
• Speed up the dimensions and cost calculations;
• Discontinue the use of deprecated technologies;
• Ensure rapid and smooth scalability;
• Correctly visualize different shower solutions and specifications despite not having physical access to them;
• Provide customers with an engaging user experience;
• Integrate the client’s existing CRM into the new solution.

After reviewing what was needed, it was decided to split the project into two products:

• A classic configurator that allows customers to use various filters and suggests several existing solutions to choose from;
• A custom configurator that uses even more filters and either suggests one model that perfectly fits the requirements or calculates the cost of a custom solution.

Solutions

From the very beginning, our team has been working on two configurator solutions at the same time, seamlessly switching between them to satisfy the needs of our client more efficiently. This is what we did to fulfill our duties:

• Created the flow to outline the customer journey;
• Created the project architecture to support the flow;
• Gradually added new elements to the architecture to make it meet the client’s business goals;
• Collected all the relevant data, including product specifications and prices, in one source;
• Provided the design for every element of the solution;
• Developed the front-end and back-end parts of the project;
• Used a combination of Git, Jenkins, and Laravel for migrations, ensuring the consistency and stability of the code in all three environments (development, staging, and production);
• Providing live support for the released parts of the project, alongside delivery of new work.

The choice of technologies for the back-end and front-end parts of the project was influenced by the end goals of the project, as well as the current state of the popular programming languages. Specifically, we chose Laravel as our PHP framework and Angular for the front end because both of these technologies make the most sense for enterprise software solutions and are also actively supported by their developers.

The classic configurator was completed first, whereas the custom configurator is a more complex solution and therefore required more time. The work on the custom configurator project was divided into three stages:

• Phase A: Preparing the solution to be used by the client’s sales managers.
• Phase B: Upgrading the solution for use by third-party dealers, which involves disabling certain features and introducing new price calculation functionality.
• Phase C: Releasing the version for the end customers with more opportunities to adjust the functionality — e.g. restrict access to certain areas of the app.

Throughout the whole development process, we also performed continuous software testing within the Agile methodology to ensure the consistency and quality of the solution at every step of the way. With the help of several testing types, including UI/UX, functional, performance, and compatibility testing, we verified the flawless state of each build. We also used smoke and regression testing after the development team introduced changes to the code. Moreover, we used data-driven testing to increase the efficiency of our QA efforts. Finally, we performed user acceptance testing to make sure the software successfully fulfilled its business requirements.

Because the client’s enterprise consists of several smaller entities involved in the development process, the project had a large number of stakeholders who often had differing opinions on what the solution should look and feel like. We overcame this challenge thanks to effective communication and being able to offer a solution that met the needs of all the parties involved.

Challenge

Providing QA services for Fintech and Banking solutions comes with extra responsibility, as we are operating with apps that will further process security-sensitive data. Here is the scope of work we were hired to do in this project:

• Establishing QA workflow and developing needed test-related documentation from scratch;
• Testing app’s security and high-performance level prior to the next round of work with sponsors;
• Support and continuous QA for new features and significant releases; 
• Ensuring correct operation of the KYC client identification service;
• Covering the testing load, freeing time for the product owners before the critical release.

Solutions

To guarantee client’s fintech product answers the audience’s expectations and high standards of the investment market, we’ve assembled an expert team consisting of a Team Lead, 3 manual and 1 auto QA engineers of Middle and Senior levels. All the team members had a strong background in testing multi-user mobile applications with sensitive data.

Overall, the project took more than 1 year. Work was done in several iterations, corresponding with new releases, and in total, took about 6 months of active testing time. Here is what our team has achieved so far in this project:

• Introduced essential metrics to monitor bug statistics;
• Contributed to testing workflow and provided necessary pieces of documentation necessary to ensure quality and uninterrupted QA process;
• Optimized tests’ architecture to ensure analysis is made much faster and determine where the bug or test requires re-runs;
• Organized security, performance, usability, and regression testing;
• Made sure clients’ key extra functionality — “​​recurring investments” — works as expected, charging the planned amount at the right time;
• Checked accuracy of the following functions: trackers (for a top dozen most popular stocks), instant funding, multiple income streams, etc.;
• Ensured correct operation of the KYC client identification service, overcoming challenges that came with various previously unchecked IDs provided by the client to pass the verification;
• Analyzed end-to-end user path to ensure every step is intuitive, self-explanatory, answers customers’ expectations, and performs as planned by the developers.

Automation

Automized performance and security testing:

• Automated scan of production servers, ensuring seamless deployment to live environment;
• Performance (Load) testing with SoapUI Pro. Using this commercial extension helped us to ease many recurring tasks and save time.

Technologies

MobSF and Kali Linux were employed for security testing: 

• MobSF was used in the early stage of new release development to ensure more efficient debugging and increased cost efficiency;
• Kali Linux helped us gather, analyze and report vital information about the app’s vulnerabilities. Considering the sensitive nature of investors’ financial information, we had to make sure no potential attackers could extract any critical data.

Localization-related aspects

It was not our first time working with a client from GCC, and we were ready to adjust to the local working environment and requests.

Schedule. Specialists on the client’s side were available from Sunday to Thursday, with Friday and Saturday off. We were not required to follow the same program, but planned sliding shifts to ensure one of our engineers was available for emergencies on Sundays. 

Islamic stock filter. The Gulf region is a predominantly Islamic domain shared by Sunni and Shia Muslims. Sharia law traditions are strong, and following them is vital for most of our client’s customers. In the scope of our work, we were tasked to test additional Sharia-compliant stocks functionality. We also made UI suggestions for an improved depiction of compliant stocks to attract more conservative customers. 

Solutions

We have provided this SaaS MVP project with 1 business analyst, 1 UX/UI designer, and 2 developers. Later, our client agreed that a decision to strengthen the development team with a skilled business analyst was inspired. She helped to turn generalized plans and high-level processes descriptions into precise and detailed requirements for the system’s functions, clear epics, tasks, and acceptance criteria.

Preparing a SaaS MVP development plan, we have:

• Assessed the client’s technical process and limitations that define a possible tech stack;
• Elicited and formalized business and technical requirements, established projects’ success criteria; 
• Prepared a branched user journey map;
• Assessed the value of each feature, chose the ones that will represent the solutions’ benefits the most, and defined the scale of deployment;
• Ideated subscription plans’ options;

Once the preparation stage was complete, our dedicated development team, in cooperation with the UX/UI designer, has:

• Designed the project’s architecture with all the functional, design and technical components;
• Developed a basic, but scalable, Edu-platform infrastructure;
• Ideated and realized a multi-functional dashboard;
• Ensured transferring from a monolithic to a scalable microservices architecture;
• Realized cloud functionality via Azure;
• Offered and realized the most suitable UX/UI strategy based on the market’s most profitable references and original design solutions;
• Realized skill paths functionality that the users and their supervisor can access;
• Introduced limited but highly representative gamification options (badges, ratings) to boost user’s progress and increase return ratio.

Solutions

The subject of our testing efforts is a HIPAA-compliance-certified project, so it was necessary and appropriate for engineers working on the project to understand and comply with HIPAA regulations. Therefore, we made sure to provide the client with QA engineers who possess deep domain knowledge and experience in Healthcare software development and testing.

We provided the project with 2 part-time QA engineers and 1 full-time dedicated QA engineer. The project was using the Scrum framework for project management.

Despite the fact that we operated in different time zones, together with the client’s in-house QA Team Lead, our QA engineers established transparent, efficient communication with the development teams and quickly got in sync with the workflows on the project.

In close collaboration with the in-house project team, our QA engineers:

• established efficient bug tracking and reporting workflows
• created and executed test cases and performed all the necessary QA activities on a daily basis
• created new and updated the existing documentation, including test scenarios, traceability matrices, and bug reports
• automated regression testing using the Selenium IDE
• reorganized the QA process to prepare the project for further test automation
• provided recommendations on documentation and workflow optimization