Solutions
We have assembled a dedicated manual and automated testing team led by a Project Manager. Our team expanded from 3 to 20+ members, reflecting the increasing scope and complexity of the project. To ensure fruitful collaboration, we have:
- Seamlessly integrated with the client's in-house and remote development teams.
- Participated in daily stand-ups and weekly Scrum meetings, aligning with the client's 2-week sprint schedule.
We have created and over years enhanced Quality Assurance strategy at the intersection of Manual and Automated QA, Cloud infrastructure and new AI features and solutions.
Comprehensive Manual Testing
Surely, we cover all the standard bases, all the classics of manual testing, but with new features popping up and millions of orders pushing the system to its limits, every day brought fresh challenges. Our manual testing team had to be part detective, part daredevil. Here are some of the testing types we covered to keep manual QA bar high:
- Functional. Ensuring all features worked as intended across the platform.
- Ad-hoc. Identifying unexpected issues through exploratory testing.
- Acceptance. Verifying that the software met all acceptance criteria.
- Smoke. Quickly assessing the stability of new builds.
- Regression. Ensuring new changes didn't negatively impact existing functionality.
- Accessibility. Verifying the platform's usability for users with disabilities.
- UI/UX. Assessing the user interface for consistency and user-friendliness.
- Cross-platform. Verifying consistent performance across iOS, Android, and web platforms.
- Confirmation. Validating that identified issues were properly resolved.
- End-to-end. Assessing the entire user journey from product selection to order completion.
But we didn't stop there. Our manual QA team became the architects of testing, creating unified test plans that brought together both manual and automated efforts. The goal was to create a testing ecosystem that would evolve with the platform. We set up TestRail to showcase both manual and automated test results, complete with screenshots and comments when needed.
Advanced Automated Testing
With millions of customizable products and a constant stream of new features, our testing needs grew exponentially. Manual testing alone couldn't cover everything without slowing down releases. We needed a way to test more thoroughly, more frequently, and across more devices - all without ballooning our team size. Automation was the clear solution. Here's how we built our automation strategy to meet these challenges:
- Developed scripts for simultaneous testing on multiple mobile devices.
- Implemented automated consolidation of test results into comprehensive reports.
- Introduced automatic video recordings of failed tests to aid in debugging low-reproducibility issues.
- Continuously expanded the scope of automated tests to cover an increasing portion of the application.
- Сreated a mobile framework from the ground up, integrating CI/CD via Jenkins and implemented a custom device farm.
- Engineered 80 tests for iOS applications with Swift and XCUITest; optimized the CI/CD pipeline, allowing early error detection.
- Cataloged a Pytest API framework with 100+ automated backend tests.
- Set up the mobile web (mobile browsers Safari/Chrome) Appium framework from scratch (with BrowserStack, and lately, transferred to LambdaTest) and achieved 180 e2e test coverage.
By ramping up our automation efforts, we dramatically cut down on regression testing time. This was a game-changer for our manual testers. Freed from repetitive tasks, they could focus on exploratory testing of new features, edge cases, and complex scenarios that machines can't handle.
Some clients have the latest iPhones; others use older Android tablets. We needed to ensure our client's app worked great on all of them. It's not just about different screen sizes — each device has its own quirks. To address this issue, we have:
- Used over 40 iOS and Android devices, including both flagship and older models.
- Ensured consistent performance across a wide range of hardware configurations.
“Online emulators didn't cut it for the level of precision we needed. So, we created a full-scale mobile testing farm with both popular and flagman devices. This setup allowed us to catch issues that would've slipped through with virtual testing alone."
Segiy, Lead AQA
AI and Machine Learning Integration
Our client took personalization to new heights with AI. They built a smart system that does more than just sort products. This AI helps customers design items, suggesting themes, colors, and layouts. It even recommends how to place photos and add special touches. It's like having a personal designer built into the app. To ensure this AI-powered personalization worked and delivered real benefits, our team:
- Conducted comprehensive testing of new AI-driven personalization features.
- Provided regular feedback to the development team, contributing to iterative improvements.
- Ensured successful rollout of the AI-enhanced version to 100% of users.
Cloud Migration Support
For a platform handling millions of orders yearly, moving to the cloud is inevitable necessity. But migrating such a vast system, with its mountain of data and complex operations, is like performing heart surgery while the patient runs a marathon. Every piece had to move seamlessly, without disrupting the constant flow of orders.
- Assisted in the transition of servers and resources to cloud infrastructure.
- Maintained regular testing duties while supporting the migration process.
“During the cloud migration, we set up a "War Room" for a week. As we switched off old services and activated new ones, we worked round-the-clock shifts alongside other teams. This hands-on approach ensured a smooth transition and quick problem-solving at every step.”
Mykhailo, QA Lead
Scalability and Performance Testing
- Implemented load testing scenarios to simulate high-volume transaction periods.
- Verified system stability and responsiveness under peak usage conditions.
Continuous Improvement and Adaptation
- Regularly reviewed and updated testing strategies to align with the evolving product.
- Expanded our testing scope to cover newly acquired software products and integrations.
