Challenge

With hundreds of parts and components available, there could be thousands of possible pergola configurations, which was a challenge in itself. In addition, we had to:

• ensure that these modifications looked real, providing users with a vision of how the future construction would look like when installed on the site;
• anticipate and eliminate possible errors that could occur in the configuration process due to the complexity and variety of options. 

Solutions

Through years of partnership, we’ve learned to intuitively understand this client’s needs, often tackling projects, streamlining processes, and addressing pain points with minimal requirements at hand.

When we first started, the team chose Unity to design the interface. However, the decision was soon made to change it in favor of Autodesk. It is a much more user-friendly technology that not only creates more visually appealing designs but also easily combines with Revit, which was chosen as a primary tool for the project. This transition positively impacted the entire project, resulting in more realistic rendering of 3D components and a significantly improved UX. 

Furthermore, we proposed to add a “save and share via link” feature for the configurations built. This addition meant a lot in terms of user engagement and marketing. Moreover, we’ve found storage services that could convert data free of charge and be seamlessly integrated into the configurator. The client approved this solution, and the implementation of this step is scheduled. 

The composition of the team working on this project included:

1 x Front-end developer (Angular)

1 x Back-end developer (.NET), 

2 x Revit & Autodesk Visualizator, 

1 x QA engineer. 

We’ve finished a big scope of work, but our collaboration continues, with more configurations and features to be developed in the upcoming months. 

Challenge

We were entrusted with automating testing the company’s custom NetSuite application used for order management, payment processing, and other operations. 

• Our client used only manual testing without applying optimization techniques or automation;
• We had to turn test cases shortly described in Word documents into a working test automation scripts system;
• Create and run tests for two separate environments;
• Foresee potential crash scenarios and bottlenecks, find and report bugs and critical system blockers.

Solutions

Automation testing diminishes manual intervention and allows for minimizing mistakes as well as speeding up the entire process. To complete our tasks, we have provided this project with two experienced auto QA engineers, who have accomplished the following:

• Created and implemented efficient UI, and end-to-end testing plan;
• Automated 100% of test cases, taking into account NetSuite peculiarities and business logic inherent to the client’s industry;
• Launched daily test runs that allowed us to find bugs and critical blockers almost immediately;
• Implemented CI workflow for more efficient communication, a better rapport with the client’s team, and reliable continuous integration;
• Ensured faster-debugging thanks to extended test coverage;
• Defined critical blockers causing system crashes and offered ways to improve system security.

Tools and Technologies

For this case, we have chosen a combination of tools and technologies for automation testing that allowed us to enhance efficiency and minimize time. 

• Selenium WebDriver: for scalability and robust support across various platforms and operating systems, as well as creating reusable test suits. 

• TestNG: for simplified management of test cases, all the benefits of parallel testing, the possibility of parameterized and dependency tests, as well as clear, “by default” reports. 

• TestRail: for more efficient test management organization and improved organization of testing efforts according to the automation tasks our team has faced in this project.

• Jenkins: to support continuous integration and enable efficient automation process within the scope of our immediate tasks and testing strategy.

Communications

We have entered this challenge following an executive director who was our client in one of the previous automation projects. Communication with a new team was predominently held through him, which had pros and cons. On the one hand, we always had inside reach and solved any issues fast and efficiently. On the other hand, when our trusted liaison quit taking a new position elsewhere, our communication with the client’s team was crippled, and we had to establish new processes hurriedly. 

Thanks to the on-point documentation we have been creating from day one and the seamless workflow our team honed on various other projects, the transition period was brief and didn’t affect our speed or efficiency. 

Impact

Methodology. Our cooperation was first based on the Kanban methodology. It didn’t fully answer the speed and efficiency demands of most of the automation tasks and bug-fix workflow. We have offered to try Scrum workflow, which proved to be more productive and, as to our knowledge, has been used by the client’s manual testing team till now  

TestRail workflow. Our QA implemented and established the TestRail workflow for the entire team. It included all the step-by-step supporting documentation and month-long training instructions for a current manual testing team and specialists that would join the company once our project was done. 

Challenge

In the process of revolutionizing software testing through AI, we encountered a series of testing roadblocks that demanded innovative solutions. These ranged from dealing with duplicated bug issues to enhancing test coverage and refining bug report creation, all the way to reimagining requirement testing and efficient report generation.

Overcoming Key Testing Roadblocks

Bug triage. Duplicated issues were a recurring problem, resulting from multiple authors logging defects into the bug tracking system.

Test coverage. Due to high-priority test execution activities, time for comprehensive documentation creation was often compromised, predominantly covering only positive scenarios.

Bug reports creation. The issues our client faced were being discussed across various forums and thematic groups, yet the simplicity of the textual explanations from customers necessitated a conversion into a more formal bug format.

Requirements testing. There was a need for structured user stories, conforming to a common user story creation approach.

Report creation. Report creation during regression testing linked data from disparate resources, a process that was significantly time-consuming.

Facing the Challenges of AI Integration in Testing

While tackling the current software testing processes, we’ve encountered additional challenges that extended beyond their initial concerns.

Confidentiality and data security. Given the sensitive nature of the software we were working with and the proprietary technology involved, maintaining the confidentiality of their data and ensuring the security of their software was paramount. Strict protocols had to be enforced during the testing process to prevent data leaks or any potential breaches.

False positives and negatives. When utilizing AI tools for software testing, there’s always a risk of encountering false positives—issues flagged as errors that are actually correct—and false negatives—actual errors that are overlooked. Mitigating these false responses, which could lead to wasted resources or overlooked problems, was a significant challenge.

Tool integration and compatibility. Another challenge was to integrate and make compatible the different AI tools—DeepTriage, ChatGPT, Grammarly, and Microsoft Power BI—with our client’s existing system. A smooth and seamless integration was crucial for the efficient operation of the AI tools and to prevent any disruptions to the current workflows.

Continuous monitoring and adjustment. AI tools, while highly effective, need continuous monitoring and adjustments to provide optimal results. The challenge was to ensure consistent and regular oversight, tweaking the AI models when needed and ensuring they evolve with the changing software dynamics and requirements.

Addressing these additional challenges required careful planning, strategic decision-making, and effective communication with our client to ensure a successful overhaul of their software testing process.

Solutions

To navigate the mentioned challenges, QArea suggested the adoption of several AI tools:

Bug triage. We recommended the use of DeepTriage, an AI-powered tool developed by Microsoft researchers. This tool leverages machine learning to streamline the bug triage process and assign bugs efficiently.

Test coverage. We deployed ChatGPT to work with the text format, typically represented by requirements, and retrieve test cases from it.

Bug reports creation. We utilized ChatGPT to analyze, structure, and rewrite customer experience reviews into a format suitable for enhancement and issue reports.

Requirements testing. ChatGPT and Grammarly were jointly used to restructure, analyze, and improve software requirements, ensuring a consistent style and correcting spelling errors.

Report creation. For creating cohesive and efficient reports, we utilized Microsoft Power BI. This tool supports data integration from various sources, offers data modeling capabilities, and provides AI-driven insights.

Practical Approaches: Enhancing Software Testing with Focused Strategies

Our team implemented several strategic solutions to ensure a seamless transition and an efficient software testing process.

To safeguard data confidentiality, we enforced security protocols and implemented encryption techniques. Also, we have never presented AI tools with 100% true data, changing numbers, names and places, as well as any information that may have been considered sensitive. Later, we have used an algorithm to use the true data at our side and receive precise results. 

To mitigate the issue of false positives and negatives, we employed validation techniques and cross-checks in our testing process. This approach ensured that any errors flagged by AI tools were verified, and any overlooked errors were caught in subsequent testing stages.

For the smooth integration of AI tools, we performed careful compatibility testing and gradually introduced the tools, ensuring there were no disruptions to existing workflows. Additionally, we provided technical support during and after the integration process to address any emerging issues promptly.

We organized a few training sessions for the client’s team to understand and effectively use the new AI tools. Additionally, we provided detailed documentation and ongoing support to help the team adapt to the new tools and processes.

Finally, as requested by our client, we established a dedicated team to monitor the performance of the AI tools continuously, adjusting parameters and refining the AI models as needed. By maintaining a close watch on the tools’ performance and making timely adjustments, we ensured that they continued to deliver optimal results, keeping pace with the evolving dynamics and requirements of the software.

Solutions

The first part of the challenge — working with an unfamiliar product — was resolved thanks to the existing experience our team members had with the product as a test management solution. As for the second part, the expertise of our Automation QAs helped us manage the existing test cases with maximum efficiency in a limited time period.

To deliver the required scope of testing, we performed functional testing to verify the core functionality of the product, and then we used regression testing to make sure that the recent changes did not have any negative impact on the product. Other types of testing we employed in this project were:

• Acceptance testing
• Security testing
• Performance testing
• Edge case testing

This was also a project where we used both manual and automated testing. We employed automated testing with the help of Selenium to check the consistency and integrity of the test cases created by the previous team working on the project. 

Our team consisted of five QA engineers, including two Automation QAs and a QA Team Lead.

Challenge

Our main challenge was to review and update existing test cases in the scope of this project prior to executing them. We had to review test cases for both bug fixes and smoke verification, check their consistency, and make necessary updates. All of that, including executing those tests and documenting new bugs, had to be done within a short period of time. 

Solutions

Due to the intense project implementation process, the QA activities were attached to the development sprints and strictly planned into limited timeframes. Thanks to the efficient collaboration with the client’s development team and our usage of ready templates for presenting the review results, we’ve managed to complete all necessary activities before the deadline.

Our QA team consisted of: 

• 2 QA engineers
• 1 QA automation engineer
• 1 QA Team Lead

The team effortlessly fit into the client’s operations and performed every single type of testing that was requested from us, including:

• Exploratory testing
• Functional testing
• Smoke testing
• Regression testing
• UI/UX testing
• Confirmation testing
• Cross-browser testing
• Cross-platform testing

The team members would also reach out to the client whenever we had questions or suggestions about the project, and we have always been available in case the client wanted to make additional requests or comments.

Solutions

We chose React to keep the fundament of the project stable but flexible and capable of changeable UI elements ensuring visualization process. This library works with a declarative method, hence enables much fitting construction for the system with models. Our team has developed these elements, together with visualization mechanics, as an optimized alternative for the boards of cards. Such tailor-made patterns ensure the high level of freedom for the user, for instance, custom reporting and personalized assessment formula.

Complementing React with Redux, we leveled-up the app’s inner constitution. Let’s look behind the curtains. Since the default intricacy of bindings between components causes obscurity, we got the clearing solution. Redux creates a separate store that clews up the app’s states to influence components directly. That helps to avoid chaos, structuring the complex system into straightforward integrity, so the app reacts faster and more reliable to the user’s inputs. That’s merely fitting for the stressful atmosphere around the project often including rush and harsh changes in the process.

Since a PM and any other user should’ve driven the application with events (to schedule), we exploited NodeJS. This JavaScript runtime gave an opportunity to apply the language for both front-end and back-end with no additional concernings. Applying the API optimized for massive use (your team can be as big as you need!) and the loop to keep an eye on events constantly, NodeJS served as a firm brick in our coding.

MongoDB clarified objects with classified documentation, so we fulfilled the tasks much more efficiently, and thus quicker.

Solutions

Our team followed the client’s requirements and suggested a few unique solutions. Key amongst them was the creation of a new database structure for all future clients. 

While it wasn’t possible to migrate all the data from the legacy database, our team set up a Master-Slave database architecture. This allowed the legacy database to perform at the significantly higher speed of the new database.