Complementing In-House Expertise with Targeted Test Automation Skills

This time, our team tested a desktop application designed for remote work management. The software is available on all major operating systems (Windows, MacOS, and Linux). This application provides analysis and actionable insights into the productivity of remote employees, as well as the efficiency of workflows.

Client's Needs

Ensure that the application works seamlessly across all major operating systems.
Achieve uninterrupted interaction of the app with various other desktop applications and websites.
Set up automated tests for complex scenarios, such as multithreading, handling asynchronous events, and ensuring accurate tracking of user actions.
Integrate new automated tests into a CI/CD pipeline to ensure that code changes are continuously tested and validated.

Business Challenges

Even though the client had an in-house team of Automation QA engineers, they needed additional expertise. In particular, they required specialists with extensive experience in automating tests for such products and deep knowledge of automation tools for desktop applications across all operating systems.

Cross-Platform Compatibility

Each OS has its own peculiarities, such as system architecture, available libraries, and APIs. This increases the complexity and effort required for thorough compatibility testing.

Interaction with Other Programs

Automating these interactions is challenging as it requires reliable modeling of user actions and processing of various actions by different software. In addition, any updates to third-party applications or websites may disrupt the automation scripts, requiring frequent updates and maintenance.

Complex Test Scenarios

Developing and maintaining complex test scenarios requires advanced technical skills and significant experience in desktop automation.

Continuous Integration and Deployment

Setting up and maintaining a proper CI/CD integration requires substantial effort, especially when dealing with desktop applications that need specific configurations for different operating systems.

Time Constraints

The client didn't have much time and resources allocated to hiring and onboarding new QA specialists. This is why the company needed a flexible team of QA engineers with previous experience on such projects to provide the necessary testing services as soon as possible.

Preparation for Testing

We followed the next steps to prepare for testing:

Analyzed the scope for automation and involved 2 independent Senior Automation QA specialists to prepare a comprehensive estimation of the time required for testing.
Defined the optimal programming language and tools for desktop test automation, which work natively with every OS.
Created a customized test automation framework based on PyTest and configured test reporting with the possibility of recording test results.
Provided the client with a Proof of Concept within a month and confirmed the next steps.
During the POC process, provided suggestions for improvements - both for processes and the product itself.

Realization

First of all, we created a Proof of Concept (POC) to demonstrate the feasibility and effectiveness of our approach. The POC involved setting up the testing environment, simulating and automating user interactions, and verifying testing results.
After this, we set up an automated test to check the connection stability of the application. It runs 24/7 and notifies us immediately in case of connection loss.
For the test coverage, we started by automating high-priority tests and integrating them into the release process. Before every release, we run a smoke test to ensure quick detection of possible bugs.
Then, we proceeded with other tests and covered all new features. This approach ensures the high quality of the application and a positive user experience after the release.

Test Documentation

Our specialists prepared comprehensive test documentation to support the QA process, including:

Detailed test plans and cases.
Test reports, along with video documentation of failed test results, for rapid issue investigation.
Progress coverage reports for easy and regular tracking of testing results.
Configuration guides for the testing environment.
Step-by-step automation framework setup instructions.
Reporting templates and logging procedures.
Transparent daily and weekly reports.

Key Results for the Client

Increased test automation coverage: achieved 85% of coverage with automated tests, significantly reducing the risk of bugs.
Efficient test reporting: implemented robust reporting mechanisms, providing clear insights into the test results and system performance.
Seamless integration: the application demonstrated smooth interaction with other software and operating systems.
No data loss: ensured data integrity and reliability across all test scenarios.
Full integration in the client's team: our independent QA engineers participated in all team meetings and communications.
Improved testing process: involved an experienced project manager who was responsible for setting up the testing process, organizational improvements, reporting, and seamless communication.