Automated testing is essential to modern software development, ensuring stability and reducing manual effort. However, test scripts frequently break due to UI changes, such as modifications in element attributes, structure, or identifiers. Traditional test automation frameworks rely on static locators, making them vulnerable to these changes. AI-powered self-healing automation addresses this challenge by dynamically selecting and adapting locators based on real-time evaluation.
Self-healing is crucial for automation testing because it significantly reduces the maintenance overhead associated with test scripts by automatically adapting to changes in the application's user interface. This allows tests to remain reliable and functional even when the underlying code or design is updated, thus saving time and effort for testers while improving overall test stability and efficiency.
When UI elements change (like button IDs or class names), self-healing mechanisms can automatically update the test script to locate the new element, eliminating the need for manual updates and preventing test failures due to outdated locators.
By dynamically adjusting to changes, self-healing tests are less prone to "flaky" failures caused by minor UI modifications, leading to more reliable test results.
With less time spent on test maintenance, developers can focus on building new features and delivering software updates faster.
Modern applications often have dynamic interfaces where elements change frequently, making self-healing capabilities vital for maintaining test accuracy.
Initialize a playwright project and install cucumber dependencies by executing the command:
Cucumber allows for writing tests in Gherkin syntax, making them readable and easier to maintain for non-technical stakeholders.
This function is designed to "self-heal" by trying multiple alternative selectors when attempting to click an element. If one selector fails (for example, due to a change in the page's structure), it automatically tries the next one until one succeeds or all have been tried.
The cucumber.js file is the configuration file for Cucumber.js, which allows you to customize how your tests are executed.
As you see in the above screenshot, the code tried to find the selector and when it couldn't find the selector, the code moved on to finding the next alternative selector , which was successful; hence, clicking the element using the selector
Let's explore with an example on how to incorporate multiple locator strategies into AI-powered self-healing tests with a fallback method. The idea is to try each known locator in a predefined order before resorting to the ML-based fallback when all known locators fail.
Below is an example of the AI-powered self-healing approach, showing how to integrate TensorFlow.js (specifically ) to perform a real machine-learning-based fallback. We'll extend the findElementUsingML function to load an ML model, run inference on a screenshot, and parse the results to find the target UI element.
Note: In a real-world scenario, you'd have a trained object detection or image classification model that knows how to detect specific UI elements (e.g., "Add to Cart" button). For illustration, we'll show pseudo-code for loading a model and parsing bounding box predictions. The actual model and label mapping will depend on your training data and approach.
Let's begin by setting up the playwright project and installing the dependencies (Cucumber and Tensorflow).
Create the folder structure below and add the required files:
This file contains a helper function to find an element using multiple locator strategies. If all fail, it delegates to the AI fallback.
In util/aiLocator.js, we simulate an ML-based locator. In a production implementation, you'd load your trained ML model (for example, using TensorFlow.js) to process the screenshot and return the location (bounding box) of the "Add to Cart" button.
Below is a mock example of how you might load and run inference with TensorFlow.js (using ), parse bounding boxes, and pick the coordinates for the "Add to Cart" button.
Disclaimer: The code below shows the overall structure. You'll need a trained model that can detect or classify UI elements (e.g., a custom object detection model). The actual code for parsing predictions will depend on how your model outputs bounding boxes, classes, and scores.
Using TensorFlow.js for self-healing tests involves:
This approach provides a robust fallback that can adapt to UI changes if your ML model is trained to recognize the visual cues of your target elements. As your UI evolves, you can retrain the model or add new examples to improve detection accuracy, thereby continuously "healing" your tests without needing to hardcode new selectors.
