Running Parallel Tests with Cucumber on AWS Device Farm

Automated testing is an essential part of modern software development. It ensures code quality, reduces bugs, and accelerates delivery. However, testing on a wide range of devices and browsers can be challenging. AWS Device Farm offers a solution by providing a cloud-based service that allows you to test your applications on a variety of real devices. In this article, we’ll explore how to run parallel tests with Cucumber on AWS Device Farm.

Why Use AWS Device Farm?

AWS Device Farm is a service that lets you test your mobile and web applications across an extensive range of devices, browsers, and operating systems. Key benefits include:

Diverse Device Availability: Test on real devices instead of emulators or simulators.
Scalability: Run tests in parallel across multiple devices to speed up your testing process.
Integration: Seamlessly integrate with your CI/CD pipelines.
Detailed Reports: Obtain comprehensive reports with logs, screenshots, and videos.

Getting Started with AWS Device Farm

Before diving into parallel testing with Cucumber, you need to set up AWS Device Farm and your testing environment.

Prerequisites

AWS Account: Ensure you have an active AWS account.
AWS CLI: Install and configure the AWS CLI on your local machine.
Cucumber Setup: Set up Cucumber in your project. If you’re new to Cucumber, you can follow the official documentation.

Setting Up AWS Device Farm

Create a Project: Log in to the AWS Device Farm console and create a new project.
Upload Your Application: Upload your application (APK for Android or IPA for iOS) to the project.
Create a Test Suite: Create a new test suite and select Cucumber as your test framework. You will need to upload your test scripts.

Configuring Parallel Tests

To run tests in parallel, you need to configure your Cucumber setup to distribute tests across multiple devices. Here’s a step-by-step guide to achieve this.

Step 1: Set Up Your Test Runner

Create a test runner class that specifies how to run your Cucumber tests. This class should use JUnit to manage test execution.

import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(
    features = "src/test/resources/features",
    glue = "stepDefinitions",
    plugin = {"pretty", "html:target/cucumber-reports"},
    monochrome = true
)
public class TestRunner {
}

Step 2: Parallel Execution Configuration

To enable parallel execution, use the cucumber-jvm-parallel-plugin. Add the following dependency to your pom.xml if you are using Maven.

<dependency>
    <groupId>com.github.temyers</groupId>
    <artifactId>cucumber-jvm-parallel-plugin</artifactId>
    <version>5.0.0</version>
</dependency>

Next, configure the plugin in your pom.xml:

<build>
    <plugins>
        <plugin>
            <groupId>com.github.temyers</groupId>
            <artifactId>cucumber-jvm-parallel-plugin</artifactId>
            <version>5.0.0</version>
            <executions>
                <execution>
                    <id>generateRunners</id>
                    <phase>generate-test-sources</phase>
                    <goals>
                        <goal>generateRunners</goal>
                    </goals>
                    <configuration>
                        <outputDirectory>${project.build.directory}/generated-test-sources/cucumber</outputDirectory>
                        <cucumberOutputDir>${project.build.directory}/cucumber-reports</cucumberOutputDir>
                        <featuresDirectory>src/test/resources/features</featuresDirectory>
                        <glue>stepDefinitions</glue>
                        <parallelScheme>SCENARIO</parallelScheme>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

This configuration will generate test runners for each scenario, allowing parallel execution.

Step 3: Integrate with AWS Device Farm

To run these tests on AWS Device Farm, follow these steps:

Package Your Tests: Ensure your project is packaged correctly. This usually involves generating a JAR or ZIP file that includes all dependencies.
Upload Tests to AWS Device Farm: In the AWS Device Farm console, navigate to your project and upload your test package.
Configure Test Execution: Select the devices you want to test on and configure the test to run in parallel. AWS Device Farm will handle the distribution and execution of your tests across the selected devices.

Step 4: Analyze Test Results

After the tests have completed, AWS Device Farm provides detailed reports including:

Logs: Detailed logs of the test execution.
Screenshots and Videos: Visual proof of the tests running on real devices.
Performance Data: Metrics on CPU, memory, and network usage.

Example Test Scenario

Here is a simple Cucumber feature and step definition example:

Feature File: login.feature

Feature: Login functionality

  Scenario: Successful login with valid credentials
    Given the user is on the login page
    When the user enters valid credentials
    Then the user should be redirected to the homepage

Step Definitions: LoginSteps.java

import cucumber.api.java.en.Given;
import cucumber.api.java.en.When;
import cucumber.api.java.en.Then;
import static org.junit.Assert.assertTrue;

public class LoginSteps {
    
    @Given("the user is on the login page")
    public void userOnLoginPage() {
        // Code to navigate to the login page
    }

    @When("the user enters valid credentials")
    public void userEntersValidCredentials() {
        // Code to enter username and password
    }

    @Then("the user should be redirected to the homepage")
    public void userRedirectedToHomepage() {
        // Code to check redirection
        assertTrue(true);
    }
}

Conclusion

Running parallel tests with Cucumber on AWS Device Farm can significantly reduce your testing time and improve the quality of your applications. By leveraging the power of cloud-based real device testing and the flexibility of Cucumber, you can ensure your application works flawlessly across a wide range of devices and configurations. With detailed reporting and easy integration into CI/CD pipelines, AWS Device Farm is a powerful tool for modern software development teams.

If you found this article helpful and are interested in integrating Cucumber Automation Framework with AWS Device Farm, I suggest you check out some of the other articles I’ve written for this series:

If you’re looking for a deeper dive into some of the concepts and specifics discussed in my article, feel free to reach out to me directly or as always you can checkout the official AWS Device Farm Developer Documentation for more information.