Scaling Cucumber Test Suites with AWS CodePipeline and EC2 Auto Scaling

Continuous integration and continuous deployment (CI/CD) are crucial for maintaining high-quality code and ensuring rapid delivery of features. Cucumber, a popular tool for behavior-driven development (BDD), allows teams to write human-readable test cases that can be executed automatically. As your project grows, the number of test cases increases, and so does the need for an efficient testing infrastructure. This is where AWS CodePipeline and EC2 Auto Scaling come into play.

In this blog post, we’ll explore how to scale your Cucumber test suites using AWS CodePipeline and EC2 Auto Scaling. We’ll delve into the architecture, setup, and provide detailed code examples to help you get started.

Architecture Overview

AWS CodePipeline: Automates the build, test, and deploy phases of your release process.
EC2 Auto Scaling: Automatically adjusts the number of EC2 instances in your fleet based on demand.
Cucumber: BDD tool for running human-readable test cases.

Prerequisites

Basic knowledge of AWS services (CodePipeline, EC2, Auto Scaling).
Familiarity with Cucumber and Java.
An AWS account with necessary permissions.
A repository with Cucumber tests and a Maven/Gradle build file.

Step 1: Setting Up the AWS CodePipeline

First, let’s create a simple AWS CodePipeline to handle the CI/CD process.

Create a Pipeline:
- Go to the AWS CodePipeline console and click “Create pipeline.”
- Enter a name for your pipeline and choose a new service role.
Source Stage:
- Select your source provider (e.g., AWS CodeCommit, GitHub, Bitbucket).
- Connect your repository and specify the branch to monitor.
Build Stage:
- Add a build stage using AWS CodeBuild.
- Create a new build project, specifying the environment as Ubuntu with Java as the runtime.

Here’s a sample buildspec.yml for running Cucumber tests:

version: 0.2

phases:
  install:
    runtime-versions:
      java: openjdk11
    commands:
      - echo "Installing dependencies..."
      - mvn install

  pre_build:
    commands:
      - echo "Pre-build steps..."

  build:
    commands:
      - echo "Running tests..."
      - mvn test

artifacts:
  files:
    - target/**/*

reports:
  cucumber-json:
    files:
      - target/cucumber/*.json

Step 2: Configuring EC2 Auto Scaling

Next, we’ll set up EC2 Auto Scaling to ensure we have enough instances to handle our test load.

Create an AMI:
- Launch an EC2 instance with the necessary environment to run your tests (Java, Maven, Cucumber).
- Install all required dependencies and configure the instance.
- Create an AMI from this instance.
Launch Configuration:
- Go to the EC2 Auto Scaling console and create a launch configuration using the AMI created in the previous step.
- Specify instance type, security groups, and IAM role.
Auto Scaling Group:
- Create an Auto Scaling group using the launch configuration.
- Specify the desired capacity, minimum, and maximum number of instances.
- Configure scaling policies based on CloudWatch metrics (e.g., CPU utilization).

Step 3: Integrating EC2 Auto Scaling with CodePipeline

We need to integrate EC2 Auto Scaling with CodePipeline to ensure that tests are distributed across multiple instances.

Modify Buildspec for Parallel Testing:
- Update your buildspec.yml to distribute tests across multiple instances.
- Use environment variables to control which tests run on each instance.

Here’s an example of distributing tests using tags:

version: 0.2

phases:
  install:
    runtime-versions:
      java: openjdk11
    commands:
      - echo "Installing dependencies..."
      - mvn install

  pre_build:
    commands:
      - echo "Pre-build steps..."

  build:
    commands:
      - echo "Running tests..."
      - mvn test -Dcucumber.options="--tags @instance${INSTANCE_ID}"

artifacts:
  files:
    - target/**/*

reports:
  cucumber-json:
    files:
      - target/cucumber/*.json

Parallel Execution:
- Modify your Cucumber tests to support parallel execution.
- Use the @instance tag to filter tests for each instance.

Step 4: Scaling Cucumber Tests

Now, let’s dive into the code for scaling Cucumber tests.

1. Tagging Tests:

Tag your Cucumber scenarios to distribute them across instances.

@RunWith(Cucumber.class)
@CucumberOptions(
    features = "src/test/resources/features",
    glue = "com.example.stepdefs",
    plugin = {"json:target/cucumber/report.json"},
    tags = "@instance1 or @instance2 or @instance3"
)
public class RunCucumberTest {
}

2. Step Definitions:

Implement your step definitions in Java.

package com.example.stepdefs;

import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;

import static org.junit.Assert.assertEquals;

public class StepDefinitions {

    private int x;
    private int y;
    private int result;

    @Given("two numbers {int} and {int}")
    public void two_numbers_and(int x, int y) {
        this.x = x;
        this.y = y;
    }

    @When("I add them")
    public void i_add_them() {
        result = x + y;
    }

    @Then("the result should be {int}")
    public void the_result_should_be(int expected) {
        assertEquals(expected, result);
    }
}

Step 5: Monitoring and Optimization

CloudWatch Metrics:
- Monitor EC2 instance metrics using CloudWatch.
- Create alarms to trigger scaling actions based on CPU utilization, memory usage, etc.
Cost Optimization:
- Use Spot Instances to reduce costs.
- Set up lifecycle policies to terminate instances when not in use.

Conclusion

By leveraging AWS CodePipeline and EC2 Auto Scaling, you can efficiently scale your Cucumber test suites to handle growing test loads. This setup ensures that your tests run in parallel, reducing execution time and providing faster feedback. The combination of CI/CD automation and dynamic scaling allows your development team to maintain high-quality code and deliver features rapidly.

Remember to monitor your infrastructure and optimize for cost and performance continuously. With the right setup, you can achieve a scalable, efficient, and cost-effective testing environment.

If you found this article helpful and are interested in integrating Cucumber Automation Framework with AWS CodePipeline, I suggest you check out some of the other articles I’ve written for this series:

If you’re looking for a deeper dive into some of the concepts and specifics discussed in my article, feel free to reach out to me directly or as always you can checkout the official AWS CodePipeline Developer Documentation for more information.