Scalable Test Data Management with AWS DynamoDB and Cucumber

Managing test data efficiently is crucial for automated testing. AWS DynamoDB, a fully managed NoSQL database, is a great choice for scalable test data management due to its high performance and low latency. This guide will walk you through setting up and managing test data with DynamoDB and integrating it with Cucumber for automated tests.

Prerequisites

AWS Account: Ensure you have an AWS account with necessary permissions.
IAM Role: Create an IAM role with permissions for DynamoDB.
Cucumber Project: Have a working Cucumber project set up.

Step 1: Set Up DynamoDB Table

Navigate to DynamoDB Console: Go to the DynamoDB console in your AWS Management Console.
Create a Table:
- Table Name: Enter a name for your table (e.g., TestData).
- Primary Key: Define the primary key (e.g., testId of type String).
- Table Settings: Use the default settings or configure as needed for your workload.
Create the Table: Click “Create” to create the table.

Step 2: Configure IAM Role and Policies

Create an IAM Role:
- Role Type: Choose AWS service role and select Lambda (or EC2, depending on your execution environment).
- Attach Policies: Attach the AmazonDynamoDBFullAccess policy (or a more restrictive policy as needed).
- Create Role: Complete the role creation process.
IAM Policy Example: If you prefer a more restrictive policy, create a custom policy like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "dynamodb:PutItem",
                "dynamodb:GetItem",
                "dynamodb:UpdateItem",
                "dynamodb:DeleteItem",
                "dynamodb:Scan",
                "dynamodb:Query"
            ],
            "Resource": "arn:aws:dynamodb:<region>:<account-id>:table/TestData"
        }
    ]
}

Step 3: Integrate DynamoDB with Cucumber

1. Add AWS SDK Dependency: Add the AWS SDK for Java to your pom.xml in your Cucumber project.

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-dynamodb</artifactId>
    <version>1.12.200</version>
</dependency>

2. Create a DynamoDB Utility Class:

package com.example.utils;

import com.amazonaws.auth.DefaultAWSCredentialsProviderChain;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.model.PutItemRequest;
import com.amazonaws.services.dynamodbv2.model.PutItemResult;

public class DynamoDBUtil {
    private static final AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()
            .withRegion(Regions.US_WEST_2)
            .withCredentials(new DefaultAWSCredentialsProviderChain())
            .build();
    private static final DynamoDB dynamoDB = new DynamoDB(client);
    private static final String TABLE_NAME = "TestData";

    public static void putItem(String testId, String data) {
        Table table = dynamoDB.getTable(TABLE_NAME);
        Item item = new Item()
                .withPrimaryKey("testId", testId)
                .withString("data", data);
        table.putItem(item);
    }

    public static String getItem(String testId) {
        Table table = dynamoDB.getTable(TABLE_NAME);
        Item item = table.getItem("testId", testId);
        return item != null ? item.getString("data") : null;
    }

    public static void deleteItem(String testId) {
        Table table = dynamoDB.getTable(TABLE_NAME);
        table.deleteItem("testId", testId);
    }
}

3. Use DynamoDB in Cucumber Step Definitions:

package com.example.steps;

import com.example.utils.DynamoDBUtil;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.When;
import io.cucumber.java.en.Then;
import static org.junit.Assert.assertEquals;

public class TestSteps {
    @Given("test data with ID {string} and data {string} is stored in DynamoDB")
    public void storeTestData(String testId, String data) {
        DynamoDBUtil.putItem(testId, data);
    }

    @When("I retrieve the test data with ID {string}")
    public void retrieveTestData(String testId) {
        String data = DynamoDBUtil.getItem(testId);
        ScenarioContext.setContext("retrievedData", data);
    }

    @Then("the retrieved data should be {string}")
    public void verifyRetrievedData(String expectedData) {
        String retrievedData = (String) ScenarioContext.getContext("retrievedData");
        assertEquals(expectedData, retrievedData);
    }
}

4. Scenario Context for Sharing Data:

package com.example.steps;

import java.util.HashMap;
import java.util.Map;

public class ScenarioContext {
    private static Map<String, Object> context = new HashMap<>();

    public static void setContext(String key, Object value) {
        context.put(key, value);
    }

    public static Object getContext(String key) {
        return context.get(key);
    }

    public static void clear() {
        context.clear();
    }
}

5. Cucumber Feature File Example:

Feature: DynamoDB Integration

Scenario: Store and retrieve test data from DynamoDB
  Given test data with ID "12345" and data "test data" is stored in DynamoDB
  When I retrieve the test data with ID "12345"
  Then the retrieved data should be "test data"

Step 4: Running Tests

1. Compile and Run Tests:

mvn compile
mvn test

2. Verify the Results: Ensure that the tests interact with DynamoDB as expected, storing, retrieving, and verifying data correctly.

Considerations and Best Practices

Security: Use IAM roles and policies to restrict access to DynamoDB tables as needed. Avoid hardcoding credentials; use AWS credentials provider chain.
Data Cleanup: Implement cleanup logic in your tests to delete or archive test data after use to prevent clutter and maintain performance.
Environment Configuration: Use environment variables or configuration files to manage different settings for development, testing, and production environments.
Error Handling: Implement robust error handling in your utility methods to manage DynamoDB operations gracefully.
Monitoring and Logging: Use AWS CloudWatch to monitor and log DynamoDB operations and performance.
Scaling: Design your DynamoDB tables and indexes to handle the anticipated load and scale as needed.

By following this guide, you can efficiently manage test data using AWS DynamoDB and integrate it with Cucumber for robust and scalable automated testing. This setup will help ensure that your tests are consistent, repeatable, and capable of handling varying amounts of test data seamlessly.

If you found this article helpful and are interested in integrating Cucumber Automation Framework with AWS DynamoDB, I suggest you check out some of the other articles I’ve written for this series:

If you’re looking for a deeper dive into some of the concepts and specifics discussed in my article, feel free to reach out to me directly or as always you can checkout the official AWS DynamoDB Developer Documentation for more information.