Skip to main content

Setup Private Runner AWS

This document provides a guide to setting up Private Runner Groups on the StackGuardian platform, focusing on configuring storage backend using AWS S3 for seamless task execution within your organization's infrastructure.

Configure a Private Runner Group on AWS​

1. Creating an S3 Bucket​

Before creating a private runner, ensure the following prerequisites are set up:

1.1 Create AWS S3 bucket​

Follow the AWS S3 bucket creation guide to set up an AWS S3 bucket. The AWS S3 storage backend template in StackGuardian can also be used to provision a S3 bucket in your AWS account as well.

1.2 Configuring CORS Policy for S3 Bucket​

Establishing a CORS (Cross-Origin Resource Sharing) policy for the S3 bucket associated with the Private Runner is important to ensure that the runner can securely access and interact with the required resources. CORS facilitates web applications running in one domain (origin) to access resources from another domain. Within the context of Private Runner, the CORS policy enables the runner to make authenticated API requests to the S3 bucket.

[
  {
    "ExposeHeaders": [],
    "AllowedMethods": ["GET", "HEAD", "PUT"],
    "AllowedHeaders": ["*"],
    "AllowedOrigins": ["https://app.stackguardian.io"]
  }
]

1.3 Create a Role to Access the S3 Bucket​

Create an IAM role that will allow the private runner to access the S3 bucket. Follow these steps:

AWS Permission Policy: Attach the following policy to the role to grant the necessary permissions to access the S3 bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "s3:DeleteObjectTagging",
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObjectVersion",
        "s3:GetObjectVersionTagging",
        "s3:PutObjectVersionTagging",
        "s3:GetObjectTagging",
        "s3:ListBucket",
        "s3:PutObjectTagging",
        "s3:DeleteObjectVersionTagging",
        "s3:DeleteObject"
      ],
      "Resource": ["arn:aws:s3:::<BUCKET_NAME>", "arn:aws:s3:::<BUCKET_NAME>/*"]
    }
  ]
}
note

Replace <BUCKET_NAME> with the actual name of your S3 bucket.

1.4 Create the Connector for the Role in StackGuardian​

Set up the connector in the StackGuardian platform using the IAM role created in step 1.3 so the private runner can access the S3 bucket.

2. Setting Up Private Runner Group in StackGuardian​

To use AWS S3 as a storage backend for a Private Runner Group in StackGuardian, follow these steps:

2.1. Under Organization Settings in the StackGuardian platform , navigate to the Runner Groups section and click on "Create New Runner Group".
2.2. In the form, provide the following details:

  • Resource Name: Give the private runner group a name, e.g., "CI/CD Pipeline Group".
    • Resource ID: Auto-generated from the runner group name. You can customize it using only letters, numbers, underscores (_), or dashes (-). This cannot be changed after creation.
  • Description: Briefly explain the group's intention.
  • Add Tag: Optionally, add tags to categorize or label the runner group.

2.3. Under "Storage Backend Configuration" dropdown, enter the following details:

  • Type: Select "AWS S3".
  • Auth: Select the connector previously configured in point 1.4
  • Bucket Region: Select the appropriate region from the dropdown.
  • S3 Bucket Name: Enter the name of the S3 bucket to be used for storing artifacts and logs. 2.4. Click on the "Create" button to create your private runner group.
Setting Up Private Runner Group in StackGuardian

Setting Up Private Runner Group in StackGuardian

3. Creating the EC2 Instance​

Private runner groups support multiple runner instances and the StackGuardian platform takes care to balance the load between them. In this step, one or multiple instances will be created that become part of the private runner group.

3.1 The IAM Role That Will Be Attached to the Instance​

Create an IAM role specifically as an EC2 Role to be attached to the EC2 instance.

{
"Version": "2012-10-17",
"Statement": [
  {
      "Sid": "Statement1",
      "Effect": "Allow",
      "Action": [
          "sts:AssumeRole"
      ],
      "Resource": [
          "<ARN_OF_BUCKET_ROLE>"
      ]
  }
]
}

3.2 Requirements for Private Runner​

For each runner following CPU architectures and operating systems are supported:

  • Architecture: x86_64
  • OS: CentOS/RHEL 7, 8, 9, Ubuntu 20, 22, 24 Amazon Linux 2, 2023

The VM requirements depend on the workloads that will be running inside the workflows. Ensure that the VM meets the following minimum requirements:

  • CPU: 4 cores
  • Memory: 8 GB
  • Disc: 100 GB
/var Directory Storage Allocation

In a standard deployment, the 100 GB of storage is dynamically assigned, requiring no additional configuration. However, if you are utilizing multiple disks for the private runner instance, please ensure that 80 GB of storage is specifically allocated to the /var directory on the VM. This directory is crucial as it is used by Docker and other private runner processes. To maintain optimal performance, we automatically clean up unused images and containers every 4 hours. Allocating 80 GB provides a sufficient buffer to accommodate these operations.

3.3 Creating and Configuring the Private Runner​

Most of the pre-required commands come built-in but you will need to install docker, crontab and jq inside the virtual machine you created using SSH. In addition, for Amazon Linux OS, gnupg2 and cronie libraries are required as well.

3.4 Attaching the IAM Role to the runner​

Once the private runner instance(s) are created, please attach the created role from step 3.1 to each runner.

3.5 Registering the runners with SG platform​

Once logged into the SG platform, navigate to Profile > Settings > Runner Groups, open your runner group and click the 'View' button to see the connection string starting with wget .... Use this command while logged into the runner CLI to register your runner. After successful registration your runner(s) will show up in the UI under Runner Instances.