エクサウィザーズ Engineer Blog

株式会社エクサウィザーズのエンジニアチームブログ

Creating CI / CD pipeline using GitHub Actions self-hosted runners on AWS ECS

f:id:tadashi-nemoto0713:20201019135904p:plain

This is English version of this article.

techblog.exawizards.com


Hello, I'm Tadashi Nemoto from the DevOps team.

I joined ExaWizards this year in July in order to improve CI / CD promote the usage of automated testing in product development.

In this article, I will demonstrate how to create GitHub Actions with self-hosted runners on AWS ECS.

GitHub Actions and self-hosted runners

You may already know of or use GitHub Actions if you are an active GitHub user.

Combined with the Actions available on GitHub Marketplace, you can easily build a variety of powerful CI/CD pipelines.

In addition, instead of using runners provided by GitHub, you can also prepare your own runner (self-hosted runners).

About self-hosted runners - GitHub Docs

If you use GitHub Actions on a GitHub provided runner you will be charged on a free-to-use plus pay-as-you-go basis.

However, there is no additional charge for using GitHub Actions with a self-hosted runner on your own infrastructure.

You can set up new self-hosted runners from the Repository or Organization settings.

f:id:tadashi-nemoto0713:20201016201036p:plain

Once the setup is complete, you will see that it has been added as a runner.

Runners configured in a repository can be executed in that repository, and runners configured in an organization can be executed in all the repositories of that organization.

f:id:tadashi-nemoto0713:20201016201348p:plain

Finally, in your GitHub Actions configuration file, you should indicate that you want it to run using your self-hosted runners.

jobs:
  build:
    runs-on: self-hosted

It is true that you will need to set up and maintain self-hosted runners by yourself.

However, I believe it's attractive that we don't have to prepare and maintain the management part of a CI / CD workflow (like a Jenkins master instance). GitHub offers this functionality for free.

Running self-hosted runners on Docker

The source code of the self-hosted runner agent is available as open source, but GitHub currently doesn't provide a Docker image for it.

There are open source projects to get self-hosted runners running on Docker and Kubernetes.


In this article, we will work together to create a CI / CD pipeline using GitHub Actions with self-hosted runners on AWS ECS.

  1. Running self-hosted runners on Docker

  2. Running self-hosted runners on AWS ECS

  3. Creating a CI / CD pipeline to deploy an application to AWS ECS using those self-hosted runners


First, we'll try to run self-hosted runners on our local machine using the below Docker image.

github.com

In order to launch a runner for an organization, run the following docker command:

docker run -d --restart always --name github-runner \
  -e RUNNER_NAME_PREFIX="myrunner" \
  -e ACCESS_TOKEN="footoken" \
  -e RUNNER_WORKDIR="/tmp/github-runner-your-repo" \
  -e ORG_RUNNER="true" \
  -e ORG_NAME="octokode" \
  -e LABELS="my-label,other-label" \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /tmp/github-runner-your-repo:/tmp/github-runner-your-repo \
  myoung34/github-runner:latest

Because we want to run workflows such as docker build, we need to be able to use Docker commands inside this container.

You can solve this by sharing the Docker daemon on the host machine (-v /var/run/docker.sock:/var/run/docker.sock part, Docker outside of Docker)

Using Docker-in-Docker for your CI or testing environment? Think twice.

Running self-hosted runners on AWS ECS

Next, we'll use the Docker image from earlier and run it in AWS ECS.

In this article, I will focus on three points using AWS CDK (Typescript).


The first is how to achieve Docker outside of Docker (DooD) in AWS ECS (the -v /var/run/docker.sock:/var/run/docker.sock part).

In AWS ECS, you can solve this problem by adding a Volume to the Task side and mounting that Volume on the Container side.

taskDefinition.addVolume({
  name: "docker_sock",
  host: {
    sourcePath: "/var/run/docker.sock"
  }
})
containerDefinition.addMountPoints({
  containerPath: "/var/run/docker.sock",
  sourceVolume: "docker_sock",
  readOnly: true
})

AWS ECS offers two startup types, Fargate and EC2, but as Fargate is not currently supported to do the above, I chose the EC2 startup type this time.


The second is about the role you give to the ECS Task.

Of course, you can do this by storing AWS access keys in GitHub and passing them to GitHub Actions.

However, you can eliminate the need to store AWS access keys on the GitHub side by giving the self-hosted runner’s container itself the role it needs for the above.

const taskRole = new iam.Role(this, 'GitHubActionsSelfhostRunnerTaskRole', {
  assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'),
  managedPolicies: [
    iam.ManagedPolicy.fromAwsManagedPolicyName('AmazonEC2ContainerServiceFullAccess'),
    iam.ManagedPolicy.fromAwsManagedPolicyName('AmazonEC2ContainerRegistryPowerUser'),
  ],
})

const taskDefinition = new ecs.Ec2TaskDefinition(
  this,
  `GitHubActionsTaskDefinition`, {
    taskRole: taskRole
  }
);


The third is about spot instances.

Self-hosted runners are used for CI/CD, so you can leverage spot instances to keep costs low.

With AWS CDK, you can use a spot instance by setting the spotInstanceDraining property to true.

cluster.addCapacity('GitHubActionsCapacity', {
  instanceType: new ec2.InstanceType("t3.small"),
  spotInstanceDraining: true
});

Create CI / CD pipeline to deploy an application to AWS ECS

This time, we will create CI / CD pipeline to deploy an application to AWS ECS using that GitHub Actions self-hosted runners.

It's complicated because it's the same AWS ECS, but it's assumed that the cluster running self-hosted runners and the cluster running the application are separate.

f:id:tadashi-nemoto0713:20201020153820p:plain

Docker build and push to ECR
↓
Edit Task Definition file to newer docker image
↓
Register Task Definition and wait Service to update

When combined with the steps of GitHub Actions provided by AWS, it looks like this.

name: Deploy to ECS

on:  
  push:
    branches:
      - master

jobs:
  build:
    runs-on: self-hosted

    steps:
    - uses: actions/checkout

    - name: Configure AWS Credentials
      uses: aws-actions/configure-aws-credentials
      with:
        aws-region: ap-northeast-1

    - name: Login to Amazon ECR
      id: login-ecr
      uses: aws-actions/amazon-ecr-login

    - name: Build, tag, and push image to Amazon ECR
      id: build-image
      env:
        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
        ECR_REPOSITORY: ${{ secrets.AWS_ECR_REPO_NAME }}
      run: |
        docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
        echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

    - name: Fill in the new image ID in the Amazon ECS task definition
      id: task-def
      uses: aws-actions/amazon-ecs-render-task-definition
      with:
        task-definition: task-definition.json
        container-name: hoge-container
        image: ${{ steps.build-image.outputs.image }}

    - name: Deploy Amazon ECS task definition
      uses: aws-actions/amazon-ecs-deploy-task-definition
      with:
        task-definition: ${{ steps.task-def.outputs.task-definition }}
        service: hoge-service
        cluster: hoge-cluster
        wait-for-service-stability: true

As mentioned earlier, you'll need to store and pass your AWS access key to GitHub in aws-actions/configure-aws-credentials step.

- name: Configure AWS Credentials
   uses: aws-actions/configure-aws-credentials
   with:
     aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
     aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
     aws-region: ap-northeast-1

However, this is not necessary this time because we have given self-hosted runners the privileges they need to do so.

Summary

In this article, I have demonstrated how to use GitHub Actions self-hosted runners on AWS ECS. I believe it would be beneficial for the following use cases:

  • You're using GitHub Actions to deploy an application to AWS, but you don't want to pass AWS access keys to GitHub.

  • You want to create a cost-efficient CI/CD environment by utilizing spot instances (especially if you expect to significantly exceed your free quota).

  • You want to run your CI/CD environment on a machine with higher specs than GitHub's runner.


Our team plans to create better CI/CD pipelines based on GitHub Actions in the future.


hrmos.co