Security-conscious cloud deployments from Github Actions via OpenID Connect

Goals

This ticket is focused on how we can securely deploy to a major cloud provider environment (e.g. AWS, Azure, GCP) from within our Github Actions workflows.

Why is this challenging?

A naive solution to this problem is to generate some cloud provider credentials (e.g. AWS Access Keys) and to store them as a Github Secret. Our Github Actions can then utilize these credentials in its workflows. However, this technique contains a number of concerns:

It is reliant on long-standing credentials being stored in Github Actions. Some environments are unable to generate such long-standing credentials without serious admin intervention (e.g. environments using CloudTamer/Kion).
It grants any user with write-access to the repo with full use of the possibly wide-scoped credentials. Currently, within Github there are not a sufficient ways to limit who or what can be done with the credentials. For example, any user with write-access to the repo could create a workflow action that references the production credentials and uses them to teardown the production environment. This is due to the combination of two factors:
1. The instructions run during the build are entirely specified within the Github repo. This means that anyone can alter them as they see fit.
2. The cloud providers lacks information about the context of a build (e.g. git branch or github user), and is therefore unable to apply or enforce any sort of restrictions regarding what a build can do.
Github Environments solves some of these problems, however at time of writing it is only available on public repositories or private Github Enterprise account repositories, making it an unvialable solution for many of our partners.

Solution: OpenID Connect

On Nov 23, 2021 Github Actions announced the general availability of support for OpenID Connect (OIDC). For an in-depth understanding of this, I recommend reviewing the following links:

High level summary

With OIDC, you can register Github as an Identity Provider within your cloud platform of choice. When your Github Action workflows run, they can be setup to request short-lived credentials from your cloud provider. When the cloud provider grants the access token, it will be associated with a particular IAM Role. That IAM Role should be set up with the permissions necessary for deploying your application.

No need to store credentials. Github Actions workflows will request a short-lived access token at runtime.
When a short-lived access token is requested, Github Actions sends an OIDC token with claims describing the context of the workflow (link). These claims can be interogated by the cloud provider and used to determine whether or not a token should be granted. This allows us to hardcode security requirements (e.g. limiting particular IAM Roles to specific Github branches, only allowing executions triggered by specified github usernames) in the cloud provider, providing guard-rails to limit what can be done by any particular user with write-access to a Github repository.

Example with AWS

Setup AWS

Setting up OIDC with AWS is described in depth here, however the following is a quick summary:

Add Github as an Identity provider (docs).
Console Screenshot

Create an IAM Policy for deployment executions. See recommendations for tips on how to craft this policy. Below is an example policy for frontend static website deployments:

Example policy

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "SyncS3Bucket",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:PutObject",
        "s3:PutObjectAcl",
        "s3:DeleteObject"
      ],
      "Resource": [
        "arn:aws:s3:::my-staging-bucket",
        "arn:aws:s3:::my-staging-bucket/*"
      ]
    },
    {
      "Sid": "InvalidateCloudfrontDistribution",
      "Effect": "Allow",
      "Action": ["cloudfront:CreateInvalidation"],
      "Resource": "*"
    }
  ]
}

Create role for deployment executions.
Console Screenshot
Attach your revelant policies. Optionally, specify the role’s permission boundaries.
Console Screenshot
For this example, we’ll be naming our role Frontend-Staging-Deployment-Role
Console Screenshot

By default, the IAM Role created for our OIDC Web Identity contains a condition where the aud claim in our token should match sts.amazonaws.com. However, it is here that we will enforce more strict conditions. The sub claim in our token contains information about the reposoitory and branch. As such, we need to customize our trust relationship to encorce our custom conditions.

Console Screenshot

In the following example, we configure the trust relationship to enforce that this role can only be used on builds on the my-org/my-repo repository’s staging branch:

Example conditions

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::123456789000:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
          "token.actions.githubusercontent.com:sub": "repo:my-org/my-repo:ref:refs/heads/staging"
        }
      }
    }
  ]
}

Setup GitHub Actions Workflow

Workflows utilizing OIDC need a few particular elements.

We need to customize the permissions of our GITHUB_TOKEN via the permissions block. The workflow will need to be able to write an id-token along with the default permissions of reading the contents of the repository:
Permissions block
1 2 3
permissions: id-token: write contents: read

Add tooling to request the an access token from AWS. For this, the AWS’ offical “Configure AWS Credentials” Action works well. To use this, you must provide the ARN of the role that you would like to assume in your execution:

AWS Configuration step

1
2
3
4
5
- name: Configure AWS credentials
  uses: aws-actions/configure-aws-credentials@v1
  with:
    role-to-assume: arn:aws:iam::123456789000:role/Frontend-Staging-Deployment-Role
    aws-region: us-west-2

Full example

A complete example workflow

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
name: Deploy Staging Frontend

on:
  push:
    branches:
      - staging

permissions:
  id-token: write
  contents: read

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Setup Node.js
        uses: actions/setup-node@v2
        with:
          node-version: 12

      - name: Check out repository code
        uses: actions/checkout@v2

      - name: Install dependencies
        run: npm install

      - name: Build code
        run: npm run build

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          role-to-assume: arn:aws:iam::123456789000:role/Frontend-Staging-Deployment-Role
          aws-region: us-west-2

      - name: Sync with S3 bucket
        env:
          BUCKET: my-staging-bucket
        run: |
          aws s3 sync \
            ./build "s3://${BUCKET}" \
            --acl public-read \
            --follow-symlinks \
            --delete          

      - name: Invalidate CloudFront
        env:
          DISTRIBUTION: EDFDVBD6EXAMPLE
        run: |
          aws cloudfront create-invalidation \
            --distribution-id $DISTRIBUTION \
            --paths "/*"          

In the above example, we have hardcoded the Role ARN, S3 Bucket, and Cloudfront Distribution ID in the workflow file. However, you may prefer to store these values as Github Secrets. This allows the values to be changed without a code change and additionally helps avoid data-leak. An example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
- name: Configure AWS credentials
  uses: aws-actions/configure-aws-credentials@v1
  with:
    role-to-assume: ${{ secrets.STAGING_CD_ROLE_ARN }}
    aws-region: us-west-2

- name: Sync with S3 bucket
  env:
    BUCKET: ${{ secrets.STAGING_BUCKET_NAME }}
  run: |
    aws s3 sync \
      ./build "s3://${BUCKET}" \
      --acl public-read \
      --follow-symlinks \
      --delete    

- name: Invalidate CloudFront
  env:
    DISTRIBUTION: ${{ secrets.STAGING_DISTRIBUTION_ID }}
  run: |
    aws cloudfront create-invalidation \
      --distribution-id $DISTRIBUTION \
      --paths "/*"    

Recommendations

Each IAM role should relate to a single deployment. For example, you may have a Service-X-Frontend-Staging-Deployment role and a Service-X-Frontend-Production-Deployment role, each referencing specific IAM policies that specify the minimal permissions needed to deploy to its respective environment. Each role should specify which repositories and branches can use the role.
Configuring the IAM role’s trust relationship is key to enforcing logic around deployment permissions. Understanding the Condition block and the Github OIDC token is paramount.

Goals#

Why is this challenging?#

Solution: OpenID Connect#

High level summary#

Example with AWS#

Setup AWS#

Setup GitHub Actions Workflow#

Full example#

Recommendations#