Using CloudFormation's Fn::Sub with Bash parameter substitution

Let’s say that you need to inject a large bash script into a CloudFormation AWS::EC2::Instance Resource’s UserData property. CloudFormation makes this easy with the Fn::Base64 intrinsic function:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
AWSTemplateFormatVersion: '2010-09-09'

Resources:
  VPNServerInstance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: ami-efd0428f
      InstanceType: m3.medium
      UserData:
        Fn::Base64: |
          #!/bin/sh
          echo "Hello world"          

In your bash script, you may even want to reference a parameter created elsewhere in the CloudFormation template. This is no problem with Cloudformation’s Fn::Sub instrinsic function:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
AWSTemplateFormatVersion: '2010-09-09'

Parameters:
  Username:
    Description: Username
    Type: String
    MinLength: '1'
    MaxLength: '255'
    AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
    ConstraintDescription: must begin with a letter and contain only alphanumeric
      characters.

Resources:
  VPNServerInstance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: ami-efd0428f
      InstanceType: m3.medium
      UserData:
        Fn::Base64: !Sub |
          #!/bin/sh
          echo "Hello ${Username}"

The downside of the Fn::Sub function is that it does not play nice with bash’ parameter substitution expressions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
AWSTemplateFormatVersion: '2010-09-09'

Parameters:
  Username:
    Description: Username
    Type: String
    MinLength: '1'
    MaxLength: '255'
    AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
    ConstraintDescription: must begin with a letter and contain only alphanumeric
      characters.

Resources:
  VPNServerInstance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: ami-efd0428f
      InstanceType: m3.medium
      UserData:
        Fn::Base64: !Sub |
          #!/bin/sh
          echo "Hello ${Username}"
          FOO=${FOO:-'bar'}

The above template fails validation:

1
2
3
$ aws cloudformation validate-template --template-body file://test.yaml

An error occurred (ValidationError) when calling the ValidateTemplate operation: Template error: variable names in Fn::Sub syntax must contain only alphanumeric characters, underscores, periods, and colons

The work-around is to rely on another intrinsic function: Fn::Join:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
AWSTemplateFormatVersion: '2010-09-09'

Parameters:
  Username:
    Description: Username
    Type: String
    MinLength: '1'
    MaxLength: '255'
    AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
    ConstraintDescription: must begin with a letter and contain only alphanumeric
      characters.

Resources:
  VPNServerInstance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: ami-efd0428f
      InstanceType: m3.medium
      UserData:
        Fn::Base64: !Join
          - '\n'
          - - !Sub |
              #!/bin/sh
              echo "Hello ${Username}"
            - |
              FOO=${FOO:-'bar'}              

This allows you to mix CloudFormation substitutions with Bash parameter substititions.

Bonus

While we’re talking about CloudFormation, another good trick comes from cloudonaut.io regarding using a Optional Parameter in CloudFormation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Parameters:
  KeyName:
    Description: (Optional) Select an ssh key pair if you will need SSH access to the machine
    Type: String

Conditions:
  HasKeyName:
    Fn::Not:
    - Fn::Equals:
      - ''
      - Ref: KeyName

Resources:
  VPNServerInstance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: ami-efd0428f
      InstanceType: m3.medium
      KeyName:
        Fn::If:
          - HasKeyName
          - !Ref KeyName
          - !Ref AWS::NoValue

Note that the KeyName has Type: String. While Type: AWS::EC2::KeyPair::KeyName would likely be a better user experience as it would render a dropdown of all keys, it does not allow for empty values:

… if you use the AWS::EC2::KeyPair::KeyName parameter type, AWS CloudFormation validates the input value against users’ existing key pair names before it creates any resources, such as Amazon EC2 instances.

Bonus#

Bonus