Enabling AWS Lambda in a Private Subnet to Access AWS Secrets Manager

To Nha Notes | Jan. 6, 2025, 7:08 p.m.

When deploying an AWS Lambda function inside a private subnet, it cannot access the internet or public AWS services directly. To enable the Lambda function to access AWS Secrets Manager securely, you can create a VPC Endpoint (VPCe) with the appropriate service type. This ensures that your Lambda function communicates with AWS Secrets Manager within the private AWS network.

Steps to Configure VPC Endpoint for AWS Secrets Manager

1. Create a VPC Endpoint

To allow access to AWS Secrets Manager from your Lambda in a private subnet:

1. Navigate to the VPC Management Console.

2. Select Endpoints from the left-hand menu.

3. Click on Create Endpoint.

4. Configure the endpoint:

   • Service Name: Choose the service type com.amazonaws.<region>.secretsmanager.

     • Replace <region> with your AWS Region (e.g., us-east-1).

   • VPC: Select the VPC where your Lambda function resides.

   • Subnets: Choose the private subnets where the Lambda function operates.

   • Security Groups: Attach a security group that allows inbound traffic from your Lambda function's security group on port 443 (HTTPS).

5. Click Create Endpoint to finalize the setup.

2. Update Lambda Security Group

Ensure the Lambda function's security group allows outbound traffic to the VPC Endpoint:

• Protocol: TCP

• Port Range: 443 (HTTPS)

• Destination: Security group of the VPC Endpoint or a CIDR range in your VPC.

3. Verify Lambda Configuration

• Ensure your Lambda function is associated with the correct subnets and security groups in its VPC Configuration.

• No additional changes are needed in the Lambda code. AWS SDKs (e.g., boto3) will automatically use the VPC Endpoint to communicate with AWS Secrets Manager.

template.yaml

Resources:
  SecretsManagerVPCEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      ServiceName: !Sub "com.amazonaws.${AWS::Region}.secretsmanager"
      VpcId: !Ref VpcId
      VpcEndpointType: Interface
      SubnetIds:
        - !Ref VpcSubnetId
      SecurityGroupIds:
        - !Ref VpcSecurityGroupId
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-secretsmanager-vpce"
          

  LambdaSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow outbound traffic for the lambda SnowflakeSQLExecutorFunction
      VpcId: !Ref VpcId
      SecurityGroupEgress:
        - IpProtocol: -1
          CidrIp: 0.0.0.0/0
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-snowflake-sql-executor-lambda-sg"
  
  
  SnowflakeSQLExecutorFunction:
    Type: AWS::Serverless::Function 
    Properties:
      FunctionName: !Sub "${AWS::StackName}-snowflake-sqls-executor"
      CodeUri: functions/snowflake_sqls_executor/
      Handler: app.lambda_handler
      Layers:
      - !Ref DependencyLayer
      Environment:
        Variables:
          ENV_NAME: !Ref EnvName
          SNOWFLAKE_CONNECTION_URI_SECRETS_ID: !Ref SnowflakeConnectionUriSecretsId
          SNOWFLAKE_CONNECTION_PK_SECRETS_ID: !Ref SnowflakeConnectionPkSecretsId
          DWH_BUCKET_NAME: !Ref DWHBucketName
          SNS_TOPIC_ARN: !Ref SnsTopicArn
          SLACK_INCOMING_WEBHOOK_URL: !Ref SlackIncomingWebhookUrl
          PAGERDUTY_INTEGRATION_KEY: !Ref PagerdutyIntegrationKey
      Runtime: python3.12
      MemorySize: 1024
      Timeout: 900
      Architectures:
      - x86_64
      Policies:
        - Statement:
          - Sid: LambdaInvokePolicy
            Effect: Allow
            Action:
              - lambda:InvokeFunction
            Resource: '*'
          - Sid: SecretsManagerGetSecretValuePolicy
            Effect: Allow
            Action:
              - secretsmanager:GetSecretValue  # pragma: allowlist secret
            Resource: !Sub "arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:/mp/maf/snowflake/*"
          - Sid: S3ReadOnlyAccessPolicy
            Effect: Allow
            Action:
              - s3:Abort*
              - s3:GetBucket*
              - s3:GetObject*
              - s3:GetObjectVersion
              - s3:HeadObject
              - s3:List*
            Resource:
              - !Sub "arn:aws:s3:::mp-${Stage}-dwh-code/*"
      VpcConfig:
        SecurityGroupIds:
          - !Ref LambdaSecurityGroup
        SubnetIds:
          - !Ref VpcSubnetId

4. Test the Setup

Deploy and execute your Lambda function to ensure it can access secrets stored in AWS Secrets Manager. Use the following sample code to confirm:

import boto3

secrets_client = boto3.client('secretsmanager')

response = secrets_client.get_secret_value(SecretId='your-secret-id')
print("Secret Value:", response['SecretString'])

Key Benefits

• Enhanced Security: Communication stays within the AWS network, reducing exposure to the internet.

• Simplified Management: No need for NAT Gateways or internet gateways for Secrets Manager access.

• Cost Efficiency: Avoid costs associated with NAT Gateway data processing.

Cost Considerations

Using a VPC Endpoint incurs additional charges based on:

• Hourly Endpoint Charges: Fixed hourly cost for the VPC Endpoint.

• Data Processing Charges: Per GB of data transferred through the VPC Endpoint.

For example, as of this writing:

• VPC Endpoint (Interface) costs $0.01 per hour.

• Data transfer costs $0.01 per GB.

Conclusion

By configuring a VPC Endpoint for AWS Secrets Manager, you can securely and efficiently enable your Lambda function in a private subnet to retrieve secrets without requiring internet access. This approach ensures a secure, reliable, and cost-effective architecture for your serverless applications.