Setup CI & CD for MWAA

To Nha Notes | Dec. 23, 2021, 1:13 p.m.

Create GitHub Actions workflow

Create a .github/workflows/ folder to store the GitHub S3 Sync Action file.

Create a .yml file in the .github/workflows/ sub folder with the following contents:

name: ci

on:

  push:

    paths:

      - 'dags/**'

  pull_request:

    branches:

      - develop

 

jobs:

  test:

    runs-on: ubuntu-latest

    steps:

    - uses: actions/checkout@v2

    - name: Set up Python

      uses: actions/setup-python@v2

      with:

        python-version: '3.7'

    - name: Install dependencies

      run: |

        python -m pip install --upgrade pip

        pip install -r dags/requirements.txt

        pip check

    - name: Lint with Flake8

      run: |

        pip install flake8

        flake8 --ignore E501 dags --benchmark -v

    - name: Confirm Black code compliance (psf/black)

      run: |

        pip install black

        black dags --check

 

References:

https://programmaticponderings.com/category/devops/continuous-delivery/

https://github.com/aws-samples/amazon-mwaa-automating-dag-deployment

https://aws.amazon.com/blogs/opensource/automating-a-dag-deployment-with-amazon-managed-workflows-for-apache-airflow/

https://aws.amazon.com/blogs/opensource/deploying-to-amazon-managed-workflows-for-apache-airflow-with-ci-cd-tools/

https://github.com/aws-samples/amazon-mwaa-automating-dag-deployment