To Nha Notes | June 29, 2022, 2:18 p.m.
1. If we got error when login to Airflow web UI first time we can try blow options.
- Check security group to ensure inbound allow access to posgres metadata database via port 5432.
- Try to upgrade to more power instance class to get it more capacity for db migration first time, then downgrade.
https://docs.aws.amazon.com/mwaa/latest/userguide/t-apache-airflow-202.html#error-scheduler-202
2. If get below error when trigger a DAG
we were able to see that this was down to the wrong KMS key being used to encrypt the logs. This can happen if you have destroyed/deployed your MWAA multiple times. Every time you do that cycle, the KMS key gets deleted/recreated. Unless you delete the CloudWatch log groups, they appear to maintain the originally configured KMS key.
*** Reading remote log from Cloudwatch log_group: airflow-seam-mwaa-cdk-demo-Task log_stream: hello_world_ondemand/hello_task/2021-12-14T15_30_49.698602+00_00/1.log. Could not read remote logs from log_group: airflow-seam-mwaa-cdk-demo-Task log_stream: hello_world_ondemand/hello_task/2021-12-14T15_30_49.698602+00_00/1.log.
We were able to see that this was down to the wrong KMS key being used to encrypt the logs. This can happen if you have destroyed/deployed your MWAA multiple times. Every time you do that cycle, the KMS key gets deleted/recreated. Unless you delete the CloudWatch log groups, they appear to maintain the originally configured KMS key.
The resolution was to do a clean deploy. To do this, we first deleted the MWAA environment (cdk destroy..) and then deleted the log groups within CloudWatch. Once cleared, we redeployed the MWAA environment and the issue was resolved.
Source: https://dev.to/aws/setting-up-mwaa-to-use-a-kms-key-4p6b