The first time you run Airflow, it will create a file called airflow.cfg in
your $AIRFLOW_HOME directory (~/airflow by default). This file contains Airflow’s configuration and you
can edit it to change any of the settings. You can also set options with environment variables by using this format:
$AIRFLOW__{SECTION}__{KEY} (note the double underscores).
For example, the
metadata database connection string can either be set in airflow.cfg like this:
[core]
sql_alchemy_conn = my_conn_string
or by creating a corresponding environment variable:
AIRFLOW__CORE__SQL_ALCHEMY_CONN=my_conn_string
You can also derive the connection string at run time by appending _cmd to the key like this:
[core]
sql_alchemy_conn_cmd = bash_command_to_run
-But only three such configuration elements namely sql_alchemy_conn, broker_url and result_backend can be fetched as a command. The idea behind this is to not store passwords on boxes in plain text files. The order of precedence is as follows -