Airflow needs to know how to connect to your environment. Information such as hostname, port, login and passwords to other systems and services is handled in the
Admin->Connections section of the UI. The pipeline code you will author will reference the ‘conn_id’ of the Connection objects.
Connections can be created and managed using either the UI or environment variables.
See the Connections Concepts documentation for more information.
Admin->Connections section of the UI. Click the
Create link to create a new connection.
Fill in the
Conn Id field with the desired connection ID. It is recommended that you use lower-case characters and separate words with underscores.
Choose the connection type with the
Conn Type field.
Fill in the remaining fields. See Connection Types for a description of the fields belonging to the different connection types.
Save button to create the connection.
Admin->Connections section of the UI. Click the pencil icon next to the connection you wish to edit in the connection list.
Modify the connection properties and click the
Save button to save your changes.
Connections in Airflow pipelines can be created using environment variables. The environment variable needs to have a prefix of
AIRFLOW_CONN_ for Airflow with the value in a URI format to use the connection properly.
When referencing the connection in the Airflow pipeline, the
conn_id should be the name of the variable without the prefix. For example, if the
conn_id is named
postgres_master the environment variable should be named
AIRFLOW_CONN_POSTGRES_MASTER (note that the environment variable must be all uppercase).
Airflow assumes the value returned from the environment variable to be in a URI format (e.g.``postgres://user:password@localhost:5432/master`` or
s3://accesskey:secretkey@S3). The underscore character is not allowed in the scheme part of URI, so it must be changed to a hyphen character (e.g. google-compute-platform if conn_type is google_compute_platform). Query parameters are parsed to one-dimensional dict and then used to fill extra.