Airflow: essentials of workflow orchestrator for ETL pipelines

[sanghisha145]

Title of the talk/workshop
Airflow: essentials of workflow orchestrator for ETL pipelines

Abstract of the talk/workshop
Apache Airflow has emerged as a powerful tool for orchestrating and automating data workflows. This talk will introduce participants to the core concepts of Apache Airflow, including DAGs, operators, schedulers, and its UI. We will explore practical examples of setting up workflows, monitoring tasks, and leveraging Airflow for efficient data pipeline management. Whether you're new to Airflow or looking to deepen your understanding, this session aims to equip you with the knowledge to streamline your data engineering processes.

Category of the talk/workshop
Data Engineering

Duration (including Q&A)
30min

Level of Audience
Beginner

Speaker Bio

1. Speaker Bio (Brief) -> Divya Sanghi is a senior software engineer who is interested in Big Data and processing, cloud computing, containerization, distributed systems, design system.
2. Company/College -> Walmart Labs
3. Email -> [email protected]
4. Years of Exp ->3

Prerequisites(if any)

1. standalone airflow installed
2. python version 3.8+