A deep dive into Airflow’s CI: Docker for reproducible environments, Breeze to unify local and CI workflows, and Selective Checks to run only the tests that matter.

PGMQ-SQLAlchemy 0.2.0 introduces support for ACID transactions, enabling developers to handle business logic and message queue operations within the same transaction, ensuring consistency and reliability of data and messages.

Sharing my experience designing and implementing the chaos scenario rollback feature for the CNCF Krkn project during the LFX Mentorship, including technical solutions, collaboration with the Red Hat Chaos Engineering team, and insights from the application and participation process.

This article provides an in-depth exploration of Airflow 2’s Dag Processing mechanism and summarizes best practices related to Dag Processing.

Exploring the AIP-82 External Event Driven Scheduling proposal in Airflow 3, the implementation details and use cases of the Common Message Queue, and how to enable real-time event-driven workflow scheduling in Airflow.

What is Apache Airflow? What problems does it solve? Understand its common use cases, architecture, and limitations.

How a Complete Beginner in Data Engineering Became an Apache Airflow Committer in 5 Months with 70+ PRs and 300 Hours of Contributions

How a beginner with no prior Data Engineering experience managed to contribute over 50 PRs in just 3 months

How to reduce context switching in CLI workflows on Mac through push-based notifications

A brief introduction to Embedded Database and concept of RocksDB

How to use uv to manage Python packages and projects

Reflection on my backend summer internship in the Data Platform department at Appier