China’s localized data stack, streaming-first infrastructure, enterprise investment in open source, open governance, and lessons for Taiwan’s tech communities.
A deep dive into Airflow’s CI: Docker for reproducible environments, Breeze to unify local and CI workflows, and Selective Checks to run only the tests that matter.
PGMQ-SQLAlchemy 0.2.0 introduces support for ACID transactions, enabling developers to handle business logic and message queue operations within the same transaction, ensuring consistency and reliability of data and messages.
Sharing my experience designing and implementing the chaos scenario rollback feature for the CNCF Krkn project during the LFX Mentorship, including technical solutions, collaboration with the Red Hat Chaos Engineering team, and insights from the application and participation process.
This article provides an in-depth exploration of Airflow 2’s Dag Processing mechanism and summarizes best practices related to Dag Processing.
Exploring the AIP-82 External Event Driven Scheduling proposal in Airflow 3, the implementation details and use cases of the Common Message Queue, and how to enable real-time event-driven workflow scheduling in Airflow.
What is Apache Airflow? What problems does it solve? Understand its common use cases, architecture, and limitations.
How a Complete Beginner in Data Engineering Became an Apache Airflow Committer in 5 Months with 70+ PRs and 300 Hours of Contributions
How a beginner with no prior Data Engineering experience managed to contribute over 50 PRs in just 3 months
How to reduce context switching in CLI workflows on Mac through push-based notifications
A brief introduction to Embedded Database and concept of RocksDB
How to use uv to manage Python packages and projects
Reflection on my backend summer internship in the Data Platform department at Appier
2024 Dcard Backend Intern Assignment
2024 TSMC Summer Intern Interview
2024 Dcard Summer Intern Interview
2024 Appier Summer Intern Interview
2024 Software Summer Intern Interview - Appier / Dcard / TSMC
Record some internship experiences
Resolve psycopg2 Install Error on MacOS
When your architecture only involves Postgres, why introduce an additional MQ? This article explores the advantages of PGMQ for lightweight asynchronous tasks, teaches you how to deploy it quickly with Docker, and integrates it into Python projects.
How to get a slice from a specific range in Hugo?
Detailed explanation of Redis persistence settings: RDB and AOF. Pros and cons of RDB and AOF. Detailed implementation with Redis source code.
How to transaction in SqlAlchemy
Read file (BinaryIO) multiple time in Python. Solution to prevent empty content in the second read.
Solving Django backend DB connection overload with PgBouncer
Setup Cloudflare Tunnel for NAT, an alternative to Ngrok
Common tmux commands Cheat Sheet
FastAPI Testing: Mock AWS S3 Boto3 With Moto
Kubernetes Cheat Sheet: Extract ConfigMap or Secret to .env file
What did I do in the first semester of freshman year?
