Luminousmen Blog - Python, Data Engineering & Machine Learning

Most of Your Backfills Didn't Have to Happen

Design Patterns Suck

If Your Data Engineer Can Be Replaced by an Agent, You Already Hired Badly

Mastering Theta Sketches for Cardinality Estimation

Stop Feeding Me AI Slop

Why Your 5-Second BigQuery Query Isn’t Cheap

What Do You Think?

AI Fluency Expectations

Compaction in Lakehouse

How the Community Turned Into a SaaS Commercial

Choosing the Right Compression Codec

Words Are the New Bytecode

Only the Squeaky Wheel Gets the Oil

Data Partitioning: Partition. Regret. Repeat

Why Parquet Is the Go-To Format for Data Engineers

Data Partitioning: Slice Smart, Sleep Better

Data Engineering: Now with 30% More Bullshit

How Not to Partition Data in S3 (And What to Do Instead)

How to Drop a BigQuery Sharded Table

Data Warehouse, Data Lake, Data Lakehouse, Data Mesh: What They Are and How They Differ

Nobody Actually Cares

Mastering Database Design: Denormalization