Data Engineering Insights

Explore articles on data engineering, big data, and analytics.

10 articles

All articles open on Medium in a new tab.

Optimising Multi Cloud Data Pipelines (Without Losing Your Shit)
Medium
Optimising Multi Cloud Data Pipelines (Without Losing Your Shit)

Two years ago I moved a “simple” pipeline from one cloud to another for a project . By week two my Slack looked like a heart monitor… Continue reading on Towards Data Engineering »

aws
apache-iceberg
data-science
10/13/2025
1 min read
Medium
Careers in Data Engineering 2025: New Roles and Skill Paths
Medium
Careers in Data Engineering 2025: New Roles and Skill Paths

“Do you see yourself as a guardian of data pipelines or the bridge between code and business strategy? In 2025, you can be both.” Continue reading on Data Engineer Things »

data-analytics
artificial-intelligence
data-engineer
8/28/2025
1 min read
Medium
The Data Engineer’s Toolkit: 12 Free Monitoring Dashboards You Didn’t Know Existed
Medium
The Data Engineer’s Toolkit: 12 Free Monitoring Dashboards You Didn’t Know Existed

How I stopped firefighting and reclaimed my weekends, without spending a dime Continue reading on Data Engineer Things »

data-engineering
data-science
towards-data-science
7/16/2025
1 min read
Medium
The Night I Caught a Fraud Ring in Real Time Using Kafka Streams & Neo4j
Medium
The Night I Caught a Fraud Ring in Real Time Using Kafka Streams & Neo4j

How I turned data chaos into instant alerts and how you can. Continue reading on Towards Data Engineering »

data-visualization
data-analytics
data-engineering
7/16/2025
1 min read
Medium
My 5 Step Blueprint for Building a Scalable Feature Store with LakeFS & Feast on AWS
Medium
My 5 Step Blueprint for Building a Scalable Feature Store with LakeFS & Feast on AWS

How I architected a rock solid feature store that scales to millions of features and how you can too Continue reading on Towards Data Engineering »

data-analytics
data-science
amazon-web-services
7/10/2025
1 min read
Medium
Why Gen AI Isn’t Coming for Your Data Engineering Job (And How to Thrive Anyway)
Medium
Why Gen AI Isn’t Coming for Your Data Engineering Job (And How to Thrive Anyway)

“When a seasoned data scientist told me Gen AI would replace data engineers, my confidence cratered. I spent days questioning my decade in… Continue reading on Data Engineer Things »

data-engineering
artificial-intelligence
data
6/22/2025
1 min read
Medium
10 Hard Topics in Data Engineering That Will Skyrocket Your Career
Medium
10 Hard Topics in Data Engineering That Will Skyrocket Your Career

Ever scroll through Reddit and wonder, “What actually separates a good data engineer from a great one?” I did after stumbling on a… Continue reading on Towards Data Engineering »

data-engineer
data
data-science
6/22/2025
1 min read
Medium
7 Brutal Truths About Data Engineering Certifications (Most of Them Are Useless!)
Medium
7 Brutal Truths About Data Engineering Certifications (Most of Them Are Useless!)

I spent months and hundreds of dollars chasing certs here’s what actually landed me interviews (and what almost cost me my sanity) Continue reading on Medium »

certification
data
data-science
6/22/2025
1 min read
Medium
16 Must-Have Free Data Quality Checkers You Should Be Using Today
Medium
16 Must-Have Free Data Quality Checkers You Should Be Using Today

Ever spent hours debugging a broken pipeline only to discover a single malformed value was the culprit? Continue reading on Data Engineer Things »

data
artificial-intelligence
data-analysis
6/22/2025
1 min read
Medium
11 High Performance Query Engines I Wish I Knew About Sooner
Medium
11 High Performance Query Engines I Wish I Knew About Sooner

How I stopped waiting and started analyzing Victor Oketch Sabare · 6 min read · Jun 14, 2025 Continue reading on Towards Data Engineering »

big-data
sql
query-optimization
6/14/2025
1 min read
Medium