↓ Skip to main content

AI Is Too Accessible: A Data Engineer's Case for Caution

·1140 words·6 mins

This is the opposite of a “Data Engineering is DEAD” post!

“AI isn’t going to replace data engineers.”

My worry is that too many businesses are acting as if it already has. I use AI every day and it has changed how fast I can build scripts and slide decks, and it even helps me plan fun ideas with my kids! That’s all to say: this isn’t an anti-AI post. It’s a caution about who is using it, and how.

If you’d rather hear this as a ranty video, check it out here:

How I actually use it
#

I’m not doing data engineering work daily anymore, but when building out Analytics Masterminds’ Fabric framework I treated Claude like a junior developer. I gave it guardrails of what I’m building, what the framework should look like, which component to work on, and a steer to keep it simple and approachable rather than technically perfect.

I deliberately kept it away from my repo to begin with. I took what it wrote, reviewed it and built it in myself. I wanted to make sure I knew every part of this so I could build upon it easily. It also meant I could learn from it too, and most importantly, check the code was right.

And it often wasn’t. Roughly 30% of the time, it got something wrong or used a Databricks function for example. Some of that may be my prompting, admittedly, but a lot of it was plain hallucination.

Even so, it helped immensely. I got our framework most of the way there in a couple of days, work that would have taken me weeks by hand. But I could only do that because I have the experience and knowledge to spot that 30%.

AI skipped the skills barrier
#

Most new technologies come with a barrier. You need skills, setup and support before you get any value out of them, and that keeps them just out of reach of the average person, which is both good and bad…I know!

The problem is that AI skipped all of that. I’d argue it’s almost too accessible. Anyone can open a chat window and get an answer that looks right in seconds. On top of that, we all know that when it’s wrong, it’s confidently wrong. A bad answer sounds exactly as sure of itself as a good one.

How often is it wrong?
#

I dug into this when I saw a random LinkedIn statistic. Vectara’s hallucination leaderboard gives models the easiest possible test: summarise a document using only the facts in front of you. Even then, most models invent something, with most around the 10% mark!

ModelProviderHallucination rate
GPT-5.4OpenAI7.0%
Gemini 2.5 ProGoogle7.0%
Llama 4 MaverickMeta8.2%
DeepSeek V4 ProDeepSeek8.6%
GPT-5.5OpenAI9.3%
Claude Haiku 4.5Anthropic9.8%
Gemini 3.1 Pro (preview)Google10.4%
Claude Sonnet 4.6Anthropic10.6%
Claude Opus 4.7Anthropic12.0%
Mistral Large 3Mistral14.5%
Grok 4.1 Fast (reasoning)xAI19.2%

Source: Vectara hallucination leaderboard, updated May 2026.

The stats do vary by model and by test, but my point stands: One in every ten responses you get from AI is likely wrong You need someone who knows data engineering to catch it.

The danger is the failures you don’t see
#

If a pipeline breaks as soon as you deploy some AI-written code or update, you’ll notice it right away and fix it, right?

The problem is the pipeline that deploys successfully, runs fine, and quietly calculates a KPI wrong for six months or truncates a data type. AI confidently gave you the solution and nobody checked it.

Or the one I see on Linkedin all the time about the new vibe-coded web portal everyone uses, with API keys and confidential data sitting right there in its HTML. Any experienced developer knows what happens when keys get pushed to a repo. AI isn’t going to stop you, and an inexperienced user won’t know what to check or where to look.

Put the engineering back in data engineering
#

Before AI, we had Stack Overflow. Sure, plenty of us pasted and ran code without understanding it 😅. But it was a snippet, coming from a visibly fallible human, so you still took it with a pinch of salt.

AI is trained on all of that, and it hands you the whole picture with total confidence - the code, where to put it, and everything to build around it. It feels like having an expert on tap, and don’t get me wrong, I know it’s easy to be impressed and fall into trusting it.

What’s missing is the “engineer” in data engineer: deployment, testing, source control and code review. Those software engineering habits are exactly what get dropped when you take AI’s output at face value.

You also get out what you put in. Vague prompts get vague, often wrong, answers. Being precise about what you want, and knowing what to rule out, comes from experience.

Where will the next seniors come from?
#

If businesses believe AI can do the work of junior developers, they’ll stop hiring them. Juniors learn by picking up the low-hanging fruit on the backlog: the small, routine tasks that teach you how everything fits together end to end.

Hand those tasks to agents and that learning disappears. We won’t feel it in two or three years. In five, there will be nobody left with the experience to deploy, debug and build complex systems, or to check what the AI produces.

Let AI do the dishes
#

We should have AI doing the dishes while we paint, not the other way round. The same goes for engineering.

  • Hand off: repetitive, routine work that takes up headspace without teaching you anything, like boilerplate, scaffolding and test generation.
  • Keep: the work where you learn something that makes the next job easier, and the judgement calls about design, correctness and security.

As one line I heard recently puts it:

“You can outsource your thinking, but you can’t outsource your understanding.”

Understanding is what lets you spot the risk, and then put the controls in place so it doesn’t happen.

Not a silver bullet
#

Every few years something is declared dead. When Power Apps arrived, application development was supposedly finished. Instead, it became easier to build mediocre apps that someone later had to rebuild.

AI will follow the same path. Data engineering isn’t going anywhere and, if anything, there will be plenty of work cleaning up AI-built pipelines for years to come!

So use it. I do, every day. Just treat it as a powerful tool in experienced hands, not a replacement for those hands.

How are you using AI in your data work? I’d like to hear where it’s helped you, and where it’s caught you out.

Related