ML Engineer vs. AI Engineer: What Actually Changes Day to Day
ML engineer vs. AI engineer, compared on daily work, tooling, evaluation and failure modes by someone who has shipped trained vision models and LLM tools.
Search "ML engineer vs AI engineer" and you get career sites and course pages agreeing on one sentence: ML engineers train models, AI engineers build products on top of models someone else trained. That sentence is right. What it leaves out is how differently the two jobs feel on an ordinary Tuesday: what you open first, what you change when something breaks, and how you know you are done. I spent a year as a machine learning engineer at Cowlar Design Studio, shipping computer vision to factory and edge hardware, and alongside that I have built LLM applications: an Azure OpenAI chatbot, LLM developer tools in Go and a multi-agent system. This essay compares the two kinds of work from the inside.
A caveat, the same one I gave in my comparison of AI PM and AI engineer roles: I have never held a job titled "AI Engineer". My view of that role comes from the LLM application work I describe below, not from a job under that name. Titles also vary a lot between companies, so treat what follows as my opinion, not a standard.
The difference in one line
My version: an ML engineer changes the model, and an AI engineer changes everything around the model.
When a trained vision model gets something wrong, the ML engineer's tools are the data, the architecture, the training run and the serving code. All of them are yours, and the fix usually ends in a new set of model weights.
When an LLM application gets something wrong, the model's weights are almost never yours to change. The tools are the prompt, the context you feed in, how the task is split into steps, which model you call, and the code that checks and repairs the output. The fix ends in a new prompt, a new retrieval step or a new guardrail, and the model stays the same.
Both jobs involve shipping software, measuring quality and fighting latency. The difference is which lever you pull.
What ML engineering looked like for me
From April 2024 to April 2025 I was a machine learning engineer at Cowlar Design Studio. Two projects defined the job.
The first was a fiber organizer and termination line for viaPhoton, an Amazon-partnered fiber optics supplier. I worked on the machine learning and computer vision for the machine, which is covered by viaPhoton's US Patent 12,001,073 B2. The vision system sorted 250-micron fibers with 96% accuracy and cut sorting time from 3 minutes to 8 seconds. I wrote that project up in production ML on a factory floor, and the short version is in the fiber organizer case study.
The second was a smart shopping cart for Al-Meera Supermarkets in Qatar. I optimized and improved the action recognition architecture that predicts how customers interact with the carts, reaching 95% accuracy in real time on edge devices, and I cleaned up the data pipeline through validation and testing. The stack was TensorFlow, MoViNet, MQTT, MLflow, NGINX and Docker.
Across the role, the daily work was mostly:
- Data. On the smart cart, part of the job was cleaning and improving the data pipeline through validation and testing. In my experience that is where much of a vision model's quality is decided.
- Training infrastructure. I set up Docker Swarm orchestration for distributed GPU training across nodes, which made training 3–4x faster. Faster training means more experiments a week, and in my experience that matters more than any single clever idea.
- Experiment tracking. MLflow handled experiment logging and automated model versioning. When a model is a file of weights, you need to know exactly which data and code produced it.
- Inference. I brought inference from 200 ms to under 100 ms per request using NVIDIA-enabled Docker images, code optimization and dimensionality reduction.
- Deployment. CI/CD pipelines with Docker, and load-balanced deployment servers with NGINX.
None of the vision work involved writing a prompt. The model was ours. Its weights, its failure cases and its latency were all our problem.
What AI engineering looked like for me
The LLM side started with an Azure OpenAI chatbot for an online retail customer experience, built during the same Cowlar role. I kept going with side projects:
- gitmind, a Go tool for an AI-assisted git workflow that runs offline against an Ollama-compatible local model: commit message generation, pre-commit security analysis, test stubs and splitting a large change into several commits. Its README also describes a fallback to heuristic commit messages when the model isn't configured or fails, which is the most AI-engineering detail in the whole project.
- gptree-cli, a Go CLI that exports a codebase into an LLM-ready prompt, with a directory tree, markdown output and token-aware chunking. Much of its point is fitting the right context into a limited window.
- FleadrAI, a multi-agent lead generation system that automates 70%+ of the lead lifecycle. It keeps a persistent memory vault per lead and uses DSPy-optimized prompts, with a React front end, an MCP server and FastAPI.
The daily work on these projects looked nothing like the factory work. I never trained a model. The work was:
- Context design. Deciding what the model sees: which diff, which files, which memory, and in what order. Token-aware chunking in gptree-cli exists because the window is finite and the model is only as good as what fits in it.
- Decomposition. Breaking a task into steps a model handles reliably. In FleadrAI that meant several agents with separate jobs, not one long prompt.
- Prompt and output handling. Prompts are code, and so is the code that checks what comes back. Part of DSPy's appeal, for me, is that it treats prompts as something to optimize against examples rather than hand-tune.
- Degradation paths. Deciding what happens when the model is missing, slow or wrong. A tool with no fallback fails in front of the user.
- Model choice. Picking which model to call, local or hosted, is an architecture decision with cost, privacy and latency attached. For gitmind, running offline matters because a commit hook that sends your code to a third party is a hard sell.
Side by side
This is the comparison I would hand to someone choosing between the two. It is my framing, built from the work above.
| ML engineer | AI engineer | |
|---|---|---|
| When output is wrong, you change | Data, architecture, training, serving code | Prompt, context, task decomposition, model choice, output checks |
| What you version | Datasets, code and model weights | Prompts, retrieval config and the model name or version |
| How you evaluate | Held-out test sets, accuracy and error analysis on labeled data | Example sets of inputs with graded outputs, often judged by rules, people or another model |
| Main latency cost | Your own model's inference on your own hardware | Calls to a model you don't run, plus the steps you chain together |
| Typical failure | Data in production no longer looks like training data | Plausible output that is wrong, or a prompt change that breaks a case that used to work |
Two rows deserve a comment.
Evaluation. On the factory line, "correct" had a hard definition: a fiber either was in the right position or wasn't, so accuracy meant something precise. LLM output rarely has a single right answer, so the AI engineer has to build the grading scheme before they can measure anything. In my view that is the hardest skill to pick up when moving from ML engineering, and the one most worth taking with you. The discipline I now use in clinical research, such as patient-level bootstrap intervals and paired tests, applies to an LLM eval set as much as to a vision test set.
Latency. Getting inference under 100 ms was an engineering problem I could attack directly: better images, better code, smaller inputs. With a hosted model, much of the latency is outside your control, and the main lever is making fewer and smaller calls. That changes how you design the system from the start.
Failure modes are the real difference
If I had to pick one row of that table that separates the jobs, it would be the last one.
An ML engineer's model usually fails quietly and consistently. When the inputs in production drift away from the training data, accuracy drops for a whole class of inputs, and error analysis on enough examples will show you which class. The fix is slow (collect data, retrain, validate, redeploy), but the problem is legible.
An LLM application usually fails loudly and inconsistently. An answer can be fluent, confident and wrong. The same input can produce different output on two runs. A prompt edit that fixes one case can quietly break three others you weren't watching. The fix is fast, often one line of text, and that speed is a trap: without a regression set, you are guessing.
So the habits differ. ML engineering trained me to distrust the dataset. LLM work trained me to distrust the output, and to write the check before I trust a change.
Which one should you choose
There is no better role, only a better fit. Here is the checklist I would use, on myself or on a job posting.
- Do you want to own the weights? If the interesting part is training, architecture and data, that is ML engineering. If the interesting part is shipping a product on top of a capable model, that is AI engineering.
- Does the problem have labels? Sorting fibers or recognizing an action has a ground truth you can label. Drafting a commit message or a sales email mostly doesn't. Problems with labels pull toward ML engineering.
- Where does the model run? On your own hardware, at the edge or in a factory, you need ML engineering and serving skills. Behind an API, the work moves to orchestration, cost and reliability.
- What does the posting ask you to build? "Training pipelines", "model optimization" and "MLOps" mean ML engineering. "RAG", "agents", "prompting" and "evals" mean AI engineering, whatever the title says.
- What transfers either way? Docker, CI/CD, versioning, latency work and honest evaluation. In my experience those carry over from vision models to LLM tools almost unchanged.
If you are an ML engineer moving toward AI engineering, start with point 5. The deployment and evaluation habits you already have are the ones many LLM projects lack. If you are going the other way, the gap is mostly data: learning to treat a dataset as something you build, test and version, not something you download. You can see more of the projects I mentioned in the also built section.