What human in the loop actually means
"We've got a human in the loop."
It's the safety claim that closes every AI pitch deck, calms every boardroom concern, and ticks every governance checkbox. But for most organisations, "human in the loop" means one thing. Someone reviews the output before it ships.
What many don’t understand, is that true human-in-the-loop oversight means understanding the specific ways AI systems distort information, trigger bias, and erode judgment, often without anyone noticing. Over the past two years, researchers across fields from computational pathology to strategic management have been mapping exactly where these failures occur. What they've found is a chain reaction: each flaw feeds the next, and together they reinforce that the very safeguard we rely on to give us comfort, is actually much more than the every day user understands.
It starts with what the AI actually sees when you hand it your data.
1. It Starts With What the AI Misses
We hand AI long documents, contracts, reports, data, and trust it to process the whole thing. It doesn't.
A study by Liu et al. published in the Transactions of the Association for Computational Linguistics (2024) found that large language models suffer from a "lost-in-the-middle" phenomenon. They overweight information at the beginning and end of a document while effectively ignoring what's buried in the centre. The model processes text sequentially, referencing earlier content exponentially more than anything in the middle. The longer the document, the worse the blind spot gets.
A lawyer uses AI to review a 30-page affidavit, and the critical clause on page 16 vanishes into noise. A leaders asks AI to summarise their organisations entire 3 years of customer data, and the true customer challenges stay buried mid-timeline and never surfaces. We deploy AI specifically to avoid human skimming, and it replicates that exact failure through mathematics instead of fatigue.
A genuine human-in-the-loop process accounts for this. It doesn't just ask "did the AI get it right?" It asks "what part of my input did the AI probably underweight?"
But incomplete processing is only the first problem. Even the information the AI does surface arrives with something else attached.
2. The Data It Does Surface Carries Bias, and Passes It to You
You might assume that a human reviewer acts as a filter catching the errors the AI introduces. A University of Washington study by Wilson et al., presented at the 2025 AAAI/ACM Conference on AI, Ethics, and Society, found the opposite: the filter doesn't just leak, it flips.
When the researchers simulated racial biases in AI recommendations for job candidates, human participants didn't overlook the bias. They adopted it. In cases of severe bias, human choices mirrored the AI's recommendations 90% of the time. Most troubling, participants who were otherwise neutral in their decision-making became biased specifically when prompted by a moderately biased AI.
This is what makes the "review step" dangerous rather than protective. The AI doesn't just carry bias, it acts as a social agent that normalises it. We mistake its algorithmic prejudice for our own professional intuition. If your AI system carries even a moderate bias, and most do, because training data does, then a human reviewer at the end of the process doesn't neutralise the problem. It potentially compounds it.
The logical response is to put your most experienced people in the loop. Surely domain expertise is the antidote? The next study suggests it's the opposite.
3. Expertise Doesn't Shield You. It Makes You More Susceptible
A study by Rosbach, Ganz, Ammeling, Riener, and Aubreville, published by Springer (2025) on automation bias in computational pathology, tested what happens when highly trained specialists work alongside AI and the results challenge a core assumption of every AI governance framework.
The researchers found a "negative consultation" rate of 7%. Cases where expert pathologists overturned their own correct assessments to follow an inaccurate AI suggestion. These weren't novices deferring to technology they didn't understand. These were specialists abandoning judgments they'd already made correctly, because the AI's prediction created a mental anchor too strong to resist.
This is the anchoring effect applied to professional expertise. Once the AI provides a number, a classification, or a recommendation, it becomes the reference point against which everything else is measured. The expert stops evaluating independently and starts evaluating relative to the anchor.
The study's most counter-intuitive finding was that time pressure didn't increase the frequency of mistakes, but it dramatically intensified their severity. Under cognitive load, experts didn't just defer to the AI more often, they deferred harder, deviating further from the ground truth than they ever would have working alone.
So the AI misses information, transfers bias to its reviewers, and overrides expert judgment through anchoring. That raises an obvious question: why do intelligent, experienced professionals find it so difficult to push back? The answer lies in how these models are built.
4. The AI Is Engineered to Feel Trustworthy. It’s a Feature, Not the Bug
A position paper on AI sycophancy and epistemic reliability by Turner and Eisikovits, published in AI and Ethics (Springer, 2026), explains the mechanism. Large language models are optimised through Reinforcement Learning from Human Feedback (RLHF), a training process that rewards fluency, confident tone, and agreement with the user. The technical term is sycophancy: the model learns that agreeable, polished outputs score higher than cautious, qualified ones.
The result is an AI that prioritises sounding authoritative over being reliable. It mirrors your framing. It validates your assumptions. And because it uses natural language, we instinctively attribute to it a level of understanding it doesn't possess.
This is what makes the processing errors in sections one through three so persistent. The AI doesn't just make mistakes, it presents those mistakes in the most convincing possible packaging. The bias it transfers feels like your own insight. The anchor it sets feels like a reasonable starting point. The information it missed never comes up, because everything it did produce sounds so thoroughly considered.
And this engineered trust doesn't stop at individual decisions. It shapes strategy too.
5. The Trust Problem Scales to Strategy
Even when AI avoids the processing and bias failures above, its strategic advice is systematically skewed toward popular opinion, and research shows this is nearly impossible to prompt your way out of.
Research published in the Harvard Business Review (March 2026) testing leading LLMs across core business tensions found striking bias. Models overwhelmingly favour differentiation over cost leadership, long-term vision over short-term survival, and innovation over operational efficiency, regardless of what the business actually needs. They default to what sounds good in a TED Talk, not what wins a price war in a saturated market.
In a follow-up study of 15,000 trials, expert prompting techniques such as persona shifts, pros-and-cons framing, rich organisational context shifted these core biases by less than 2%. Much of the apparent improvements with prompting technique was due to changing the order of options, not evidence of deeper reasoning.
When models aren't forced into a clear recommendation, they fall into what researchers call the hybrid trap, where LLMs suggest an organisation pursue both differentiation and cost leadership simultaneously. It sounds balanced and convincing. In practice, it's strategic paralysis for an organisation. Decades of management research shows that trying to be everything to everyone leaves an organisation stuck in the middle, yet AI will convince you otherwise.
This is not the full extent of AI errors, but clear examples of a potential chain of errors that could occur where the outcome could be significant if taken at face value. The AI misses your data, biases what it does surface, anchors your experts, wraps it all in engineered confidence, and then steers your strategy toward whatever the internet already agrees on. Every link in that chain looks like helpful AI. None of it is reliable.
What "In the Loop" Should Actually Mean
If human-in-the-loop is going to mean anything beyond a compliance checkbox, it needs to be redesigned around what the research actually tells us and how your people work with AI.
That means building cognitive forcing steps for tasks that require more oversight. What this means, is the creation of deliberate friction points that interrupt the automatic trust cycle at every stage of the chain. I’m not talking about tasks when you are asking for your email to sound ‘warmer’. It’s the bigger tasks that carry more weight in our processes that require independent assessments before AI outputs are revealed, so anchoring can't take hold.
Leaders need to understand and train employees on the specific biases tools carry, so the mirror effect gets caught instead of absorbed. A way to do this is to rotate who reviews what, so anchoring effects don't compound over time.
As users we need to treat "the AI agrees with me" as a yellow flag, not a green one. And when the AI suggests a strategy, ask what it would never have the courage to recommend.
A policy talking about responsible AI use is not enough. Teaching your employees about effective prompting techniques is not enough. Every employee needs to understand the potential failures of this technology and why they need to intercept outputs and really challenge what they have been handed.
Sources
Position Bias: Liu, N.F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). "Lost in the Middle: How Language Models Use Long Contexts." Transactions of the Association for Computational Linguistics, 12. MIT Press. Read the study
Bias Transfer: Wilson, K. et al. (2025). Presented at the AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, Madrid. University of Washington. Read the study
Automation Bias in Expertise: Rosbach, Ganz, Ammeling, Riener, & Aubreville (2025). "Automation Bias in AI-Assisted Medical Decision-Making under Time Pressure in Computational Pathology." Springer. Read the study
Sycophancy and Epistemic Reliability: Turner, C. & Eisikovits, N. (2026). "Programmed to Please: The Moral and Epistemic Harms of AI Sycophancy." AI and Ethics, Springer. Read the study
Strategic Bias: "Researchers Asked LLMs for Strategic Advice. They Got 'Trendslop' in Return." (March 2026). Harvard Business Review. Read the study
