In discussions about AI ethics, "human in the loop" is a common refrain that implies human oversight of AI-driven processes and systems. The phrase sounds straightforward: by ensuring humans can observe and have authority over an AI process, we can avoid the worst inaccuracies of AI while still benefitting from its efficiency gains.
However, AI presents unique strengths and challenges that distinguish it from previous technological improvements. AI can be deployed at a scale that can very easily overwhelm human oversight, and due to its "black box" nature, has opaque reasoning that makes its behavior difficult for humans to troubleshoot. Because of these challenges and others, there is real nuance in the phrase "human in the loop" that is worth exploring further.
The European Commission's Ethics Guidelines for Trustworthy AI (2019) offers a useful structure for this discussion. It distinguishes three modalities of human oversight: Human-In-The-Loop, Human-On-The-Loop, and Human-In-Command. These are sometimes misunderstood as a hierarchy of human involvement; in reality, they reflect distinct oversight functions operating at different levels of the AI lifecycle. Applied to healthcare, the appropriateness of each model is contingent on the nature and severity of risk.
HITLHuman-In-The-Loop
HITL is required in patient-specific contexts — diagnostic support, treatment recommendations, or triage decisions — where clinical judgment, contextual interpretation, and professional accountability are indispensable. Its effectiveness depends immensely on the quality of human engagement.
Ethical considerations cannot be satisfied merely by adding a person somewhere near the output. A clinician who is overloaded, under-informed, or overly deferential to the system may add little real protection, even where formal approval is required. The adequacy of oversight therefore depends on whether that human has the competence, authority, context, and time needed to exercise genuine oversight — otherwise HITL risks degenerating into rubber-stamping, actually undermining the control function it is meant to provide.
HOTLHuman-On-The-Loop
By contrast, HOTL is better suited to large-scale or system-level applications — population health monitoring, workflow optimization, or anomaly detection. Its principal advantage is scalability and continuous system evaluation.
Yet it introduces distinct risks, such as automation bias or diminished vigilance, as human operators transition from active decision-makers to passive supervisors. Effective HOTL therefore depends on robust system design: transparency, auditability, calibrated alert mechanisms, and clear intervention thresholds.
HICHuman-In-Command
HIC addresses a different dimension of ethical control, where responsibility remains anchored at the organisational level — with the governance authority determining whether, where, and under what constraints a system should be used at all. It is particularly important in healthcare, where significant harms may arise from flawed deployment decisions (such as inadequate validation across populations or inappropriate clinical integration) rather than from wrong outputs.
HIC works in tandem with the previous paradigms to ensure processes are continuously evaluated for efficacy and safety. Ultimately, it preserves the capacity to constrain, suspend, or terminate the use of an unsafe system.
The real question is not where the human sits in relation to the system in the abstract, but whether human judgment is meaningfully exercised at the points in the lifecycle where it matters most.
Creating AI-enhanced systems with an appropriate "human in the loop" is destined to fail unless the appropriate model of oversight is applied. Each model carries its own failure mode — and each, applied at the wrong layer, can give the illusion of control while providing none.
See also the World Health Organization's Ethics and Governance of Artificial Intelligence for Health (2021).
Oversight happens at multiple layers
missing
Human oversight operates at multiple layers — each bringing a different form of judgment. Together, they ensure AI systems are not only effective, but safe and accountable in practice.
AI in Bio Explained
This is the first article in AI in Bio Explained, a BioLiterate series that unpacks the concepts, frameworks, and vocabulary shaping AI in the life sciences — translating them into plain, practical terms for everyone to understand. Each piece stands on its own, taking one idea at a time and grounding it in how AI actually shows up in biomedical practice.