I was honoured to attend this year’s World Patient Safety Congress in Amsterdam, representing the Sagittal team. Of the many panels and discussions hosted by leaders of the industry all around the world, it was clear to me that there has been a rapid shift in conversation from generative AI last year towards agentic AI this year. Across the presentations and panel discussions, speakers repeatedly emphasised our familiar principles of validation, governance, accountability and of course, human-in-the-loop oversight.
On the stages, and on the panels, the message was generally reserved and cautious. Agentic AI may be capable of taking on increasingly complex pharmacovigilance tasks, but it must operate within clearly defined boundaries. Humans remain responsible for oversight, validation and scientific judgement; the technology should support the experts and not simply replace them. If you walk just a few meters to the exhibition hall however, the emphasis felt rather different.
At the vendor stands, the most impressive and emphasised selling points were how little human intervention the technology required. Agents can independently triage cases, extract information, classify literature, coordinate workflows and are trusted with increasingly sophisticated decisions. Though different vendors offered different levels of autonomy, the implied direction of innovation was that the cleverer the agent, the less it needs someone constantly looking over its shoulder.
Many of my conversations felt almost contradictory:
“Our agents can handle the entire workflow autonomously!”
“So where does the human come in?”
“We value human-in-the-loop governance as fundamental to our approach.”
“And the agent?”
“It can independently complete the entire process end-to-end by itself!”
This is not to suggest that human oversight and increasing autonomy are necessarily incompatible. An agent could operate independently for thousands of routine activities while humans are left to define its boundaries, monitor performance, review exceptions and retain ultimate responsibility. But it does feel to me that “human-in-the-loop” is becoming an increasingly ceremonial tick-box phrase.
Historically, human-in-the-loop meant a person was actively reviewing or approving machine generated outputs. The model now being presented by many vendors is closer to humans governing the system rather than participating in each decision. It may be closer to humans over the loop than inside it.
The more interesting question therefore may no longer be whether PV keeps a human in the loop, but how far away from the loop that human can be permitted to move.
By Ben Triggs
