Jeff Sovis and Matt push on a tension anyone who has ever run a validation project has felt: the documents say the system is fit for use, but the actual issues never showed up in the documents. This episode is about that gap. Jeff frames it as operational drift — the slow separation between what the paperwork claims and what the organization is actually doing. The conversation moves through GAMP Category 3 SaaS validation and the checkbox trap, into the on-prem → SaaS → AI paradigm collision, and lands on what "human in the loop" actually has to mean when a third, unknown entity is making decisions inside the process. Practical throughout — not a theory episode.
Jeff Sovis is founder and principal consultant of Mindful FDA Compliance LLC, which he started in 2015. He began his career in the lab at Genzyme Biosurgery manufacturing cell-therapy products for burn victims, then moved into life sciences quality and validation consulting — roughly 12 years of it, specializing in MasterControl, Veeva Vault, and ETQ eQMS implementations, CSV, SOP harmonization, and audit remediation.
Six or seven years ago Jeff was validation lead on a single-instance MasterControl harmonization at one of the largest companies in the world. He had visibility into both the validation work and the executive-level operations meetings — and the real issues almost never surfaced in the validation document. They showed up in operational communication, in decisions made by stakeholders operating in silos, in documents that got split or lost. That gap between the paperwork and the operation is what he calls operational drift.
For GAMP 5 Category 3 systems — where most SaaS lives — Jeff argues validation has drifted into a pass-through exercise instead of a tool for actually finding the seams in an implementation. His framing: risk correlates directly to the business process, so validation should be used to work out ways into and out of every foreseeable situation. Post-go-live matters just as much as day one because features keep landing.
Matt traces how the industry dragged on-premise validation habits into SaaS and made a mess, then dragged that mess into AI. Jeff recalls the 2017 V-model era — test scripts due before anyone had access to the system, no sandbox — as an exercise that turned validation into paperwork theater. Today's ability to spin up sandbox environments changes the ceiling if the operating model catches up.
Jeff's line: automation you can prove works correctly. AI is different — it introduces a third unknown entity making final decisions. His starting pattern for organizations is AI as an outside system, not connected to production, used for education and trust-building. The human moving data across the boundary eliminates most of the risk. Matt pushes on what happens when we let agents operate where humans did — the scale of a mistake changes fast when the click-throughs disappear.
Both agree the "validate on install and walk away" model is dead for modern systems. Matt frames it as always-on monitoring; Jeff notes he's hearing "continuous validation" more and more. This aligns with the direction of the FDA's Computer Software Assurance guidance, finalized in September 2025.
Matt goes off on the lifecycle math: a biotech product spans years — sometimes a decade — from lab to marketed product. Assuming a single human holds the information across that arc is fragile. Jeff picks up the thread with the "last mile" problem: workarounds harden into someone's job. He reframes it as misallocation rather than overallocation — that resource could be improving the system instead of patching around it.
Biotech organizations weren't built for this. Ten to fifteen years ago the enterprise ran on file cabinets. The transition to modern software requires people who understand a little of everything — quality, IT, business, GxP — rather than departments operating in silos. Jeff has seen audit findings tied to putting non-GxP experienced people into GxP roles as a fix.
Jeff's concrete offering: a two-to-three-week engagement, a roughly 20-page report focused on systems, testing, and procedures, with actionable feedback. Positioned as a low-investment way to align philosophies before something breaks in an audit.
Jeff: "It was almost never in the validation document."
Jeff: "Validation has kind of become like a checkbox exercise."
Jeff: "With AI there's a third unknown entity making final decisions."
Jeff: "I still think we're basically at square one."
Jeff: "It's not overallocation… it's misallocation."
Matt: "The longer you're not operating at an optimal level, the longer you're creating risk."
Matt: "We have to hold it to a higher standard."
Quality and validation leaders, regulatory operations professionals, life sciences IT, GxP software vendors and implementers, and anyone stewarding an eQMS through go-live and beyond. Especially useful if your organization is caught between traditional CSV instincts and the reality of SaaS plus AI.
Guest
Guidance and Standards
Concepts Referenced