Home / Insights
Healthcare AI Vendor Change Control: What to Require After Go-Live
Set healthcare AI vendor change control after go-live: review evidence, data changes, decision owners, monitoring, rollback and decommissioning for providers.
Healthcare AI Vendor Change Control: What to Require After Go Live A healthcare AI implementation does not become static once the contract is signed and the workflow goes live. Vendors may change a model, feature, interface, data flow, hosting arrangement, subprocessor, or support process. Local teams may also change configuration, expand users, connect new systems, or apply the output to a different decision. The practical question is not whether every change requires the same committee or evidence pack. It is whether the organisation can recognise a material change, understand its effect on the deployed workflow, and make an accountable decision before risk quietly moves downstream. A healthcare AI vendor change control programme is an operating practice. It should be proportionate to the workflow, the people affected, the data involved, and the consequence of an incorrect or unavailable output. It is not a universal federal checklist, a claim that every healthcare AI product is a regulated device, or a substitute for legal, clinical, privacy, security, procurement, or regulatory advice. Start with applicability, not a generic checklist Different authorities apply to different facts. For covered entities and business associates handling electronic protected health information (ePHI), the HIPAA Security Rule requires policies and documentation to be updated in response to environmental or operational changes that affect safeguard implementation. Required documentation must be retained for six years. [1] The rule also addresses, among other safeguards, device and media controls, including final disposition policies for ePHI and the media that contain it. [2] Those requirements are important, but they do not prescribe an AI model change notice, a particular monitoring threshold, or a universal vendor evidence packet. The organisation still needs to map the obligation to its own workflow and risk analysis. The NIST Artificial Intelligence Risk Management Framework (AI RMF) is voluntary guidance. Its GOVERN, MAP, MEASURE, and MANAGE functions provide a useful structure for lifecycle governance, including monitoring, documented response, incident handling, recovery, and decommissioning. [3] It is not a healthcare AI certification or a binding law. FDA guidance has a different and narrower scope. FDA’s final guidance on predetermined change control plans addresses certain planned modifications for AI enabled device software functions in defined marketing submission pathways. [4] It should not be presented as a universal provider side requirement for every AI feature. Similarly, ONC’s HTI 1 requirements apply within the Health IT Certification Program and do not govern every AI product or local workflow. [5] Eunoia recommendation: establish one repeatable change control path that identifies which laws, contracts, standards, and internal controls are relevant to the individual change. The path should make its assumptions visible rather than treating every update as identical. Define a material change in operational terms A vendor release note is not automatically decision ready. Begin by asking what has actually changed and whether it could alter the deployed workflow. A material change may involve the model or prompting approach, intended use, data inputs or outputs, integration, access role, hosting location, subprocessor, user interface, training content, or expected operating behaviour. For each potential change, record the affected use case, sites, roles, patient or operational populations, connected systems, and decision point. This prevents a common failure mode: a technically small release changes an operational dependency that is significant locally. For example, a change to an administrative drafting feature may require a lighter review than a change to a tool that influences clinical documentation, patient access, revenue operations, or a prioritisation workflow. The organisation should make that judgement explicitly. A vendor’s own description of a change is useful evidence, but it is not the final determination of local impact. Ask for a concise, versioned evidence packet An evidence packet does not have to be lengthy. It should be detailed enough to support a proportionate decision. A practical packet can include the following information: A version identifier, release date, and plain language description of what changed. The stated rationale, affected capability, and known compatibility dependencies. The intended use, affected users, and any change to what the output can influence. A summary of validation or release testing, including its limits and relevance to the deployed setting. Changes to inputs, outputs, data flows, retention, access controls, hosting, or subprocessors. Updated user guidance, training, known limitations, and reported failure modes. The vendor’s monitoring, incident notification, disablement, rollback, or recovery approach. NIST’s AI RMF supports lifecycle documentation, monitoring, and risk response, but it does not prescribe these exact artefacts. [3] The list is an Eunoia operating recommendation. The appropriate evidence depends on the organisation’s risk tolerance and the actual workflow. A useful evidence register records the source of each assertion. Rather than writing “validated” or “secure,” link the statement to a dated vendor artefact, identify what it demonstrates, note what it does not demonstrate locally, and assign an accountable reviewer. This keeps a marketing statement from becoming an unexamined operating assumption. Use a cross functional review gate when the change affects care or operations A review gate should include only the functions that need to make a decision. For a material healthcare AI change, that commonly means the operational owner, clinical or informatics lead where relevant, privacy, security, procurement or vendor management, and a technical integration owner. Higher impact workflows may also require quality, compliance, legal, or executive oversight. The group should answer a short set of questions: 1. Does the change alter intended use, user population, output, data flow, or the role of human review? 2. Could it affect clinical, access, scheduling, revenue, documentation, privacy, security, or downtime risk? 3. Do configuration, integration, training, policy, contract, or communication materials need to change? 4. What evidence is sufficient for a pilot, full release, deferral, restriction, or pause? 5. Who owns the decision, the implementation tasks, and the next review? This is not a claim that HIPAA mandates an AI governance committee. It is a way to bring the people who own the consequences into the decision before the release becomes routine production behaviour. Plan monitoring, override, and incident response before releasing A change control decision is incomplete if it ends at approval. The implementation record should identify what will be monitored after release, who will review it, and what would trigger intervention. Depending on the use case, that may include user feedback, correction or override patterns, workflow exceptions, data quality signals, access events, interface failures, complaints, vendor notices, and incident reports. The aim is not to set one universal performance threshold. It is to give the organisation a way to notice whether the change is behaving differently in its own setting. NIST’s AI RMF describes post deployment monitoring, documented response, appeals and override, incident response, recovery, and decommissioning as lifecycle risk management outcomes. [3] A rollback or disablement plan deserves the same specificity. Name the clinical or business decision authority, the technical owner, the safe manual fallback, relevant communication path, and how affected records or transactions will be reconciled where appropriate. Not every platform can reverse a change instantly. A credible plan identifies that limitation before an incident forces the question. For teams serving multi site healthcare organisations in New York, San Francisco, and San Jose, this discipline also makes local variation visible. A shared policy does not remove the need to understand site specific configuration, data flows, user roles, and downtime plans. Keep end of life and decommissioning in scope An AI workflow may be retired because the product changes, the organisation changes direction, performance concerns arise, a contract ends, or a connected system is replaced. Decommissioning should not be treated as an afterthought. A practical plan identifies access revocation, interface shutdown, required records retention, data return or handling obligations, training and workflow changes, archival of governance decisions, and the owner responsible for confirming that the process is complete. Where HIPAA applies, final disposition of ePHI and applicable documentation retention should be assessed against the organisation’s responsibilities and the facts of the arrangement. [1] [2] This does not create a universal requirement to delete every item of information or imply that one vendor attestation resolves all records management obligations. It establishes a visible decision path. A quarterly question set for live AI workflows At least when a material release occurs, healthcare leaders can ask their vendor and internal owners: What changed, which version is live, and which local workflow does it affect? What evidence supports the vendor’s assertion, and what remains untested in our environment? Did data use, retention, access, integration, hosting, or subcontractor arrangements change? Does the change require an updated risk, privacy, security, clinical, or contract review? What will we monitor, who can override or pause use, and what is the fallback if the workflow fails? What must be retained, transitioned, or decommissioned if this capability is withdrawn? Eunoia Consulting Co. helps healthcare organisations turn vendor updates into evidence based governance, operational review, and accountable implementation decisions. To discuss a focused Vendor Change Control Evidence Checklist for a live workflow, book a strategy conversation or explore AI governance advisory. This is advisory support for internal readiness, not a compliance determination or legal sign off. This article was produced by the Eunoia Consulting Co. Editorial Team. Eunoia Consulting Co. specialises in AI governance, healthcare operations, and data strategy for healthcare and veterinary organisations. Sources [1] 45 CFR § 164.316 — Policies and Procedures and Documentation Requirements [2] 45 CFR § 164.310 — Physical Safeguards [3] NIST AI 100 1 — Artificial Intelligence Risk Management Framework [4] FDA — Marketing Submission Recommendations for a Predetermined Change Control Plan for AI Enabled Device Software Functions [5] ONC — HTI 1 Final Rule