LLM vendor risk is the exposure an organization takes on when a third party supplies the model, the platform or the conversational agent it puts in front of customers. The core problem is that approval is a point in time, while vendor behavior is not. Models are updated, data practices shift and prompts change, so a vendor that passed review in spring can behave differently by autumn. The organization stays accountable either way. The practical answer is a three-part routine: assess the vendor before approval, write the protections into the contract, and verify real behavior independently after launch. Lexic Compass, the AI Agent Audit platform from Lexic, exists for that last part: it assesses conversational AI agents from any vendor by interacting with them as real users would.
TL;DR
- LLM vendor risk covers data use, security, unexpected model behavior, supply-chain dependencies and availability. It changes after approval, not only before it.
- Accountability stays with the deploying organization. Vendor documentation and assurances are inputs, not proof.
- Contracts should cover data-use limits, subprocessors, incident notice, model-change notice, audit evidence and exit support.
- Tier vendors by what their AI can touch (customer data, customer conversations, regulated decisions) and reassess when versions, data practices or permissions change.
- Independent testing of the live agent closes the gap that questionnaires leave open. Lexic’s audits found that 85% of the agents assessed fail to escalate issues to a human representative appropriately.
Table of Contents
- What LLM vendor risk includes
- Why an approved vendor’s risk keeps moving
- Accountability does not transfer to the vendor
- Assess the vendor before approval
- Tier vendors by what the AI can touch
- Put the protections in the contract
- Monitor after deployment
- Prepare for breach, outage and discontinuation
- Document decisions so you can show them
- Perspective
- How Lexic supports vendor oversight
- FAQ
- Sources
What LLM vendor risk includes
The UK National Cyber Security Centre’s guidelines for secure AI system development treat the AI supply chain as part of the security perimeter. For an organization buying LLM capability, the exposure falls into five groups:
- AI-specific vulnerabilities. Attacks such as prompt injection target how models process instructions, and ordinary application security tests do not cover them.
- Supply-chain risk. The vendor you contract with often depends on a foundation model provider, a hosting provider and other subprocessors.
- Sensitive-data exposure. Prompts, files and outputs may be stored, logged or used for training.
- Unexpected behavior. The agent says something wrong, refuses to hand over to a person, or fails to disclose that it is an AI.
- Availability. Outages, deprecations and retired models interrupt customer service.
The Cloud Security Alliance makes the inheritance point directly: using an LLM through a third party can carry over the underlying model’s security vulnerabilities, privacy concerns during training or inference, and bias and fairness risks. A standard supplier security questionnaire does not surface these on its own.
Why an approved vendor’s risk keeps moving
The NCSC guidance notes that changes to data, models or prompts can change how a system behaves. It advises treating major updates as new versions and supporting evaluation through measures such as preview access and versioned APIs. It also recommends monitoring outputs for sudden or gradual behavioral change, which may signal compromise or natural drift.
In practice, four events should reopen a review:
- The vendor changes the model version or swaps the underlying provider.
- The vendor changes how it handles data, including retention or training use.
- The agent gains new permissions, such as access to account data or the ability to issue refunds.
- The business extends the agent to a new use, channel, language or market.
TechTarget’s AI vendor risk framework lists the same triggers: model versions, data practices, permissions and intended uses. It also recommends tracking incidents, control failures, vendor changes and dependencies after onboarding.
Accountability does not transfer to the vendor
Buying AI functionality does not buy the legal responsibility that comes with it. Cleary’s governance analysis states that organizations remain accountable for legal compliance when they source AI from third-party vendors. It recommends governance, tailored diligence, contract protections and ongoing oversight.
For US companies selling into Europe, this is the practical meaning of the EU AI Act. The European Commission states that Article 50 of the AI Act applies from 2 August 2026. It sets transparency obligations for providers and deployers of interactive and generative AI, including that people are told when they interact directly with an AI system. A customer-service agent built by a vendor and deployed by you puts you in the deployer role. Whether the agent actually discloses itself in live conversations is a behavior to verify, not a feature to assume.
In the United States, the NIST AI Risk Management Framework is the common voluntary reference. It is organized around four functions: Govern, Map, Measure and Manage. It gives a shared vocabulary for vendor reviews across a US and EU footprint.

Assess the vendor before approval
Good due diligence asks for specifics and separates evidence from assurance. TechTarget’s framework recommends distinguishing independently verified vendor evidence from vendor assurances. A policy PDF is an assurance. A test result produced by a third party is evidence.
The UK Information Commissioner’s Office, in its guidance on AI contracts and third parties, asks organizations to request comprehensive documentation from model developers, including the training process. It also asks for evidence on accuracy, bias, fairness, explainability and user testing. Its audience is UK organizations, but the questions apply to any market.
A workable due-diligence set covers six areas:
| Area | What to ask |
|---|---|
| Data use | Are prompts, files or outputs used to train or improve models? Does the vendor hold rights to its own training data? |
| Location and subprocessors | Where is data processed and stored? Which subprocessors are involved? |
| Retention | How long are conversations kept, and how is deletion confirmed at contract end? |
| Model changes | Which model versions run, and how much notice precedes a change? |
| Security testing | What testing has been done, by whom, and can results be shared? |
| Logs | Are logs visible to the customer for investigations? |
These questions follow the lists in Morgan Lewis’s vendor-evaluation guidance, which covers how a vendor collects, uses, stores and protects data, and in DigiTrust Asia’s third-party AI module.
The OECD Due Diligence Guidance for Responsible AI adds a risk-based method for identifying, preventing, mitigating and accounting for adverse impacts linked to AI operations and business relationships. For teams that need the underlying logic of how AI automation fits into business processes before mapping risk, this overview of how AI automation works gives a plain starting point.
Tier vendors by what the AI can touch
Not every LLM vendor needs the same depth of review. Tier by exposure, then match the assessment and monitoring to the tier.
| Tier | Typical profile | Review depth |
|---|---|---|
| High | Agent talks directly to customers, handles personal data or acts in a regulated sector | Full due diligence, contract protections, independent testing of the live agent, continuous monitoring |
| Medium | Internal tool with access to confidential data, or customer-facing content with human review | Standard due diligence, periodic reassessment |
| Low | No personal data, no customer exposure, easily replaced | Lightweight review, contract basics |
Customer-facing conversational agents sit in the top tier almost by definition. They speak for the brand, collect personal data and must hand over to people when a situation requires it.
Put the protections in the contract
The ICO guidance calls for written terms covering processing details, authorized subprocessors, technical and organizational security measures, assistance with data-protection duties, and return or deletion of personal information at contract end. The DigiTrust module extends the list to AI-specific points. A contract for an LLM vendor should address:
- Restrictions on using your data for training or improvement.
- Retention and deletion, with confirmation.
- Security commitments and incident notification, with a stated window.
- Notice of model changes, and notice of new subprocessors with a right to object.
- Audit evidence and regulatory cooperation.
- Customer-accessible logs.
- Service availability, transition support and exit.
TechTarget’s framework places service continuity, incident cooperation, audit rights, data portability and exit provisions among the resilience matters to settle before approval. Audit rights matter most for the argument in this article: they let you test the live agent, and the contract should say so explicitly.
Monitor after deployment
Monitoring has two layers. The vendor’s own dashboards report what the vendor chooses to measure, such as uptime, latency and usage. They are useful for operations. They do not independently answer whether the agent behaves as your policies and the law expect.
The second layer is external: testing the agent the way a customer meets it. The questions are concrete.
- Does the agent state that it is an AI when it should?
- Does it escalate to a human when a customer asks, or when the case demands it?
- Does it resist attempts to extract data or override its instructions?
- Does it give accurate answers on policy, pricing and regulated topics?
- Does it behave the same way after a model update?
These checks connect directly to the NCSC’s advice to watch for sudden or gradual change. Teams designing the product side of this monitoring, including how AI behaviors surface to users, can draw on these LLM UX patterns for SaaS.
Lexic’s own audits show why the second layer matters: 85% of the agents assessed fail to escalate issues to a human representative appropriately. That is a behavior visible only in conversation, and a configuration review would not necessarily reveal it.
Prepare for breach, outage and discontinuation
Plan for the vendor failing before it does. DigiTrust’s module recommends incident-notification and cooperation duties, retained customer-accessible logs, and contracted notice for model changes along with transition support and exit. The NCSC recommends failover to alternatives for mission-critical systems when security criteria are not met, and documenting and protecting AI-related assets such as prompts, data, models, logs and assessments.
Four decisions are worth making in advance:
- Fallback path. What happens to customer conversations if the agent is switched off: queue to humans, a simpler flow or a notice?
- Evidence retention. Which logs and test results you hold independently of the vendor.
- Decision owner. Who can suspend the agent, and on what trigger.
- Exit route. How prompts, configurations and data come back to you.
Document decisions so you can show them
The ICO advises documenting the controller and processor relationship and the processing activities, evidencing the assessment in records such as emails, minutes or system-design documents, and reviewing written contracts periodically. A defensible file for each LLM vendor holds the tier and its reasoning, the diligence responses, the contract terms, the test results and the dates of each reassessment.
Evidence quality matters when a regulator or a customer asks. Lexic’s positioning is that findings should be traceable: it uses third-party cryptographic hashing to verify evidence integrity and support traceability.
Perspective
Vendor risk reviews were built for software that behaves the same way each time it runs. Conversational AI does not. The same agent can answer differently after a model update, and a security questionnaire completed at signing says little about what a customer hears in month nine.
We see the gap between what a vendor reports and what an agent does as the central issue in LLM vendor risk. That is why Lexic Compass interacts with agents the way real users do and assesses Customer Experience, Security, Technical Reliability and Regulatory Compliance together. The output is evidence an organization can use to govern its agents in production and to hold vendors accountable.
Independence is the point. A vendor measuring its own agent is useful for operations. An assessment from a party with no stake in the result is what a board, a regulator or a customer can rely on.
Sergio Llorens
How Lexic supports vendor oversight
Lexic Compass audits conversational AI agents from any vendor, covering Customer Experience, Security, Technical Reliability and Regulatory Compliance, including blind spots that traditional monitoring tools may miss. Engagements are available as a Flash Preview, an Audit Sprint or Enterprise Continuous Trust. See the audit options or the Lexic Compass product page.

For teams that also want to learn from the conversations themselves, Lexic Pulse analyzes customer interactions and generates sentiment, churn and opportunity insights.
FAQ
What is LLM vendor risk?
It is the set of exposures that come from relying on an outside provider for a language model or an AI agent. These include data exposure, security vulnerabilities, unexpected behavior, supply-chain dependencies and outages. The deploying organization remains accountable for the outcome.
How often should an LLM vendor be reassessed?
Reassess on a regular schedule and whenever something material changes. Triggers include a new model version, a change in data practices, new permissions for the agent or a new use case. The ICO also recommends periodic review of outsourced services.
Does the EU AI Act apply to a US company using an LLM vendor?
It can, when the AI system is used in the EU. The Commission states that Article 50 transparency obligations apply from 2 August 2026 to providers and deployers of certain AI systems, including interactive ones. Confirm your role and scope with qualified legal counsel.
What is the difference between vendor assurances and evidence?
Assurances are the vendor’s statements: policies, questionnaires and certifications it supplies. Evidence is independently verified, such as results from testing the live agent. TechTarget’s framework recommends keeping the two apart when evaluating an AI supplier.
Sources
- Guidelines for secure AI system development (NCSC)
- Revamping Third Party Vendor Assessments for the Age of Large Language Models (Cloud Security Alliance)
- AI vendor risk assessment: A framework for CISOs (TechTarget)
- Managing AI Risk: Legal and Governance Imperatives for the Board (Cleary)
- Guidelines on transparency obligations under the AI Act (European Commission)
- AI Risk Management Framework (NIST)
- Contracts and third parties (ICO)
- Key Considerations for Evaluating an AI Vendor (Morgan Lewis)
- Third-party AI: due diligence, contracts and shared responsibility (DigiTrust Asia)
- OECD Due Diligence Guidance for Responsible AI

