Technical Due Diligence Red Flags for AI Deals

Technical due diligence is usually described as a search for defects. That is only partly right.

The better version is a search for mismatch. The product claim says one thing, the architecture shows another. The revenue story assumes adoption that the logs do not support. The roadmap promises enterprise control, but the current system depends on manual work by two people. The platform is described as AI-native, but nobody can explain how prompts, models, data sources, evaluations, permissions, and incident response are owned after launch.

In AI deals, that mismatch can hide under impressive surface behavior. A model answers well in a demo. A vendor has customers. The dashboard shows usage. The deck says the platform is scalable, secure, and easy to integrate. None of that is useless evidence, but none of it is enough by itself.

If a company is buying an AI platform, acquiring a software business, expanding a vendor contract, or turning an internal pilot into a production dependency, the due diligence should answer a concrete question: can the operating facts support the business promise?

Here is the red-flag register I would put in front of the team before the first serious review.

That table is not a legal checklist. It is a way to keep the technical review tied to the business decision.

Due diligence should inspect the operating model

The most common mistake in technical due diligence is treating the product as the whole system.

In modern AI work, the product is only the visible part. Behind it sit model providers, prompts, retrieval pipelines, vector indexes, tool permissions, document parsers, data contracts, evaluation sets, logging, human review, customer support, security controls, cloud infrastructure, and vendor agreements. A buyer who inspects only the interface may miss the system that makes the interface possible.

This is why NIST’s guidance on cybersecurity supply chain risk matters beyond traditional security review. NIST SP 800-161 describes supply chain risk as a visibility problem around how acquired technology is developed, integrated, deployed, secured, made resilient, and maintained. That framing is useful for AI because many risks are not visible in the demo. They live in the supply chain, the operating process, and the assumptions around ownership.

An AI vendor may say the platform is scalable. Due diligence should ask which layer scales. The model API may scale, but the data ingestion process may not. The chat interface may scale, but the human review workflow may not. The product may support many tenants, but the customer-specific implementation work may still depend on a small services team.

In teaching AI and data work, I often see a smaller version of this pattern: people inspect the model answer before they inspect the workflow around the answer. That is understandable, because the answer is visible. But the workflow is usually where the risk hides.

The practical move is to ask for an operating model, not only an architecture diagram. Who owns each layer? What is automated? What is manual? What changes when usage doubles? What breaks if usage drops? What has to happen every week for the system to keep working?

If the answer is mostly “our team handles that,” keep asking until the work becomes visible.

Scale is not one question

People often ask whether a system scales as if scale has only one direction.

In due diligence, scale has several meanings. Can the system handle more users? Can it handle more documents, customers, requests, regions, integrations, languages, workflows, or compliance requirements? Can the support team handle more exceptions? Can the cost model survive heavier usage? Can the product still make sense if the buyer starts with a narrow rollout instead of the large deployment assumed in the sales case?

The last question is easy to miss. Some AI products look attractive only when usage is high enough to absorb platform, integration, and support costs. Others look cheap during a pilot and become expensive when long prompts, repeated tool calls, premium models, observability, and human review are added at production volume. Both are due-diligence issues.

This connects directly to the point in How to Test AI Vendor Claims Before You Buy: a claim should be tested in the buyer’s environment. For this article, the narrower point is that the test should include volume assumptions.

Ask for three versions of the business case:

• Low volume: What happens if adoption is slower than planned?
• Expected volume: What cost, staffing, and latency are assumed in the base case?
• High volume: What happens if the product succeeds faster than the team can support?

For each version, inspect cost per completed workflow, not only monthly license price or model-call cost. A completed workflow may include document ingestion, retrieval, model calls, validation, human review, audit logging, support, and rework. If the vendor or internal team cannot model that, the business case is still immature.

Scale also includes reliability under messy use. A retrieval assistant over 500 clean documents is not the same as a permission-aware assistant over 200,000 documents with duplicates, old policies, regional differences, and different content owners. A support agent that handles common tickets is not the same as one that touches refunds, account changes, or regulated decisions.

Due diligence should not reward the largest claim. It should reward the clearest boundary.

Vague versions are a serious warning

When a seller, vendor, or internal platform team cannot describe the software and model versions behind the product, the issue is not clerical. It may be operational.

AI products can depend on many moving parts: foundation models, embeddings, rerankers, orchestration frameworks, vector databases, parsers, browser automation tools, hosted connectors, open-source packages, cloud services, evaluation libraries, and security filters. Some are vendor-managed. Some are customer-managed. Some are pinned. Some update silently. Some are no longer maintained.

If the review cannot identify those dependencies, it cannot assess patching, vulnerability exposure, supportability, cost, portability, or reproducibility.

NIST’s Secure Software Development Framework says software purchasers and consumers can use its common vocabulary when communicating with suppliers during acquisition and management activities. That is the right spirit for AI deals. You do not need every executive to read dependency manifests, but the buyer needs a disciplined way to ask what the system is made of and how it is kept secure.

For AI systems, the version inventory should include more than code packages:

• model names and provider terms
• prompt and system-instruction versions
• embedding and reranking models
• vector store and schema details
• tool schemas and permissions
• evaluation datasets and scoring logic
• data sources and freshness rules
• deployment regions and runtime dependencies
• logging and trace retention policies

OWASP’s 2025 Top 10 for LLM and GenAI applications is useful here because several risks map directly to due diligence: prompt injection, sensitive information disclosure, supply chain risk, excessive agency, vector and embedding weaknesses, misinformation, and unbounded consumption. A buyer does not need to memorize every category, but the categories make one thing clear: an AI application is not a single model. It is a chain of components and permissions.

If the team cannot explain the chain, it cannot manage the risk.

Staffing evidence matters as much as architecture

Architecture diagrams are often cleaner than organizations.

A due-diligence review may show a reasonable system design while the people model tells a different story. Maybe one engineer owns ingestion, retrieval, evaluation, deployment, and incident response. Maybe the person who built the most important component has left. Maybe the customer success team is quietly doing manual cleanup that the product roadmap describes as automated. Maybe the AI evaluation process exists only because one senior person reviews examples every Friday.

That is not automatically a deal breaker. Early products often rely on concentrated knowledge. Internal platforms often begin with a small team. The red flag is pretending that this concentration is not a dependency.

The buyer should ask:

• Which components have named owners?
• Which work depends on contractors or informal help?
• Who handles customer-impacting AI failures?
• How often does support escalate issues to engineering?
• What changed after the last key employee left?
• What documentation would help a new engineer maintain the system?
• Which roadmap commitments require hiring before they are credible?

This is also where business and technical due diligence should meet. A revenue forecast may assume faster customer onboarding. The staffing evidence may show that onboarding still requires custom integration, data cleanup, prompt tuning, or manual validation by scarce specialists. That mismatch matters.

The same issue appears inside enterprises. A company may approve an internal AI assistant because the pilot worked, but the pilot worked because a data engineer manually cleaned the knowledge base, a security lead personally reviewed access, and a product manager handled user feedback one by one. If broader rollout does not fund those activities, the “successful” pilot becomes a fragile program.

Good due diligence counts the invisible labor.

Rights, data, and provenance cannot stay fuzzy

AI deals create rights questions that are easy to postpone and expensive to discover later.

The traditional software questions still matter: who owns the code, who wrote it, whether contractor agreements assigned the work properly, which open-source licenses apply, and whether third-party components can be used commercially. AI adds more questions: what data was used for training or fine-tuning, whether customer content is retained or used to improve models, whether generated outputs carry restrictions, whether embeddings contain sensitive material, and whether external providers can process the data in the required region.

The FTC and SEC have already shown that unsupported AI claims can become enforcement issues, not only marketing embarrassment. The FTC’s 2024 AI enforcement sweep included a case where a company allegedly claimed an AI legal product could substitute for professional legal expertise without adequate testing. The SEC’s 2024 actions against investment advisers focused on false and misleading statements about their use of AI. The exact legal context will differ by industry, but the lesson for technical buyers is straightforward: claims about AI capability and process need evidence.

For due diligence, evidence means provenance.

Where did the data come from? Who had rights to use it? Which customer data was excluded? Which datasets trained, tuned, evaluated, or monitored the system? Which subcontractors touched the code, data, prompts, documentation, or implementation? Which licenses restrict redistribution, commercial use, model training, or hosted-service use?

Do not accept “we own the IP” as the final answer for a serious AI product. Ask for the path of ownership. Ask for exceptions. Ask for the parts that are licensed, rented, generated, open source, customer supplied, or built by outside firms.

The goal is not to turn technical leaders into lawyers. The goal is to make sure the legal review is pointed at the real system, not an oversimplified description of it.

Resilience should be proven, not promised

Many weak systems look healthy until someone asks how they fail.

For AI platforms, resilience includes normal software concerns: backups, disaster recovery, access control, monitoring, rollback, incident response, support coverage, and tested recovery time. It also includes AI-specific concerns: model-provider outage, prompt regression, retrieval-index corruption, runaway tool calls, malformed structured outputs, stale documents, unsafe user inputs, cost spikes, and review overload.

NIST’s AI Risk Management Framework describes trustworthy AI work as something organizations incorporate into design, development, use, and evaluation. Its Generative AI Profile goes further by treating generative AI risk as a lifecycle issue across design, deployment, operation, and decommissioning. That matters for due diligence because a system is not reliable just because it launched. It has to keep being managed.

Ask for the evidence:

• When was the last restore tested?
• What happens if the model provider is unavailable?
• How are prompt, model, and retrieval changes tested before release?
• Which logs let the team reconstruct a bad answer or action?
• What is the process for disabling a tool, connector, or workflow?
• What happens when cost exceeds the expected range?
• Who communicates with customers or employees during an AI incident?

These questions often reveal the difference between a product and a prototype.

They also connect with the broader DataTweets note on the hidden work behind reliable AI projects. Reliability is not a feature name. It is a set of practices that must be funded, owned, and rehearsed.

Business claims should map to technical facts

The most important due-diligence red flag is the gap between the business claim and the technical facts.

A company says it provides 24/7 intelligent support, but the system has no clear escalation path when answers are unsupported. A vendor says its agent automates operations, but write actions require manual work outside the product. A platform says it is enterprise-ready, but audit logs do not show retrieved context, tool calls, model versions, or approval decisions. A startup says its AI improves productivity, but active usage is concentrated in a few friendly accounts and the support burden rises with adoption.

None of these examples proves bad intent. They prove the need for evidence.

A useful review should create a claim-to-evidence map:

This map helps avoid two bad outcomes. The first is naive trust: believing the claim because the product looked good. The second is vague skepticism: rejecting the claim because the reviewer is uncomfortable but cannot name the missing proof.

Evidence makes the conversation more precise.

The output should be a decision record

Due diligence should not end with scattered notes, meeting impressions, and a yes-or-no recommendation.

It should produce a decision record that can survive after the excitement, pressure, or fear of the deal fades. The record does not need to be long, but it should be explicit.

Write down:

• which product, vendor, platform, or acquisition is being assessed
• which business claims matter to the decision
• which claims were supported by evidence
• which claims remain unproven
• which risks are acceptable for a pilot but not for broad rollout
• which controls must exist before expansion
• which assumptions should be revisited after real usage
• which deal terms should reflect technical uncertainty
• who owns the system after the decision

This record is closely related to Avoid AI Platform Lock-In Before It Becomes Policy. The first decision often becomes the default operating model. If the team does not write down why it accepted a risk, when to revisit it, and what would make it stop or renegotiate, the risk becomes background noise.

A decision record also protects useful innovation. It lets a team say, “This is promising, but only for this workflow, at this volume, with these controls, and with these open questions.” That is a stronger decision than a broad yes or a defensive no.

The practical standard is evidence before commitment

Technical due diligence is not about assuming people are lying.

It is about refusing to let important decisions depend on unsupported alignment between a story and a system. The story may be honest and still incomplete. The system may be strong and still unsuitable for the buyer’s workflow. The product may be useful and still require more integration, staffing, governance, resilience, or rights review than the business case admits.

AI raises the stakes because the product surface is persuasive and the operating surface is wide. Models, data, prompts, tools, workflows, users, costs, security, and rights can all change the real risk of a deal. A buyer who inspects only the demo or the contract will miss too much.

The better habit is simple: translate every important claim into evidence.

If the claim is scale, ask for volume tests and unit economics. If the claim is security, ask for controls and incident history. If the claim is ownership, ask for the chain of rights. If the claim is automation, ask where the humans still sit. If the claim is reliability, ask for recovery tests and monitoring. If the claim is AI capability, ask what was tested, on which data, under which constraints, and how failures are handled.

Some red flags will be acceptable. Early products are imperfect. Internal platforms grow in phases. Vendors can improve. Acquisitions always include uncertainty. The point is not to eliminate all risk before acting. The point is to know which risk you are accepting.

A good AI deal can survive serious questions. A weak one depends on nobody asking them carefully enough.