A constraint-first scorecard for choosing technology projects that improve a measurable business outcome instead of adding another disconnected pilot.
Before approving the next AI project, put one sentence at the top of the proposal:
This operation cannot improve because ______ reaches its limit first.
Do not fill the blank with a technology. “We do not have an agent” is not a business constraint. Neither is “our data platform is old” or “competitors use generative AI.” Those statements may describe the environment, but they do not identify what prevents a better result.
A useful answer sounds more like this: qualified support cases wait two days because only three specialists can resolve them; analysts spend most of the reporting cycle reconciling conflicting customer definitions; sales cannot quote complex orders quickly because approval evidence is scattered across systems; or engineers lose recovery time because incident context arrives through several unconnected channels.
Each statement names a flow, a limit, and a consequence. That is enough to begin a serious project conversation. It is not enough to justify AI.
This distinction matters in a market where teams can produce convincing prototypes quickly. A model can summarize a ticket, draft a query, search documents, or call a tool before anyone has measured whether that task controls the outcome that the organization cares about. The demo can work while the business remains unchanged.
The central discipline is simple: find the active constraint, verify it with evidence, and design the smallest intervention that can change it. Only then decide whether the intervention needs an LLM, conventional software, better data, process redesign, training, or no new technology at all.
The following table is designed for a first project review. Score each dimension from 0 to 2. A project does not win because it has the highest total alone; the scores expose what the team knows and what it is merely assuming.
| Test | 0 points | 1 point | 2 points |
|---|---|---|---|
| Outcome | Described as activity or technology | Outcome is named but weakly measured | One business outcome and baseline are clear |
| Constraint | Based on opinion or executive pressure | Plausible bottleneck with partial evidence | Flow data shows where work queues, fails, or waits |
| Causality | No reason the project changes the outcome | Connection is indirect | A testable mechanism connects intervention to outcome |
| Frequency | Rare or exceptional task | Recurring but low-volume work | Frequent work where improvement can accumulate |
| Readiness | Ownership, data, or integration is unclear | Important gaps are known | Required data, owner, users, and controls are available |
| Safety | Harm and approval boundaries are undefined | Risks are known but controls are incomplete | Failure limits, human review, and rollback are explicit |
| Learning speed | Value appears only after a long rollout | An intermediate signal is available | A small experiment can test the main assumption quickly |
| Durability | Creates another isolated tool | Partly fits an owned workflow | Strengthens a workflow or capability the business will maintain |
A candidate that scores well on excitement but poorly on outcome, constraint, and causality should not enter delivery. It needs discovery. A candidate with strong business evidence but low readiness may deserve preparatory work: clean the source data, establish ownership, simplify the workflow, or define access controls. A candidate with high readiness and little outcome value should remain a convenience improvement, not be presented as transformation.
This is different from asking teams to calculate a fictional return on investment before they learn anything. Early estimates are uncertain. The scorecard asks for something more honest: evidence that the proposed work touches an important limit and a practical way to learn whether the intervention changes it.
The most visible pain often attracts the first automation proposal. A team handles thousands of emails, so someone proposes summarization. A finance group maintains a large spreadsheet, so someone proposes an analytics assistant. Customer support has a growing queue, so someone proposes an autonomous agent.
Volume is a clue, not a diagnosis.
Imagine that support agents spend ten minutes writing each response. A drafting assistant could reduce that to four minutes. That sounds valuable. But if most cases wait twelve hours for an account permission held by another team, writing speed does not control resolution time. The tool improves one task without improving the flow.
The reverse can also happen. A small manual step may be the real constraint because every case must pass through it. A compliance specialist may review only a short section of each request, but limited reviewer capacity controls the entire queue. In that case, the valuable project might assemble evidence, classify risk, and route routine cases for faster review. It should not quietly automate the accountable decision.
Look for constraint signals in operational evidence:
Interviewing users matters, but memory and frustration can overemphasize recent incidents. Pair interviews with timestamps, queue sizes, error categories, conversion stages, service-level breaches, support escalations, or workflow traces. The goal is not perfect measurement. It is enough evidence to distinguish a governing limit from an annoying task.
Project proposals become confused when these three layers are collapsed.
The outcome is the result worth improving: faster resolution, fewer preventable defects, higher successful completion, lower loss, more reliable forecasting, or safer access to knowledge.
The constraint is the factor currently limiting that result: specialist capacity, missing information, unreliable definitions, an approval delay, high error rates, poor discoverability, or a system that cannot handle peak demand.
The intervention is what the team will change: restructure a process, improve a data product, create a deterministic service, add retrieval, deploy an assistant, or let an agent take bounded actions.
Keeping the layers separate prevents a common reasoning error. If the proposal starts with “build a RAG assistant,” the team can make every observation support that solution. If it starts with “reduce unsupported policy answers from the service desk,” retrieval is only one candidate. The team may discover that the documents conflict, ownership is missing, or users need a decision tree rather than generated prose.
This is where business strategy has to become usable by technical teams. Leaders should define the priority and boundary without choosing the architecture by slogan. Technical teams should explain which intervention fits the evidence without pretending that technical feasibility proves business value.
Microsoft’s current Cloud Adoption Framework guidance for AI follows a similar practical sequence: assess use cases against maturity and resources, rank them by strategic value and feasibility, define success criteria, and use focused proofs of concept to test feasibility and value. The useful point is not the cloud platform. It is that a proof of concept should resolve a decision, not merely demonstrate that a model can respond.
Before the team writes a backlog, prepare a one-page constraint brief. It should contain seven items:
For a document-heavy support workflow, the brief might target median resolution time while identifying specialist search and verification as the constraint. The hypothesis could be that permission-aware retrieval with citations reduces evidence-gathering time. The guardrails might include unsupported-answer rate, escalation accuracy, access violations, and user correction rate. The decision date might follow four weeks of controlled use with one support group.
Notice what the brief does not claim. It does not promise that an assistant will transform customer service. It does not equate adoption with success. It creates a falsifiable explanation of how the project may improve a measured result.
In my teaching of data and AI, the distinction I keep returning to is that knowing how a tool works is not the same as knowing where it belongs. A technically correct model attached to the wrong part of a workflow is still a weak project. Learners and working teams both benefit when the business mechanism is explained as clearly as the code.
That lesson also changes how a portfolio should be discussed. AI strategy requires choosing what not to build, but saying no becomes easier when proposals contain comparable evidence. The conversation moves away from who has the most persuasive demo and toward which constraint is important, observable, and ready for a responsible experiment.
Once the constraint brief is credible, shrink the project until it tests the causal link.
If the hypothesis is that better retrieval will reduce specialist search time, do not begin by replacing the entire service desk. Select a stable document set, a defined question category, a small user group, and a representative evaluation set. Compare time, evidence quality, and escalation behavior with the current workflow.
If the hypothesis is that structured extraction will increase invoice throughput, test the document categories that create the queue. Measure fields requiring correction, processing time, and the downstream cost of errors. A model that extracts 95 percent of fields correctly may still be unusable if the remaining errors affect payment details and are hard to detect.
If the hypothesis is that an agent can reduce incident triage delay, limit it to gathering evidence and proposing next actions. Log its tool calls, set step limits, and keep execution behind approval. The initial test should establish whether the agent assembles useful context more consistently than the existing process. Autonomous remediation is a different hypothesis with a different risk profile.
DORA’s guidance on working in small batches connects small changes with faster feedback and lower risk. For constraint-first work, the benefit is sharper learning. A broad transformation changes too many variables at once. A small batch makes it easier to see whether the proposed mechanism actually moved the limiting factor.
Small does not mean trivial. The experiment must touch real work, representative data, and meaningful failure cases. A toy dataset can verify code paths; it cannot verify business value. Scope should be narrow enough to learn quickly and real enough that the result can change a decision.
This complements pull-first project scoping: define the future workflow, then bring in only the data, tools, and controls it requires. Constraint thinking adds another filter. Of all the future improvements the team can imagine, which one changes the limit that governs performance now?
Improving the constraint can create damage elsewhere if the team measures only one number.
An AI drafting tool may increase case throughput while sending more subtle errors to reviewers. A text-to-SQL assistant may reduce analyst requests while increasing compute spend and inconsistent metric use. Automated intake may make submission faster while flooding a scarce approval team. A coding agent may increase code volume while review queues and change failure rates worsen.
Every constraint metric therefore needs balancing measures. Track speed with quality, automation with exception load, adoption with successful completion, cost reduction with reliability, and output volume with downstream rework. Human review is not proof that risk is controlled if reviewers receive more work than they can examine carefully.
This is also why the intervention should live inside an owned product or workflow. A prototype can be temporary; responsibility cannot. Someone must own the data sources, evaluation cases, permissions, model or prompt changes, user feedback, cost, and retirement decision. The note on treating internal AI systems like products goes deeper into that operating responsibility.
Do not reward a project for moving its local metric while exporting cost, delay, or risk to another team. The business constraint belongs to a system, not a dashboard tile.
A successful intervention changes the system. That means the original constraint may stop being the most important one.
Suppose retrieval cuts evidence-gathering time enough that support specialists can handle twice as many cases. The next limit may be document freshness, approval capacity, customer follow-up, or the rate of exceptions. Continuing to optimize retrieval because that team owns the technology would produce diminishing value.
The correct response is not to declare the program finished or to keep scaling the same feature. Remeasure the flow.
After each meaningful release, ask:
This review keeps a successful project from turning into a permanent solution in search of more problems. It also creates a healthier relationship between business leaders and technical teams. Leaders bring priorities and operating evidence. Teams bring implementation options and learning. Both remain willing to change the diagnosis.
An organization can complete many technology projects without improving the result that matters most. Activity is easy to count: pilots launched, licenses assigned, models tested, agents demonstrated, dashboards delivered, and features shipped. Business movement requires a stronger chain of evidence.
Start with the outcome. Observe the flow. Locate the active constraint. Separate it from the proposed technology. Write a testable hypothesis, choose guardrails, and run the smallest real experiment that can prove or weaken the explanation. If the result improves, measure where the limit moved next.
AI expands the set of possible interventions, but it does not remove the need to choose well. Sometimes the right project will use retrieval, structured outputs, or a bounded agent. Sometimes it will repair data ownership, simplify an approval path, or replace a spreadsheet with ordinary software. The technology earns its place by changing a meaningful constraint safely and measurably.
That is a more demanding standard than a successful demo. It is also a more useful one.