Evidence.
Context. Implications.

Time saved is an input to a business case. It becomes financial value only when the organisation can put that time to productive use, avoid expenditure or improve an outcome customers value.

Separate four kinds of benefit

Measure the complete workflow

Include time spent preparing inputs, checking outputs, correcting errors and handling exceptions. Compare like-for-like work and record adoption: a tool’s potential benefit and its realised benefit are different quantities.

Build the cost side with the same discipline. Account for integration, training, supervision, software and ongoing maintenance. Avoid counting the same released capacity as both a cash saving and additional revenue.

What the evidence tells us

Research by Brynjolfsson, Li and Raymond on an AI assistant in customer support found productivity improvements, with effects that varied across workers. It provides evidence about a particular deployment and outcome, rather than a universal forecast of company-wide returns.

Evidence and scope

This briefing is an analytical interpretation of existing research, not an original MABI field study. Its measurement framework should be adapted to the organisation and tested against observed results.

1. Brynjolfsson, E., Li, D. & Raymond, L. Generative AI at Work. NBER Working Paper 31161, 2023.

An AI system may help people retrieve documented knowledge. That does not establish that the organisation can operate effectively without the person who originally supplied it.

Identify the kind of dependence

Start with four questions. Is the dependency on information that can be recorded? On judgement in unusual situations? On relationships and trust? Or on the authority to make a decision? Each calls for a different intervention.

A knowledge assistant might make a procedure easier to find. It cannot, by itself, transfer a customer relationship, delegate decision rights or demonstrate that another employee can handle an unfamiliar exception.

Test continuity, not document volume

Ask another qualified person to complete representative work using the proposed system, under appropriate supervision. Record accuracy, escalation needs and the time required. Include exceptions, not just the most familiar tasks.

Then examine new dependencies: who maintains the knowledge base, checks its accuracy, manages access and restores the service if it fails? Retain a workable route for critical activities when the system is unavailable.

Evidence and scope

This is a conceptual perspective, not evidence that AI has reduced key person risk in a particular business. NIST’s AI Risk Management Framework provides a useful basis for considering roles, monitoring and ongoing risk management; it does not establish the return from a knowledge assistant.

1. NIST. Artificial Intelligence Risk Management Framework, 2023.

A useful pilot resolves an important uncertainty. Before choosing a tool, define the business outcome, the current baseline and the conditions under which the organisation would invest further.

Six questions before approval

1. What problem are we solving? Specify the workflow, affected users and commercial consequence.
2. What is the baseline? Record current time, cost, quality and failure rates where relevant.
3. What are the alternatives? Compare AI with process redesign, existing software, training or leaving the process unchanged.
4. What will implementation require? Include data access, integration, employee adoption and an accountable operating owner.
5. What will the evidence look like? Set outcome measures, an evaluation period and a meaningful comparison.
6. What would make us stop? Define unacceptable cost, quality and risk outcomes before results arrive.

Make uncertainty explicit

Separate facts from assumptions in the investment case. Use a range for uncertain benefits and costs. A small trial should help narrow that range; it should not be treated as proof of performance at a different scale.

Basis and limitations

This is a proposed MABI decision framework, not a validated scoring instrument. It draws on general evaluation principles and NIST’s approach to mapping, measuring and managing AI risk. The criteria must reflect the specific business and use case.

1. NIST. AI RMF Core: Govern, Map, Measure and Manage.

An AI system can perform well on one task and poorly on another that appears similar. Implementation should therefore begin with the work itself, rather than an assumption that every task will benefit equally.

What the research shows

A field experiment involving 758 BCG consultants examined AI-assisted performance on knowledge-work tasks. For tasks within the model’s capabilities, the researchers reported improvements in speed, quality and completion. Their “jagged technological frontier” describes the uneven boundary between tasks AI can and cannot handle well.

Implications for implementation

Break a workflow into its component tasks. Identify where outputs can be checked reliably, where errors have significant consequences and where exceptions require experienced judgement. Test representative work before expanding access.

Measure the human-AI workflow as a whole. Improvements in drafting can be offset by review and correction. Staff need clear guidance on when to use the system, when to question it and when to escalate.

Evidence and scope

The study concerns a particular model, task set and professional setting. It does not establish the result for every occupation, current model or organisation. The implementation recommendations here are MABI’s interpretation of that evidence.

1. Dell’Acqua, F. et al. Navigating the Jagged Technological Frontier. Harvard Business School research summary, 2023.