AI DeliveryPreview Learning Center

AI engineering

The shapes an AI system can take

Before the lifecycle there is a drawing: how many model calls, in what order, and who decides the next step. Seven shapes cover most systems. Turn each one, play a request through it, and see where an attack gets in. Further down: spec-driven development with a spec builder, and the forward deployed engineer with a plan you can build for the customer in front of you.

Architecture

Seven shapes, from one call to an agent

Drag the picture to turn it.

  • Model
  • Tool or data
  • Check
  • Goes in, comes out

What it is

When to use it

What to secure

The seven shapes, their names and when to use each are from Anthropic: Building effective agents (Erik S. and Barry Zhang, 2024-12-19), read 2026-11-11. It separates workflows, where models and tools follow code paths written ahead, from agents, where the model directs its own steps, and its advice is to start simple and add a multi-step system only when the simple one falls short. "What to secure", the step-by-step walk through each shape and the marks for where attacks enter are this center's own.

Deployment

Where the model runs

Hosted API

You call the model maker's service.

Fastest to start and always the newest model. Your prompts and data leave your network, so the contract, the retention setting and the key are the controls.

Your cloud's model service

The same models, served inside your cloud account.

Identity, network rules and logs are the ones you already run. The model list and the release dates are the cloud's, not the maker's.

Self-hosted open weights

You run the model on machines you control.

Nothing leaves, and nobody patches it but you: the model file, the serving software and the GPU hosts are all yours to verify and update.

On the device

A small model on the laptop or phone.

Works offline and keeps data local. The device is now the security boundary, and the model can be read by whoever holds it.

Spec-driven development

Write the spec first, then let the agent build

A coding agent builds what it is told, quickly, including what you did not mean. Spec-driven development puts a written spec between the idea and the code: what the system must do, in sentences that can be tested, agreed by a person before any code exists. This is the method as its makers and one careful reviewer describe it, its limits, and a builder that writes a first spec with you.

What it is

Writing a spec before writing code with AI, so that the spec becomes the source of truth for the person and for the AI [5]. GitHub describes the spec as a contract for how the code should behave, which tools and agents use to generate, test and validate it [4].

Four phases

Specify, Plan, Tasks, Implement. Each phase produces a Markdown file that feeds the next [1]: spec.md, plan.md and tasks.md in Spec Kit [2]. Specify is about users and outcomes, not the technology; Plan adds the stack, the architecture and the constraints; Tasks cuts the work into small pieces that can each be tested [4].

Your job

To steer. The agent does most of the writing, and at each phase the person reads what it produced and corrects it before moving on [4]. In GitHub's words: "The AI generates the artifacts; you ensure they're right" [4].

Where it fits

GitHub names three cases: a new project, a new feature in an existing system, which it calls the case where the method is most powerful, and modernizing an old system [4]. For existing codebases it adds that more work on context may be needed [4].

Three levels

Spec-first: write a good spec and use it for the task in hand. Spec-anchored: keep the spec afterwards and maintain the feature through it. Spec-as-source: people edit only the spec and never the code. Every approach the author found is spec-first; not all aim for the other two [5].

The tools

Spec Kit is GitHub's open-source toolkit and lists 38 integrations with coding agents [1]; its project rules live in a file it calls the constitution [3]. Kiro writes three files per spec: requirements.md (or bugfix.md), design.md and tasks.md [6]. AGENTS.md is a different thing: a README for agents with the build commands, the style and the security notes, used by over 60,000 open-source projects by its own count [7]. It is standing context, where a spec describes one piece of behavior [5].

Four phases, and a person between each

The agent drafts every file. Nothing moves to the next phase until a person has read the last one and agreed [4].

1. SpecifyWhat and why,for whomYou check: is thiswhat we want?2. PlanStack, design,constraintsYou check: does itfit our rules?3. TasksSmall pieces,each one testableYou check: can eachone be tested?4. ImplementThe agent builds,task by taskYou check: testspass, diff readTHE AGENT WRITES EACH FILEA PERSON CHECKS IT BEFORE THE NEXT PHASE
The four phases are from Spec Kit [1] and the GitHub article [4]; the wording of the four checks is this center's own.

Where it falls short

One reviewer tried three of the tools and wrote down what went wrong [5]. Read these before adopting the method for everything.

The word is loose

There is no agreed definition of a spec, and some people use the word for a long prompt [5].

Too heavy for small work

A small bug fix came back from one tool as 4 user stories and 16 acceptance criteria, and another tool produced many repetitive Markdown files for a small feature [5].

A false sense of control

The agent did not always follow the spec: in one run it ignored its own research notes and wrote duplicate classes [5].

Harder in old code

For two of the three tools, bringing them into an existing codebase looked harder than starting fresh [5].

Small steps still win

The reviewer prefers small iterative steps and doubts that a lot of design up front is a good idea. Her notes are from September 2025, and the tools were changing quickly [5].

Sentences that can be tested

A requirement is only useful if someone can check it. EARS, the Easy Approach to Requirements Syntax, is a set of five sentence patterns made by Alistair Mavin and colleagues at Rolls-Royce and first published in 2009 [8]. Each one names the system and says what it shall do.

Use it forThe pattern
AlwaysThe <system> shall <response>.
When something happensWhen <trigger>, the <system> shall <response>.
While a state holdsWhile <state>, the <system> shall <response>.
If something goes wrongIf <trigger>, then the <system> shall <response>.
Where a feature is includedWhere <feature>, the <system> shall <response>.

Build a spec

Fill in what you know. The spec is written on the right as you type, in the EARS patterns above, and the check beside it tells you what a reviewer would send back: a missing owner, no failure case, a word nobody can test. Copy it or save it as spec.md for your agent. Nothing you type leaves your browser. The security rules, the check and the list of vague words are this center's own.

Acceptance criteria

One behavior per line. Pick the pattern, fill the blanks, and the sentence writes itself.

Security rules to write in

Sources for this section, read 2026-11-15; a number in brackets points to the one a statement was read on. For where the spec sits in the whole lifecycle, see the Plan stage.

  1. GitHub Spec Kit documentation (home page, last updated 2026-08-21)
  2. GitHub Spec Kit documentation: What is Spec-Driven Development?
  3. GitHub Spec Kit repository: spec-driven.md
  4. The GitHub Blog: Spec-driven development with AI: get started with a new open source toolkit (Den Delimarsky, 2025-09-02)
  5. martinfowler.com: Understanding Spec-Driven-Development: Kiro, spec-kit, and Tessl (Birgitta Böckeler, Thoughtworks, 2025-10-15)
  6. Kiro documentation: Specs (page updated 2026-11-12)
  7. AGENTS.md
  8. Alistair Mavin: EARS, the Easy Approach to Requirements Syntax

Forward deployment

The engineer who ships inside the customer

The forward-deployed AI engineer is the person who makes an AI product work in one real organization. This is the role in detail: what it is, the roles next to it, how an engagement runs and what gets built, each with a diagram, then a step-by-step engagement plan and eight customer scenarios, from old systems with no interface to a pilot that stalled.

What it is

A forward-deployed engineer works inside the customer's team instead of behind a ticket queue: on the customer's data, in the customer's systems, building the thing that makes the product work there, and carrying what was learned back to the product team.

Why AI brought it back

An AI system is finished by the place it runs in. The prompts, the retrieval sources, the tools it may call and the cases it is graded on all come from one organization's work, so someone has to sit where that work happens.

What the engineer builds

The connectors to the customer's systems, the evaluation set made from the customer's real cases, the guardrails that match the customer's rules, and the runbook for the day the agent does something nobody asked for.

What to settle on day one

Whose credentials the agent uses and how they are revoked. Which customer data may reach which model. Who approves a change to a prompt or a tool. Where the logs live and who reads them. What leaves with the engineer when the engagement ends.

The role, as the sources describe it

The role in one line

A software engineer who alternates between being embedded with a customer's team and working with the core product team, doing a mix of software, sales and platform engineering. The aim is to make the customer succeed by building on top of the company's own product [1].

Where it came from

Palantir, founded in 2003, named the role Delta, and until about 2016 it had more forward-deployed engineers than ordinary software engineers. The article says no company employs more of them [1].

How fast it is spreading

Job postings for the title went from fewer than 10 in September 2020 to almost 50 in September 2025, in figures the paper takes from a16z and Bloomberry [2]. OpenAI's group started with two engineers and had more than 10 by August 2025, in eight cities on three continents; Ramp had about 15, working in pods [1].

Who gets hired

A solid software engineering background plus some real-world experience. Palantir's posting asks for as little as a year after college; Ramp prefers five or more years for senior roles [1]. The paper describes engineers who ship rough but right code quickly under constraints, are comfortable throwing away the first version, and can deploy in secure environments [2].

Time on site

Around 25 percent of the time at the customer in a Palantir posting, and up to 50 percent in a Commure posting [1].

What the company gets back

The customer work doubles as product discovery from inside the enterprise: the first bespoke version proves the outcome, and the product team then turns the pattern into something maintainable for the next customers. The paper warns that the early phases can be unprofitable per account [2].

Where the role sits

Two teams work at the customer and one at home. What the field team learns becomes the product; what the product team builds comes back as something the next customer does not need built by hand.

The customerTheir dataTheir systemsTheir people and rulesForward deployedAnalyst or account lead:finds the use caseEngineer: builds thefirst working versionThe product teamTurns the patterninto the platformfor the next customersreal problemworking systemwhat it learnedthe productONE CUSTOMER, MANY CAPABILITIESONE CAPABILITY, MANY CUSTOMERSbuilds the gravel roadpaves it
The loop between the customer, the forward-deployed team and the product team. The gravel road and paving picture is from the paper [2]; the two mottos are how Palantir tells its field and product engineers apart, as quoted in the article [1].

The roles around it

Titles differ from company to company. What separates them is where the person sits, whether they write code on the customer's systems, and whether what they learn changes the product.

RoleWhere it sitsWhat it doesHow it differs
Forward-deployed engineerWith one customerBuilds the first working version on the customer's data and systems, then brings the pattern home. Palantir's name for it is Delta.One customer, many capabilities [1]. Works with the customer and also contributes to the product [1].
Embedded analyst or account leadWith one customerThe domain expert who finds the first high-value use case, keeps the stakeholders aligned and shapes the demo. The paper calls this team Echo.Owns the problem and the people, not the code [2].
Core product engineerAt headquartersTakes the pattern the field team proved and builds it into the platform as a general, maintainable capability.One capability, many customers [1]. The paper's picture: the field builds the gravel road, headquarters paves it [2].
Solutions architectBetween sales and the customerDesigns how the product fits the customer's systems and advises the customer's own engineers.At OpenAI, as the article describes it, a separate role that rarely writes code on the customer's infrastructure; the forward-deployed engineer is much more hands-on [1].
Sales engineer, technical delivery engineer, agent engineerVaries by companyTitles the article lists as coming close to the forward-deployed engineer.The difference it draws: the forward-deployed engineer also contributes to the product the company sells [1].
ConsultantOutside bothStudies the problem and recommends what to do.Makes one-off recommendations, where a forward-deployed engineer generally works with the customer long term [1].

An engagement, phase by phase

The three phases are how the article describes OpenAI's forward-deployed team working [1]. Under each one is this center's own note on the security work that belongs there.

1. Early scopingA couple of dayson siteCredentials, dataand who approves2. ValidationAgree criteria, buildevals, final reportReal cases plusattacks in the evals3. DeliveryOn site part of theweek, build, demoApprovals, tracing,runbook, exitTHE ENGAGEMENTWHAT TO SECURE IN EACH PHASE
Three phases of an engagement, and what to secure in each. The phases are from the article [1]; the lower row is this center's own.

1. Early scoping

A couple of days on site to understand the problem and the place it lives in [1].

What to secure

Settle whose credentials the agent will use, which data may reach which model, and who can say yes to a change.

2. Validation

Agree the criteria the system will be judged by, build the evals, move fast to find any brick wall, and present a final report [1].

What to secure

Build the eval set from the customer's real cases and add the attacks: injected documents, requests for another user's data, tools called out of turn.

3. Delivery

On site a few days a week, with the building done between visits, ending in a demo [1].

What to secure

Turn on approvals, budgets and tracing before the demo, write the runbook, and plan the exit: keys revoked, access removed, an owner named.

What the AI engineer builds on site

The model arrives finished. Everything around it is built at the customer, and each part is also a place an attack can enter, which is why the security work is the engineer's own work and not a review at the end.

The customer's agentmodel, loop and toolsConnectorsTools and MCP serverswith narrow identitiesContextRetrieval, prompts,instruction filesEvalsReal cases, graded,before every changeGuardrailsChecks, sandbox,budgets, approvalOperationsTracing, alerts,runbook, owner
Five things built around the customer's agent. This breakdown is this center's own; each part is explained in the vocabulary below.

Connectors

The tools and MCP servers that reach the customer's systems, each with its own narrow identity.

Context

What the agent reads: the retrieval sources, the prompts and the instruction files, kept in version control.

Evals

The customer's real cases as a graded set, run before every change to a prompt, a tool or a model.

Guardrails

Input and output checks, a sandbox, budgets, and a person's approval before anything that cannot be undone.

Operations

Tracing of every model and tool call, alerts, the runbook, and a hand-over to a named owner.

The engagement plan, step by step

The three phases above say how the time is spent. This is the work inside them, in order, for any company the engineer walks into: what to do, which framework or method to lean on, what to look out for and what to leave each step with. Walk it one stop at a time. The plan is this center's own.

Stop 1 of 9Prepare

What you do

Before day one: read what the company publishes, learn the rules of its industry, and agree the scope, the access you will get and who your sponsor is. A stakeholder map and a RACI chart are enough.

Look out for

Access promised but not granted, and no named sponsor. Either one stalls the first week.

You leave with

A one-page charter: the goal, the sponsor, the access and the end date.

Stop 2 of 9Discover

What you do

Sit with the people who do the work. Map the real workflow, find where time and money are lost, and write down what better means in the customer's own numbers.

Lean on

CRISP-DM

Look out for

The problem the executive describes and the problem the staff have are often different. A use case with no current measurement cannot be shown to have improved.

You leave with

Problem statements, each with a baseline measurement.

Stop 3 of 9Assess readiness

What you do

Check four things: the data (quality, access, sensitivity), the systems (interfaces, identity, logging), the people (skills, owners) and how much risk the customer accepts.

Look out for

Data nobody owns, systems with no test environment, and AI tools staff already use without approval.

You leave with

A readiness scorecard and a list of blockers with an owner each.

Stop 4 of 9Prioritize

What you do

Rank the candidate use cases by value and by how feasible they are, pick one or two, and say no to the rest in writing.

Look out for

The most exciting use case is rarely the right first one. Choose one with clean data, a willing owner and a result you can measure in weeks.

You leave with

A ranked list and one chosen pilot with a pass mark.

Stop 5 of 9Design

What you do

Choose the simplest shape that works (see the seven shapes above), decide what the system may read and do, draw the threat model, and write the spec and the evaluation set before any building (see Spec-driven development above).

Look out for

Reaching for an agent when a single call or a fixed workflow would do. Every tool and every data source added here is something to secure later.

You leave with

An architecture sketch, a threat model, the evaluation set and the points where a person approves.

Stop 6 of 9Build a thin slice

What you do

Ship one path end to end, on real data, with a few real users, inside the customer's environment.

Look out for

Shared or personal credentials used to move fast, and demo data that hides what real data does.

You leave with

A working pilot that a few people use for real work.

Stop 7 of 9Evaluate and harden

What you do

Measure against the baseline, attack it yourself, then add the guardrails, spending limits, logging and a way to roll back.

Look out for

Passing your own tests and nothing else. Add the injected document, the request for another user's data and the tool called out of turn.

You leave with

A go or no-go report with the evidence.

Stop 8 of 9Deploy and adopt

What you do

Roll out in stages, train the users, and track whether people use it as well as whether it is right.

Look out for

A system that works and that nobody uses. Adoption is a result to measure, the same as accuracy.

You leave with

The system in production, monitored, with named owners.

Stop 9 of 9Hand off and scale

What you do

Transfer ownership, write the runbook, remove your own access, and take what you learned back to the product team and into the next use case.

Look out for

Keys, test accounts and wide permissions left behind when the engineer leaves.

You leave with

A hand-over pack, your access removed, and a roadmap for the next use cases.

See all nine steps in one table
StepWhat you doLean onLook out forYou leave with
0. PrepareBefore day one: read what the company publishes, learn the rules of its industry, and agree the scope, the access you will get and who your sponsor is. A stakeholder map and a RACI chart are enough.NIST AI Risk Management FrameworkAccess promised but not granted, and no named sponsor. Either one stalls the first week.A one-page charter: the goal, the sponsor, the access and the end date.
1. DiscoverSit with the people who do the work. Map the real workflow, find where time and money are lost, and write down what better means in the customer's own numbers.CRISP-DMThe problem the executive describes and the problem the staff have are often different. A use case with no current measurement cannot be shown to have improved.Problem statements, each with a baseline measurement.
2. Assess readinessCheck four things: the data (quality, access, sensitivity), the systems (interfaces, identity, logging), the people (skills, owners) and how much risk the customer accepts.NIST AI Risk Management Framework, ISO/IEC 42001, EU AI Act, CISA and partners: AI Data SecurityData nobody owns, systems with no test environment, and AI tools staff already use without approval.A readiness scorecard and a list of blockers with an owner each.
3. PrioritizeRank the candidate use cases by value and by how feasible they are, pick one or two, and say no to the rest in writing.NIST AI Risk Management FrameworkThe most exciting use case is rarely the right first one. Choose one with clean data, a willing owner and a result you can measure in weeks.A ranked list and one chosen pilot with a pass mark.
4. DesignChoose the simplest shape that works (see the seven shapes above), decide what the system may read and do, draw the threat model, and write the spec and the evaluation set before any building (see Spec-driven development above).OWASP Top 10 for LLM Applications, OWASP Top 10 for Agentic Applications, MITRE ATLAS, MCP security best practicesReaching for an agent when a single call or a fixed workflow would do. Every tool and every data source added here is something to secure later.An architecture sketch, a threat model, the evaluation set and the points where a person approves.
5. Build a thin sliceShip one path end to end, on real data, with a few real users, inside the customer's environment.OWASP MCP Top 10, MCP security best practicesShared or personal credentials used to move fast, and demo data that hides what real data does.A working pilot that a few people use for real work.
6. Evaluate and hardenMeasure against the baseline, attack it yourself, then add the guardrails, spending limits, logging and a way to roll back.NIST AI Risk Management Framework, OWASP AI Testing Guide, MITRE ATLASPassing your own tests and nothing else. Add the injected document, the request for another user's data and the tool called out of turn.A go or no-go report with the evidence.
7. Deploy and adoptRoll out in stages, train the users, and track whether people use it as well as whether it is right.Prosci ADKARA system that works and that nobody uses. Adoption is a result to measure, the same as accuracy.The system in production, monitored, with named owners.
8. Hand off and scaleTransfer ownership, write the runbook, remove your own access, and take what you learned back to the product team and into the next use case.NIST AI Risk Management Framework, ISO/IEC 42001Keys, test accounts and wide permissions left behind when the engineer leaves.A hand-over pack, your access removed, and a roadmap for the next use cases.

Five rules hold in every engagement. Measure the current process first, or the improvement cannot be shown. Write the evaluation set before the code: it is the specification the customer signs. Start the security work at step 2, not step 6. Have one named owner on the customer's side. Use the simplest shape that passes the evaluation.

Build the plan for the customer in front of you

Real customers are several situations at once. Tick everything you found and the page puts together one plan: the nine steps, the extra steps for each situation, the frameworks to lean on and what to look out for. Copy it, save it, or share the link: the address remembers your choices. Nothing you tick leaves your browser.

What did you find?

Eight scenarios, and how to work each one

No two customers look alike, but the situations repeat. Open each one for what you find, the use case that fits, the steps in order, the frameworks to use and what to look out for. The scenarios are this center's own sketches of common situations, not accounts of named companies.

01An old system with no interface for programs

What you find

A mainframe or a decades-old application that people reach through terminal screens, batch jobs and nightly files. Nothing offers an API, and the people who understand it are few.

The use case

Staff retype data between the old system and everything else. An assistant that reads the nightly extract and answers questions, or drafts the entry for a person to confirm, removes the retyping without touching the old system.

Frameworks and methods

Strangler Fig (Martin Fowler), Anti-Corruption Layer (Microsoft Azure Architecture Center), NIST AI Risk Management Framework, OWASP Top 10 for LLM Applications

Step by step

  1. Start read-only: take the files and reports the system already produces, and leave the system itself alone.
  2. Put an adapter between the old system and the AI that translates its codes and field names into plain, documented terms.
  3. Build the assistant on that adapter. Let it draft, and let a person confirm every entry.
  4. Only when the read-only version is trusted, add writing back, one transaction type at a time, through the same path a person uses.
  5. Move functions off the old system piece by piece as each one proves itself.

What to look out for

  • Screen scraping breaks when a screen changes. Prefer files and reports to screens.
  • Batch data is a day old. Say so in every answer that depends on it.
  • The old system's access rules do not travel with an extract. Rebuild them in the adapter, or people will read what they could not read before.
  • Nobody may be able to tell you what a field means. Budget time for that, and write it down.
02A large old codebase nobody fully understands

What you find

A monolith that runs the business, written over many years, with thin tests and little documentation. Every change is slow because nobody is sure what it will break.

The use case

A coding agent that reads the codebase and explains it, writes the missing tests, and then helps move one module at a time to something maintainable.

Frameworks and methods

Strangler Fig (Martin Fowler), OWASP Top 10 for Agentic Applications, OWASP Top 10 for LLM Applications, OWASP AI Testing Guide

Step by step

  1. Have the agent map the code and write the documentation first. Nothing changes yet.
  2. Add tests that record what the system does today, so a change that alters behavior is caught.
  3. Pick one module with clear edges and move it, with the old and the new running side by side until the results match.
  4. Run the agent in a sandbox with no production credentials, and send every change through review and the same pipeline as any other.

What to look out for

  • Tests written by a model can agree with the bugs. Have someone who knows the business review what they assert.
  • Old repositories hold old secrets. Scan before an agent or a model provider reads them.
  • Files in a repository can carry instructions for the agent. Treat the code as untrusted input.
  • A rewrite that looks finished is not the same as one that behaves the same. Compare outputs on real data.
03Integrations of every kind

What you find

Some systems speak REST, some SOAP, some drop CSV files on a file server, some post to a message queue, and one partner still sends EDI. Each was added in a different decade.

The use case

One assistant or agent that can answer and act across them, for example looking up an order, its shipment and its invoice in a single question.

Frameworks and methods

Anti-Corruption Layer (Microsoft Azure Architecture Center), MCP security best practices, OWASP MCP Top 10, OWASP Top 10 for Agentic Applications

Step by step

  1. List every interface: who owns it, how it authenticates, how fast it may be called and what a wrong call costs.
  2. Wrap each one as a small tool with a narrow job and its own identity, read-only first.
  3. Translate each system's terms into one shared vocabulary inside the tool, so the model never sees five names for a customer.
  4. Add write tools last, one at a time, with a person's approval for anything that cannot be undone.
  5. Record every tool call so a wrong answer can be traced to the system it came from.

What to look out for

  • One service account with access to everything is the quick way and the dangerous one. Give each tool its own.
  • Old interfaces have no rate limits. An agent in a loop can take one down.
  • Text that arrives from a partner file or a message is untrusted. It can carry instructions.
  • Two systems will disagree about the same record. Decide which one is right before the model has to.
04A dozen cloud applications and no shared view

What you find

Sales, support, finance and HR each bought their own cloud software. The data is modern but scattered, and each product now ships its own AI assistant.

The use case

An assistant that works across the applications with the user's own permissions, so a support agent sees the customer's contract and open tickets in one place.

Frameworks and methods

NIST AI Risk Management Framework, MCP security best practices, OWASP Top 10 for LLM Applications, ISO/IEC 42001

Step by step

  1. Use the vendors' own connectors where they exist, and review what each one can reach before turning it on.
  2. Have the assistant act as the signed-in user, never as an administrator, so each person sees only what they already could.
  3. Ask for the narrowest permission each connector needs, and write down who approved it.
  4. Decide which built-in vendor assistants are allowed, and what data each vendor may keep.

What to look out for

  • Connectors ask for broad permissions by default. Narrow them.
  • A vendor's AI feature sends your data to that vendor's model provider. Ask where it goes and how long it is kept.
  • Old sharing mistakes become visible: an assistant will find the folder that was open to everyone.
  • Tokens granted to a pilot keep working after the pilot ends. Revoke them.
05Knowledge scattered across documents

What you find

Policies, contracts, manuals and old tickets sit in shared drives, a wiki and email. People ask a colleague because searching does not work.

The use case

A question-answering assistant over the documents that cites the page it read.

Frameworks and methods

CRISP-DM, CISA and partners: AI Data Security, OWASP Top 10 for LLM Applications, OWASP AI Testing Guide

Step by step

  1. Pick one body of documents with one owner. Do not start with everything.
  2. Remove what is out of date or duplicated before indexing. The assistant repeats whatever it is given.
  3. Carry each document's access rules into the index, and filter by the person asking.
  4. Build the evaluation set from real questions staff asked, with the right answer and its source.
  5. Show the source with every answer, and say plainly when nothing was found.

What to look out for

  • An index without access rules is a search engine for confidential files.
  • A document can carry hidden instructions. Anyone who can add a document can try to steer the assistant.
  • Two versions of a policy give two answers. Keep one.
  • Without a named owner the index goes stale and trust goes with it.
06A regulated or closed environment

What you find

A bank, a hospital, a public agency or a plant where data may not leave, the network is closed or tightly filtered, and every change needs sign-off.

The use case

The same use cases as anywhere else, with the model run where the rules allow and the evidence kept for the auditor.

Frameworks and methods

EU AI Act, ISO/IEC 42001, NIST AI Risk Management Framework, CISA and partners: AI Data Security

Step by step

  1. Bring risk, legal and security in during the first week, not at the end.
  2. Place the use case under the rules that apply, and write down the reasoning.
  3. Choose where the model runs first (see Where the model runs above): the customer's own cloud account, or open weights on their machines.
  4. Keep the evidence as you go: the evaluations, the approvals, the logs and who changed what.
  5. Plan for a person's review of any output that affects a customer, a patient or a citizen.

What to look out for

  • A model you host yourself is yours to patch and to verify, including the model file itself.
  • Approval takes longer than building. Start it early and build the low-risk part meanwhile.
  • A closed network also blocks updates and package downloads. Agree how they come in.
  • Logs of prompts and answers hold sensitive data. Protect them like the records they came from.
07A team that wants an agent to act, not only answer

What you find

Support or operations staff spend the day on repeat tasks: refunds, password resets, updating records. The customer wants an agent that does them.

The use case

An agent that reads the request, gathers what it needs and carries out the task within set limits.

Frameworks and methods

OWASP Top 10 for Agentic Applications, MITRE ATLAS, NIST AI Risk Management Framework, Prosci ADKAR

Step by step

  1. Begin with drafting: the agent proposes the action and a person clicks to approve.
  2. Measure how often the person agrees. Automate only the task types where they almost always do.
  3. Set limits in code, not in the prompt: amounts, how many actions per hour, which accounts.
  4. Keep approval for anything that cannot be undone or that moves money.
  5. Give the agent its own identity, a spending cap and a switch that stops it.

What to look out for

  • The request is written by an outsider and may be an attack. Test with requests that ask for someone else's refund.
  • An instruction in the prompt is not a control. The limit has to hold when the model is talked out of it.
  • Agents can loop. Cap the calls and the cost.
  • Staff who fear the agent will work around it. Involve them in choosing what it takes over.
08A pilot that already stalled

What you find

The customer built a demo months ago. It impressed in the meeting and never reached production, or it reached production and nobody trusts it.

The use case

Getting the existing work over the line, or stopping it for a stated reason.

Frameworks and methods

NIST AI Risk Management Framework, OWASP AI Testing Guide, OWASP Top 10 for LLM Applications, Prosci ADKAR

Step by step

  1. Find out why it stalled before changing anything: no owner, no measurement, no security sign-off, or data it could not reach.
  2. Write the evaluation set it never had, from real cases, and run it. Now there is a number.
  3. Fix the largest gap the number shows, and nothing else, then measure again.
  4. Do the security review the pilot skipped: what it reads, what it can do, whose credentials it uses.
  5. Name an owner on the customer's side. If nobody will take it, recommend stopping.

What to look out for

  • Pilots are built with whatever credentials were at hand. Expect a personal key in the code.
  • The model or the prompt may be changing with no record. Put both under version control first.
  • A demo's test data flatters it. Real data is the test.
  • Stopping a project that cannot pass is a result. Say it early.

The frameworks and methods the plan names

Each links to its own page. The security and governance frameworks are also in the Learning Center's Frameworks list; the four working methods at the end were read on 2026-11-15. The note beside each says how this plan uses it and is this center's own.

  • NIST AI Risk Management Framework: The backbone of the plan. Its four functions, Govern, Map, Measure and Manage, give the order: who decides, what the system touches, how it is tested, how it is run.
  • ISO/IEC 42001: When the customer wants AI run as a managed system with policies, objectives and audits, this is the standard its auditors will ask about.
  • EU AI Act: For a customer that operates in the EU: place the use case in a risk tier before design, because the tier decides the duties.
  • OWASP Top 10 for LLM Applications: The checklist for any feature that puts a model in front of users or documents.
  • OWASP Top 10 for Agentic Applications: The checklist once the system calls tools or acts by itself.
  • OWASP MCP Top 10: The checklist for every connector built as an MCP server.
  • MCP security best practices: How to build those connectors: narrow scopes, no token passthrough, no confused deputy.
  • MITRE ATLAS: The attacker's side of the threat model: real techniques against AI systems to test the design against.
  • OWASP AI Testing Guide: The test plan before go-live, layer by layer: application, model, infrastructure and data.
  • CISA and partners: AI Data Security: For the data the system is trained on or reads: where it came from, who can change it, how drift is noticed.
  • CRISP-DM: A six-phase process for data projects that starts with understanding the business and the data. Use it to keep discovery ahead of building.
  • Strangler Fig (Martin Fowler): Modernize an old system gradually: build the new part alongside it and move behavior over piece by piece, instead of replacing it in one go.
  • Anti-Corruption Layer (Microsoft Azure Architecture Center): An adapter between two systems that do not mean the same thing by the same words. It translates, so the old system stays unchanged and the new one stays clean.
  • Prosci ADKAR: A model of how one person takes up a change: awareness, desire, knowledge, ability, reinforcement. Use it to plan the rollout to the people who will use the system.

Sources for this section, read 2026-11-12; a number in brackets points to the one a statement was read on. Statements with no number, the security notes, the five things built on site, the engagement plan and the eight scenarios are this center's own explanations.

  1. The Pragmatic Engineer: What are Forward Deployed Engineers, and why are they so in demand? (Gergely Orosz, 2025-08-12; the free part of the article)
  2. Alvarez & Marsal: The Rise and Role of the Forward Deployed Engineer (Robert Kreft, Joe Perovich and Michael Roth, 2026; a consulting firm's paper)

Vocabulary

Twelve words the work runs on

Context engineering

Choosing what goes into the model's window for this step, and what stays out. Most failures and most leaks start here.

Retrieval (RAG)

Looking up passages at question time and placing them in the prompt. Whatever is retrieved is read as if it were trusted.

Tool use

The model asks for a function to be run and gets the result back. The tool's permissions are the agent's permissions.

Memory

What the system keeps between sessions. A poisoned memory is an instruction that arrives later.

Evals

A fixed set of cases the system is graded on, run before every change. The AI counterpart of a test suite.

Guardrails

Checks on what goes in and what comes out, written in code or run by a second model. Useful, and never the only control.

Human approval

A person confirms before an action that costs money, deletes data or cannot be undone.

Sandbox

The box the agent's code and commands run in, with its own files and its own network rules.

Tracing

A record of every model call and tool call in a run, so a bad outcome can be followed back to its step.

Model routing

Sending each request to the cheapest model that can do it, with a fallback when one is down.

Budgets

Hard limits on tokens, steps, time and spend per task, set before the agent starts.

Agent-computer interface

How tools are named, documented and shaped for the model. The source asks for the same care here as for an interface made for people.

Where the model runs and the vocabulary are this center's own explanations, written for this page; they state no figures. For the attacks on each of these parts, with sources, see the agent loop and the best practices in the Learning Center, and for the delivery steps see the lifecycle.