Ask the demo to survive a shift change

Imagine a planner asking for the latest work-order status and a draft note for the next shift. The information sits in a maintenance system, an approved instruction library and a technician’s recent comment. In this example, the agent may assemble the note. It may not close the work order or declare equipment fit for service. Every supplier gets the same useful task, with no authority over production.

Prepare approved sample records: order and equipment identifiers, relevant procedure revisions, an old status and a comment that conflicts with it. Keep the expected answer and forbidden actions in your review notes. Ask the supplier to identify prepared connections, custom scripts and any work a person does behind the scenes. Run the exercise in a test environment and note where that setup differs from the one you would buy.

Microsoft documents knowledge access tied to the signed-in user. Google lists a federated BigQuery connector. Glean supports custom actions defined through OpenAPI.^1,2,3

Ask how each step gets its information: search, an index holding a copy, a live query or a tool able to change records. Put the source record and timestamp beside the answer. Change a harmless sample status, allow for the documented refresh behavior, then ask again. You want to see which record the agent used and whether it is current, not simply watch another confident answer appear.^1,2,3

Repeat the request with an account that cannot open one document. The draft should explain that evidence is missing without revealing the restricted contents. A summary containing a protected technician note fails this proposed rule, even if everything else is right. Use the kind of account operators will have on the floor. A demonstration with broad service-account access cannot answer their permission questions.

Try an unavailable maintenance system, similar equipment names, an outdated procedure and an event arriving twice. Agree the expected response before the run. Depending on the case, the agent should stop visibly, ask for clarification or hand the task to someone. A plausible guess should not count as recovery when the information needed to decide is missing. Keep the awkward cases beside the ordinary ones in your notes.

UiPath describes approval pauses and low-confidence cases passed to people with context. Nexus also describes passing uncertain cases with their context. Relevance AI lists escalation through email and Slack.^4,5,6,7

Leave a case unresolved with the first operator, then ask another authorized operator to pick it up. They should find the sources, attempted actions, unanswered question and permission needed to continue. An alert alone is not the handover. The case needs to remain saved and assigned after the first person leaves. Ask the supplier to show that whole sequence; the escalation claims do not establish every step.^5,6,7

Workato describes human approvals and detailed access controls. UiPath describes logs containing human approvers and tool calls. Microsoft documents approval gates for releasing changes to agents.^8,9,10

Ask two separate questions: who approves a new agent version, and who approves the action it takes? Try closing the sample work order and expect it to stop, because closure is outside this demo’s scope. If a later pilot permits updates, have the review show the target record, proposed values and destination. Reject an update and inspect the receiving system. Approval to launch an agent cannot serve as approval of all its later transactions.^8,9,10

Keep the audit entry with the final business record. Note the account used, source revision, proposed action, decision, result and anything unresolved. Remove sensitive sample details before circulating the notes. A colleague who was not at the demonstration should be able to follow what happened without asking the presenter to explain it from memory. That is the useful test of the record you take away.

Keep a row for each essential step: requirement, setup used, observed result, evidence, gap and owner. For example: retrieve current work-order status; connection and account named; revised sample value shown; source and run log saved; write-back not shown; plant IT to request a follow-up. Mark an unshown action as unanswered. That is different from a demonstrated failure or an established capability.

Move forward when the required reads, approvals and handoff work in the setup being proposed. Arrange a focused follow-up if an essential step used an unquoted add-on, a different account or custom work. A smooth conversation cannot compensate for a requirement the supplier has not shown. Before the next buying decision, compare the demonstration notes with the price, scope and acceptance plan so missing work remains visible.