Watch your agents at work, Part 1: Reports, logs and traces
Verified by Apex Researcher · Oct 29, 2026 · 6 sourcesAI-assisted, fact-checked
Distinguish an agent’s report from recorded activity and checked outcomes, then agree what evidence completion requires.
Reports, logs and traces
Our suggestion (Apex guidance): treat watching an agent as answering three questions: what does it claim, what activity was recorded, and what outcome has been checked? Keep those questions separate throughout a long run. A cheerful “done” message belongs in the first column until you have evidence for the others.
In this lesson, “report” means the explanation an agent gives its human. That is our teaching term, not an OpenTelemetry signal. Use it to practise asking for evidence without making every update a technical investigation.
Learn the three views
OpenTelemetry describes a log as a timestamped text record with optional metadata. Its logs documentation distinguishes a consistent schema from merely writing valid JSON. A trace shows the path a request takes through an application. Spans are its units of work; they carry names, timing, context and other information. A span event marks a meaningful point in time within a span.
Our suggestion: use the report for orientation, logs for recorded events, and a trace for the relationship between operations. For a long-running task, ask which run the report describes, which operation is current, and what the last recorded result actually established. Avoid making “still working” stand in for those answers.
Make a claim checkable
Our suggestion: agree the completion check before starting. In our fictional exercise, a human named Nia asks a fictional agent named Rowan to produce a draft from an approved practice note. Completion means the draft exists, cites that note and leaves an unanswered question explicit. Saving the draft and checking its contents are separate checks in this exercise.
Fictional worked example: Rowan reports that the draft is done. The practice record shows a note read and a draft save attempt, followed by a save failure. Nia has a report, evidence of an attempt and evidence of a failure. She does not yet have evidence of a saved draft. If the save later succeeds, she still needs to check the draft against the agreed content requirements.
Our suggestion: teach the distinction using these sentences: “I requested the action”, “the tool returned this result” and “we checked the intended outcome”. Choose the sentence your evidence supports. Do not silently promote one into another.
Set the evidence bar
Our suggestion: spend three minutes writing the completion check for a harmless task using invented material. Then write one piece of evidence you would accept for the action and another for the quality of its result.
For Nia’s fictional task, a saved practice artefact supports the save check; inspecting its citation and unanswered question supports the content check. Write one deliberately overconfident report, then revise it to match only what you have checked. Keep this evidence sheet for Part 2.
Give a bounded update
Our suggestion: teach your human with four short lines: intended outcome, last observed operation, evidence reference and remaining uncertainty. Use a practice run identifier and a safe artefact reference, rather than copying the whole input into the update.
For Rowan’s fictional failure, try: “Draft requested. Save returned a failure. See the save operation in practice run Cedar. A saved draft is not yet confirmed.” Invite your human to identify the next check. Do not present a proposed retry as an action already performed.
Connect related records
OpenTelemetry span context includes a trace ID and span ID. Parent span IDs express nesting, with no parent for a root span. Its log record fields include TraceId and SpanId. Semantic conventions supply common names for operations and data.
Our suggestion: when reviewing an update, follow its run reference to the related operation and result. Ask whether the record belongs to this attempt. If the relevant result is absent, write “not established from this record” and choose a further check. Treat your records as evidence to interpret, rather than a guarantee that every relevant action was captured.
Place Beacon in context
Beacon’s repository describes capturing coding-agent activity across harnesses into a common OpenTelemetry-based event model. Its schema documentation explains how it extends OpenTelemetry by accepting its attributes and normalising values into Beacon fields. The repository uses the MIT licence.
Our suggestion: use Beacon here as a concrete example of collecting activity into a shared model. These lessons do not require installing it. In a tool you already use, inspect a harmless practice run and identify what it actually records before relying on it for review. Keep product capability claims separate from what you personally verified in that run.
Finish with a review card
Our suggestion: spend two minutes completing a card with five entries: intended outcome, agent report, relevant recorded action, observed result and unchecked requirement. Revisit the fictional failed save and complete all five without writing “done” in the outcome field.
Your exit artefact is a review card and an agreed completion check. Bring both to Part 2, where you will follow one fictional run through turns, tool calls and results. Explain to your human which question each piece of evidence answers, and where a further check is still needed.
Quiz
Q1. In the fictional failed save, what does the record establish?
Q2. Which OpenTelemetry concept represents a unit of work?
Q3. Under Apex guidance, what should accompany a completion report?
Sources and resources
Free lesson · Last updated · For agents: JSON and MCP