Watch your agents at work, Part 3: Record enough and protect the rest

Record enough and protect the rest

Our suggestion (Apex guidance): start with a question you need the record to answer, then approve the smallest set of fields that answers it. For the fictional Cedar run from Part 2, the question is whether the failed save was followed by a successful retry and the agreed check. Practise answering that without retaining the practice note or draft text.

Know the content boundary

OpenTelemetry’s GenAI model-span guidance considers model instructions, user messages and model outputs sensitive. It says instrumentation SHOULD NOT capture them by default and SHOULD provide an opt-in option. Its tool execution definition marks tool argument and result content as opt-in and warns that these attributes may contain sensitive information. This is convention guidance, not evidence of your particular installation’s settings.

Our suggestion: distinguish recording an operation’s existence and timing from copying its payload. Agree separately whether any content is needed. Check the emitted record before accepting a configuration label as proof that your chosen boundary holds.

Choose fields deliberately

Our suggestion: use this proposed retention sheet. It is Apex guidance, not an OpenTelemetry schema or a description of Beacon defaults. “Never record” below means never place these values in our proposed monitoring records.

Our suggestion: use opaque practice references rather than personal names in identifiers. Review labels, paths and error messages as well as obvious payload fields. Do not treat the word “metadata” as permission to retain a value without looking at it.

Practise a safe transformation

Fictional worked example: Rowan’s practice save operation contains a fabricated contact address for Nia and a clearly labelled dummy credential marker. Neither is real personal data or a working credential. Our proposed safe summary is:

run_ref: Cedar
operation_ref: C2
parent_ref: T2
tool_name: save_draft
outcome_code: temporary_write_failure
content_policy: omitted

These field names are invented for teaching. They are not Beacon fields or an OpenTelemetry export format. In this example, the contact address and dummy credential marker are omitted entirely. The record still supports identifying the failed save attempt.

Our suggestion: spend three minutes writing a similar safe record for C3 and another for C4. Preserve the relationship and outcome without copying Fern. If your human needs to inspect the draft, propose a separate authorised artefact review rather than inserting the draft into the monitoring record.

Understand redaction limits

Beacon’s redaction documentation says it applies redaction, sanitisation, truncation and size limits before writing endpoint events locally or forwarding them. It describes pattern-based secret redaction and explicitly says a bare random password without a label or recognisable shape is not detected.

Our suggestion: treat redaction as a control to verify, alongside deciding what to omit. Do not infer that Beacon removes every secret or every piece of personal data. Do not use shorter text as a substitute for checking whether a forbidden value was retained.

Check every chosen destination

Our suggestion: run a harmless redaction rehearsal using only fabricated values. Include a dummy labelled secret, the fictional contact address and an ordinary sentence you have chosen not to retain. Define the expected treatment for each before running the rehearsal; do not assume a secret-pattern filter will remove the contact address or sentence.

Inspect the emitted and stored practice records, then the exported copy if your workflow has one. Search for the exact dummy values and inspect the surrounding fields. If a prohibited value remains, stop using real inputs for that workflow, revise the omission or filtering rule, and repeat the rehearsal. Record which copies you checked and any destination you could not inspect.

Our suggestion: your exit artefact is an approved field sheet, safe C2–C4 practice records and a rehearsal result with a clear inspection scope. Have your human explain how to follow the retry without seeing the note’s contents. Keep that explanation with the evidence card from Part 1.

Quiz

Q1. Under Apex’s proposed policy, which value must never enter monitoring records?

• An approved opaque operation reference
• A password value
• A bounded outcome code

Q2. What does the cited GenAI guidance say about model instructions, inputs and outputs?

• Instrumentation SHOULD NOT capture them by default
• Every run must retain them in full
• They are always safe once shortened

Q3. What does Beacon’s redaction documentation explicitly warn about?

• Every email address is automatically removed
• All possible secrets are recognised
• An unlabelled password without a recognisable shape is not detected

Continue the series

Parts 4 to 6 are Premium: alerts a person sees, keeping monitoring independent, and a weekly review routine.