EU AI Act

How to Build AI Agent Compliance Evidence Packages for EU AI Act Audits in H2 2026

Scott Miller

28 Aug 2026 — 9 min read

The second half of 2026 is not a grace period. With the EU AI Act's high-risk AI system obligations now in full enforcement mode, enterprise legal and compliance teams are scrambling to answer a question that most engineering organizations have never had to face before: Can you prove, right now, exactly what your AI agent did, why it did it, and which model version made that decision?

If your answer involves manually digging through CloudWatch logs, Slack messages from engineers, and a spreadsheet someone built in 2024, you are already behind. Regulators under the EU AI Act can demand audit evidence on short notice, and the documentation standard is not "we tried our best." It is structured, traceable, and technically verifiable.

This tutorial walks you through building an AI Agent Compliance Evidence Package (CEP): an automated pipeline that continuously compiles inference logs, tool call traces, and model version attestations into a single, audit-ready documentation artifact that your legal team can hand to a notified body or national market surveillance authority without breaking a sweat.

Why H2 2026 Is the Real Deadline That Matters

The EU AI Act's phased rollout has been well-documented, but H2 2026 is the inflection point where enforcement teeth appear for high-risk AI systems deployed in sectors like HR, credit scoring, critical infrastructure management, and legal decision support. National competent authorities across EU member states have been given the green light to conduct conformity assessments, issue corrective orders, and levy fines of up to 3% of global annual turnover for non-compliance with obligations under Articles 9 through 17.

Those articles are not abstract. They require:

A documented risk management system that is continuously updated
Technical documentation sufficient to assess conformity before market deployment
Automatic logging of operations to the extent necessary to identify risks
Human oversight measures that are verifiable, not just stated in a policy PDF
Accuracy, robustness, and cybersecurity measures with supporting evidence

For enterprises running AI agents (autonomous or semi-autonomous systems that chain LLM calls with tool use, memory retrieval, and external API interactions), the logging and attestation requirements are especially complex. A single user session may involve dozens of inference calls, multiple tool invocations, and dynamic model routing. Your CEP must capture all of it.

What a Compliance Evidence Package Actually Contains

Before building the pipeline, you need a clear definition of the artifact you are producing. A CEP for an AI agent system should contain four core layers:

Layer 1: Inference Logs

A structured, tamper-evident record of every inference call made by the agent. This includes the prompt (or a privacy-safe hash of it), the model endpoint called, input token count, output token count, latency, the raw completion (or a semantic summary where PII scrubbing is required), and a confidence or uncertainty signal where available.

Layer 2: Tool Call Traces

A sequential, timestamped record of every external tool, API, or function the agent invoked during a session. This includes the tool name, input parameters, return values, execution duration, and any error or retry events. For agents using retrieval-augmented generation (RAG), this layer must also capture document chunk IDs and retrieval scores.

Layer 3: Model Version Attestations

A cryptographically signed record of which model version, fine-tune checkpoint, or system prompt configuration was active at the time of each inference. This is the layer most teams neglect, and it is the one regulators care about most when investigating an adverse outcome.

Layer 4: Human Oversight Signals

Evidence that human review mechanisms were in place and functioning. This includes override events, escalation triggers, human approval timestamps, and any cases where the agent was stopped or corrected by a human operator.

Step 1: Instrument Your Agent for Structured Emission

The foundation of your CEP pipeline is structured log emission at the agent framework level. Whether you are using LangGraph, AutoGen, CrewAI, or a custom orchestration layer, every agent action must emit a structured event in a consistent schema before it reaches any downstream system.

Here is a recommended base schema in JSON for an inference event:

{
  "event_type": "inference",
  "session_id": "uuid-v4",
  "trace_id": "uuid-v4",
  "span_id": "uuid-v4",
  "timestamp_utc": "ISO-8601",
  "agent_id": "string",
  "model_id": "string",
  "model_version_hash": "sha256-of-model-card",
  "system_prompt_hash": "sha256",
  "input_token_count": "integer",
  "output_token_count": "integer",
  "latency_ms": "integer",
  "completion_hash": "sha256",
  "pii_scrubbed": "boolean",
  "risk_classification": "string",
  "environment": "production | staging"
}

And a corresponding tool call event:

{
  "event_type": "tool_call",
  "session_id": "uuid-v4",
  "trace_id": "uuid-v4",
  "span_id": "uuid-v4",
  "parent_inference_span_id": "uuid-v4",
  "timestamp_utc": "ISO-8601",
  "tool_name": "string",
  "tool_version": "string",
  "input_params_hash": "sha256",
  "output_hash": "sha256",
  "execution_duration_ms": "integer",
  "success": "boolean",
  "retry_count": "integer",
  "human_readable_summary": "string"
}

The critical design principle here is span parenting. Every tool call must reference the inference span that triggered it. This creates a directed acyclic graph (DAG) of causality that an auditor can traverse to reconstruct exactly why the agent took a specific action.

Step 2: Build the Model Version Attestation Service

Model version attestation is the part of compliance that feels like an engineering problem but is actually a legal one. The question it answers is: "On March 14th at 14:32 UTC, which exact model, with which exact system prompt, produced this output?" Without a dedicated attestation service, that question is nearly impossible to answer reliably, especially in organizations that do frequent prompt updates or use model routing layers.

Here is how to build a lightweight attestation service:

The Model Registry

Maintain a versioned model registry that records, for each deployment event: the model provider and model identifier, the model card hash (a SHA-256 of the official model card document), the system prompt text and its SHA-256 hash, the deployment timestamp, the deploying engineer's identity (for chain of custody), and the approval ticket reference from your change management system.

The Attestation Sidecar

Deploy a lightweight sidecar service alongside your agent runtime. At inference time, before the call is made, the sidecar queries the registry for the current active model configuration and appends a signed attestation token to the inference event. The token is signed with a private key managed by your HSM or a cloud KMS (AWS KMS, Azure Key Vault, GCP Cloud HSM are all appropriate). The corresponding public key is registered with your compliance authority record.

The attestation token payload should include:

The model registry entry ID
The system prompt hash
The timestamp of the configuration's last change
A reference to the change management ticket that authorized this configuration
The signer's key ID

This token travels with every inference event log entry, creating an unbreakable chain between a specific output and a specific, auditable model configuration.

Step 3: Centralize Into a Compliance Data Lake

Raw structured events are not a CEP. They are raw materials. The next step is centralizing them into a purpose-built compliance data lake that is separated from your operational data infrastructure. This separation matters for two reasons: it prevents operational teams from accidentally modifying compliance records, and it allows you to apply stricter access controls and retention policies.

Recommended architecture:

Event Bus: Use Apache Kafka or AWS EventBridge to stream all structured agent events in real time. Apply schema validation at the bus level so malformed events are flagged immediately rather than silently corrupting your audit trail.
Immutable Storage: Land events in an append-only object store with object lock enabled. AWS S3 Object Lock in WORM (Write Once Read Many) mode, Azure Immutable Blob Storage, or GCP Bucket Lock all satisfy the tamper-evidence requirement. Partition by date and session ID.
Metadata Index: Maintain a searchable index (OpenSearch or a dedicated compliance database) that allows legal teams to query by date range, agent ID, session ID, model version, tool name, or risk classification without touching the raw immutable store.
Integrity Verification: Run a nightly job that recomputes SHA-256 hashes of stored event batches and compares them against a separately stored hash manifest. Any discrepancy triggers an immediate alert and a compliance incident record.

Step 4: Automate the CEP Assembly Pipeline

This is the step that transforms your data lake from a passive archive into an active compliance tool. The CEP Assembly Pipeline is a scheduled or on-demand process that, given a session ID or a date range and agent ID, automatically compiles all four layers of evidence into a single structured document package.

The pipeline should execute the following stages:

Stage 1: Evidence Retrieval

Query the metadata index for all events matching the requested scope. Retrieve the corresponding raw event records from the immutable store. Verify integrity hashes before proceeding. If any hash fails, halt the pipeline and raise a compliance incident rather than producing a potentially corrupted package.

Stage 2: Trace Reconstruction

Using the span parent relationships, reconstruct the full execution DAG for each session. This produces a human-readable (and machine-parseable) timeline that shows: which inference calls were made in sequence, which tool calls each inference triggered, what the outcomes were, and where human oversight events occurred.

Stage 3: Attestation Verification

For each inference event, verify the cryptographic signature on the attestation token using the registered public key. Resolve the model registry entry referenced in the token and include the full model configuration snapshot in the package. Flag any events where attestation verification fails.

Stage 4: Risk Annotation

Apply your organization's risk classification rules to the reconstructed trace. Flag any inference or tool call that matches known high-risk patterns: decisions affecting individuals, calls to external financial or legal data sources, outputs that were overridden by human operators, or sessions that triggered escalation rules.

Stage 5: Document Rendering

Render the compiled evidence into the output formats your legal team needs. At minimum, produce: a structured JSON manifest (machine-readable, for submission to technical auditors), a PDF narrative report (human-readable, for legal review and notified body submission), and a SARIF-compatible findings file if your organization uses security tooling that ingests that format.

The PDF narrative report should follow a structure that maps directly to the EU AI Act's technical documentation requirements under Annex IV. Include sections for: system description, risk management evidence, data governance evidence, transparency and logging evidence, human oversight evidence, and accuracy and robustness evidence. Each section should cite specific event records from the package by their unique IDs.

Step 5: Implement Continuous Compliance Monitoring

A CEP built on demand is reactive. A mature compliance program also needs a proactive monitoring layer that detects compliance gaps before an auditor does.

Set up the following continuous monitors:

Attestation Gap Monitor: Alert if any inference event arrives in the data lake without a valid attestation token. This indicates either a misconfigured agent or a potential attempt to run an unregistered model.
Log Completeness Monitor: Compare the count of inference events against the count of corresponding tool call events. Gaps in the expected ratio (based on your agent's typical behavior profile) may indicate logging failures.
Model Drift Detector: Alert if a model version hash in production does not match any registered entry in the model registry. This catches unauthorized model changes before they accumulate into a large undocumented exposure.
Human Oversight Frequency Monitor: Track the rate at which human oversight events occur relative to total agent sessions. A sudden drop may indicate that oversight mechanisms have been bypassed or are malfunctioning.
Retention Policy Enforcer: EU AI Act Article 12 requires that logs for high-risk AI systems be retained for at least 6 months. Automate retention policy enforcement and produce a monthly attestation that all required records are intact.

Step 6: Structure the Legal Team Interface

All of the above engineering work is wasted if your legal team cannot actually use the output. The final step is building a lightweight interface that allows non-technical legal and compliance professionals to self-serve CEP generation without needing to understand the underlying infrastructure.

At minimum, this interface should support:

Scoped package generation: Legal can specify a date range, agent ID, or specific session ID and trigger package generation without engineering involvement.
Regulatory framework mapping: The interface should allow legal to tag a package with the specific regulatory inquiry it is responding to (for example, a national market surveillance authority request under Article 65). This metadata is included in the generated package for chain-of-custody purposes.
Redaction controls: Legal must be able to apply PII redaction rules before a package is exported for external submission, without altering the underlying immutable records.
Package signing: Each exported CEP should be signed with a legal entity certificate (not just the technical attestation key) to create a formal record of the organization's attestation that the package is complete and accurate.
Submission tracking: Maintain a log of which packages were generated, when, by whom, and to which authority they were submitted. This is itself part of your compliance record.

Common Pitfalls to Avoid

Having outlined the full pipeline, here are the most common mistakes enterprises make when building CEP infrastructure for the first time:

Treating operational logs as compliance logs: Operational logs are optimized for debugging. Compliance logs are optimized for legal defensibility. They have different schemas, different retention requirements, and different access control models. Do not conflate them.
Neglecting system prompt versioning: Many teams version their model but not their system prompts. A changed system prompt is effectively a changed AI system under the EU AI Act. Every system prompt change must be registered and attested.
Logging completions without hashing: Storing raw completions creates PII risk. Storing only hashes without the original (or a privacy-safe summary) creates evidentiary gaps. The right balance is a hash plus a structured semantic summary that captures the nature of the output without reproducing sensitive content.
Skipping the DAG reconstruction step: A flat list of events is not a trace. Without the causal DAG, an auditor cannot determine why the agent took a specific action. Span parenting is not optional.
Building for today's agent architecture: Agent frameworks are evolving rapidly. Build your CEP pipeline to be framework-agnostic by emitting events through a standardized interface layer rather than instrumenting each framework directly.

Conclusion: Compliance as a First-Class Engineering Concern

The EU AI Act has fundamentally changed the relationship between AI engineering and legal compliance. In the pre-enforcement era, compliance was something you documented after the fact. In H2 2026 and beyond, compliance evidence must be generated continuously, automatically, and with cryptographic integrity guarantees.

Building an AI Agent Compliance Evidence Package pipeline is not a small project. It requires coordination between engineering, legal, security, and data governance teams. But the alternative, which is attempting to reconstruct audit evidence manually after a regulatory inquiry arrives, is far more expensive and far less likely to succeed.

The good news is that the same infrastructure that makes your agents compliant also makes them more observable, more debuggable, and more trustworthy to the enterprise customers who are increasingly demanding this level of transparency as a procurement requirement. Compliance is not just a legal obligation in 2026. It is a competitive differentiator.

Start with Step 1 today. Instrument one agent. Build the attestation sidecar for one model. Generate one CEP manually. Then automate. The pipeline you build incrementally over the next quarter will be the one that keeps your organization out of the enforcement headlines in Q4 2026.