IONSEC TRACE: Leave No Model Untraced — Open-Source Forensics for the AI Harness

Executive summary

Every AI tool an employee installs leaves a forensic trail. The problem is that nobody was collecting it.

The past year made the endpoint’s AI surface impossible to ignore. Agent runtimes moved the evidence boundary off the host and into vendor control planes ^[1]. Autonomous agents demonstrated the ability to chain ordinary misconfigurations into multi-day intrusions at machine speed ^[1]. And in parallel, a quieter wave was building on the workstation itself: developers and analysts installing local inference engines, agent frameworks, and AI coding assistants — often without IT approval, almost never with a security review, and always leaving artifacts behind.

That second wave is shadow AI, and it is the gap IONSEC TRACE is built to close.

TRACE — Tool for Reconnaissance of AI & Compute Evidence — is an open-source, forensically sound collector and analyzer for AI harness evidence. It reads artifacts without modifying them, hashes each one with SHA-256, writes a chain-of-custody manifest, and turns the result into a case file: indicators, leaked credentials, reconstructed conversations, MITRE ATLAS and ATT&CK mappings, a kill chain, a risk score and prioritized actions.

The two builds have converged. The Go binary now has near-identical capabilities to the Python CLI — the same platform catalog, the same 103-rule secret detector, the same analyzers, the same three report formats and the same DFIR-IRIS integration, over one shared forensic data model. Evidence collected by either is interchangeable, and either can report on the other’s output. The one exception: the Go binary collects SQLite conversation stores but does not parse them — use the Python CLI when you need SQLite conversation parsing. Alongside that: conversation secret hunting with leak direction and per-field provenance, a JavaScript-free HTML report, and 15 Velociraptor artifacts for fleet deployment.

This post is written for responders and platform teams. It covers what TRACE is, the shadow-AI problem it addresses, how collection and analysis work, the forensic-soundness guarantees, and the numbers from a live end-to-end run on a working analyst workstation — including what those numbers do not mean.

1. The problem: shadow AI leaves artifacts nobody collects

Shadow AI is unsanctioned AI software that employees install without IT approval. It is the AI-era analogue of shadow IT — except the stakes are higher, because the artifacts are richer.

A local inference engine like Ollama or LM Studio stores model manifests, signing keys, conversation databases, and CLI history. An agent framework like Hermes or AutoGPT persists sessions, state databases, memories, cron jobs, secrets, and skills. An AI coding assistant like Cursor or Claude Code keeps chat history, project rules, and — critically — authentication tokens. Each of these is a forensic goldmine: they record what the model was asked, what it was told, and what it did.

The problem is that none of it is collected by standard endpoint tooling. Traditional DFIR collectors know where browsers, shells, and mail clients keep their data. They do not know where Ollama keeps its signing keys, or that Hermes persists a state.db alongside its memories, or that Cursor’s globalStorage SQLite holds the conversation history of an AI that may have been given production credentials.

Takeaway 1 — Shadow AI is not a policy problem, it is an evidence problem. You cannot govern what you cannot inventory, and you cannot investigate what you cannot collect. The first step is detection and acquisition, and that is exactly what TRACE automates.

The forensic value is not hypothetical. In the reference agentic-AI incident of 2026, the reconstruction of a 4.5-day intrusion was built from agent logs and control-plane records — evidence that only existed because someone had the foresight to collect it ^[1]. On the endpoint, the same logic applies in miniature: the conversation history of an unsanctioned coding assistant may be the only record of what an AI was told to do with a set of credentials.

2. What TRACE is

TRACE is a cross-platform Python CLI and a self-contained Go binary, plus a Velociraptor artifact pack. Its tagline is the promise: “Leave no model untraced.”

The workflow is a standard four-step forensic pipeline:

# 1. Discover AI platforms on the system
trace discover

# 2. Collect all forensic artifacts (read-only, hashed, chain-of-custody)
trace collect --output /evidence --deep

# 3. Analyze collected evidence (IOCs, secrets, conversations, MITRE, risk)
trace analyze /evidence --mitre-atlas --mitre-attack --risk-score --secret-hunt

# 4. Generate reports (HTML, JSON, STIX 2.1)
trace report /evidence --format all

The output is a self-contained evidence package: a CHAIN_OF_CUSTODY.json manifest with per-file SHA-256 hashes and UTC timestamps, an analysis_results.json record of every derived finding, and the three reports.

2.1 One tool, two builds, near-identical capabilities

The Go binary began as a lean detector — useful for a fast sweep on a host with no Python, but not a substitute for the full pipeline. That asymmetry is largely gone. Both builds now implement the same platform catalog, the same secret rules, the same analyzers, the same reporters and the same IRIS integration over one shared data model. The one remaining difference: the Go binary collects SQLite conversation stores but does not parse them — use the Python CLI when you need SQLite conversation parsing.

This matters operationally. The moment you are on a host where installing a package is not an option — a locked-down endpoint, a server mid-incident, a machine you must not change — you should not have to accept a reduced collection. The Go binary has a single pure-Go dependency — a Zstandard decoder, for compressed transcripts — so it still cross-compiles for macOS, Linux and Windows on both architectures with no cgo, and runs the same pipeline end to end (it collects SQLite conversation stores but does not parse them):

# One-shot: discover → collect → analyze → HTML + JSON + STIX
./trace run -o /evidence

2.2 The collector surface

TRACE ships 27 collectors across four categories, plus four live-detection layers and a shadow-AI sweep that recognises 47 further tools by their on-disk footprint:

Each collector is read-only, knows the platform’s artifact paths on Linux, macOS and Windows, and records the original path, source OS, platform, artifact type, size and SHA-256 for every file it copies.

Beyond the static artifact collectors, four live-detection layers close the gap most shadow-AI tools leave open:

• Network AI collector — correlates running processes with their outbound connections and classifies destination domains against a catalog of 100+ AI providers. It answers the question static collectors cannot: which process is talking to which AI provider, right now.
• Code Scanner collector — scans source for AI framework imports, MCP server registrations and hardcoded API keys. A hardcoded key in a repository becomes a credential-exposure finding with a hash and a custody entry, not just a grep hit.
• Docker AI collector — Docker’s built-in AI assistant (Gordon), whose threads/ directory holds full conversation history, plus hosted LLM images, containers and the model registry.
• Browser AI collector — browser-based assistant evidence: Brave Leo’s AIChat/ store, browser history queried for AI-site URLs, and per-site IndexedDB conversation stores.

2.3 DeepSeek Harness

The newest addition is DeepSeek Harness (dsh), DeepSeek’s open-source agent harness. It is worth calling out because it breaks two assumptions most collectors make.

Its transcripts are Zstandard-compressed by default — sessions/--<cwd>--/<id>/session.jsonl.zstd, one frame appended per write batch — so a collector that reads lines finds a binary blob, and a decoder that stops after the first frame recovers only the session header. TRACE decompresses across frames in both builds.

Its records are also envelopes rather than messages: {type, seq, time, data}, with no role at the top level, the model under message.source, and each tool invocation logged twice — once as a nested content block and once as a standalone tool/call event. A generic role/content parser reads nothing from that, and a naive one double-counts every tool call. Session identity comes from the header record, because every transcript on disk is named session.jsonl.

Beyond transcripts, dsh keeps provider API keys in ~/.dsh/.credentials.yaml, registers MCP servers inside its Cordis plugin patch rather than a separate manifest, and reads skills from a shared ~/.agents root — which TRACE collects once dsh is confirmed present, but never treats as evidence that it is.

3. Secret hunting inside conversations

The most valuable artifact on an AI-enabled endpoint is the transcript, because it is the only place that records what a human told a model. It is also where credentials go to die: pasted into a prompt to “just fix the connection”, echoed back by the assistant in a code block, embedded in a tool-call argument.

TRACE hunts them specifically.

3.1 The detector

The shared secret detector carries 103 rules — provider keys, cloud credentials, VCS and registry tokens, webhooks, database DSNs, private keys — each with a cheap substring pre-filter, a per-rule entropy floor and a severity. Around it sit the heuristics that separate a useful detector from an alert cannon:

• Entropy gating per rule. A distinctive prefix (AKIA, ghp_, sk-proj-) is stronger evidence than any entropy measurement, so those rules bypass the gate. Generic rules must clear a threshold.
• Path confidence. The same string is more likely to be a live credential in ~/dev/ than in node_modules/. Thresholds shift accordingly.
• A context layer. Secrets with no distinctive shape are caught by the key name on an assignment line — client_secret, db_password, webhook_url — gated on entropy so that password = "changeme" stays quiet.
• Allowlists. Lockfiles, vendored trees, minified bundles, $VAR and {{ }} placeholders, format verbs and stopwords are excluded outright.

3.2 Redaction is not optional

The detector never emits a raw secret. Every match is reduced to a redacted preview and a salted fingerprint before it leaves the engine, which is what allows the same credential to be counted once across a case without ever being stored. A TRACE report can be attached to a ticket without becoming a second credential leak.

3.3 Direction and provenance

For conversation findings, TRACE records two things most scanners do not:

• Leak direction — did the credential travel user_to_model (someone pasted it into a prompt) or model_to_user (the assistant produced it)? These are different incidents with different responses.
• Evidence field — was it in the message content, in a recorded tool command, or in a structured tool-call argument? A key in a tool argument means something executed with it.

Takeaway 2 — A secret in a transcript is an incident with a timestamp. Unlike a key sitting in a config file, a pasted credential tells you exactly when it was exposed, to which vendor, and — through the surrounding turns — what was done with it next.

4. Forensic soundness: built for the courtroom, not just the SOC

TRACE is designed to produce evidence that survives scrutiny. The guarantees are explicit:

• Read-only collection — every collector reads without modifying the source. SQLite databases are opened in immutable mode. No source file is touched.
• SHA-256 per file — every collected file is hashed at collection time and the hash recorded in the manifest.
• Chain of custody — a CHAIN_OF_CUSTODY.json manifest records the tool version, collection timestamp, host and per-file hashes.
• UTC timestamps — all timestamps are ISO 8601 UTC, so evidence from different platforms and timezones correlates cleanly.
• Append-only — the tool has no deletion capability. The output package is additive by design.
• Minimal footprint — no agents installed, no registry modifications, no persistent processes, no network calls. TRACE leaves the endpoint as it found it.

The design principle is simple: the tool should be able to testify. If a finding ever reaches a courtroom, the examiner needs to be able to say exactly what was collected, when, from where, and that nothing was altered. TRACE’s manifest is built to support that statement.

One deliberate omission belongs here. TRACE does not verify credentials against their providers. Live verification would tell you whether a key is still valid — and would also transmit collected evidence to a third party from the examiner’s machine, mid-investigation. That trade is not worth making, so the detector stays offline.

5. The analysis engine: from files to findings

Collection is only half the job. The analysis engine turns raw artifacts into a structured, scored and mapped picture of the AI surface.

5.1 Unified timeline

Every collected artifact and parsed conversation is placed on a single timeline, so a responder can see the sequence of AI activity across platforms — what was installed, when a model was pulled, when a conversation happened, when a credential appeared. Collection events are explicitly marked, so the act of investigating can never be mistaken for user activity.

5.2 IOC extraction

The IOC extractor pulls indicators from conversations, configs and logs: IP addresses, URLs, domains, file paths, email addresses, suspicious command strings, MD5/SHA-1/SHA-256 hashes, credentials, and data-exfiltration patterns such as base64 encoding piped to a network destination.

Indicators seen on more than one platform are escalated automatically. Shared infrastructure across two AI tools is a stronger signal than the same string in one place.

5.3 MITRE ATLAS and ATT&CK mapping

Findings are mapped to MITRE ATLAS — the AI-specific threat framework — covering prompt injection (AML.T0010), LLM jailbreak (AML.T0011), modify model (AML.T0025), AI tool integration (AML.T0048), LLM data exfiltration (AML.T0050) and LLM credential theft (AML.T0055), among others. Those mappings then derive the MITRE ATT&CK techniques they imply, so an AI incident lands in the same language as the rest of the program.

5.4 Risk scoring

The risk scorer produces a 0–100 score across four categories, 25 points each:

The report expands this into an eight-category breakdown that adds network exposure, supply chain, data integrity and compliance. The score is a triage primitive — it tells a responder where to look first, not what to conclude. Section 8 shows exactly how much that distinction matters.

5.5 Kill chain, narratives and priority actions

The analyzer adds the responder-facing layer: a kill chain showing which of the seven stages the evidence supports, attack narratives that assemble corroborated findings into a story, cross-platform correlations, and priority actions ranked so the first five are the ones worth doing today.

6. Reporting: HTML, JSON, STIX 2.1

TRACE produces three report formats, each for a different consumer:

• HTML — a dark, self-contained forensic report across fifteen tabs: executive summary, attack surface map, findings, IOCs, timeline, MITRE, kill chain, actions, narratives, correlations, conversations, secret hunt, risk, evidence manifest and appendices. The charts render in pure CSS, so the report needs no JavaScript to read, has no CDN dependencies, opens offline and prints as-is.
• JSON — structured output for SIEM/SOAR ingestion and programmatic analysis.
• STIX 2.1 — a threat-intelligence bundle for sharing with MISP, OpenCTI and partner teams.

The STIX output is worth emphasising. AI-harness evidence is a new intelligence domain, and STIX 2.1 gives teams a standard way to share it — a finding about a jailbreak pattern or a leaked API key becomes an object other teams can consume and correlate.

For teams already running DFIR-IRIS, trace iris pushes the whole case in one command: the source host as an asset, every indicator as an IOC, the timeline as case events, findings as notes and priority actions as tasks.

7. Velociraptor artifact pack: fleet deployment

A single-endpoint CLI is useful; a fleet capability is what platform teams need. TRACE ships 15 Velociraptor artifacts for deployment across an estate from an existing Velociraptor server — covering inference engines, agents, dev tools, API keys, HuggingFace, network, processes, browser, Docker, code scanning and the shadow-AI sweep.

Three of them — TRACE.AI.Binary.Linux, TRACE.AI.Binary.macOS and TRACE.AI.Binary.Windows — fetch and run the Go binary on each endpoint and return the results. That is the whole fleet story: no new infrastructure, no agent to deploy, and the same evidence model as a hands-on collection.

8. Validation: a live end-to-end run

TRACE was run end to end against a live analyst workstation (macOS, arm64) on 15 August 2026 — a working developer machine with a heavy AI footprint, which is precisely the profile the tool exists for.

Where the evidence concentrated: the shadow-AI sweep (485 artifacts), Cursor (356), Antigravity (275), Claude Code (68), Hermes (30), Ollama (18), LM Studio (11), then Docker AI, HuggingFace, the code scanner, llama.cpp and Unsloth.

The indicator distribution is a useful picture of what an AI surface actually contains:

The secret hunt found 621 findings across 150 turns, 254 of them unique — 30 critical and 591 high — with 618 travelling user_to_model. That ratio is the story of the modern workstation: credentials are not leaking out of the model, they are being pasted into it.

The kill chain resolved 5 of 7 stages, and MITRE mapping produced 10 ATT&CK techniques led by T1071 (application layer protocol), T1105 (ingress tool transfer) and T1552 (unsecured credentials).

8.1 Reading the score honestly

The overall risk score came back 100/100 (critical), with all four categories maxed out. That number needs a caveat, and withholding it would be the easy dishonesty here.

This is a developer workstation running a dozen AI tools daily. The scoring model was designed for endpoints where credentials, outbound AI traffic, tool execution and prompt-pattern hits are exceptional. On a machine where all four are routine, every category saturates — and a saturated model stops discriminating. A 100 here means “this endpoint has a very large AI surface”, not “this endpoint is compromised”.

Likewise, the 1,767 conversation findings are pattern matches, not confirmed attacks. A developer discussing a jailbreak, writing a prompt-injection test, or asking an assistant to read a .env file trips the same patterns as a genuine incident. Findings are leads for an examiner, not verdicts.

Takeaway 3 — A saturating score is a finding about your model, not just your endpoint. Baseline the score per machine class before treating any absolute value as meaningful. What is worth watching is the delta from a known-good baseline for that role, and the specific findings underneath it — a credential fingerprint appearing for the first time is worth more than the score that contains it.

That is also why the report leads with prioritized actions rather than the gauge. On this run the top three were: rotate the exposed credentials, investigate the exfiltration indicators, and harden the guardrails around the tools showing jailbreak-pattern turns.

9. What this means for a DFIR team

TRACE is not a replacement for a full incident-response platform. It is a purpose-built acquisition and analysis layer for a class of evidence that standard tooling ignores. Three ways to use it:

1. Shadow-AI inventory. Run the sweep across the fleet via Velociraptor to answer “what AI is installed where”. This is the prerequisite for every AI governance decision.
2. Incident scoping. When an AI tool is implicated — a leaked key, a jailbroken assistant, an agent that went somewhere it should not — TRACE collects the transcripts, config and credentials that tell the story, with the custody record to back them.
3. Baseline and trend. Run it per machine class quarterly. Watch the categories move, and treat a new credential fingerprint or a newly detected kill-chain stage as the signal.

Takeaway 4 — The evidence window for AI artifacts is short, and it is closing. Conversations rotate, caches clear, and ephemeral agents tear down. The tool that collects AI evidence has to be run before the incident, or the artifacts are gone. TRACE is designed to be run as a routine sweep, not just a response tool.

10. Get started

IONSEC TRACE is open source under AGPL-3.0-or-later, and we want the community to extend it. The collector surface is deliberately modular — adding a platform means implementing a collector against a documented template, on both sides of the catalog.

# Python CLI
pip install ionsec-trace
trace discover
trace collect -o /evidence --deep
trace analyze /evidence --secret-hunt
trace report /evidence --format all

# Or the single Go binary — no runtime required
./trace run -o /evidence

• Site: ionsec.github.io/trace — capabilities, deployment and a walkthrough of the report.
• Repository: github.com/ionsec/trace, with prebuilt binaries for macOS, Linux and Windows on the releases page.
• Documentation: installation guide, user guide and end-to-end walkthrough, plus an artifact reference documenting the paths and evidence value of every platform TRACE targets.

Leave no model untraced. If you are running AI on your endpoints — and you almost certainly are — TRACE is the tool that tells you what is there, what it did, and what it left behind.

If a sweep turns up something you cannot explain, talk to the IONSEC DFIR team. We built TRACE for our own investigations, and we run those investigations for other people too.

References

1. IONSEC — The Machine Has No Disk: Forensic Readiness for AI Agent Runtimes, 4 August 2026. https://www.ionsec.io/blog/ai-agent-runtime-forensics