Detections

Detections are rules that run against your traces and flag spans that match. They appear as annotations on top of your traces in the dashboard — so you can understand what happened in a run at a glance, without opening individual spans.

The key benefit: if you've created detections, you can scan your entire traces list and immediately see which runs had high token usage, which outputs contained PII, which agent decisions matched a pattern you care about — all without digging into the trace itself. Detections give you a summary layer on top of your data.

---

How detections work

You define a detection rule — one of four types: regex, condition, PII detection, or classifier. Neatlogs evaluates it against spans as they come in. When a span matches, the detection appears as a badge on that trace in the traces list and inside the span detail panel.

A single trace can trigger multiple detections. Each one is labelled with the rule that triggered it, so you can filter the traces list by detection type.

---

Types of detections

Regex

Match against the text content of a span's input or output using a regular expression.

Use this when you're looking for a specific string pattern in what your agent produced — an action label, a structured output format, an error message, or a keyword that signals something worth reviewing.

Examples:

• Flag any trace where the agent output contains TASK: — useful if your agent assigns tasks and you want to track every run that produced one
• Detect when a tool response contains "status": "failed"
• Match a specific error string coming back from an API call

Condition

Flag spans based on numeric or boolean attributes — token counts, latency, cost, or any custom attribute you've set on a span.

Use this to surface operational signals: runs that were unusually expensive, slow, or that hit a threshold you care about.

Examples:

• total_tokens > 8000 — flag runs where the model consumed significantly more tokens than usual
• latency_ms > 5000 — catch responses that took longer than 5 seconds
• tool_call_count > 10 — detect agents that looped more than expected
• cost_usd > 0.10 — flag any single run that exceeded a cost threshold

PII detection

A built-in check that flags spans where the model input or output contains personally identifiable information: names, email addresses, phone numbers, financial data, and similar. No configuration needed — enable it and Neatlogs scans span content automatically.

Use this when your agent handles user-submitted text and you need visibility into whether PII is flowing through prompts or responses.

Classifier

Run a custom semantic check against span content. Classifiers go beyond exact pattern matching — they evaluate the meaning or category of the text rather than its literal value.

Examples:

• Flag traces where the agent output is off-topic or out of scope
• Detect when a retriever returned documents that don't match the query semantically
• Classify agent outputs by sentiment to surface negative user interactions
• Flag any span where the model response falls into a category you define

---

What you see in the dashboard

Detections appear in two places:

1. Traces list — each trace shows its active detections as labelled badges. You can filter the entire list by detection type, so "show me all runs where high token usage was detected" is a single click.

2. Span detail — inside a trace, the specific span that triggered the detection is marked. You can see exactly which step caused the flag and what the matched content was.

---

Example: meeting task agent

An agent that reads a meeting transcript and assigns tasks to team members might have:

• A regex detection matching ASSIGNED TO: to track every trace that produced a task assignment
• A condition on total_tokens > 6000 to flag unusually long transcripts that drove up cost
• A PII detection to catch any run where participant names or email addresses leaked into the output
• A classifier to flag runs where the assigned tasks were off-topic or outside the agent's expected scope

Without opening a single trace, you can see: how many runs produced assignments today, which ones were expensive, whether any output contained PII, and whether the agent stayed on task — just from the traces list.