Investigations, fix briefs, and verification

An investigation, a fix brief, and verification answer three different questions:

Keeping these stages separate prevents a plausible explanation from silently becoming an approved code change. It also prevents a merged change from being mistaken for a verified improvement.

Follow one failure from report to verification

Suppose a refund assistant cites an expired policy.

1. A teammate opens the affected trace and identifies the incorrect sentence.
2. They state the expected behavior: use the policy active on the date of the request.
3. An investigation reads the failed run and searches for useful comparison evidence.
4. It finds that successful runs include the request date in retrieval, while failed runs omit it.
5. The report cites that divergence and recommends passing the date into the retrieval filter.
6. A person checks the evidence and sends a focused brief through the team's normal implementation workflow, for example with Fix in or Push to on the incident.
7. The original failure and nearby valid cases are tested.
8. New production traces are monitored for the same detection or failed Eval.

Each stage narrows uncertainty, but no stage makes the next one automatic.

What starts an investigation

An investigation always anchors on evidence Neatlogs can read: one or more traces, or an incident that groups them. You start one by:

• selecting Investigate further on an incident;
• asking Neat AI to investigate a run or an incident in investigate, or through Ask neat AI on a trace.

A detection match, a failed Eval, a customer or teammate report, or a production search result is a good reason to start. Point the investigation at the affected runs, or at the incident that groups them.

Start with the observed problem and expected behavior. “The answer used a policy that had expired before the request date” gives the investigation something testable. “The agent is bad at refunds” is too broad to support a focused cause.

Add business context that changes how evidence should be interpreted, such as the applicable policy, customer impact, known constraints, or a previous team decision.

What an investigation can read

The investigation begins with scoped evidence available in Neatlogs:

• the anchor traces and their spans, including prompts, outputs, tool calls, and retrieval;
• similar runs, and the workflow's error, latency, and token metrics compared with the previous 7 days;
• detection matches and Eval results for those runs;
• user feedback signals and earlier incidents in the project;
• read-only data from apps you have connected, such as the repository file behind a failing step.

Project, environment, workflow, customer, and time-range boundaries affect what comparisons are valid. Broad access can add noise, while a scope that is too narrow can hide the useful contrast.

Compare failed and successful behavior

A failed run shows where the symptom occurred. A successful comparison helps distinguish a likely cause from a fact that appears in every run.

Failed run
request date → retrieval without date filter → expired policy → wrong answer

Successful run
request date → retrieval with date filter → current policy → correct answer
                               ↑
                 earliest supported divergence

Useful comparison dimensions include:

• input and customer context;
• workflow, prompt, and model version;
• retrieved documents and ranking;
• tool arguments, results, retries, and errors;
• span order and missing steps;
• environment and deployment timing;
• final output and Eval verdicts.

The earliest meaningful divergence is often more useful than the last visible error. A wrong final answer may have started with missing context several spans earlier.

Correlation is not enough. If all successful and failed runs use the same prompt version, that version does not explain the difference on its own.

When a person must supply context

An investigation runs in the background and does not stop to ask questions. It works from the evidence it can read and reports its confidence. In chat, Neat AI asks one short question first only when it cannot tell what to investigate.

Some facts only your team knows, and they can change the conclusion:

• Which policy applied on the request date?
• Was the tool allowed to retry in this workflow?
• Is this customer-specific behavior expected?
• Which of two conflicting outputs is authoritative?

Supply the narrowest useful context or a source. Include it when you ask for the investigation in chat, or after a report, select Give feedback on the incident to open the report chat, where you can correct the report or run the investigation again. Missing context should lower confidence, not invite a guess.

Possible investigation outcomes

Not every investigation ends with a confident root cause.

Each run appears on the incident's Investigations tab, numbered when there is more than one. No cause found is a valid result. It is different from an investigation that was never run and safer than an unsupported explanation.

What the report should contain

A Neatlogs investigation report has these parts. You can edit the report on the incident, and edits are recorded on the Activity tab.

Root cause

A concise statement of the earliest supported cause, at the level where a change can be considered.

Evidence & reasoning

Citations to traces, spans, results, or connected sources, plus the reasoning that links those facts to the conclusion.

Confidence

A score with the factors behind it, such as evidence strength, pattern consistency, and detection alignment. Read it alongside what evidence is missing and which alternative explanations remain plausible.

Recommended action

A bounded direction that addresses the supported cause: ordered steps, how to verify the fix worked, what could go wrong, and how to roll back. It should identify relevant constraints and avoid expanding into unrelated cleanup.

A report should let a reviewer move from each claim to its evidence. A detailed paragraph without inspectable grounding is not a strong investigation.

Review before handoff

The reviewer should:

1. Confirm that the reported problem and expected behavior are correct.
2. Open the strongest citations.
3. Check that comparisons use similar workflows, environments, and time periods.
4. Look for alternative explanations the report may have missed.
5. Confirm that the recommended action addresses the supported cause.
6. Check that the verification plan can fail, not merely confirm the recommendation.

Correct the context or reasoning when needed. Rerun the investigation if new evidence could materially change the conclusion.

What belongs in a fix brief

A fix brief is the focused handoff from investigation to implementation. It should contain enough context to make the intended change unambiguous without copying the full investigation transcript.

In the product, this material is the incident's Recommended action and the context Neatlogs sends with a handoff, rather than a separate object named “fix brief.”

What happens before anything changes

An investigation does not edit code. Tracking or resolving an incident does not deploy a fix.

A person chooses whether to proceed and picks a destination on the incident:

• Fix in opens the work in a coding tool such as Cursor, Codex, or Claude Code, or sends it to a Cursor cloud agent.
• Push to creates a Linear or Jira ticket in a connected workspace.
• Copy context copies the incident's context to paste anywhere.

Fix in and Push to appear once an investigation has produced a report.

Repository permissions, branch protections, code review, CI, deployment approvals, and rollback remain part of the team's normal controls.

The destination confirms that a handoff was received. It does not prove the resulting implementation is correct.

Verification starts before implementation

Write the verification plan while the failure is still understood. A plan created only after the change can easily be shaped to whatever the implementation already does.

Use three layers:

1. Reproduce the known failure

Run the exact affected cases or the closest reproducible equivalents. The previously wrong behavior should now meet the expected standard.

2. Protect nearby behavior

Test successful and boundary cases that should remain unchanged. For the refund example, check current policies, missing request dates, ineligible items, and tool failures.

3. Watch production evidence

Apply the relevant Evals, monitor the original detection or failure pattern, and inspect new traces after release. Compare like periods and account for traffic changes. Neatlogs has no verified status: when the evidence is sufficient, mark the incident Resolved.

When to reopen an incident

Reopen or continue the incident when:

• the same failure returns;
• the detection or failed Eval rises again;
• the implementation passed tests but production behavior did not improve;
• a nearby regression appears;
• new evidence invalidates the original root cause.

Do not rely on a resolved incident reopening on its own. Change its status back to Open from the Status menu. Recurrence is useful evidence: keep it on the original incident when it represents the same underlying problem, and track a separate incident when the cause or impact differs.

Frequently asked questions

Is an investigation report a guarantee?

No. It is a cited analysis based on available evidence. Review it more carefully when evidence is sparse, contradictory, or high impact.

Does tracking or resolving an incident change code?

No. A handoff through Fix in or Push to is a separate action, followed by the team's normal implementation and review process.

Can I add evidence after an investigation completes?

Yes. Use Give feedback to add context in the report chat, or select Investigate further to run a new investigation. Each run is kept on the Investigations tab, so you can compare the revised conclusion with the earlier one.

What if no clear cause is found?

Preserve the evidence and uncertainty. Add the missing context, wait for recurrence, improve instrumentation, or narrow the question before trying again.

Is verification automatic?

No. Neatlogs can collect signals and run configured Evals, and each report includes verification steps, but nothing marks a fix as verified. Your team decides which cases, duration, and production evidence are sufficient, then resolves the incident.

Next step

Investigate a failure →