Start with a failure
Turn one wrong production outcome into an evidence-backed investigation report.
The fastest way to understand Neatlogs is to use a real failure. You will anchor the report to the exact run, explain the expected behavior, investigate the evidence, and decide whether the result is strong enough to hand off.
Time: about 10 minutes once the affected run is available.
Before you start
You need a project with traces and one outcome that a customer, reviewer, or teammate considers wrong. Native and imported traces both work; the investigation can only use fields that the source captured.
Write down these three facts before opening Neatlogs:
| Fact | Example |
|---|---|
| What happened | The assistant quoted a policy that expired last month. |
| What should have happened | It should use the policy effective on the request date. |
| Why it matters | The customer was offered an invalid refund option. |
That distinction matters. “The retrieval code is broken” is a theory; “the answer used an expired policy” is an observable failure.
Investigate the run
Find the affected run. Open Traces, select the project and time range, then search by workflow, end user, session, error, or distinctive output. Open the trace and confirm its first input and final output match the report.
Locate the failing evidence. Follow the execution from the final output backward. Inspect the model response, tool result, or retrieved document that first made the outcome inevitable. Do not assume the last span is the cause.
Flag the problem. Attach the report to the smallest useful evidence: open Comments on the trace for an end-to-end failure, or use the span's Comment action for one model or tool decision. State the observed result, expected result, and impact.
Start the investigation. If the run already belongs to an incident, open it from Incidents, review its linked evidence, and select Investigate (Investigate again if it was investigated before). Otherwise select Ask neat AI on the trace and ask it to investigate this run. The investigation runs in the background and produces a report with a root cause, evidence, a recommended action with verification steps, and a confidence score.
Answer only questions that require human context. If neat AI pauses with a question about a policy, customer constraint, or missing permission, choose or type the smallest factual answer, or skip a question you can't answer. A clarification is safer than an invented assumption.
Review the result. Open every important citation. Confirm that the root cause explains the symptom, the action addresses that cause, and the verification plan would fail if the problem still existed.
An investigation should make its evidence and intermediate work inspectable, not return an unsupported answer.
Use a precise investigation request
The incident or trace you start from provides context, but a precise request keeps the analysis focused.
Investigate why this run answered from an expired policy.
Expected: select the policy version effective on the request date.
Impact: the customer received an invalid refund option.
Compare with successful refund runs from the same workflow in the last 7 days.Avoid requests such as Why is this bad? or Fix the RAG. The first omits the standard; the second assumes a cause and a solution before the evidence has been read.
Decide whether the report is ready
Use this review gate before creating a ticket or coding-agent handoff:
- Root cause: names the earliest supported failure, not just the visible symptom.
- Evidence: links to spans, traces, or connected sources that directly support the conclusion.
- Uncertainty: distinguishes observed facts from plausible inference.
- Scope: explains whether the behavior is isolated or appears in related runs.
- Action: is narrow enough for an implementer to act on without replaying the investigation.
- Verification: names concrete cases and judgments that could disprove the fix.
Do not hand off a change when the conclusion depends on one ambiguous output, a missing policy, or an uncited assumption. Add a successful comparison run or authoritative business context, then investigate again (Investigate again on the incident).
If you cannot find the run
Use Search across production when the report describes a behavior but does not include a trace ID. Search by customer, workflow, tool, time range, or plain-English behavior, then inspect candidate runs before starting the investigation.
Frequently asked questions
Do I need a detection or Eval first?
No. Start from a trace. Create a detection when the failure can be recognized as a repeatable condition; create an Eval when it requires a repeatable quality judgment.
Can I investigate an imported trace?
Yes. Results depend on the structure and fields included by the source platform.
What if several causes remain possible?
Keep them as hypotheses. Ask which evidence would distinguish them and gather that evidence before choosing an action.
Does Neatlogs change my code?
No. Investigation produces a reviewable report. A person separately chooses whether to dispatch work, approve code, and deploy it.