Preview a bounded evaluator draft
POST /api/v1/public/evaluators/preview
POST /api/v1/public/evaluators/previewCreate or trigger evaluators preview.
Operation ID: post_api_v1_public_evaluators_preview.
Authentication
Choose a credential. Each row is an accepted alternative; access also depends on current roles, project bindings, and entitlements.
| Credential | Carrier | OAuth scopes |
|---|---|---|
OAuth2 | Authorization: Bearer | evaluation:write |
For OAuth and service-account credentials, send the selected project UUID in x-project-id, including when the header is optional for legacy keys.
Availability
This operation requires the tier-c-compute capability to be enabled for the deployment. Its presence in the contract does not guarantee availability. See operation availability.
Request parameters
| Name | Location | Required | Type / allowed values | Description / constraints |
|---|---|---|---|---|
x-project-id | header | Yes | string | x-project-id header parameter; Required service-account or OAuth project selection; format: uuid; Pattern constrained; see the downloadable schema. |
Request body
Body required: yes.
application/json
| Field | Type / allowed values | Required in parent | Description / constraints |
|---|---|---|---|
$ | object | Yes | additionalProperties: false |
definition | object | Yes | additionalProperties: false |
definition.maxBudgetUsd | number | null | No | — |
definition.maxBudgetUsd (anyOf 1) | number | Yes | minimum: 0; maximum: 1 |
definition.maxToolCalls | integer | No | default: 12; minimum: 1; maximum: 12 |
definition.modelName | string | null | No | — |
definition.modelName (anyOf 1) | string | Yes | maxLength: 120 |
definition.modelProvider | string | null | No | — |
definition.modelProvider (anyOf 1) | string | Yes | maxLength: 40 |
definition.outputSpec | array<object> | Yes | maxItems: 50 |
definition.outputSpec[] | object | Yes | additionalProperties: false |
definition.outputSpec[].key | string | Yes | minLength: 1; maxLength: 80; Pattern constrained; see the downloadable schema. |
definition.outputSpec[].label | string | Yes | minLength: 1; maxLength: 200 |
definition.outputSpec[].answerType | string | No | maxLength: 40 |
definition.outputSpec[].scoreType | "boolean" | "numeric" | "categorical" | "text" | No | — |
definition.outputSpec[].options | array<string> | No | maxItems: 50 |
definition.outputSpec[].options[] | string | Yes | maxLength: 200 |
definition.outputSpec[].required | boolean | No | — |
definition.reasoningEnabled | boolean | No | default: false |
definition.systemPrompt | string | No | default: ``; maxLength: 20000 |
definition.toolGrants | array<object> | No | maxItems: 50 |
definition.toolGrants[] | object | Yes | additionalProperties: false |
definition.toolGrants[].config | object | No | — |
definition.toolGrants[].config.* | any | No | — |
definition.toolGrants[].provider | "neatlogs_mcp" | "composio" | "http_get" | "custom_mcp" | Yes | — |
definition.toolGrants[].toolName | string | null | No | — |
definition.toolGrants[].toolName (anyOf 1) | string | Yes | minLength: 1; maxLength: 160 |
definition.type | "llm_judge" | "agent_judge" | No | default: llm_judge |
item | object | Yes | additionalProperties: false |
item.attachmentIds | array<string> | No | maxItems: 5 |
item.attachmentIds[] | string | Yes | format: uuid; Pattern constrained; see the downloadable schema. |
item.inputText | string | null | No | — |
item.inputText (anyOf 1) | string | Yes | — |
item.outputText | string | null | No | — |
item.outputText (anyOf 1) | string | Yes | — |
item.spanId | string | null | No | — |
item.spanId (anyOf 1) | string | Yes | minLength: 1; maxLength: 64 |
item.traceId | string | No | default: ``; maxLength: 255 |
skillVersionIds | array<string> | No | maxItems: 10 |
skillVersionIds[] | string | Yes | format: uuid; Pattern constrained; see the downloadable schema. |
Request example
Replace placeholder IDs and environment variables with values from your authorized project. Credentials are expanded into curl's stdin configuration, not its command arguments. Keep shell tracing off and do not log this configuration. For requests with a body, put a payload matching the schema in request-body.txt; form requests use URL-encoded content.
curl --fail-with-body --silent --show-error --config - <<CURL_CONFIG
request = "POST"
header = "Authorization: Bearer ${NEATLOGS_TOKEN}"
header = "x-project-id: ${NEATLOGS_PROJECT_ID}"
header = "Content-Type: application/json"
data-binary = "@request-body.txt"
url = "https://app.neatlogs.com/api/v1/public/evaluators/preview"
CURL_CONFIGResponses
| Status | Description | Content type |
|---|---|---|
200 | Evaluator preview verdict | application/json |
400 | Invalid public API request | application/problem+json |
401 | Missing or invalid public API credential | application/problem+json |
403 | Public API request is not authorized | application/problem+json |
404 | Evaluator input not found | application/problem+json |
409 | Public API request conflict | application/problem+json |
413 | Public API request body too large | application/problem+json |
415 | Unsupported public API request body | application/problem+json |
422 | Evaluator execution is unavailable or too large | application/problem+json |
429 | Public API rate limit exceeded | application/problem+json |
500 | Internal server error | application/problem+json |
503 | Public API credential, project context, or rate limiting unavailable. Backend is draining; retry after the Retry-After delay. | application/problem+json, application/json |
200 response fields
Content type: application/json.
| Field | Type / allowed values | Required in parent | Description / constraints |
|---|---|---|---|
$ | object | Yes | additionalProperties: false |
data | object | Yes | additionalProperties: false |
data.error | string | null | Yes | — |
data.error (anyOf 1) | string | Yes | maxLength: 2000 |
data.fields | array<object> | Yes | maxItems: 50 |
data.fields[] | object | Yes | additionalProperties: false |
data.fields[].confidence | number | null | No | — |
data.fields[].confidence (anyOf 1) | number | Yes | minimum: 0; maximum: 1 |
data.fields[].key | string | Yes | minLength: 1; maxLength: 80 |
data.fields[].score | number | null | No | — |
data.fields[].score (anyOf 1) | number | Yes | minimum: 0; maximum: 1 |
data.fields[].value | any | Yes | — |
data.overall | string | null | Yes | — |
data.overall (anyOf 1) | string | Yes | maxLength: 20000 |
data.overallScore | number | null | Yes | — |
data.overallScore (anyOf 1) | number | Yes | minimum: 0; maximum: 1 |
data.status | "failed" | "succeeded" | "unknown" | Yes | — |
data.toolCalls | integer | Yes | minimum: 0; maximum: 12 |
requestId | string | Yes | Pattern constrained; see the downloadable schema. |
success | true | Yes | — |
Response headers
| Header | Description / constraints |
|---|---|
RateLimit-Limit | Effective request ceiling for the current window; minimum: 1 |
RateLimit-Remaining | Requests remaining in the current window; minimum: 0 |
RateLimit-Reset | Unix timestamp in seconds when the current window resets; minimum: 0 |
X-NeatLogs-RateLimit-Degraded | Whether bounded emergency enforcement is active; — |
X-NeatLogs-RateLimit-Limit | Effective request ceiling for the current window; minimum: 1 |
X-NeatLogs-RateLimit-Policy | Applied pre-authentication or authenticated rate class; — |
X-NeatLogs-RateLimit-Remaining | Requests remaining in the current window; minimum: 0 |
X-NeatLogs-RateLimit-Reset-After | Whole seconds until the current window resets; minimum: 1 |
X-NeatLogs-RateLimit-Version | NeatLogs rate-limit contract version; — |
X-Request-Id | Server-owned request correlation identifier; Pattern constrained; see the downloadable schema. |
Failures use Problem Details. For exact validation patterns and all response schemas, download the OpenAPI specification.