evaluations
Creates, updates, deletes, gets or lists an evaluations resource.
Overview​
| Name | evaluations |
| Type | Resource |
| Id | typesafe.systemone.evaluations |
Fields​
The following fields are returned by SELECT queries:
- evaluate
| Name | Datatype | Description |
|---|---|---|
answers | object | Answers keyed by the question names supplied in the request. Each answer's type matches its question's type. (title: Answers) |
model | string | Name of the model that answered the questions. May differ from the alias supplied in the request. (title: Model) |
usage | object | Input and output token counts for this evaluation. (title: Usage) |
Methods​
The following methods are available for this resource:
| Name | Accessible by | Required Params | Optional Params | Description |
|---|---|---|---|---|
evaluate | select | model, questions, state | Answer one or more questions about the content supplied in state.You can mix question types in one request. Answers use the same names as the questions, so you can match each result to its question. The response also includes the model used and token usage. |
Parameters​
Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.
| Name | Datatype | Description |
|---|---|---|
model | string | Name or alias of the model to use. Available names are returned by GET /v1/models. |
questions | object | Questions to ask about the content, each with a name you choose. The response uses those names to identify the answers. |
state | string | The content all questions in this request refer to. |
SELECT examples​
- evaluate
Answer one or more questions about the content supplied in state.
You can mix question types in one request. Answers use the same names as the
questions, so you can match each result to its question. The response also includes
the model used and token usage.
SELECT
answers,
model,
usage
FROM typesafe.systemone.evaluations
WHERE model = '{{ model }}' -- required
AND questions = '{{ questions }}' -- required
AND state = '{{ state }}' -- required
;