Skip to main content

evaluations

Creates, updates, deletes, gets or lists an evaluations resource.

Overview​

Nameevaluations
TypeResource
Idtypesafe.systemone.evaluations

Fields​

The following fields are returned by SELECT queries:

NameDatatypeDescription
answersobjectAnswers keyed by the question names supplied in the request. Each answer's type matches its question's type. (title: Answers)
modelstringName of the model that answered the questions. May differ from the alias supplied in the request. (title: Model)
usageobjectInput and output token counts for this evaluation. (title: Usage)

Methods​

The following methods are available for this resource:

NameAccessible byRequired ParamsOptional ParamsDescription
evaluateselectmodel, questions, stateAnswer one or more questions about the content supplied in state.

You can mix question types in one request. Answers use the same names as the
questions, so you can match each result to its question. The response also includes
the model used and token usage.

Parameters​

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

NameDatatypeDescription
modelstringName or alias of the model to use. Available names are returned by GET /v1/models.
questionsobjectQuestions to ask about the content, each with a name you choose. The response uses those names to identify the answers.
statestringThe content all questions in this request refer to.

SELECT examples​

Answer one or more questions about the content supplied in state.

You can mix question types in one request. Answers use the same names as the
questions, so you can match each result to its question. The response also includes
the model used and token usage.

SELECT
answers,
model,
usage
FROM typesafe.systemone.evaluations
WHERE model = '{{ model }}' -- required
AND questions = '{{ questions }}' -- required
AND state = '{{ state }}' -- required
;