List

List Evaluation Test Cases

agents.evaluation_test_cases.list() -> evaluation_test_caseslistEvaluationTestCaseListResponse

get/v2/gen-ai/evaluation_test_cases

To list all evaluation test cases, send a GET request to /v2/gen-ai/evaluation_test_cases.

Returns

EvaluationTestCaseListResponseclass

Hide ParametersShow Parameters

evaluation_test_caseslist

optional

Optional[List[archived_atdatetimecreated_atdatetimecreated_by_user_emailstrcreated_by_user_idstrdatasetDatasetdataset_namestrdataset_uuidstrdescriptionstrlatest_version_number_of_runsintmetricslistnamestrstar_metricAPIStarMetrictest_case_uuidstrtotal_runsintupdated_atdatetimeupdated_by_user_emailstrupdated_by_user_idstrversionintAPIEvaluationTestCase]]

Alternative way of authentication for internal usage only - should not be exposed to public api

Hide ParametersShow Parameters

archived_atdatetime

optional

formatdate-time

created_atdatetime

optional

formatdate-time

created_by_user_emailstr

optional

created_by_user_idstr

optional

formatuint64

datasetcreated_atdatetimedataset_namestrdataset_uuidstrfile_sizestrhas_ground_truthboolrow_countintDataset

optional

Hide ParametersShow Parameters

created_atdatetime

optional

Time created at.

formatdate-time

dataset_namestr

optional

Name of the dataset.

dataset_uuidstr

optional

UUID of the dataset.

file_sizestr

optional

The size of the dataset uploaded file in bytes.

formatuint64

has_ground_truthbool

optional

Does the dataset have a ground truth column?

row_countint

optional

Number of rows in the dataset.

formatint64

dataset_namestr

optional

dataset_uuidstr

optional

descriptionstr

optional

latest_version_number_of_runsint

optional

formatint32

metricslist

optional

Optional[List[descriptionstrinvertedboolmetric_namestrmetric_typeliteralmetric_uuidstrmetric_value_typeliteralrange_maxfloatrange_minfloatAPIEvaluationMetric]]

Hide ParametersShow Parameters

descriptionstr

optional

invertedbool

optional

If true, the metric is inverted, meaning that a lower value is better.

metric_namestr

optional

metric_typeliteral

optional

Optional[Literal["METRIC_TYPE_UNSPECIFIED", "METRIC_TYPE_GENERAL_QUALITY", "METRIC_TYPE_RAG_AND_TOOL"]]

Hide ParametersShow Parameters

"METRIC_TYPE_UNSPECIFIED"

"METRIC_TYPE_GENERAL_QUALITY"

"METRIC_TYPE_RAG_AND_TOOL"

metric_uuidstr

optional

metric_value_typeliteral

optional

Optional[Literal["METRIC_VALUE_TYPE_UNSPECIFIED", "METRIC_VALUE_TYPE_NUMBER", "METRIC_VALUE_TYPE_STRING", "METRIC_VALUE_TYPE_PERCENTAGE"]]

Hide ParametersShow Parameters

"METRIC_VALUE_TYPE_UNSPECIFIED"

"METRIC_VALUE_TYPE_NUMBER"

"METRIC_VALUE_TYPE_STRING"

"METRIC_VALUE_TYPE_PERCENTAGE"

range_maxfloat

optional

The maximum value for the metric.

formatfloat

range_minfloat

optional

The minimum value for the metric.

formatfloat

namestr

optional

star_metricAPIStarMetric

optional

test_case_uuidstr

optional

total_runsint

optional

formatint32

updated_atdatetime

optional

formatdate-time

updated_by_user_emailstr

optional

updated_by_user_idstr

optional

formatuint64

versionint

optional

formatint64

HTTPPython

from do_gradientai import GradientAI

client = GradientAI()
evaluation_test_cases = client.agents.evaluation_test_cases.list()
print(evaluation_test_cases.evaluation_test_cases)

200 Example

{
  "evaluation_test_cases": [
    {
      "archived_at": "2023-01-01T00:00:00Z",
      "created_at": "2023-01-01T00:00:00Z",
      "created_by_user_email": "[email protected]",
      "created_by_user_id": "\"12345\"",
      "dataset": {
        "created_at": "2023-01-01T00:00:00Z",
        "dataset_name": "\"example name\"",
        "dataset_uuid": "\"123e4567-e89b-12d3-a456-426614174000\"",
        "file_size": "\"12345\"",
        "has_ground_truth": true,
        "row_count": 123
      },
      "dataset_name": "\"example name\"",
      "dataset_uuid": "\"123e4567-e89b-12d3-a456-426614174000\"",
      "description": "\"example string\"",
      "latest_version_number_of_runs": 123,
      "metrics": [
        {
          "description": "\"example string\"",
          "inverted": true,
          "metric_name": "\"example name\"",
          "metric_type": "METRIC_TYPE_UNSPECIFIED",
          "metric_uuid": "\"123e4567-e89b-12d3-a456-426614174000\"",
          "metric_value_type": "METRIC_VALUE_TYPE_UNSPECIFIED",
          "range_max": 123,
          "range_min": 123
        }
      ],
      "name": "\"example name\"",
      "star_metric": {
        "metric_uuid": "\"123e4567-e89b-12d3-a456-426614174000\"",
        "name": "\"example name\"",
        "success_threshold": 123,
        "success_threshold_pct": 123
      },
      "test_case_uuid": "\"123e4567-e89b-12d3-a456-426614174000\"",
      "total_runs": 123,
      "updated_at": "2023-01-01T00:00:00Z",
      "updated_by_user_email": "[email protected]",
      "updated_by_user_id": "\"12345\"",
      "version": 123
    }
  ]
}