Update Data Source options

knowledge_bases.data_sources.update(, ) -> DataSourceUpdateResponse

put/v2/gen-ai/knowledge_bases/{knowledge_base_uuid}/data_sources/{data_source_uuid}

To update a data source (e.g. chunking options), send a PUT request to /v2/gen-ai/knowledge_bases/{knowledge_base_uuid}/data_sources/{data_source_uuid}.

ParametersExpand Collapse

knowledge_base_uuid: str

data_source_uuid: str

chunking_algorithm: Optional[Literal["CHUNKING_ALGORITHM_UNKNOWN", "CHUNKING_ALGORITHM_SECTION_BASED", "CHUNKING_ALGORITHM_HIERARCHICAL", 2 more]]

The chunking algorithm to use for processing data sources.

Note: This feature requires enabling the knowledgebase enhancements feature preview flag.

Accepts one of the following:

"CHUNKING_ALGORITHM_UNKNOWN"

"CHUNKING_ALGORITHM_SECTION_BASED"

"CHUNKING_ALGORITHM_HIERARCHICAL"

"CHUNKING_ALGORITHM_SEMANTIC"

"CHUNKING_ALGORITHM_FIXED_LENGTH"

chunking_options: Optional[ChunkingOptions]

Configuration options for the chunking algorithm.

Note: This feature requires enabling the knowledgebase enhancements feature preview flag.

child_chunk_size: Optional[int]

Hierarchical options

formatint64

max_chunk_size: Optional[int]

Section_Based and Fixed_Length options

formatint64

parent_chunk_size: Optional[int]

Hierarchical options

formatint64

semantic_threshold: Optional[float]

Semantic options

formatfloat

data_source_uuid: str

knowledge_base_uuid: str

ReturnsExpand Collapse

class DataSourceUpdateResponse: …

Update a data source of a knowledge base with change in chunking algorithm/options

knowledge_base_data_source: Optional[APIKnowledgeBaseDataSource]

Data Source configuration for Knowledge Bases

aws_data_source: Optional[AwsDataSource]

AWS S3 Data Source for Display

bucket_name: Optional[str]

Spaces bucket name

item_path: Optional[str]

region: Optional[str]

Region of bucket

bucket_name: Optional[str]

Name of storage bucket - Deprecated, moved to data_source_details

chunking_algorithm: Optional[Literal["CHUNKING_ALGORITHM_UNKNOWN", "CHUNKING_ALGORITHM_SECTION_BASED", "CHUNKING_ALGORITHM_HIERARCHICAL", 2 more]]

The chunking algorithm to use for processing data sources.

Note: This feature requires enabling the knowledgebase enhancements feature preview flag.

Accepts one of the following:

"CHUNKING_ALGORITHM_UNKNOWN"

"CHUNKING_ALGORITHM_SECTION_BASED"

"CHUNKING_ALGORITHM_HIERARCHICAL"

"CHUNKING_ALGORITHM_SEMANTIC"

"CHUNKING_ALGORITHM_FIXED_LENGTH"

chunking_options: Optional[ChunkingOptions]

Configuration options for the chunking algorithm.

Note: This feature requires enabling the knowledgebase enhancements feature preview flag.

child_chunk_size: Optional[int]

Hierarchical options

formatint64

max_chunk_size: Optional[int]

Section_Based and Fixed_Length options

formatint64

parent_chunk_size: Optional[int]

Hierarchical options

formatint64

semantic_threshold: Optional[float]

Semantic options

formatfloat

created_at: Optional[datetime]

Creation date / time

formatdate-time

dropbox_data_source: Optional[DropboxDataSource]

Dropbox Data Source for Display

folder: Optional[str]

file_upload_data_source: Optional[APIFileUploadDataSource]

File to upload as data source for knowledge base.

original_file_name: Optional[str]

The original file name

size_in_bytes: Optional[str]

The size of the file in bytes

formatuint64

stored_object_key: Optional[str]

The object key the file was stored as

google_drive_data_source: Optional[GoogleDriveDataSource]

Google Drive Data Source for Display

folder_id: Optional[str]

folder_name: Optional[str]

Name of the selected folder if available

item_path: Optional[str]

Path of folder or object in bucket - Deprecated, moved to data_source_details

last_datasource_indexing_job: Optional[APIIndexedDataSource]

completed_at: Optional[datetime]

Timestamp when data source completed indexing

formatdate-time

data_source_uuid: Optional[str]

Uuid of the indexed data source

error_details: Optional[str]

A detailed error description

error_msg: Optional[str]

A string code provinding a hint which part of the system experienced an error

failed_item_count: Optional[str]

Total count of files that have failed

formatuint64

indexed_file_count: Optional[str]

Total count of files that have been indexed

formatuint64

indexed_item_count: Optional[str]

Total count of files that have been indexed

formatuint64

removed_item_count: Optional[str]

Total count of files that have been removed

formatuint64

skipped_item_count: Optional[str]

Total count of files that have been skipped

formatuint64

started_at: Optional[datetime]

Timestamp when data source started indexing

formatdate-time

status: Optional[Literal["DATA_SOURCE_STATUS_UNKNOWN", "DATA_SOURCE_STATUS_IN_PROGRESS", "DATA_SOURCE_STATUS_UPDATED", 4 more]]

Accepts one of the following:

"DATA_SOURCE_STATUS_UNKNOWN"

"DATA_SOURCE_STATUS_IN_PROGRESS"

"DATA_SOURCE_STATUS_UPDATED"

"DATA_SOURCE_STATUS_PARTIALLY_UPDATED"

"DATA_SOURCE_STATUS_NOT_UPDATED"

"DATA_SOURCE_STATUS_FAILED"

"DATA_SOURCE_STATUS_CANCELLED"

total_bytes: Optional[str]

Total size of files in data source in bytes

formatuint64

total_bytes_indexed: Optional[str]

Total size of files in data source in bytes that have been indexed

formatuint64

total_file_count: Optional[str]

Total file count in the data source

formatuint64

region: Optional[str]

Region code - Deprecated, moved to data_source_details

spaces_data_source: Optional[APISpacesDataSource]

Spaces Bucket Data Source

bucket_name: Optional[str]

Spaces bucket name

item_path: Optional[str]

region: Optional[str]

Region of bucket

updated_at: Optional[datetime]

Last modified

formatdate-time

uuid: Optional[str]

Unique id of knowledge base

web_crawler_data_source: Optional[APIWebCrawlerDataSource]

WebCrawlerDataSource

base_url: Optional[str]

The base url to crawl.

crawling_option: Optional[Literal["UNKNOWN", "SCOPED", "PATH", 3 more]]

Options for specifying how URLs found on pages should be handled.

UNKNOWN: Default unknown value
SCOPED: Only include the base URL.
PATH: Crawl the base URL and linked pages within the URL path.
DOMAIN: Crawl the base URL and linked pages within the same domain.
SUBDOMAINS: Crawl the base URL and linked pages for any subdomain.
SITEMAP: Crawl URLs discovered in the sitemap.

Accepts one of the following:

"UNKNOWN"

"SCOPED"

"PATH"

"DOMAIN"

"SUBDOMAINS"

"SITEMAP"

embed_media: Optional[bool]

Whether to ingest and index media (images, etc.) on web pages.

exclude_tags: Optional[SequenceNotStr[str]]

Declaring which tags to exclude in web pages while webcrawling

Update Data Source options

from gradient import Gradient

client = Gradient(
    access_token="My Access Token",
)
data_source = client.knowledge_bases.data_sources.update(
    path_data_source_uuid="123e4567-e89b-12d3-a456-426614174000",
    path_knowledge_base_uuid="123e4567-e89b-12d3-a456-426614174000",
)
print(data_source.knowledge_base_data_source)

{
  "knowledge_base_data_source": {
    "aws_data_source": {
      "bucket_name": "example name",
      "item_path": "example string",
      "region": "example string"
    },
    "bucket_name": "example name",
    "chunking_algorithm": "CHUNKING_ALGORITHM_SECTION_BASED",
    "chunking_options": {
      "child_chunk_size": 350,
      "max_chunk_size": 750,
      "parent_chunk_size": 1000,
      "semantic_threshold": 0.5
    },
    "created_at": "2023-01-01T00:00:00Z",
    "dropbox_data_source": {
      "folder": "example string"
    },
    "file_upload_data_source": {
      "original_file_name": "example name",
      "size_in_bytes": "12345",
      "stored_object_key": "example string"
    },
    "google_drive_data_source": {
      "folder_id": "123e4567-e89b-12d3-a456-426614174000",
      "folder_name": "example name"
    },
    "item_path": "example string",
    "last_datasource_indexing_job": {
      "completed_at": "2023-01-01T00:00:00Z",
      "data_source_uuid": "123e4567-e89b-12d3-a456-426614174000",
      "error_details": "example string",
      "error_msg": "example string",
      "failed_item_count": "12345",
      "indexed_file_count": "12345",
      "indexed_item_count": "12345",
      "removed_item_count": "12345",
      "skipped_item_count": "12345",
      "started_at": "2023-01-01T00:00:00Z",
      "status": "DATA_SOURCE_STATUS_UNKNOWN",
      "total_bytes": "12345",
      "total_bytes_indexed": "12345",
      "total_file_count": "12345"
    },
    "region": "example string",
    "spaces_data_source": {
      "bucket_name": "example name",
      "item_path": "example string",
      "region": "example string"
    },
    "updated_at": "2023-01-01T00:00:00Z",
    "uuid": "123e4567-e89b-12d3-a456-426614174000",
    "web_crawler_data_source": {
      "base_url": "example string",
      "crawling_option": "UNKNOWN",
      "embed_media": true,
      "exclude_tags": [
        "example string"
      ]
    }
  }
}

Returns Examples

{
  "knowledge_base_data_source": {
    "aws_data_source": {
      "bucket_name": "example name",
      "item_path": "example string",
      "region": "example string"
    },
    "bucket_name": "example name",
    "chunking_algorithm": "CHUNKING_ALGORITHM_SECTION_BASED",
    "chunking_options": {
      "child_chunk_size": 350,
      "max_chunk_size": 750,
      "parent_chunk_size": 1000,
      "semantic_threshold": 0.5
    },
    "created_at": "2023-01-01T00:00:00Z",
    "dropbox_data_source": {
      "folder": "example string"
    },
    "file_upload_data_source": {
      "original_file_name": "example name",
      "size_in_bytes": "12345",
      "stored_object_key": "example string"
    },
    "google_drive_data_source": {
      "folder_id": "123e4567-e89b-12d3-a456-426614174000",
      "folder_name": "example name"
    },
    "item_path": "example string",
    "last_datasource_indexing_job": {
      "completed_at": "2023-01-01T00:00:00Z",
      "data_source_uuid": "123e4567-e89b-12d3-a456-426614174000",
      "error_details": "example string",
      "error_msg": "example string",
      "failed_item_count": "12345",
      "indexed_file_count": "12345",
      "indexed_item_count": "12345",
      "removed_item_count": "12345",
      "skipped_item_count": "12345",
      "started_at": "2023-01-01T00:00:00Z",
      "status": "DATA_SOURCE_STATUS_UNKNOWN",
      "total_bytes": "12345",
      "total_bytes_indexed": "12345",
      "total_file_count": "12345"
    },
    "region": "example string",
    "spaces_data_source": {
      "bucket_name": "example name",
      "item_path": "example string",
      "region": "example string"
    },
    "updated_at": "2023-01-01T00:00:00Z",
    "uuid": "123e4567-e89b-12d3-a456-426614174000",
    "web_crawler_data_source": {
      "base_url": "example string",
      "crawling_option": "UNKNOWN",
      "embed_media": true,
      "exclude_tags": [
        "example string"
      ]
    }
  }
}