Data Sources

List Data Sources for a Knowledge Base

client.KnowledgeBases.DataSources.List(ctx, knowledgeBaseUuid, query) (*KnowledgeBaseDataSourceListResponse, error)

get/v2/gen-ai/knowledge_bases/{knowledge_base_uuid}/data_sources

Add Data Source to a Knowledge Base

client.KnowledgeBases.DataSources.New(ctx, knowledgeBaseUuid, body) (*KnowledgeBaseDataSourceNewResponse, error)

post/v2/gen-ai/knowledge_bases/{knowledge_base_uuid}/data_sources

Delete a Data Source from a Knowledge Base

client.KnowledgeBases.DataSources.Delete(ctx, knowledgeBaseUuid, dataSourceUuid) (*KnowledgeBaseDataSourceDeleteResponse, error)

delete/v2/gen-ai/knowledge_bases/{knowledge_base_uuid}/data_sources/{data_source_uuid}

Create Presigned URLs for Data Source File Upload

client.KnowledgeBases.DataSources.NewPresignedURLs(ctx, body) (*KnowledgeBaseDataSourceNewPresignedURLsResponse, error)

post/v2/gen-ai/knowledge_bases/data_sources/file_upload_presigned_urls

Update Data Source options

client.KnowledgeBases.DataSources.Update(ctx, knowledgeBaseUuid, dataSourceUuid, body) (*KnowledgeBaseDataSourceUpdateResponse, error)

put/v2/gen-ai/knowledge_bases/{knowledge_base_uuid}/data_sources/{data_source_uuid}

ModelsExpand Collapse

type APIFileUploadDataSource struct{…}

File to upload as data source for knowledge base.

OriginalFileName stringoptional

The original file name

SizeInBytes stringoptional

The size of the file in bytes

formatuint64

StoredObjectKey stringoptional

The object key the file was stored as

type APIKnowledgeBaseDataSource struct{…}

Data Source configuration for Knowledge Bases

AwsDataSource APIKnowledgeBaseDataSourceAwsDataSourceoptional

AWS S3 Data Source for Display

BucketName stringoptional

Spaces bucket name

ItemPath stringoptional

Region stringoptional

Region of bucket

BucketName stringoptional

Name of storage bucket - Deprecated, moved to data_source_details

ChunkingAlgorithm APIKnowledgeBaseDataSourceChunkingAlgorithmoptional

The chunking algorithm to use for processing data sources.

Note: This feature requires enabling the knowledgebase enhancements feature preview flag.

Accepts one of the following:

const APIKnowledgeBaseDataSourceChunkingAlgorithmChunkingAlgorithmUnknown APIKnowledgeBaseDataSourceChunkingAlgorithm = "CHUNKING_ALGORITHM_UNKNOWN"

const APIKnowledgeBaseDataSourceChunkingAlgorithmChunkingAlgorithmSectionBased APIKnowledgeBaseDataSourceChunkingAlgorithm = "CHUNKING_ALGORITHM_SECTION_BASED"

const APIKnowledgeBaseDataSourceChunkingAlgorithmChunkingAlgorithmHierarchical APIKnowledgeBaseDataSourceChunkingAlgorithm = "CHUNKING_ALGORITHM_HIERARCHICAL"

const APIKnowledgeBaseDataSourceChunkingAlgorithmChunkingAlgorithmSemantic APIKnowledgeBaseDataSourceChunkingAlgorithm = "CHUNKING_ALGORITHM_SEMANTIC"

const APIKnowledgeBaseDataSourceChunkingAlgorithmChunkingAlgorithmFixedLength APIKnowledgeBaseDataSourceChunkingAlgorithm = "CHUNKING_ALGORITHM_FIXED_LENGTH"

ChunkingOptions APIKnowledgeBaseDataSourceChunkingOptionsoptional

Configuration options for the chunking algorithm.

Note: This feature requires enabling the knowledgebase enhancements feature preview flag.

ChildChunkSize int64optional

Hierarchical options

formatint64

MaxChunkSize int64optional

Section_Based and Fixed_Length options

formatint64

ParentChunkSize int64optional

Hierarchical options

formatint64

SemanticThreshold float64optional

Semantic options

formatfloat

CreatedAt Timeoptional

Creation date / time

formatdate-time

DropboxDataSource APIKnowledgeBaseDataSourceDropboxDataSourceoptional

Dropbox Data Source for Display

Folder stringoptional

FileUploadDataSource APIFileUploadDataSourceoptional

File to upload as data source for knowledge base.

OriginalFileName stringoptional

The original file name

SizeInBytes stringoptional

The size of the file in bytes

formatuint64

StoredObjectKey stringoptional

The object key the file was stored as

GoogleDriveDataSource APIKnowledgeBaseDataSourceGoogleDriveDataSourceoptional

Google Drive Data Source for Display

FolderID stringoptional

FolderName stringoptional

Name of the selected folder if available

ItemPath stringoptional

Path of folder or object in bucket - Deprecated, moved to data_source_details

LastDatasourceIndexingJob APIIndexedDataSourceoptional

CompletedAt Timeoptional

Timestamp when data source completed indexing

formatdate-time

DataSourceUuid stringoptional

Uuid of the indexed data source

ErrorDetails stringoptional

A detailed error description

ErrorMsg stringoptional

A string code provinding a hint which part of the system experienced an error

FailedItemCount stringoptional

Total count of files that have failed

formatuint64

IndexedFileCount stringoptional

Total count of files that have been indexed

formatuint64

IndexedItemCount stringoptional

Total count of files that have been indexed

formatuint64

RemovedItemCount stringoptional

Total count of files that have been removed

formatuint64

SkippedItemCount stringoptional

Total count of files that have been skipped

formatuint64

StartedAt Timeoptional

Timestamp when data source started indexing

formatdate-time

Status APIIndexedDataSourceStatusoptional

Accepts one of the following:

const APIIndexedDataSourceStatusDataSourceStatusUnknown APIIndexedDataSourceStatus = "DATA_SOURCE_STATUS_UNKNOWN"

const APIIndexedDataSourceStatusDataSourceStatusInProgress APIIndexedDataSourceStatus = "DATA_SOURCE_STATUS_IN_PROGRESS"

const APIIndexedDataSourceStatusDataSourceStatusUpdated APIIndexedDataSourceStatus = "DATA_SOURCE_STATUS_UPDATED"

const APIIndexedDataSourceStatusDataSourceStatusPartiallyUpdated APIIndexedDataSourceStatus = "DATA_SOURCE_STATUS_PARTIALLY_UPDATED"

const APIIndexedDataSourceStatusDataSourceStatusNotUpdated APIIndexedDataSourceStatus = "DATA_SOURCE_STATUS_NOT_UPDATED"

const APIIndexedDataSourceStatusDataSourceStatusFailed APIIndexedDataSourceStatus = "DATA_SOURCE_STATUS_FAILED"

const APIIndexedDataSourceStatusDataSourceStatusCancelled APIIndexedDataSourceStatus = "DATA_SOURCE_STATUS_CANCELLED"

TotalBytes stringoptional

Total size of files in data source in bytes

formatuint64

TotalBytesIndexed stringoptional

Total size of files in data source in bytes that have been indexed

formatuint64

TotalFileCount stringoptional

Total file count in the data source

formatuint64

Region stringoptional

Region code - Deprecated, moved to data_source_details

SpacesDataSource APISpacesDataSourceoptional

Spaces Bucket Data Source

BucketName stringoptional

Spaces bucket name

ItemPath stringoptional

Region stringoptional

Region of bucket

UpdatedAt Timeoptional

Last modified

formatdate-time

Uuid stringoptional

Unique id of knowledge base

WebCrawlerDataSource APIWebCrawlerDataSourceoptional

WebCrawlerDataSource

BaseURL stringoptional

The base url to crawl.

CrawlingOption APIWebCrawlerDataSourceCrawlingOptionoptional

Options for specifying how URLs found on pages should be handled.

UNKNOWN: Default unknown value
SCOPED: Only include the base URL.
PATH: Crawl the base URL and linked pages within the URL path.
DOMAIN: Crawl the base URL and linked pages within the same domain.
SUBDOMAINS: Crawl the base URL and linked pages for any subdomain.
SITEMAP: Crawl URLs discovered in the sitemap.

Accepts one of the following:

const APIWebCrawlerDataSourceCrawlingOptionUnknown APIWebCrawlerDataSourceCrawlingOption = "UNKNOWN"

const APIWebCrawlerDataSourceCrawlingOptionScoped APIWebCrawlerDataSourceCrawlingOption = "SCOPED"

const APIWebCrawlerDataSourceCrawlingOptionPath APIWebCrawlerDataSourceCrawlingOption = "PATH"

const APIWebCrawlerDataSourceCrawlingOptionDomain APIWebCrawlerDataSourceCrawlingOption = "DOMAIN"

const APIWebCrawlerDataSourceCrawlingOptionSubdomains APIWebCrawlerDataSourceCrawlingOption = "SUBDOMAINS"

const APIWebCrawlerDataSourceCrawlingOptionSitemap APIWebCrawlerDataSourceCrawlingOption = "SITEMAP"

EmbedMedia booloptional

Whether to ingest and index media (images, etc.) on web pages.

ExcludeTags []stringoptional

Declaring which tags to exclude in web pages while webcrawling

type APISpacesDataSource struct{…}

Spaces Bucket Data Source

BucketName stringoptional

Spaces bucket name

ItemPath stringoptional

Region stringoptional

Region of bucket

type APIWebCrawlerDataSource struct{…}

WebCrawlerDataSource

BaseURL stringoptional

The base url to crawl.

CrawlingOption APIWebCrawlerDataSourceCrawlingOptionoptional

Options for specifying how URLs found on pages should be handled.

UNKNOWN: Default unknown value
SCOPED: Only include the base URL.
PATH: Crawl the base URL and linked pages within the URL path.
DOMAIN: Crawl the base URL and linked pages within the same domain.
SUBDOMAINS: Crawl the base URL and linked pages for any subdomain.
SITEMAP: Crawl URLs discovered in the sitemap.

Accepts one of the following:

const APIWebCrawlerDataSourceCrawlingOptionUnknown APIWebCrawlerDataSourceCrawlingOption = "UNKNOWN"

const APIWebCrawlerDataSourceCrawlingOptionScoped APIWebCrawlerDataSourceCrawlingOption = "SCOPED"

const APIWebCrawlerDataSourceCrawlingOptionPath APIWebCrawlerDataSourceCrawlingOption = "PATH"

const APIWebCrawlerDataSourceCrawlingOptionDomain APIWebCrawlerDataSourceCrawlingOption = "DOMAIN"

const APIWebCrawlerDataSourceCrawlingOptionSubdomains APIWebCrawlerDataSourceCrawlingOption = "SUBDOMAINS"

const APIWebCrawlerDataSourceCrawlingOptionSitemap APIWebCrawlerDataSourceCrawlingOption = "SITEMAP"

EmbedMedia booloptional

Whether to ingest and index media (images, etc.) on web pages.

ExcludeTags []stringoptional

Declaring which tags to exclude in web pages while webcrawling

type AwsDataSource struct{…}

AWS S3 Data Source

BucketName stringoptional

Spaces bucket name

ItemPath stringoptional

KeyID stringoptional

The AWS Key ID

Region stringoptional

Region of bucket

SecretKey stringoptional

The AWS Secret Key