Epsilla
HomeDiscordTwitterGithubEmail
  • Welcome
    • Register and Login
    • Explore App Portal
  • Build Your First AI Agent
    • Create a Knowledge Base
    • Set Up Your AI Agent
    • Publish Your AI Agent
  • Knowledge Base
    • Local Files
    • Website
    • Google Drive
    • S3
    • Notion
    • Share Point
    • Google Cloud Storage
    • Azure Blob Storage
    • Confluence
    • Jira
    • Advanced Settings
      • Auto Sync
      • Embedding
      • Data Parsing
      • Data Chunking
      • Hypothetical Questions
      • Webhook
      • Meta Data
    • Data Storage
    • Programmatically Manage Knowledge Bases
  • Application
    • Create New AI Agent
    • Basic Chat Agent Config
    • Basic Smart Search Agent Config
    • Advanced Workflow Customization
    • Publish and Deployment
    • User Engagement Analytics
  • Evaluation
    • Create New Evaluation
    • Run Evaluation
    • Evaluation Run History
  • Integration
  • Team Member Management
  • Project Management
  • Billing Management
  • Release Notes
  • Epsilla Vector Database
    • Overview
    • Quick Start
      • Run with Docker
      • Epsilla Cloud
    • User Manual
      • Connect to a database
      • Create a new table
      • Drop a table
      • Delete a database
      • Insert records
      • Upsert records
      • Search the top K semantically similar records
      • Retrieve records (with filters and pagination)
      • Delete records
      • Performance Tuning
    • Advanced Topics
      • Embeddings
      • Dense vector vs. sparse vector
      • Hybrid Search
    • Integrations
      • OpenAI
      • Mistral AI
      • Jina AI
      • Voyage AI
      • Mixedbread AI
      • Nomic AI
    • Roadmap
Powered by GitBook
On this page
  • Inspect Data
  • Semantic Search
  • Add New Records
  • Delete Records
  • Inspect Table Information
  • Create New Table
  • Delete Table
  1. Knowledge Base

Data Storage

PreviousMeta DataNextProgrammatically Manage Knowledge Bases

Last updated 7 months ago

Data storage in Epsilla manages the underlying stored data records (chunks) from the knowledge base. Users can perform CRUD (Create, Read, Update, Delete) operations on these records, manage tables, and inspect the details of each record. Additionally, users can view the table schema to understand the structure and organization of stored data.

Inspect Data

If you are create a knowledge base from any data source type other than Basic (which means creating knowledge base from scratch), the knowledge_table will already be automatically . You can inspect the processed data (chunks). By default, the first 20 chunks will be visualized when you enter the tab:

Click a record to inspect the data detail:

By default, each record in Epsilla's knowledge base contains the following fields:

  • ID: A unique UUID to distinguish each record in the knowledge base.

  • Content: The chunk content as a string.

  • Filename: The file's name, the webpage's URL, or the identifier in the data source.

  • Timestamp: The epoch time (in milliseconds) when the record was loaded or updated.

  • Metadata: Additional information about the chunk, such as DataSourceType (e.g., file), FileType (e.g., pdf), and Pages (a list indicating the pages where this chunk is located, starting at page 1). Note that this metadata differs from the "Meta Data" configured in Advanced Settings.

If Meta Data is configured, additional fields mapped from that meta data will also be shown in the record details.

Use the Records per page setting to adjust how many records are displayed in each batch.

You can navigate between batches using the Prev and Next buttons to scroll through the records.

Semantic Search

You can enter a natural language query in the search box to find the top K most relevant records based on meaning rather than exact keywords; this is known as semantic search.

Click the settings button to open the advanced search setup. By default, the Content field is semantically indexed by Index. If you have added other semantic indexes to the table, you can select a different index for searching.

You can also specify a filter expression for the search:

It will be applied along with the semantic search to further narrow down the results.

Add New Records

Click the Add New Record button at bottom right corner:

And you can insert new records to the knowledge base (following the table schema):

It is recommended to use the data source's automatic processing of records instead of manually inserting records.

Delete Records

Hover over a record with your mouse, then click the Trash Can icon to delete it.

Confirm Delete:

Deleting records is irreversible. It is recommended to rely on the data source's automatic processing of records rather than manually deleting them.

Inspect Table Information

Hover over a table with your mouse, then click the Edit icon to view and inspect the table information.

The second tab displays the APIs that can be used to programmatically manipulate data in the table, allowing for operations such as creating, reading, updating, and deleting records through code:

For each API, three implementation methods are provided: cURL, Python, and JavaScript, allowing users to choose their preferred language or tool for interacting with the table data programmatically:

Create New Table

If you want to define your own knowledge schema, particularly when building from scratch using a Basic type knowledge base, click the Create Table button to create a new table:

The table fields at the "Create Table" stage define the structure and type of data each column will store. You can specify the field names, data types (such as string, integer, or boolean), and any constraints (e.g., Primary Key) to ensure the data is organized correctly and adheres to the intended format:

Delete Table

Hover over a table with your mouse, then click the Trash Can icon to delete the table.

Confirm Delete:

Read more about supported .

The first tab displays the table schema, which defines the structure of the table, including the fields, data types, and semantic indices. Read more about .

Read more about the table fields at .

cURL
Python
JavaScript
filter expression
table schema
create table