AI preprocessing Embeds files in the specified format and stores the results in a vector database

This application reads the specified format file (*1) into OpenAI, an LLM, embeds it, and converts it into a numeric vector. The converted results are stored in a vector database.
By using this application, you can efficiently prepare data, which is essential for generative AI, contributing to cost reduction and improved response quality.

• For details on supported file formats, please refer to the OpenAI website below.
  ⇒ https://platform.openai.com/docs/assistants/tools/file-search#supported-files

Script Details

Embeds a file in the specified format and stores the result in the vector database.

Embedding_and_Store_OpenAI_main

Upload the file in the specified format to LLM (OpenAI)

Embedding_and_Store_OpenAI_upload_files

Create a new vector DB or use the ID of an existing DB

Embedding_and_Store_OpenAI_prepare_store

Embedding and store the results in the vector DB

Embedding_and_Store_OpenAI_trans_to_store

Check the processing results of embedding vector DB storage

Embedding_and_Store_OpenAI_check_status

Aggregate the processing results stored in the embedding vector DB and output them to a CSV file

Embedding_and_Store_OpenAI_generate_summary

• Download specifications
• Try HULFT Square

How to install and use it