This pull request introduces a new integration with DuckDB, an in-process SQL OLAP database, to enhance support for analytical workloads and similarity search in Retrieval-Augmented Generation (RAG) workflows. The changes include the addition of a new module for DuckDB integration, an example script, tests, and updates to documentation and dependencies.

Related issue: #924

DuckDB Integration:

• New DuckDB Integration Module:

  • Added the DuckDBIntegration class in llmware/duckdb_integration.py, providing methods to initialize the database, create tables, insert data, query data, and close the connection.

• Example Script:

  • Added examples/Models/duckdb_rag_example.py, demonstrating how to use the DuckDBIntegration class for creating tables, inserting data, and querying a DuckDB database.

• Unit Tests:

  • Added tests/models/test_duckdb_integration.py to validate the functionality of the DuckDBIntegration class, including creating tables, inserting data, querying data, and closing the database connection.

Documentation and Dependencies:

• Documentation Update:

  • Updated docs/components/model_catalog.md to include a new section on DuckDB integration, detailing its features, use cases, and references to the example script and tests.

• Dependency Update:

  • Added duckdb>=0.7.1 to llmware/requirements.txt to ensure the necessary library is available for the integration.