blockoli
blockoli
🥦🔎
Blockoli is a high-performance tool for code indexing, embedding generation and semantic search tool for use with LLMs. blockoli is built in Rust and uses the ASTerisk crate for semantic code parsing. blockoli allows you to efficiently index, store, and search code blocks and their embeddings using vector similarity.
Features
- Index code blocks from a codebase 📂🔍
- Generate vector embeddings for code blocks using a pre-trained model 🤖🧠
- Store code blocks and their embeddings in a SQLite database (Support for Qdrant soon!) 💾🗄️
- Perform efficient similarity search on code blocks using vector embeddings (k-d tree algorithm) 🔎⚡
- REST API for easy integration with other tools and platforms 🌐🔗
- Fast and memory-efficient implementation using Rust ⚡💻
Installation (from source)
-
Ensure you have Rust installed on your system. You can install it from the official Rust website: https://www.rust-lang.org/tools/install
-
Clone the blockoli repository:
git clone https://github.com/stitionai/blockoli.gitcd blockoli
- Download
grammar filestree-sitter
mkdir grammars
chmod +x get-grammar.sh
./get-grammar.sh
- Build the project:
cargo build --release
- Run the server:
./target/release/blockoli <port>
Replace
with the desired port number for the server.
Usage
Blockoli provides a REST API for indexing and searching code blocks. Here are some example API endpoints:
: Create a new projectPOST /project
: Get information about a projectGET /project/{project_name}
: Delete a projectDELETE /project/{project_name}
: Generate embeddings for code blocks in a projectPOST /project/generate
: Search for similar code blocks in a projectPOST /search/{code_block}
: Get all function blocks in a projectPOST /get_blocks/{project_name}
: Search for function blocks in a projectPOST /search_blocks/{function_block}
: Search for blocks by function name in a projectPOST /search_by_function/{function_name}
Refer to the
file for detailed information about each API endpoint and its parameters.
Configuration
uses a configuration file named
for specifying indexing options. Modify this file to customize the behavior of the indexer according to your needs.
Contribution Guidelines
Contributions to Blockoli are welcome! If you find a bug, have a feature request, or want to contribute code improvements, please open an issue or submit a pull request on the GitHub repository.
When contributing code, please ensure that your changes are well-tested and follow the Rust coding conventions and style guidelines.
Contribution
Ways to contribute:
- Suggest a feature
- Report a bug
- Fix something and open a pull request
- Help document the code
- Spread the word
License
Licensed under the MIT License, see LICENSE for more information.
Liked the project?
Support the project by starring the repository. ⭐
Описание
Blockoli is a high-performance tool for code indexing, embedding generation and semantic search tool for use with LLMs.
Языки
Rust