💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
- Rust
01Обновлено 8 месяцев назад
Robust recipes to align language models with human and AI preferences
- Python
01Обновлено 8 месяцев назад
The Security Toolkit for LLM Interactions
llmtransformerslarge-language-modelschatgptllmopsprompt-engineeringllm-securityadversarial-machine-learningprompt-injectionsecurity-tools- Python
01Обновлено 7 месяцев назад
:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
pythonaimachine-learningtransformerslanguage-modelnlppytorchbertchatgptgenerative-aigpt-3large-language-modelsinformation-retrievalquestion-answeringsemantic-searchsquadsummarization- Python
01Обновлено 8 месяцев назад
Efficient Retrieval Augmentation and Generation Framework
llmtransformersnlpquestion-answeringgenerative-aiinformation-retrievalsemantic-searchdiffusionsummarizationbenchmarkcolbertknowledge-graphmulti-modalsentence-transformers- Python
01Обновлено 7 месяцев назад
OpenChat: Advancing Open-source Language Models with Imperfect Data
- Python
01Обновлено 8 месяцев назад
The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction
- Python
01Обновлено 7 месяцев назад
Lightweight demos for finetuning of instruct LLMs. Powered by transformers/accelerate and open-source datasets.
- Jupyter Notebook
01Обновлено 7 месяцев назад
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
- Python
01Обновлено 8 месяцев назад
Unify Efficient Fine-tuning of 100+ LLMs
llmgptlanguage-modelagentbaichuanchatglmfine-tuninggenerative-aiinstruction-tuninglarge-language-modelsllamaloramistralmixture-of-expertspeftqloraquantizationqwenrlhftransformers- Python
01Обновлено 8 месяцев назад
Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)
- Python
00Обновлено 7 месяцев назад
💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows
pythonllmmachine-learningnlplarge-language-modelstransformerslanguage-modelragembeddingsvector-databasesemantic-searchretrieval-augmented-generationinformation-retrievalvector-searchneural-searchsearch-enginesearchvector-search-enginesentence-embeddingstxtai- Python
00Обновлено 5 месяцев назад
Framework for probing tasks
- Python
00Обновлено 2 месяца назад
Python bindings for the Transformer models implemented in C/C++ using GGML library.
- C
00Обновлено 7 месяцев назад
Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI
transformersquestion-answeringconversational-aiinformation-retrivalnamed-entity-recognitiontext-classification- Python
00Обновлено 7 месяцев назад
🤗 Optimum Intel: Accelerate inference with Intel optimization tools
- Python
00Обновлено 7 месяцев назад
An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
- Python
00Обновлено 7 месяцев назад
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
- Python
00Обновлено 7 месяцев назад
👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
llmtransformersnlpllamabertquestion-answeringcompressionpretrained-modelssearch-enginesemantic-analysissentiment-analysisuiedistributed-trainingdocument-intelligenceembeddingernieinformation-extractionneural-searchpaddlenlp- Python
00Обновлено 7 месяцев назад
Seamlessly integrate LLMs into scikit-learn.
- Python
00Обновлено 7 месяцев назад
RayLLM - LLMs on Ray
- Python
00Обновлено 7 месяцев назад
Providing enterprise-grade LLM-based development framework, tools, and fine-tuned models.
pythonmachine-learningaitransformerspytorchnlplarge-language-modelsbertquestion-answeringgenerative-airagembeddingsinformation-retrievalsemantic-searchretrieval-augmented-generationembedding-vectorsfaissmilvusparsing- Python
00Обновлено 7 месяцев назад