Test your prompts, models, and RAGs. Catch regressions and improve prompt quality. LLM evals for OpenAI, Azure, Anthropic, Gemini, Mistral, Llama, Bedrock, Ollama, and other local & private models with CI/CD integration.
llmragllmopsprompt-engineeringtestingpromptsevaluation-frameworkevaluationllm-evalcicdci-cdcillm-evaluationllm-evaluation-frameworkprompt-testing- TypeScript
01Обновлено 6 месяцев назад
The Security Toolkit for LLM Interactions
llmtransformerslarge-language-modelschatgptllmopsprompt-engineeringllm-securityadversarial-machine-learningprompt-injectionsecurity-tools- Python
01Обновлено 7 месяцев назад
A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap
chatgptchatbotgptllmopsgpt-4llm-inferencechatllm-frameworkgpt-35-turbollm-agentagent-based-frameworkagent-oriented-programmingagenticagentic-agichat-application- Jupyter Notebook
01Обновлено 5 месяцев назад
Build Enterprise RAG (Retriver Augmented Generation) Pipelines to tackle various Generative AI use cases with LLM's by simply plugging componants like Lego pieces. This repo is intended for IBM Ecosystem partners.
- Jupyter Notebook
01Обновлено 7 месяцев назад
LLM App templates for RAG, knowledge mining, and stream analytics. Ready to run with Docker,⚡in sync with your data sources.
llmmachine-learningllmopschatbotvector-databaseragpathwayreal-timeretrieval-augmented-generationvector-indexhugging-facellm-localllm-promptingllm-securityopen-ai- Python
00Обновлено 7 месяцев назад
RayLLM - LLMs on Ray
- Python
00Обновлено 7 месяцев назад
LLMFlows - Simple, Explicit and Transparent LLM Apps
pythonllmmachine-learningaichatgptopenaillmsllmopsllm-inferencegpt-4vector-databasequestion-answeringprompt-engineering- Python
00Обновлено 7 месяцев назад
Open-source observability for your LLM application, based on OpenTelemetry
pythonllmllmopsgenerative-aiopen-sourcemonitoringobservabilitymlmetricsopentelemetry-pythonartifical-intelligencedatasciencegood-first-issuegood-first-issueshelp-wantedmodel-monitoringopen-telemetryopentelemetry- Python
00Обновлено 7 месяцев назад
AI Observability & Evaluation - Evaluate, troubleshoot, and fine tune your LLM, CV, and NLP models in a notebook.
llmopsmlopsmodel-monitoringai-monitoringai-observabilityai-roiclusteringllm-evalml-monitoringml-observabilitymodel-observabilityumap- Jupyter Notebook
00Обновлено 7 месяцев назад
🚀 Open-source platform for evaluating and monitoring LLMs. Integrates with OpenAI, Cohere and Anthropic with stable SDKs in Python and Javascript.
llmlarge-language-modelsgolangopenaillamallmopstypescriptgenerative-aiopen-sourcemonitoringobservabilitynextjsgpt-3metricsself-hostedai-observabilityanthropiccohereprimsa- TypeScript
00Обновлено 7 месяцев назад
An awesome & curated list of best LLMOps tools for developers
- Shell
00Обновлено 7 месяцев назад
A high-throughput and memory-efficient inference and serving engine for LLMs
- Python
00Обновлено 8 месяцев назад
<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.
pythonllmhacktoberfestaiopenaillmopsgpt-4artificial-intelligencenextjsagentsagipineconeautonomous-agentsartificial-general-intelligencesuperagi- Python
00Обновлено 5 месяцев назад
Operating LLMs in production
llmaillamafine-tuningmistralmlopsllama2llm-inferencellm-servingllmopsmlfalconbentomlllm-opsmodel-inferencemptopen-source-llmopenllmstablelmvicuna- Python
00Обновлено 7 месяцев назад
No-code platform to build LLM Agents, workflows and applications with your data
aillmsllmopsgenerative-aiagentsno-code-aiplatformai-agents-frameworkllm-agentsllm-chainllm-framework- Python
00Обновлено 7 месяцев назад
🪢 Open source LLM engineering platform. Observability, metrics, evals, prompt management 🍊YC W23 🤖 SDKs + integrations for Typescript, Python, OpenAI, Langchain, LlamaIndex, Litellm
llmtypescriptllamalarge-language-modelsgptopenainextjsmonitoringopen-sourcegenerative-aillmopsanalyticsobservabilitylangchaintailwindcssloggingllama-indexself-hostedt3-stackycombinator- TypeScript
00Обновлено 7 месяцев назад