ray-llm

RayLLM - LLMs on Ray

Getting started

Deploying RayLLM

Query your models

RayLLM Reference

Installing RayLLM

Running Aviary Explorer locally

Using RayLLM through the CLI

RayLLM Model Registry

Frequently Asked Questions

How do I add a new model?

How do I deploy multiple models at once?

How do I deploy a model to multiple nodes?

My deployment isn't starting/working correctly, how can I debug?

Getting Help and Filing Bugs / Feature Requests

Contributions