main
Initial inference documentation pass (#3330)
Revert "Cleanup `handle_worker()` in preparation for #2815 (Stop generation early) (#3573)" (#3644)
Introduce model configs to abstract pairings of models and hardware (#2194)
Added max messages and max message length settings for inference (#2774)
Fixes for worker prompt truncation in ChatML case (#3673)
adjusted token buffer to pop EOS
Add inference api docs (#3059)
revert postgres port in `full-dev-setup` (#3022)