main
[Inference] Fix bugs and docs for feat/online-server (#5598)
[upgrade]Upgrade transformers (#6320)
[Inference] Fix flash-attn import and add model test (#5794)
[Fix] Fix Inference Example, Tests, and Requirements (#5688)
[Inference] Add the logic of the inference engine (#5173)
[Fix/Inference] Fix format of input prompts and input model in inference engine (#5395)
[inference] Fix running time of test_continuous_batching (#5750)
[Fix] Fix & Update Inference Tests (compatibility w/ main)
[release] update version (#5752)
[Inference]Add Streaming LLM (#5745)