v0.3.9
[Fix] Fix spec-dec Glide LlamaModel for compatibility with transformers (#5837)
[Hotfix] Add missing init file in inference.executor (#5774)
[Inference]Add Streaming LLM (#5745)
[Inference]Fix readme and example for API server (#5742)
Refactor modeling by adding attention backend
[doc] updated inference readme (#5343)
Pass inference model shard configs for module init
add paged-attetionv2: support seq length split across thread block (#5707)
[fix] pytest and fix dyn grid bug
[Inference] Fix Inference Generation Config and Sampling (#5710)
[Online Server] Chat Api for streaming and not streaming response (#5470)
[Inference] Fix flash-attn import and add model test (#5794)