mistral-src

Mistral Transformer

Deployment

Installation

Download the model

Run the model

Sliding window attention

Vanilla attention

Sliding window to speed-up inference and reduce memory pressure

Rolling buffer cache

Pre-fill and chunking

Sparse Mixture of Experts (SMoE)

Pipeline Parallelism

Integrations and related projects

References