🚀 Colossal-Inference

Table of contents

Introduction

Design

Pipeline of inference:

Roadmap of our implementation

Get started

Installation

Requirements

Docker

Dive into fast-inference!

Performance

environment:

Single GPU Performance:

Llama

Bloom