Table of Contents

What Is ColossalChat And Coati ?

Online demo

Install

Install the Environment

How To Use?

RLHF Training Stage1 - Supervised Instructs Tuning

Step 1: Data Collection

Step 2: Preprocessing

Step 3: Training

RLHF Training Stage2 - Training Reward Model

Step 1: Data Collection

Step 2: Preprocessing

Step 3: Training

RLHF Training Stage3 - Proximal Policy Optimization

Step 1: Data Collection

Step 2: Data Preprocessing

Step 3: Training

Alternative Option For RLHF: Direct Preference Optimization (DPO)

DPO Training Stage1 - Supervised Instructs Tuning

DPO Training Stage2 - DPO Training

Step 1: Data Collection & Preparation

Step 2: Training

Alternative Option For RLHF: Simple Preference Optimization (SimPO)

Alternative Option For RLHF: Odds Ratio Preference Optimization (ORPO)

Alternative Option For RLHF: Kahneman-Tversky Optimization (KTO)

Inference Quantization and Serving - After Training

O1 Journey

Inference with Self-refined MCTS

Coati7B examples

Generation

Open QA

Limitation

FAQ

Real-time progress

Invitation to open-source contribution

Quick Preview

Authors

Citations

Licenses