Table of Contents

What is ColossalChat?

Online demo

Install

Install the Environment

Introduction

RLHF Training Stage1 - Supervised Instructs Tuning

RLHF Training Stage2 - Training Reward Model

RLHF Training Stage3 - Proximal Policy Optimization

Alternative Option For RLHF: Direct Preference Optimization (DPO)

Alternative Option For RLHF: Simple Preference Optimization (SimPO)

Alternative Option For RLHF: Odds Ratio Preference Optimization (ORPO)

Alternative Option For RLHF: Kahneman-Tversky Optimization (KTO)

Alternative Option For RLHF: Group Relative Policy Optimization (GRPO)

SFT for DeepSeek V3

Inference Quantization and Serving - After Training

Invitation to open-source contribution

Quick Preview

Authors

Citations

Licenses