Distributed RL Framework for Language Model Fine-Tuning

🚀 Features

🛠 Installation

Prepare Develop Environment

Architecture Design

🧠 Data Format

Math Data Format

Code Data Format

⚙️ Hyperparameters & Arguments

Distributed Training

Sampling

GRPO Specific

Logging and Checkpointing

Miscellaneous

⚙️ GRPO Settings

🔄 Constraints and Notes

🧪 Example: single machine 8-GPU Zero2 Strategy

🧪 Example: multi-machine TP+PP Strategy

Create ray cluster on multi-machine

Acknowledgement