GreedyLR: Adaptive Learning Rate Scheduler

Paper

How It Works

Key Parameters

Quick Start

Fine-tuning LLaMA 3.2 1B on simpleCoT

Pre-training LLaMA 3.2 1B on RedPajama (multi-GPU)

Standalone Usage (without Trainer)

Comparison with Other Schedulers

Using with Other Models

Citation