/
githubmirror
/
Open-Assistant
Обзор
Документация
Войти
/
githubmirror
/
Open-Assistant
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность
main
model/model_training/configs/deepspeed_rl.json
29 строк
689 B
sanagno
Rl training (#2206)
24 апр 2023, 10:49
Не верифицирован
24 апр 2023, 10:49
4f02095
Код
Авторство
О чём код?
{ "fp16": { "enabled": "true", "loss_scale": 0, "loss_scale_window": 1000, "initial_scale_power": 16, "hysteresis": 2, "min_loss_scale": 1 }, "zero_optimization": { "stage": 2, "allgather_partitions": true, "allgather_bucket_size": 2e8, "overlap_comm": false, "reduce_scatter": true, "reduce_bucket_size": 2e8, "contiguous_gradients": true, "offload_optimizer": { "device": "cpu", "pin_memory": true } }, "gradient_accumulation_steps": "8", "train_micro_batch_size_per_gpu": "auto", "train_batch_size": "auto", "gradient_clipping": "auto", "steps_per_print": 2000, "wall_clock_breakdown": false }