/
githubmirror
/
ColossalAI
Обзор
Документация
Войти
/
githubmirror
/
ColossalAI
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность
main
colossalai/legacy/nn/layer/parallel_2d/_utils.py
22 строки
853 B
Hongxin Liu
[misc] update pre-commit and run all files (#4752)
19 сен 2023, 09:20
Не верифицирован
19 сен 2023, 09:20
079bf3c
Код
Авторство
О чём код?
from colossalai.legacy.context.parallel_mode import ParallelMode from colossalai.legacy.core import global_context as gpc from colossalai.legacy.global_variables import tensor_parallel_env as env def get_summa_dim_from_env() -> int: try: summa_dim = env.summa_dim assert summa_dim > 0, "SUMMA_DIM must be larger than zero" return summa_dim except KeyError: raise EnvironmentError( "SUMMA_DIM is not found in the current environment, " "please make sure that you have used the correct process group initializer" ) def assert_summa_initialization(): assert gpc.is_initialized(ParallelMode.PARALLEL_2D_COL) and gpc.is_initialized( ParallelMode.PARALLEL_2D_ROW ), "Both TWO_DIMENSION_COL and TWO_DIMENSION_ROW must be initialized by the process group initializer"