/
githubmirror
/
ColossalAI
Обзор
Документация
Войти
/
githubmirror
/
ColossalAI
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность
main
colossalai/inference/server/utils.py
36 строк
713 B
Jianghai
[Online Server] Chat Api for streaming and not streaming response (#5470)
08 май 2024, 18:20
08 май 2024, 18:20
c064032
Код
Авторство
О чём код?
from typing import Any, Optional from pydantic import BaseModel # make it singleton class NumericIDGenerator: _instance = None def __new__(cls): if cls._instance is None: cls._instance = super(NumericIDGenerator, cls).__new__(cls) cls._instance.current_id = 0 return cls._instance def __call__(self): self.current_id += 1 return self.current_id id_generator = NumericIDGenerator() class ChatMessage(BaseModel): role: str content: Any class DeltaMessage(BaseModel): role: Optional[str] = None content: Optional[Any] = None class ChatCompletionResponseStreamChoice(BaseModel): index: int message: DeltaMessage