/
dar-sn
/
Quality-MobileFaceNet
Обзор
Документация
Войти
/
dar-sn
/
Quality-MobileFaceNet
Код
Запросы
0
Задачи
Вики
Пакеты
0
Релизы
0
CI/CD
Аналитика
Безопасность
master
split_data.py
37 строк
2 KB
dar-sn
Create: magnitudes.csv, new_path_magnitudes.csv, norm_stats.json, test.csv, train.csv, val.csv, dataset.py, ema.py, mobilefacenet.py, split_data.py, train.py, baseline_8_pretrained_best.pth
27 июн 2026, 14:54
Верифицирован
27 июн 2026, 14:54
1a78c1f
Код
Авторство
О чём код?
import pandas as pd from sklearn.model_selection import GroupShuffleSplit import json df = pd.read_csv("data/new_path_magnitudes.csv") # df['image_path'] = df['image_path'].str.replace(r'D:\Sanatina', r'C:\Users\daria', regex=False) # df.to_csv('data/new_path_magnitudes.csv', index=False) # Первое разбиение: отделяем test (15%) gss1 = GroupShuffleSplit(n_splits=1, test_size=0.15, random_state=42) train_val_idx, test_idx = next(gss1.split(df, groups=df["identity"])) df_train_val = df.iloc[train_val_idx].reset_index(drop=True) df_test = df.iloc[test_idx].reset_index(drop=True) # Второе разбиение: из оставшегося отделяем val (15% от всего ≈ 17.6% от train_val) gss2 = GroupShuffleSplit(n_splits=1, test_size=0.176, random_state=42) train_idx, val_idx = next(gss2.split(df_train_val, groups=df_train_val["identity"])) df_train = df_train_val.iloc[train_idx].reset_index(drop=True) df_val = df_train_val.iloc[val_idx].reset_index(drop=True) # Считаем статистики по train y_mean = df_train["magnitude"].mean() y_std = df_train["magnitude"].std() y_max = df["magnitude"].max() print(f"Train: {len(df_train)}, Val: {len(df_val)}, Test: {len(df_test)}") print(f"y_mean = {y_mean:.4f}, y_std = {y_std:.4f}, y_max = {y_max:.4f}") df_train.to_csv("data/train.csv", index=False) df_val.to_csv("data/val.csv", index=False) df_test.to_csv("data/test.csv", index=False) with open("data/norm_stats.json", "w") as f: json.dump({"y_mean": y_mean, "y_std": y_std, "y_max": y_max}, f)