ОбзорДокументацияВойти
/
ai-forever
/
CerberusDet
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность

CerberusDet

0

Описание

4 MiB
README
Лицензия
В избранном0
Форки0

Языки

  • Python99%
  • Shell0,8%
  • Dockerfile0,2%
1
12
0
assets
3 месяца назад
bash_scripts
2 года назад
cerberusdet
3 месяца назад
data
2 года назад
.flake8
2 года назад
.gitignore
3 месяца назад
.isort.cfg
2 года назад
.pre-commit-config.yaml
2 года назад
Dockerfile
2 года назад
LICENSE.txt
2 года назад
README.md
3 месяца назад
docker-compose.yml
2 года назад
mypy.ini
2 года назад
pyproject.toml
3 месяца назад
README.md

CerberusDet: Unified Multi-Dataset Object Detection

[

Paper
][🤗 HuggingFace Models]

The code is based on:

Install

Python>=3.8.0 is required.

Docker

Run the docker:

Data

  • Use script voc.py to download VOC dataset

For information about the VOC dataset and its creators, visit the PASCAL VOC dataset website.

  • Use script objects365_part.py to download subset of Objects365 dataset with 19 animals categories:
['Monkey', 'Rabbit', 'Yak', 'Antelope', 'Pig',  'Bear', 'Deer', 'Giraffe', 'Zebra', 'Elephant',
'Lion', 'Donkey', 'Camel', 'Jellyfish', 'Other Fish', 'Dolphin', 'Crab', 'Seal', 'Goldfish']

Along with Objects365 subset with 12 tableware categories:

  [ 'Cup', 'Plate', 'Wine Glass', 'Pot', 'Knife', 'Fork', 'Spoon', 'Chopsticks',
    'Cutting/chopping Board', 'Tea pot', 'Kettle', 'Tong']

To download full Objects365 dataset, set

DOWNLOAD_SUBSETS = False
in the script objects365_part.py.

The Objects365 dataset is available for the academic purpose only. For information about the dataset and its creators, visit the Objects365 dataset website.

Train

  • Download pretrained on COCO yolov8 weights
  • Run train process with 1 GPU
  • OR run train process with several GPUs:

By default logging will be done with tensorboard, but you can use mlflow if set --mlflow-url, e.g.

--mlflow-url localhost
.

CerberusDet model config details

Example of the model's config for 2 tasks: yolov8x_voc_obj365.yaml

  • The model config is based on yolo configs, except that the
    head
     is divided into two sections (
    neck
     and
    head
    )
  • The layers of the
    neck
     section can be shared between tasks or be unique
  • The
    head
     section defines what the head will be for all tasks, but each task will always have its own unique parameters
  • The
    from
     parameter of the first neck layer must be a positive ordinal number, specifying from which layer, starting from the beginning of the entire architecture, to take features.
  • The
    cerber
     section is optional and defines the architecture configuration for determining the neck layers to be shared among tasks. If not specified, all layers will be shared among tasks, and only the heads will be unique.
  • The CerberusDet configuration is constructed as follows:
    cerber: List[OneBranchConfig]
    , where
     
    OneBranchConfig = List[cerber_layer_number, SharedTasksConfig]
    , where
         
    cerber_layer_number
     - the layer number (counting from the end of the backbone) after which branching should occur
         
    SharedTasksConfig = List[OneBranchGroupedTasks]
    , where
               
    OneBranchGroupedTasks = [number_of_task1_head, number_of_task2_head, ...]
     - the task head numbers (essentially task IDs) that should be in the same branch and share layers thereafter

    The head numbers will correspond to tasks according to the sequence in which they are listed in the data configuration.

    Example for YOLO v8x:
    [[2, [[15], [13, 14]]], [6, [[13], [14]]]]
     - configuration for 3 tasks. Task id=15 will have all task-specific layers, starting from the 3rd. Tasks id=13, id=14 will share layers 3-6, then after the 6th, they will have their own separate branches with all layers.

Evaluation

Inference

You can run inference using either the provided bash script or directly via the Python API.

1. Using Bash Script

First, download the CerberusDet checkpoint trained on VOC and parts of the Objects365 dataset (see the Pretrained Checkpoints section below).

Then, run the detection script:

2. Using Python API

You can also integrate CerberusDet into your own code. Below is an example of how to initialize the model, preprocess images, and visualize the results.

NOTE: To run inference using standard YOLOv8 checkpoints, use the

cerberusdet.yolo_wrapper.YOLOV8ForObjectDetection
class. Please ensure the following requirements are met:

Tip: Class names for specific datasets can be found in the corresponding YAML configuration files located in the

data/
directory.

Example using the VOC_07_12_best_state_dict.pt checkpoint (Click to expand)
`

Pretrained Checkpoints

ModelTrain setsize
(pixels)		mAPval
50-95		mAPval
50		Speed
V100 b32, fp16
(ms)		params
(M)		FLOPs
@640 (B)
YOLOv8xVOC6400.7580.9165.668257.5
YOLOv8xObjects365_animals6400.430.5485.668257.5
YOLOv8xObjects365_tableware6400.560.685.668257.5
YOLOv8xObjects365_full6400.2910.3815.670267.0
CerberusDet_v8xVOC, Objects365_animals6400.751, 0.4320.918, 0.5567.2105381.3
CerberusDet_v8xVOC, Objects365_animals, Objects365_tableware6400.762, 0.421, 0.560.927, 0.541, 0.6810142505.1
CerberusDet_v8xVOC, Objects365_full6400.767, 0.3550.932, 0.4647.2107390.8

YOLOv8x models were trained with the commit: https://github.com/ultralytics/ultralytics/tree/2bc36d97ce7f0bdc0018a783ba56d3de7f0c0518

Hyperparameter Evolution

See the launch example in the bash_scripts/evolve.sh.

Notes
  • To evolve hyperparameters specific to each task, specify initial parameters separately per task and append
    --evolve_per_task
  • To evolve specific set of hyperparameters, specify their names separated by comma via the
    --params_to_evolve
     argument, e.g.
    --params_to_evolve 'box,cls,dfl'
  • Use absolute paths to configs.
  • Specify search algorith via
    --evolver
    . You can use the search algorithms of the ray library (see available values here: predefined_evolvers.py), or
    'yolov5'

License

CerberusDet is released under the GNU AGPL v.3 license.

See the file LICENSE for more details.

Citing

If you use our models, code or dataset, we kindly request you to cite our paper and give repository a ⭐