mirror of
https://github.com/macaodha/batdetect2.git
synced 2025-06-30 15:12:06 +02:00
34 lines
2.1 KiB
Markdown
34 lines
2.1 KiB
Markdown
# Evaluating BatDetect2
|
|
This script evaluates a trained model and outputs several plots summarizing the performance. It is used as follows:
|
|
`python path_to_store_images/ path_to_audio_files/ path_to_annotation_file/ path_to_trained_model/`
|
|
|
|
e.g.
|
|
`python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar`
|
|
|
|
By default this will just evaluate the set of test files that are already specified in the model at training time. However, you can also specify a single set of annotations to evaluate using the `--test_file` flag. These must be stored in one annotation file, containing a list of the individual files.
|
|
|
|
e.g.
|
|
`python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar --test_file yucatan_TEST.json`
|
|
|
|
You can also specify if the plots are saved as a .png or .pdf using `--file_type` and you can set title text for a plot using `--title_text`, e.g. `--file_type pdf --title_text "My Dataset Name"`
|
|
|
|
|
|
|
|
|
|
### Comparing to Tadarida-D
|
|
It is also possible to compare to Tadarida-D. For Tadarida-D the following steps are performed:
|
|
- Matches Tadarida's detections to manually annotated calls
|
|
- Trains a RandomForest classifier using Tadarida call features
|
|
- Evaluate the classifier on a held out set
|
|
|
|
Uses precomputed binaries for Tadarida-D from:
|
|
`https://github.com/YvesBas/Tadarida-D/archive/master.zip`
|
|
|
|
Needs to be run using the following arguments:
|
|
`./TadaridaD -t 4 -x 1 ip_dir/`
|
|
-t 4 means 4 threads
|
|
-x 1 means time expansions of 1
|
|
|
|
This will generate a folder called `txt` which contains a corresponding `.ta` file for each input audio file. Example usage is as follows:
|
|
`python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar --td_ip_dir /data1/bat_data/baselines/tadarida_D/`
|