mirror of
https://github.com/macaodha/batdetect2.git
synced 2026-07-07 21:00:10 +02:00
Compare commits
2 Commits
0adb1bbea7
...
67aee0b79c
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
67aee0b79c | ||
|
|
f4cb7d3fc1 |
@ -2,7 +2,8 @@
|
||||
|
||||
Evaluation is not just "run predictions and compute one number".
|
||||
|
||||
The reported metric depends on the evaluation task, the matching rule, and the treatment of clip boundaries and generic labels.
|
||||
The reported metric depends on the evaluation task, the matching rule, and the
|
||||
treatment of clip boundaries and generic labels.
|
||||
|
||||
## Task families answer different questions
|
||||
|
||||
@ -18,7 +19,8 @@ Choose the task that matches the scientific or engineering question.
|
||||
|
||||
## Matching matters
|
||||
|
||||
For sound-event-style tasks, predictions and annotations are matched using an affinity function.
|
||||
For sound-event-style tasks, predictions and annotations are matched using an
|
||||
affinity function.
|
||||
|
||||
Important controls include:
|
||||
|
||||
@ -27,22 +29,28 @@ Important controls include:
|
||||
- `strict_match`,
|
||||
- `ignore_start_end`.
|
||||
|
||||
Small changes here can change the reported metric without changing the underlying predictions.
|
||||
Small changes here can change the reported metric without changing the
|
||||
underlying predictions.
|
||||
|
||||
## Boundary handling matters
|
||||
|
||||
The evaluation base task can exclude events near clip boundaries through `ignore_start_end`.
|
||||
The evaluation base task can exclude events near clip boundaries through
|
||||
`ignore_start_end`.
|
||||
|
||||
This is useful when clip boundaries make matches ambiguous.
|
||||
|
||||
## Generic labels can matter in classification
|
||||
|
||||
Classification tasks can include or exclude generic targets depending on configuration.
|
||||
Classification tasks can include or exclude generic targets depending on
|
||||
configuration.
|
||||
|
||||
That affects what counts as a valid class-level comparison.
|
||||
|
||||
## Related pages
|
||||
|
||||
- Evaluate on a test set: {doc}`../tutorials/evaluate-on-a-test-set`
|
||||
- Evaluation config reference: {doc}`../reference/evaluation-config`
|
||||
- Model output and validation: {doc}`model-output-and-validation`
|
||||
- Evaluate on a test set:
|
||||
{doc}`../tutorials/evaluate-on-a-test-set`
|
||||
- Evaluation config reference:
|
||||
{doc}`../reference/configs/evaluation/evaluation-config`
|
||||
- Model output and validation:
|
||||
{doc}`model-output-and-validation`
|
||||
|
||||
@ -38,6 +38,6 @@ These are related ideas, but not necessarily one-to-one replacements.
|
||||
## Related pages
|
||||
|
||||
- Inspect detection features in Python:
|
||||
{doc}`../how_to/inspect-detection-features-in-python`
|
||||
{doc}`../how_to/analysis/inspect-detection-features-in-python`
|
||||
- Legacy migration guide:
|
||||
{doc}`../legacy/migration-guide`
|
||||
|
||||
@ -2,21 +2,25 @@
|
||||
|
||||
BatDetect2 can write predictions in several output formats.
|
||||
|
||||
Those formats are different views of the same underlying detections, not different model behaviors.
|
||||
Those formats are different views of the same underlying detections, not
|
||||
different model behaviors.
|
||||
|
||||
## Separate the underlying detection from the serialized file
|
||||
|
||||
Internally, the current stack works with clip-level detections containing geometry, detection score, class scores, and features.
|
||||
Internally, the current stack works with clip-level detections containing
|
||||
geometry, detection score, class scores, and features.
|
||||
|
||||
Output formatters then serialize those detections in different ways.
|
||||
|
||||
## Raw outputs are richest
|
||||
|
||||
The `raw` format preserves the broadest structured view of detections and is a good default when you want to inspect or reload predictions later.
|
||||
The `raw` format preserves the broadest structured view of detections and is a
|
||||
good default when you want to inspect or reload predictions later.
|
||||
|
||||
## Tabular outputs are for analysis convenience
|
||||
|
||||
The `parquet` format is convenient for data analysis workflows, but the tabular representation is only one projection of the underlying detection object.
|
||||
The `parquet` format is convenient for data analysis workflows, but the tabular
|
||||
representation is only one projection of the underlying detection object.
|
||||
|
||||
## Legacy-shaped outputs are mainly for compatibility
|
||||
|
||||
@ -26,11 +30,14 @@ Use it when you need compatibility with older downstream tools or workflows.
|
||||
|
||||
## The meaning does not come from the file extension
|
||||
|
||||
Do not assume that a `.json`, `.parquet`, or `.nc` file changes what the model predicted.
|
||||
Do not assume that a `.json`, `.parquet`, or `.nc` file changes what the model
|
||||
predicted.
|
||||
|
||||
It changes how the prediction is packaged and how much detail is retained.
|
||||
|
||||
## Related pages
|
||||
|
||||
- Output formats reference: {doc}`../reference/output-formats`
|
||||
- Outputs config reference: {doc}`../reference/outputs-config`
|
||||
- Output formats reference:
|
||||
{doc}`../reference/configs/outputs/output-formats`
|
||||
- Outputs config reference:
|
||||
{doc}`../reference/configs/outputs/outputs-config`
|
||||
|
||||
@ -1,8 +1,8 @@
|
||||
# Model output and validation
|
||||
|
||||
BatDetect2 outputs model predictions, not ground truth. The same configuration
|
||||
can behave differently across recording conditions, species compositions, and
|
||||
acoustic environments.
|
||||
BatDetect2 outputs model predictions, not ground truth.
|
||||
The same configuration can behave differently across recording conditions,
|
||||
species compositions, and acoustic environments.
|
||||
|
||||
## Why threshold choice matters
|
||||
|
||||
@ -10,8 +10,9 @@ acoustic environments.
|
||||
positives.
|
||||
- Higher thresholds reduce false positives but can miss faint calls.
|
||||
|
||||
No threshold is universally correct. The right setting depends on your survey
|
||||
objectives and tolerance for false positives versus missed detections.
|
||||
No threshold is universally correct.
|
||||
The right setting depends on your survey objectives and tolerance for false
|
||||
positives versus missed detections.
|
||||
|
||||
## Why local validation is required
|
||||
|
||||
@ -26,4 +27,4 @@ Recommended validation checks:
|
||||
3. Repeat checks across sites, seasons, and recorder setups.
|
||||
|
||||
For practical threshold workflows, see
|
||||
{doc}`../how_to/tune-detection-threshold`.
|
||||
{doc}`../how_to/inference/tune-detection-threshold`.
|
||||
|
||||
@ -1,8 +1,9 @@
|
||||
# Postprocessing and thresholds
|
||||
|
||||
After the detector runs on a spectrogram, the model output is still a set of
|
||||
dense prediction tensors. Postprocessing turns that into a final list of call
|
||||
detections with positions, sizes, and class scores.
|
||||
dense prediction tensors.
|
||||
Postprocessing turns that into a final list of call detections with positions,
|
||||
sizes, and class scores.
|
||||
|
||||
## What postprocessing does
|
||||
|
||||
@ -27,9 +28,10 @@ You can tune this behavior per run without retraining the model.
|
||||
|
||||
## Two common threshold controls
|
||||
|
||||
- `detection_threshold`: minimum score required to keep a detection.
|
||||
- `classification_threshold`: minimum class score used when assigning class
|
||||
labels.
|
||||
- `detection_threshold`:
|
||||
minimum score required to keep a detection.
|
||||
- `classification_threshold`:
|
||||
minimum class score used when assigning class labels.
|
||||
|
||||
Both settings shape the final output and should be validated on reviewed local
|
||||
data.
|
||||
@ -39,5 +41,7 @@ data.
|
||||
Tune thresholds on a representative subset first, then lock settings for the
|
||||
full analysis run.
|
||||
|
||||
- How-to: {doc}`../how_to/tune-detection-threshold`
|
||||
- CLI reference: {doc}`../reference/cli/predict`
|
||||
- How-to:
|
||||
{doc}`../how_to/inference/tune-detection-threshold`
|
||||
- CLI reference:
|
||||
{doc}`../reference/cli/predict`
|
||||
|
||||
@ -6,8 +6,9 @@ performance.
|
||||
## Why consistency matters
|
||||
|
||||
The detector is trained on spectrograms produced by a specific preprocessing
|
||||
pipeline. If inference uses different settings, the model can see a shifted
|
||||
input distribution and performance may drop.
|
||||
pipeline.
|
||||
If inference uses different settings, the model can see a shifted input
|
||||
distribution and performance may drop.
|
||||
|
||||
Typical mismatch sources:
|
||||
|
||||
@ -30,7 +31,8 @@ re-validate on reviewed local data.
|
||||
## Related pages
|
||||
|
||||
- Configure audio preprocessing:
|
||||
{doc}`../how_to/configure-audio-preprocessing`
|
||||
{doc}`../how_to/data/configure-audio-preprocessing`
|
||||
- Configure spectrogram preprocessing:
|
||||
{doc}`../how_to/configure-spectrogram-preprocessing`
|
||||
- Preprocessing config reference: {doc}`../reference/preprocessing-config`
|
||||
{doc}`../how_to/data/configure-spectrogram-preprocessing`
|
||||
- Preprocessing config reference:
|
||||
{doc}`../reference/configs/data/preprocessing-config`
|
||||
|
||||
@ -11,8 +11,8 @@ At training time, the target system:
|
||||
2. assigns a classification label (or none for non-specific class matches),
|
||||
3. maps event geometry into position and size targets.
|
||||
|
||||
This behaviour is configured through `TargetConfig`,
|
||||
`TargetClassConfig`, and ROI mapper settings.
|
||||
This behaviour is configured through `TargetConfig`, `TargetClassConfig`, and
|
||||
ROI mapper settings.
|
||||
|
||||
## Decoding path (model outputs -> tags and geometry)
|
||||
|
||||
@ -24,7 +24,8 @@ annotations.
|
||||
|
||||
## Why this matters
|
||||
|
||||
Target definitions are not just metadata. They directly shape:
|
||||
Target definitions are not just metadata.
|
||||
They directly shape:
|
||||
|
||||
- what events are treated as positive examples,
|
||||
- which class names the model learns,
|
||||
@ -34,7 +35,11 @@ Small changes here can alter both training outcomes and prediction semantics.
|
||||
|
||||
## Related pages
|
||||
|
||||
- Configure detection target logic: {doc}`../how_to/configure-target-definitions`
|
||||
- Configure class mapping: {doc}`../how_to/define-target-classes`
|
||||
- Configure ROI mapping: {doc}`../how_to/configure-roi-mapping`
|
||||
- Target config reference: {doc}`../reference/targets-config-workflow`
|
||||
- Configure detection target logic:
|
||||
{doc}`../how_to/data/configure-target-definitions`
|
||||
- Configure class mapping:
|
||||
{doc}`../how_to/data/define-target-classes`
|
||||
- Configure ROI mapping:
|
||||
{doc}`../how_to/data/configure-roi-mapping`
|
||||
- Target config reference:
|
||||
{doc}`../reference/configs/data/targets-config-workflow`
|
||||
|
||||
@ -5,50 +5,64 @@
|
||||
### Do I need Python knowledge to use batdetect2?
|
||||
|
||||
Not much.
|
||||
If you only want to run the model on your own recordings, you can use the CLI and follow the steps in {doc}`getting_started`.
|
||||
If you only want to run the model on your own recordings, you can use the CLI
|
||||
and follow the steps in {doc}`getting_started`.
|
||||
|
||||
Some command-line familiarity helps, but you do not need to write Python code for standard inference workflows.
|
||||
Some command-line familiarity helps, but you do not need to write Python code
|
||||
for standard inference workflows.
|
||||
|
||||
### Are there plans for an R version?
|
||||
|
||||
Not currently.
|
||||
Output files are plain formats (for example CSV/JSON), so you can read and analyze them in R or other environments.
|
||||
Output files are plain formats (for example CSV/JSON), so you can read and
|
||||
analyze them in R or other environments.
|
||||
|
||||
### I cannot get installation working. What should I do?
|
||||
|
||||
First, re-check {doc}`getting_started` and confirm your environment is active.
|
||||
If it still fails, open an issue with your OS, install method, and full error output: [GitHub Issues](https://github.com/macaodha/batdetect2/issues).
|
||||
If it still fails, open an issue with your OS, install method, and full error
|
||||
output:
|
||||
[GitHub Issues](https://github.com/macaodha/batdetect2/issues).
|
||||
|
||||
## Model behavior and performance
|
||||
|
||||
### The model does not perform well on my data
|
||||
|
||||
This usually means your data distribution differs from training data.
|
||||
The best next step is to validate on reviewed local data and then fine-tune/train on your own annotations if needed.
|
||||
The best next step is to validate on reviewed local data and then
|
||||
fine-tune/train on your own annotations if needed.
|
||||
|
||||
### The model confuses insects/noise with bats
|
||||
|
||||
This can happen, especially when recording conditions differ from training conditions.
|
||||
This can happen, especially when recording conditions differ from training
|
||||
conditions.
|
||||
Threshold tuning and training with local annotations can improve results.
|
||||
|
||||
See {doc}`how_to/tune-detection-threshold`.
|
||||
See {doc}`how_to/inference/tune-detection-threshold`.
|
||||
|
||||
### The model struggles with feeding buzzes or social calls
|
||||
|
||||
This is a known limitation of available training data in some settings.
|
||||
If you have high-quality annotated examples, they are valuable for improving models.
|
||||
If you have high-quality annotated examples, they are valuable for improving
|
||||
models.
|
||||
|
||||
### Calls in the same sequence are predicted as different species
|
||||
|
||||
Currently we do not do any sophisticated post processing on the results output by the model.
|
||||
Currently we do not do any sophisticated post processing on the results output
|
||||
by the model.
|
||||
We return a probability associated with each species for each call.
|
||||
You can use these predictions to clean up the noisy predictions for sequences of calls.
|
||||
You can use these predictions to clean up the noisy predictions for sequences of
|
||||
calls.
|
||||
|
||||
### Can I trust model outputs for biodiversity conclusions?
|
||||
|
||||
The models developed and shared as part of this repository should be used with caution.
|
||||
While they have been evaluated on held out audio data, great care should be taken when using the model outputs for any form of biodiversity assessment.
|
||||
Your data may differ, and as a result it is very strongly recommended that you validate the model first using data with known species to ensure that the outputs can be trusted.
|
||||
The models developed and shared as part of this repository should be used with
|
||||
caution.
|
||||
While they have been evaluated on held out audio data, great care should be
|
||||
taken when using the model outputs for any form of biodiversity assessment.
|
||||
Your data may differ, and as a result it is very strongly recommended that you
|
||||
validate the model first using data with known species to ensure that the
|
||||
outputs can be trusted.
|
||||
|
||||
### The pipeline is slow
|
||||
|
||||
@ -65,7 +79,8 @@ You can train/fine-tune with your own annotated data and species labels.
|
||||
### Does this work on frequency-division or zero-crossing recordings?
|
||||
|
||||
Not directly.
|
||||
The workflow assumes audio can be converted to spectrograms from the raw waveform.
|
||||
The workflow assumes audio can be converted to spectrograms from the raw
|
||||
waveform.
|
||||
|
||||
### Can this be used for non-bat bioacoustics (for example insects or birds)?
|
||||
|
||||
|
||||
11
docs/source/how_to/analysis-in-python.md
Normal file
11
docs/source/how_to/analysis-in-python.md
Normal file
@ -0,0 +1,11 @@
|
||||
# Analysis in Python
|
||||
|
||||
Use this section when you want to inspect model outputs more closely from
|
||||
Python, beyond the default saved predictions.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
|
||||
analysis/inspect-class-scores-in-python
|
||||
analysis/inspect-detection-features-in-python
|
||||
```
|
||||
@ -1,6 +1,7 @@
|
||||
# How to inspect detection features in Python
|
||||
|
||||
Use this guide when you want the per-detection feature vectors exposed by the current API.
|
||||
Use this guide when you want the per-detection feature vectors exposed by the
|
||||
current API.
|
||||
|
||||
## Get the feature vector for one detection
|
||||
|
||||
@ -34,16 +35,21 @@ They can be useful for:
|
||||
|
||||
They do not replace validation.
|
||||
|
||||
They also do not automatically have a one-to-one interpretation as ecological variables.
|
||||
They also do not automatically have a one-to-one interpretation as ecological
|
||||
variables.
|
||||
|
||||
## Save predictions with features included
|
||||
|
||||
If you need features on disk, use an output format that supports them, such as `raw` or `parquet`, and keep feature inclusion enabled.
|
||||
If you need features on disk, use an output format that supports them, such as
|
||||
`raw` or `parquet`, and keep feature inclusion enabled.
|
||||
|
||||
See {doc}`save-predictions-in-different-output-formats`.
|
||||
See {doc}`../inference/save-predictions-in-different-output-formats`.
|
||||
|
||||
## Related pages
|
||||
|
||||
- Understanding features and embeddings: {doc}`../explanation/extracted-features-and-embeddings`
|
||||
- Output formats reference: {doc}`../reference/output-formats`
|
||||
- API reference: {doc}`../reference/api`
|
||||
- Understanding features and embeddings:
|
||||
{doc}`../../explanation/extracted-features-and-embeddings`
|
||||
- Output formats reference:
|
||||
{doc}`../../reference/configs/outputs/output-formats`
|
||||
- API reference:
|
||||
{doc}`../../reference/api`
|
||||
16
docs/source/how_to/data-and-targets.md
Normal file
16
docs/source/how_to/data-and-targets.md
Normal file
@ -0,0 +1,16 @@
|
||||
# Data and Targets
|
||||
|
||||
Use this section when you are preparing datasets, preprocessing audio, or
|
||||
defining the targets used for training and evaluation.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
|
||||
data/configure-aoef-dataset
|
||||
data/import-legacy-batdetect2-annotations
|
||||
data/configure-audio-preprocessing
|
||||
data/configure-spectrogram-preprocessing
|
||||
data/configure-target-definitions
|
||||
data/define-target-classes
|
||||
data/configure-roi-mapping
|
||||
```
|
||||
@ -5,7 +5,8 @@ including exports from Whombat.
|
||||
|
||||
## 1) Add an AOEF source entry
|
||||
|
||||
In your dataset config, add a source with `format: aoef`.
|
||||
In your dataset config, add a source with `format:
|
||||
aoef`.
|
||||
|
||||
```yaml
|
||||
sources:
|
||||
@ -49,5 +50,7 @@ batdetect2 data summary path/to/dataset.yaml
|
||||
|
||||
## 4) Continue to training or evaluation
|
||||
|
||||
- For training: {doc}`../tutorials/train-a-custom-model`
|
||||
- For field-level reference: {doc}`../reference/data-sources`
|
||||
- For training:
|
||||
{doc}`../../tutorials/train-a-custom-model`
|
||||
- For field-level reference:
|
||||
{doc}`../../reference/configs/data/data-sources`
|
||||
@ -63,4 +63,4 @@ before full-batch runs.
|
||||
- Spectrogram settings:
|
||||
{doc}`configure-spectrogram-preprocessing`
|
||||
- Preprocessing config reference:
|
||||
{doc}`../reference/preprocessing-config`
|
||||
{doc}`../../reference/configs/data/preprocessing-config`
|
||||
@ -52,6 +52,9 @@ roi:
|
||||
|
||||
## Related pages
|
||||
|
||||
- Target definitions: {doc}`configure-target-definitions`
|
||||
- Class definitions: {doc}`define-target-classes`
|
||||
- Target encoding overview: {doc}`../explanation/target-encoding-and-decoding`
|
||||
- Target definitions:
|
||||
{doc}`configure-target-definitions`
|
||||
- Class definitions:
|
||||
{doc}`define-target-classes`
|
||||
- Target encoding overview:
|
||||
{doc}`../../explanation/target-encoding-and-decoding`
|
||||
@ -55,5 +55,7 @@ Large mismatches can degrade model performance.
|
||||
|
||||
## Related pages
|
||||
|
||||
- Why consistency matters: {doc}`../explanation/preprocessing-consistency`
|
||||
- Preprocessing config reference: {doc}`../reference/preprocessing-config`
|
||||
- Why consistency matters:
|
||||
{doc}`../../explanation/preprocessing-consistency`
|
||||
- Preprocessing config reference:
|
||||
{doc}`../../reference/configs/data/preprocessing-config`
|
||||
@ -53,6 +53,9 @@ selection logic keeps the events you expect.
|
||||
|
||||
## Related pages
|
||||
|
||||
- Class mapping: {doc}`define-target-classes`
|
||||
- ROI mapping: {doc}`configure-roi-mapping`
|
||||
- Targets reference: {doc}`../reference/targets-config-workflow`
|
||||
- Class mapping:
|
||||
{doc}`define-target-classes`
|
||||
- ROI mapping:
|
||||
{doc}`configure-roi-mapping`
|
||||
- Targets reference:
|
||||
{doc}`../../reference/configs/data/targets-config-workflow`
|
||||
@ -1,7 +1,6 @@
|
||||
# How to define target classes
|
||||
|
||||
Use this guide to map annotations to classification labels used during
|
||||
training.
|
||||
Use this guide to map annotations to classification labels used during training.
|
||||
|
||||
## 1) Add classification target entries
|
||||
|
||||
@ -54,6 +53,9 @@ classification_targets:
|
||||
|
||||
## Related pages
|
||||
|
||||
- Detection-target filtering: {doc}`configure-target-definitions`
|
||||
- ROI mapping: {doc}`configure-roi-mapping`
|
||||
- Targets config reference: {doc}`../reference/targets-config-workflow`
|
||||
- Detection-target filtering:
|
||||
{doc}`configure-target-definitions`
|
||||
- ROI mapping:
|
||||
{doc}`configure-roi-mapping`
|
||||
- Targets config reference:
|
||||
{doc}`../../reference/configs/data/targets-config-workflow`
|
||||
@ -4,12 +4,15 @@ Use this guide if your annotations are in older batdetect2 JSON formats.
|
||||
|
||||
Two legacy formats are supported:
|
||||
|
||||
- `batdetect2`: one annotation JSON file per recording
|
||||
- `batdetect2_file`: one merged JSON file for many recordings
|
||||
- `batdetect2`:
|
||||
one annotation JSON file per recording
|
||||
- `batdetect2_file`:
|
||||
one merged JSON file for many recordings
|
||||
|
||||
## 1) Choose the correct source format
|
||||
|
||||
Directory-based annotations (`format: batdetect2`):
|
||||
Directory-based annotations (`format:
|
||||
batdetect2`):
|
||||
|
||||
```yaml
|
||||
sources:
|
||||
@ -19,7 +22,8 @@ sources:
|
||||
annotations_dir: /path/to/annotation_json_dir
|
||||
```
|
||||
|
||||
Merged annotation file (`format: batdetect2_file`):
|
||||
Merged annotation file (`format:
|
||||
batdetect2_file`):
|
||||
|
||||
```yaml
|
||||
sources:
|
||||
@ -61,6 +65,9 @@ batdetect2 data convert path/to/dataset.yaml --output path/to/output.json
|
||||
|
||||
## 4) Continue with current workflows
|
||||
|
||||
- Run predictions: {doc}`run-batch-predictions`
|
||||
- Train on imported data: {doc}`../tutorials/train-a-custom-model`
|
||||
- Field-level reference: {doc}`../reference/data-sources`
|
||||
- Run predictions:
|
||||
{doc}`../inference/run-batch-predictions`
|
||||
- Train on imported data:
|
||||
{doc}`../../tutorials/train-a-custom-model`
|
||||
- Field-level reference:
|
||||
{doc}`../../reference/configs/data/data-sources`
|
||||
11
docs/source/how_to/evaluation.md
Normal file
11
docs/source/how_to/evaluation.md
Normal file
@ -0,0 +1,11 @@
|
||||
# Evaluation
|
||||
|
||||
Use this section when you want to choose evaluation tasks or understand the
|
||||
artifacts produced by an evaluation run.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
|
||||
evaluation/choose-and-configure-evaluation-tasks
|
||||
evaluation/interpret-evaluation-outputs
|
||||
```
|
||||
@ -6,25 +6,15 @@ tutorial.
|
||||
Use this section when you already know the basic workflow and want help with one
|
||||
specific task.
|
||||
|
||||
The guides are grouped by topic so you can jump straight to the part of the
|
||||
workflow you are working on.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
|
||||
choose-a-model
|
||||
choose-an-inference-input-mode
|
||||
run-batch-predictions
|
||||
tune-inference-clipping
|
||||
tune-detection-threshold
|
||||
inspect-class-scores-in-python
|
||||
inspect-detection-features-in-python
|
||||
save-predictions-in-different-output-formats
|
||||
fine-tune-from-a-checkpoint
|
||||
choose-and-configure-evaluation-tasks
|
||||
interpret-evaluation-outputs
|
||||
configure-aoef-dataset
|
||||
import-legacy-batdetect2-annotations
|
||||
configure-audio-preprocessing
|
||||
configure-spectrogram-preprocessing
|
||||
configure-target-definitions
|
||||
define-target-classes
|
||||
configure-roi-mapping
|
||||
inference-and-outputs
|
||||
analysis-in-python
|
||||
training-and-fine-tuning
|
||||
evaluation
|
||||
data-and-targets
|
||||
```
|
||||
|
||||
15
docs/source/how_to/inference-and-outputs.md
Normal file
15
docs/source/how_to/inference-and-outputs.md
Normal file
@ -0,0 +1,15 @@
|
||||
# Inference and Outputs
|
||||
|
||||
Use this section when you want to run predictions, choose how inference runs, or
|
||||
control what gets written to disk.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
|
||||
inference/choose-a-model
|
||||
inference/choose-an-inference-input-mode
|
||||
inference/run-batch-predictions
|
||||
inference/tune-inference-clipping
|
||||
inference/tune-detection-threshold
|
||||
inference/save-predictions-in-different-output-formats
|
||||
```
|
||||
@ -101,12 +101,12 @@ api = BatDetect2API.from_checkpoint(
|
||||
## Related pages
|
||||
|
||||
- Run inference on a folder:
|
||||
{doc}`../tutorials/run-inference-on-folder`
|
||||
{doc}`../../tutorials/run-inference-on-folder`
|
||||
- `BatDetect2API` reference:
|
||||
{doc}`../reference/api`
|
||||
{doc}`../../reference/api`
|
||||
- Process command reference:
|
||||
{doc}`../reference/cli/predict`
|
||||
{doc}`../../reference/cli/predict`
|
||||
- Train a custom model:
|
||||
{doc}`../tutorials/train-a-custom-model`
|
||||
{doc}`../../tutorials/train-a-custom-model`
|
||||
- Fine-tune from a checkpoint:
|
||||
{doc}`fine-tune-from-a-checkpoint`
|
||||
{doc}`../training/fine-tune-from-a-checkpoint`
|
||||
@ -68,4 +68,4 @@ recording paths before inference.
|
||||
- Tune inference clipping:
|
||||
{doc}`tune-inference-clipping`
|
||||
- Process command reference:
|
||||
{doc}`../reference/cli/predict`
|
||||
{doc}`../../reference/cli/predict`
|
||||
@ -66,8 +66,8 @@ Check a reviewed subset before applying clipping changes to a full project.
|
||||
## Related pages
|
||||
|
||||
- Inference config reference:
|
||||
{doc}`../reference/inference-config`
|
||||
{doc}`../../reference/configs/inference/inference-config`
|
||||
- Run batch predictions:
|
||||
{doc}`run-batch-predictions`
|
||||
- Understanding the pipeline:
|
||||
{doc}`../explanation/pipeline-overview`
|
||||
{doc}`../../explanation/pipeline-overview`
|
||||
10
docs/source/how_to/training-and-fine-tuning.md
Normal file
10
docs/source/how_to/training-and-fine-tuning.md
Normal file
@ -0,0 +1,10 @@
|
||||
# Training and Fine-Tuning
|
||||
|
||||
Use this section when you already have a training workflow and want help with a
|
||||
specific fine-tuning step.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
|
||||
training/fine-tune-from-a-checkpoint
|
||||
```
|
||||
@ -25,8 +25,8 @@ For more detail on the underlying approach, see the pre-print:
|
||||
[Towards a General Approach for Bat Echolocation Detection and Classification](https://www.biorxiv.org/content/10.1101/2022.12.14.520490v1)
|
||||
|
||||
```{warning}
|
||||
Treat outputs as model predictions, not ground truth.
|
||||
Always validate on reviewed local data before using results for ecological inference.
|
||||
Model outputs are predictions, not ground truth, and may be incorrect in subtle or significant ways.
|
||||
We strongly encourage validating results on reviewed local data before using them for downstream analyses.
|
||||
```
|
||||
|
||||
## What can I do with it?
|
||||
|
||||
@ -34,6 +34,6 @@ Defined in `batdetect2.api_v2`.
|
||||
- Python tutorial:
|
||||
{doc}`../tutorials/integrate-with-a-python-pipeline`
|
||||
- Outputs config reference:
|
||||
{doc}`outputs-config`
|
||||
{doc}`configs/outputs/outputs-config`
|
||||
- Output formats reference:
|
||||
{doc}`output-formats`
|
||||
{doc}`configs/outputs/output-formats`
|
||||
|
||||
@ -33,8 +33,8 @@ Prefer `batdetect2 process directory` for new workflows.
|
||||
## Related pages
|
||||
|
||||
- {doc}`../../tutorials/run-inference-on-folder`
|
||||
- {doc}`../../how_to/run-batch-predictions`
|
||||
- {doc}`../../how_to/tune-detection-threshold`
|
||||
- {doc}`../../how_to/inference/run-batch-predictions`
|
||||
- {doc}`../../how_to/inference/tune-detection-threshold`
|
||||
- {doc}`../configs`
|
||||
|
||||
```{toctree}
|
||||
|
||||
@ -3,16 +3,23 @@ Config reference
|
||||
|
||||
BatDetect2 uses separate config objects for different workflow surfaces.
|
||||
|
||||
Use the dedicated reference pages for each config family:
|
||||
Use this section when you need exact config fields for training, inference,
|
||||
evaluation, outputs, preprocessing, postprocessing, or target definitions.
|
||||
|
||||
- model config
|
||||
- training config
|
||||
- logging config
|
||||
- inference config
|
||||
- evaluation config
|
||||
- outputs config
|
||||
- preprocessing config
|
||||
- postprocess config
|
||||
- targets config workflow
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
configs/training/model-config
|
||||
configs/training/training-config
|
||||
configs/training/logging-config
|
||||
configs/inference/inference-config
|
||||
configs/evaluation/evaluation-config
|
||||
configs/outputs/outputs-config
|
||||
configs/outputs/output-formats
|
||||
configs/outputs/output-transforms
|
||||
configs/data/data-sources
|
||||
configs/data/preprocessing-config
|
||||
configs/data/postprocess-config
|
||||
configs/data/targets-config-workflow
|
||||
|
||||
Example config files live under `example_data/configs/`.
|
||||
|
||||
@ -24,16 +24,19 @@ Optional fields:
|
||||
- `description`
|
||||
- `filter`
|
||||
|
||||
`filter` is only used when `annotations_path` points to an
|
||||
`AnnotationProject`.
|
||||
`filter` is only used when `annotations_path` points to an `AnnotationProject`.
|
||||
|
||||
AOEF filter options:
|
||||
|
||||
- `only_completed` (default: `true`)
|
||||
- `only_verified` (default: `false`)
|
||||
- `exclude_issues` (default: `true`)
|
||||
- `only_completed` (default:
|
||||
`true`)
|
||||
- `only_verified` (default:
|
||||
`false`)
|
||||
- `exclude_issues` (default:
|
||||
`true`)
|
||||
|
||||
Use `filter: null` to disable project filtering.
|
||||
Use `filter:
|
||||
null` to disable project filtering.
|
||||
|
||||
## Legacy per-file (`format: batdetect2`)
|
||||
|
||||
@ -65,12 +68,15 @@ Optional fields:
|
||||
|
||||
Legacy filter options:
|
||||
|
||||
- `only_annotated` (default: `true`)
|
||||
- `exclude_issues` (default: `true`)
|
||||
- `only_annotated` (default:
|
||||
`true`)
|
||||
- `exclude_issues` (default:
|
||||
`true`)
|
||||
|
||||
Use `filter: null` to disable filtering.
|
||||
Use `filter:
|
||||
null` to disable filtering.
|
||||
|
||||
## Related guides
|
||||
|
||||
- {doc}`../how_to/configure-aoef-dataset`
|
||||
- {doc}`../how_to/import-legacy-batdetect2-annotations`
|
||||
- {doc}`../../../how_to/data/configure-aoef-dataset`
|
||||
- {doc}`../../../how_to/data/import-legacy-batdetect2-annotations`
|
||||
@ -18,14 +18,20 @@ Defined in `batdetect2.postprocess.config`.
|
||||
|
||||
## Defaults
|
||||
|
||||
- `detection_threshold`: `0.01`
|
||||
- `classification_threshold`: `0.1`
|
||||
- `top_k_per_sec`: `100`
|
||||
- `detection_threshold`:
|
||||
`0.01`
|
||||
- `classification_threshold`:
|
||||
`0.1`
|
||||
- `top_k_per_sec`:
|
||||
`100`
|
||||
|
||||
`nms_kernel_size` defaults to the library constant used by the NMS module.
|
||||
|
||||
## Related pages
|
||||
|
||||
- Threshold behaviour: {doc}`../explanation/postprocessing-and-thresholds`
|
||||
- Threshold tuning workflow: {doc}`../how_to/tune-detection-threshold`
|
||||
- CLI predict options: {doc}`cli/predict`
|
||||
- Threshold behaviour:
|
||||
{doc}`../../../explanation/postprocessing-and-thresholds`
|
||||
- Threshold tuning workflow:
|
||||
{doc}`../../../how_to/inference/tune-detection-threshold`
|
||||
- CLI predict options:
|
||||
{doc}`../../cli/predict`
|
||||
@ -8,9 +8,12 @@ Defined in `batdetect2.audio.loader`.
|
||||
|
||||
Fields:
|
||||
|
||||
- `samplerate` (int): target audio sample rate in Hz.
|
||||
- `resample.enabled` (bool): whether to resample loaded audio.
|
||||
- `resample.method` (`poly` or `fourier`): resampling method.
|
||||
- `samplerate` (int):
|
||||
target audio sample rate in Hz.
|
||||
- `resample.enabled` (bool):
|
||||
whether to resample loaded audio.
|
||||
- `resample.method` (`poly` or `fourier`):
|
||||
resampling method.
|
||||
|
||||
## Model preprocessing config (`PreprocessingConfig`)
|
||||
|
||||
@ -18,11 +21,16 @@ Defined in `batdetect2.preprocess.config`.
|
||||
|
||||
Top-level fields:
|
||||
|
||||
- `audio_transforms`: ordered waveform transforms.
|
||||
- `stft`: STFT parameters.
|
||||
- `frequencies`: spectrogram frequency range.
|
||||
- `spectrogram_transforms`: ordered spectrogram transforms.
|
||||
- `size`: final resize settings.
|
||||
- `audio_transforms`:
|
||||
ordered waveform transforms.
|
||||
- `stft`:
|
||||
STFT parameters.
|
||||
- `frequencies`:
|
||||
spectrogram frequency range.
|
||||
- `spectrogram_transforms`:
|
||||
ordered spectrogram transforms.
|
||||
- `size`:
|
||||
final resize settings.
|
||||
|
||||
### `audio_transforms` built-ins
|
||||
|
||||
@ -44,7 +52,8 @@ Top-level fields:
|
||||
### `spectrogram_transforms` built-ins
|
||||
|
||||
- `pcen`
|
||||
- `scale_amplitude` (`scale: db|power`)
|
||||
- `scale_amplitude` (`scale:
|
||||
db|power`)
|
||||
- `spectral_mean_subtraction`
|
||||
- `peak_normalize`
|
||||
|
||||
@ -55,7 +64,9 @@ Top-level fields:
|
||||
|
||||
## Related pages
|
||||
|
||||
- Audio preprocessing how-to: {doc}`../how_to/configure-audio-preprocessing`
|
||||
- Audio preprocessing how-to:
|
||||
{doc}`../../../how_to/data/configure-audio-preprocessing`
|
||||
- Spectrogram preprocessing how-to:
|
||||
{doc}`../how_to/configure-spectrogram-preprocessing`
|
||||
- Why consistency matters: {doc}`../explanation/preprocessing-consistency`
|
||||
{doc}`../../../how_to/data/configure-spectrogram-preprocessing`
|
||||
- Why consistency matters:
|
||||
{doc}`../../../explanation/preprocessing-consistency`
|
||||
@ -0,0 +1,78 @@
|
||||
# Targets config workflow reference
|
||||
|
||||
This page summarizes the target-definition configuration used by batdetect2.
|
||||
|
||||
## `TargetConfig`
|
||||
|
||||
Defined in `batdetect2.targets.config`.
|
||||
|
||||
Fields:
|
||||
|
||||
- `detection_target`:
|
||||
one `TargetClassConfig` defining detection eligibility.
|
||||
- `classification_targets`:
|
||||
list of `TargetClassConfig` entries for class encoding/decoding.
|
||||
- `roi`:
|
||||
ROI mapping config with `default` mapper and optional per-class `overrides`.
|
||||
|
||||
## `TargetClassConfig`
|
||||
|
||||
Defined in `batdetect2.targets.classes`.
|
||||
|
||||
Fields:
|
||||
|
||||
- `name`:
|
||||
class label name.
|
||||
- `tags`:
|
||||
tag list used for matching (shortcut for `match_if`).
|
||||
- `match_if`:
|
||||
explicit condition config (`match_if` is accepted as alias).
|
||||
- `assign_tags`:
|
||||
tags used when decoding this class.
|
||||
|
||||
`tags` and `match_if` are mutually exclusive.
|
||||
|
||||
## Supported condition config types
|
||||
|
||||
Built from `batdetect2.data.conditions`.
|
||||
|
||||
- `has_tag`
|
||||
- `has_all_tags`
|
||||
- `has_any_tag`
|
||||
- `duration`
|
||||
- `frequency`
|
||||
- `all_of`
|
||||
- `any_of`
|
||||
- `not`
|
||||
|
||||
## ROI mapper config
|
||||
|
||||
`roi.default` and each `roi.overrides.<class_name>` entry support built-in
|
||||
mappers including:
|
||||
|
||||
- `anchor_bbox`
|
||||
- `peak_energy_bbox`
|
||||
|
||||
Key `anchor_bbox` fields:
|
||||
|
||||
- `anchor`
|
||||
- `time_scale`
|
||||
- `frequency_scale`
|
||||
|
||||
Top-level ROI mapping shape:
|
||||
|
||||
- `default`:
|
||||
fallback mapper used for all classes.
|
||||
- `overrides`:
|
||||
optional mapping from class name to mapper config.
|
||||
|
||||
## Related pages
|
||||
|
||||
- Detection target setup:
|
||||
{doc}`../../../how_to/data/configure-target-definitions`
|
||||
- Class setup:
|
||||
{doc}`../../../how_to/data/define-target-classes`
|
||||
- ROI setup:
|
||||
{doc}`../../../how_to/data/configure-roi-mapping`
|
||||
- Concept overview:
|
||||
{doc}`../../../explanation/target-encoding-and-decoding`
|
||||
@ -1,6 +1,7 @@
|
||||
# Evaluation config reference
|
||||
|
||||
`EvaluationConfig` defines which evaluation tasks run and which plots they generate.
|
||||
`EvaluationConfig` defines which evaluation tasks run and which plots they
|
||||
generate.
|
||||
|
||||
Defined in `batdetect2.evaluate.config`.
|
||||
|
||||
@ -41,6 +42,9 @@ The default evaluation config starts with:
|
||||
|
||||
## Related pages
|
||||
|
||||
- Choose and configure evaluation tasks: {doc}`../how_to/choose-and-configure-evaluation-tasks`
|
||||
- Evaluation concepts: {doc}`../explanation/evaluation-concepts-and-matching`
|
||||
- Evaluate CLI reference: {doc}`cli/evaluate`
|
||||
- Choose and configure evaluation tasks:
|
||||
{doc}`../../../how_to/evaluation/choose-and-configure-evaluation-tasks`
|
||||
- Evaluation concepts:
|
||||
{doc}`../../../explanation/evaluation-concepts-and-matching`
|
||||
- Evaluate CLI reference:
|
||||
{doc}`../../cli/evaluate`
|
||||
@ -1,6 +1,7 @@
|
||||
# Inference config reference
|
||||
|
||||
`InferenceConfig` controls how files are clipped and batched during prediction-time workflows.
|
||||
`InferenceConfig` controls how files are clipped and batched during
|
||||
prediction-time workflows.
|
||||
|
||||
Defined in `batdetect2.inference.config`.
|
||||
|
||||
@ -37,5 +38,7 @@ Override `InferenceConfig` when:
|
||||
|
||||
## Related pages
|
||||
|
||||
- Tune inference clipping: {doc}`../how_to/tune-inference-clipping`
|
||||
- Predict CLI reference: {doc}`cli/predict`
|
||||
- Tune inference clipping:
|
||||
{doc}`../../../how_to/inference/tune-inference-clipping`
|
||||
- Predict CLI reference:
|
||||
{doc}`../../cli/predict`
|
||||
@ -70,6 +70,6 @@ It can also write legacy `_cnn_features.csv` sidecars when
|
||||
- Outputs config:
|
||||
{doc}`outputs-config`
|
||||
- Save predictions in different output formats:
|
||||
{doc}`../how_to/save-predictions-in-different-output-formats`
|
||||
{doc}`../../../how_to/inference/save-predictions-in-different-output-formats`
|
||||
- Understanding formatted outputs:
|
||||
{doc}`../explanation/interpreting-formatted-outputs`
|
||||
{doc}`../../../explanation/interpreting-formatted-outputs`
|
||||
@ -33,5 +33,7 @@ Built-in examples include:
|
||||
|
||||
## Related pages
|
||||
|
||||
- Outputs config: {doc}`outputs-config`
|
||||
- Understanding outputs: {doc}`../explanation/interpreting-formatted-outputs`
|
||||
- Outputs config:
|
||||
{doc}`outputs-config`
|
||||
- Understanding outputs:
|
||||
{doc}`../../../explanation/interpreting-formatted-outputs`
|
||||
@ -38,4 +38,4 @@ If no outputs config is provided, the CLI still uses its command defaults.
|
||||
- Output transforms:
|
||||
{doc}`output-transforms`
|
||||
- Save predictions in different output formats:
|
||||
{doc}`../how_to/save-predictions-in-different-output-formats`
|
||||
{doc}`../../../how_to/inference/save-predictions-in-different-output-formats`
|
||||
@ -30,8 +30,8 @@ Example files live under `example_data/configs/`, including
|
||||
## Related pages
|
||||
|
||||
- Preprocessing config:
|
||||
{doc}`preprocessing-config`
|
||||
{doc}`../data/preprocessing-config`
|
||||
- Postprocess config:
|
||||
{doc}`postprocess-config`
|
||||
{doc}`../data/postprocess-config`
|
||||
- Train command reference:
|
||||
{doc}`cli/train`
|
||||
{doc}`../../cli/train`
|
||||
@ -43,8 +43,8 @@ Example files live under `example_data/configs/`, including
|
||||
## Related pages
|
||||
|
||||
- Evaluation config:
|
||||
{doc}`evaluation-config`
|
||||
{doc}`../evaluation/evaluation-config`
|
||||
- Train command reference:
|
||||
{doc}`cli/train`
|
||||
{doc}`../../cli/train`
|
||||
- Fine-tune from a checkpoint:
|
||||
{doc}`../how_to/fine-tune-from-a-checkpoint`
|
||||
{doc}`../../../how_to/training/fine-tune-from-a-checkpoint`
|
||||
@ -11,18 +11,6 @@ details, or Python API entries.
|
||||
cli/index
|
||||
api
|
||||
detections
|
||||
model-config
|
||||
training-config
|
||||
logging-config
|
||||
inference-config
|
||||
evaluation-config
|
||||
outputs-config
|
||||
output-formats
|
||||
output-transforms
|
||||
data-sources
|
||||
preprocessing-config
|
||||
postprocess-config
|
||||
targets-config-workflow
|
||||
configs
|
||||
targets
|
||||
```
|
||||
|
||||
@ -1,67 +0,0 @@
|
||||
# Targets config workflow reference
|
||||
|
||||
This page summarizes the target-definition configuration used by batdetect2.
|
||||
|
||||
## `TargetConfig`
|
||||
|
||||
Defined in `batdetect2.targets.config`.
|
||||
|
||||
Fields:
|
||||
|
||||
- `detection_target`: one `TargetClassConfig` defining detection eligibility.
|
||||
- `classification_targets`: list of `TargetClassConfig` entries for class
|
||||
encoding/decoding.
|
||||
- `roi`: ROI mapping config with `default` mapper and optional per-class
|
||||
`overrides`.
|
||||
|
||||
## `TargetClassConfig`
|
||||
|
||||
Defined in `batdetect2.targets.classes`.
|
||||
|
||||
Fields:
|
||||
|
||||
- `name`: class label name.
|
||||
- `tags`: tag list used for matching (shortcut for `match_if`).
|
||||
- `match_if`: explicit condition config (`match_if` is accepted as alias).
|
||||
- `assign_tags`: tags used when decoding this class.
|
||||
|
||||
`tags` and `match_if` are mutually exclusive.
|
||||
|
||||
## Supported condition config types
|
||||
|
||||
Built from `batdetect2.data.conditions`.
|
||||
|
||||
- `has_tag`
|
||||
- `has_all_tags`
|
||||
- `has_any_tag`
|
||||
- `duration`
|
||||
- `frequency`
|
||||
- `all_of`
|
||||
- `any_of`
|
||||
- `not`
|
||||
|
||||
## ROI mapper config
|
||||
|
||||
`roi.default` and each `roi.overrides.<class_name>` entry support built-in
|
||||
mappers including:
|
||||
|
||||
- `anchor_bbox`
|
||||
- `peak_energy_bbox`
|
||||
|
||||
Key `anchor_bbox` fields:
|
||||
|
||||
- `anchor`
|
||||
- `time_scale`
|
||||
- `frequency_scale`
|
||||
|
||||
Top-level ROI mapping shape:
|
||||
|
||||
- `default`: fallback mapper used for all classes.
|
||||
- `overrides`: optional mapping from class name to mapper config.
|
||||
|
||||
## Related pages
|
||||
|
||||
- Detection target setup: {doc}`../how_to/configure-target-definitions`
|
||||
- Class setup: {doc}`../how_to/define-target-classes`
|
||||
- ROI setup: {doc}`../how_to/configure-roi-mapping`
|
||||
- Concept overview: {doc}`../explanation/target-encoding-and-decoding`
|
||||
@ -41,7 +41,8 @@ For an example, see `example_data/dataset.yaml`.
|
||||
|
||||
If you need help creating the dataset config, follow the dataset section in
|
||||
{doc}`train-a-custom-model`.
|
||||
For more detail on dataset source formats, see {doc}`../reference/data-sources`.
|
||||
For more detail on dataset source formats, see
|
||||
{doc}`../reference/configs/data/data-sources`.
|
||||
|
||||
Use a dataset that was not used for training or tuning.
|
||||
|
||||
@ -56,7 +57,7 @@ batdetect2 evaluate \
|
||||
|
||||
If you do not pass `--model`, BatDetect2 uses the built-in default UK model.
|
||||
If you want to choose a different checkpoint, alias, or Hugging Face model, see
|
||||
{doc}`../how_to/choose-a-model`.
|
||||
{doc}`../how_to/inference/choose-a-model`.
|
||||
|
||||
If you want to save the results somewhere else, add `--output-dir`:
|
||||
|
||||
@ -121,17 +122,17 @@ So, depending on your evaluation config, you may see files such as:
|
||||
- saved prediction files.
|
||||
|
||||
If you want to control which tasks run and which plots are generated, see
|
||||
{doc}`../reference/evaluation-config` and
|
||||
{doc}`../how_to/choose-and-configure-evaluation-tasks`.
|
||||
{doc}`../reference/configs/evaluation/evaluation-config` and
|
||||
{doc}`../how_to/evaluation/choose-and-configure-evaluation-tasks`.
|
||||
|
||||
## Common next steps
|
||||
|
||||
- Choose a different model:
|
||||
{doc}`../how_to/choose-a-model`
|
||||
{doc}`../how_to/inference/choose-a-model`
|
||||
- Configure evaluation tasks:
|
||||
{doc}`../how_to/choose-and-configure-evaluation-tasks`
|
||||
{doc}`../how_to/evaluation/choose-and-configure-evaluation-tasks`
|
||||
- Interpret evaluation artifacts:
|
||||
{doc}`../how_to/interpret-evaluation-outputs`
|
||||
{doc}`../how_to/evaluation/interpret-evaluation-outputs`
|
||||
- Learn the evaluation concepts:
|
||||
{doc}`../explanation/evaluation-concepts-and-matching`
|
||||
- Check full evaluate options:
|
||||
|
||||
@ -1,10 +1,12 @@
|
||||
# Integrate with a Python pipeline
|
||||
|
||||
This tutorial shows a simple Python workflow for loading audio, running BatDetect2, and inspecting the detections.
|
||||
This tutorial shows a simple Python workflow for loading audio, running
|
||||
BatDetect2, and inspecting the detections.
|
||||
|
||||
Use it when you want to work directly in Python rather than through the CLI.
|
||||
|
||||
If you mainly want to run the model on recordings, start with {doc}`run-inference-on-folder` instead.
|
||||
If you mainly want to run the model on recordings, start with
|
||||
{doc}`run-inference-on-folder` instead.
|
||||
|
||||
## Before you start
|
||||
|
||||
@ -21,7 +23,8 @@ By the end of this tutorial you will have:
|
||||
- run inference on one file,
|
||||
- inspected detections, scores, and features,
|
||||
- used lower-level audio and spectrogram methods for more control,
|
||||
- identified the next API workflows for batch processing, training, fine-tuning, and evaluation.
|
||||
- identified the next API workflows for batch processing, training, fine-tuning,
|
||||
and evaluation.
|
||||
|
||||
## 1. Create the API instance
|
||||
|
||||
@ -34,11 +37,13 @@ from batdetect2 import BatDetect2API
|
||||
api = BatDetect2API.from_checkpoint()
|
||||
```
|
||||
|
||||
If you want to use a different checkpoint later, see {doc}`../how_to/choose-a-model`.
|
||||
If you want to use a different checkpoint later, see
|
||||
{doc}`../how_to/inference/choose-a-model`.
|
||||
|
||||
## 2. Run inference on one file
|
||||
|
||||
`process_file` is the simplest Python entry point when you want one prediction object per recording.
|
||||
`process_file` is the simplest Python entry point when you want one prediction
|
||||
object per recording.
|
||||
|
||||
```python
|
||||
from batdetect2 import BatDetect2API
|
||||
@ -57,7 +62,8 @@ for detection in prediction.detections:
|
||||
`prediction` is a `ClipDetections` object.
|
||||
See {doc}`../reference/detections` for the full reference.
|
||||
|
||||
Very briefly, `ClipDetections` represents all detections for one processed clip or recording.
|
||||
Very briefly, `ClipDetections` represents all detections for one processed clip
|
||||
or recording.
|
||||
It includes:
|
||||
|
||||
- the clip metadata,
|
||||
@ -76,10 +82,13 @@ Each `Detection` includes:
|
||||
|
||||
The detection score and the class scores answer different questions.
|
||||
|
||||
- `detection_score` is about whether the model thinks there is a call at that time-frequency location.
|
||||
- `class_scores` are about which class the model prefers for that detected event.
|
||||
- `detection_score` is about whether the model thinks there is a call at that
|
||||
time-frequency location.
|
||||
- `class_scores` are about which class the model prefers for that detected
|
||||
event.
|
||||
|
||||
So a detection can have a fairly strong detection score, but still have a more uncertain class ranking.
|
||||
So a detection can have a fairly strong detection score, but still have a more
|
||||
uncertain class ranking.
|
||||
|
||||
```python
|
||||
for detection in prediction.detections:
|
||||
@ -90,7 +99,8 @@ for detection in prediction.detections:
|
||||
print(f" {class_name}: {score:.3f}")
|
||||
```
|
||||
|
||||
If you want more detail on class-score inspection, see {doc}`../how_to/inspect-class-scores-in-python`.
|
||||
If you want more detail on class-score inspection, see
|
||||
{doc}`../how_to/analysis/inspect-class-scores-in-python`.
|
||||
|
||||
## 5. Inspect the detection features
|
||||
|
||||
@ -104,15 +114,20 @@ They can be useful for things like:
|
||||
- comparing detections across files,
|
||||
- building downstream analysis pipelines.
|
||||
|
||||
They are useful descriptors, but they are not direct ecological labels by themselves.
|
||||
They are useful descriptors, but they are not direct ecological labels by
|
||||
themselves.
|
||||
|
||||
For more detail, see {doc}`../how_to/inspect-detection-features-in-python` and {doc}`../explanation/extracted-features-and-embeddings`.
|
||||
For more detail, see
|
||||
{doc}`../how_to/analysis/inspect-detection-features-in-python` and
|
||||
{doc}`../explanation/extracted-features-and-embeddings`.
|
||||
|
||||
## 6. Use lower-level audio and spectrogram methods for more control
|
||||
|
||||
If you want finer control over what gets processed and when, the API also lets you work step by step.
|
||||
If you want finer control over what gets processed and when, the API also lets
|
||||
you work step by step.
|
||||
|
||||
For example, you can load the audio yourself, inspect the waveform length, generate the spectrogram, and then run detection on that spectrogram:
|
||||
For example, you can load the audio yourself, inspect the waveform length,
|
||||
generate the spectrogram, and then run detection on that spectrogram:
|
||||
|
||||
```python
|
||||
from batdetect2 import BatDetect2API
|
||||
@ -136,12 +151,14 @@ This is helpful when you want to:
|
||||
- control which audio segment is processed,
|
||||
- run only part of the pipeline in custom code.
|
||||
|
||||
You can also call `process_audio(audio)` directly if you already have the waveform array in memory.
|
||||
You can also call `process_audio(audio)` directly if you already have the
|
||||
waveform array in memory.
|
||||
|
||||
## 7. Use the wider API workflows
|
||||
|
||||
The Python API is not only for single-file inference.
|
||||
It also exposes methods for batch processing, training, evaluation, and fine-tuning.
|
||||
It also exposes methods for batch processing, training, evaluation, and
|
||||
fine-tuning.
|
||||
|
||||
Examples:
|
||||
|
||||
@ -152,9 +169,15 @@ Examples:
|
||||
|
||||
Useful next pages:
|
||||
|
||||
- Choose a different model: {doc}`../how_to/choose-a-model`
|
||||
- Run batch predictions: {doc}`../how_to/run-batch-predictions`
|
||||
- Train a custom model: {doc}`train-a-custom-model`
|
||||
- Evaluate on a test set: {doc}`evaluate-on-a-test-set`
|
||||
- Fine-tune from a checkpoint: {doc}`../how_to/fine-tune-from-a-checkpoint`
|
||||
- API reference: {doc}`../reference/api`
|
||||
- Choose a different model:
|
||||
{doc}`../how_to/inference/choose-a-model`
|
||||
- Run batch predictions:
|
||||
{doc}`../how_to/inference/run-batch-predictions`
|
||||
- Train a custom model:
|
||||
{doc}`train-a-custom-model`
|
||||
- Evaluate on a test set:
|
||||
{doc}`evaluate-on-a-test-set`
|
||||
- Fine-tune from a checkpoint:
|
||||
{doc}`../how_to/training/fine-tune-from-a-checkpoint`
|
||||
- API reference:
|
||||
{doc}`../reference/api`
|
||||
|
||||
@ -1,10 +1,13 @@
|
||||
# Run BatDetect2 on a folder of audio files
|
||||
|
||||
This tutorial shows how to run BatDetect2 on a folder of recordings from the command line.
|
||||
This tutorial shows how to run BatDetect2 on a folder of recordings from the
|
||||
command line.
|
||||
|
||||
Use it when you want a first pass over a folder of audio recordings and want to see what BatDetect2 finds.
|
||||
Use it when you want a first pass over a folder of audio recordings and want to
|
||||
see what BatDetect2 finds.
|
||||
|
||||
If you want to follow the tutorial exactly, you can use the example recordings that come with the repository.
|
||||
If you want to follow the tutorial exactly, you can use the example recordings
|
||||
that come with the repository.
|
||||
|
||||
## Before you start
|
||||
|
||||
@ -18,7 +21,8 @@ If you have not installed BatDetect2 yet, start with {doc}`../getting_started`.
|
||||
|
||||
## Optional: use the repository example files
|
||||
|
||||
If you want to follow the steps with the same paths shown here, clone the repository and move into it:
|
||||
If you want to follow the steps with the same paths shown here, clone the
|
||||
repository and move into it:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/macaodha/batdetect2.git
|
||||
@ -57,7 +61,8 @@ project/
|
||||
If `outputs/` does not exist yet, that is fine.
|
||||
BatDetect2 can create it.
|
||||
|
||||
If you are using the repository example files, your layout already looks like this:
|
||||
If you are using the repository example files, your layout already looks like
|
||||
this:
|
||||
|
||||
```text
|
||||
batdetect2/
|
||||
@ -95,7 +100,8 @@ What this does:
|
||||
You do not need to choose a model for this first run.
|
||||
If you do nothing, BatDetect2 uses the built-in default UK model.
|
||||
|
||||
If you want to use a different model later, see {doc}`../how_to/choose-a-model`.
|
||||
If you want to use a different model later, see
|
||||
{doc}`../how_to/inference/choose-a-model`.
|
||||
|
||||
## 3. Check the output files
|
||||
|
||||
@ -104,7 +110,8 @@ After the command finishes, look in your output folder.
|
||||
By default, the CLI writes predictions in the `batdetect2` output format.
|
||||
This is a JSON-based format used for BatDetect2-style outputs.
|
||||
|
||||
With the default settings, you will usually see one `.json` file and one `_detections.csv` file per recording.
|
||||
With the default settings, you will usually see one `.json` file and one
|
||||
`_detections.csv` file per recording.
|
||||
|
||||
For the repository example run, that means files like:
|
||||
|
||||
@ -141,16 +148,20 @@ One of the JSON files will look roughly like this:
|
||||
|
||||
Very briefly:
|
||||
|
||||
- `annotated: false` means this is a prediction file, not a reviewed annotation file.
|
||||
- `annotated:
|
||||
false` means this is a prediction file, not a reviewed annotation file.
|
||||
- `annotation` holds the list of detections.
|
||||
- Each detection includes a predicted class, detection score, class score, time bounds, and frequency bounds.
|
||||
- Each detection includes a predicted class, detection score, class score, time
|
||||
bounds, and frequency bounds.
|
||||
|
||||
For more detail, see {doc}`../explanation/interpreting-formatted-outputs`.
|
||||
If you want to save results in another format, see {doc}`../how_to/save-predictions-in-different-output-formats`.
|
||||
If you want to save results in another format, see
|
||||
{doc}`../how_to/inference/save-predictions-in-different-output-formats`.
|
||||
|
||||
## 4. Run the same folder with a higher threshold
|
||||
|
||||
If you want, you can also run the same folder again with a higher detection threshold and save that run in a separate output folder.
|
||||
If you want, you can also run the same folder again with a higher detection
|
||||
threshold and save that run in a separate output folder.
|
||||
|
||||
```bash
|
||||
batdetect2 process directory \
|
||||
@ -206,12 +217,18 @@ batdetect2 process file_list \
|
||||
example_outputs/selected_outputs
|
||||
```
|
||||
|
||||
This is useful when your recordings are spread across folders, or when you only want to run a chosen subset.
|
||||
This is useful when your recordings are spread across folders, or when you only
|
||||
want to run a chosen subset.
|
||||
|
||||
## Common next steps
|
||||
|
||||
- If your recordings are not all in one folder, or you want to compare input modes, see {doc}`../how_to/choose-an-inference-input-mode`.
|
||||
- If you want to save results in another format, see {doc}`../how_to/save-predictions-in-different-output-formats`.
|
||||
- If you want to choose a different model, see {doc}`../how_to/choose-a-model`.
|
||||
- If you already write code and want more control from Python, see {doc}`integrate-with-a-python-pipeline`.
|
||||
- If you want the full command reference, including `--model`, see {doc}`../reference/cli/predict`.
|
||||
- If your recordings are not all in one folder, or you want to compare input
|
||||
modes, see {doc}`../how_to/inference/choose-an-inference-input-mode`.
|
||||
- If you want to save results in another format, see
|
||||
{doc}`../how_to/inference/save-predictions-in-different-output-formats`.
|
||||
- If you want to choose a different model, see
|
||||
{doc}`../how_to/inference/choose-a-model`.
|
||||
- If you already write code and want more control from Python, see
|
||||
{doc}`integrate-with-a-python-pipeline`.
|
||||
- If you want the full command reference, including `--model`, see
|
||||
{doc}`../reference/cli/predict`.
|
||||
|
||||
@ -1,8 +1,10 @@
|
||||
# Train a custom model
|
||||
|
||||
This tutorial walks through a first custom training run using your own annotations.
|
||||
This tutorial walks through a first custom training run using your own
|
||||
annotations.
|
||||
|
||||
Use it when you already have labelled recordings and want to train a model for your own data.
|
||||
Use it when you already have labelled recordings and want to train a model for
|
||||
your own data.
|
||||
|
||||
## Before you start
|
||||
|
||||
@ -19,7 +21,8 @@ Use {doc}`run-inference-on-folder` for that.
|
||||
|
||||
## Optional: use the repository example files
|
||||
|
||||
If you want to follow the steps with the same files shown here, clone the repository and move into it:
|
||||
If you want to follow the steps with the same files shown here, clone the
|
||||
repository and move into it:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/macaodha/batdetect2.git
|
||||
@ -40,10 +43,12 @@ By the end of this tutorial you will have:
|
||||
|
||||
The dataset config explicitly declares what data you want to use for training.
|
||||
It is a YAML file.
|
||||
If YAML is new to you, see [Learn YAML in Y Minutes](https://learnxinyminutes.com/yaml/).
|
||||
If YAML is new to you, see
|
||||
[Learn YAML in Y Minutes](https://learnxinyminutes.com/yaml/).
|
||||
|
||||
In the dataset config, you list one or more data sources.
|
||||
Each source tells `batdetect2` where the audio recordings live and where the matching annotations are stored.
|
||||
Each source tells `batdetect2` where the audio recordings live and where the
|
||||
matching annotations are stored.
|
||||
|
||||
BatDetect2 can read annotations from different source formats.
|
||||
In this example, we use the example data in the `batdetect2` format.
|
||||
@ -62,21 +67,25 @@ sources:
|
||||
```
|
||||
|
||||
For your own project, the main thing to change is the file paths.
|
||||
If you have several collections of recordings, you can add more than one source to the same dataset config.
|
||||
If you have several collections of recordings, you can add more than one source
|
||||
to the same dataset config.
|
||||
That lets you describe the full training data you want to use in one place.
|
||||
|
||||
If you need more detail on dataset source formats, see {doc}`../reference/data-sources`.
|
||||
If you need more detail on dataset source formats, see
|
||||
{doc}`../reference/configs/data/data-sources`.
|
||||
|
||||
## 2. Define a targets config
|
||||
|
||||
The targets config tells BatDetect2 how to turn your annotations into training targets.
|
||||
The targets config tells BatDetect2 how to turn your annotations into training
|
||||
targets.
|
||||
|
||||
It defines two main things:
|
||||
|
||||
- what should count as a detection,
|
||||
- which classes the model should learn to predict.
|
||||
|
||||
In practice, this means the targets config maps the labels in your annotations to the detection and classification outputs used during training.
|
||||
In practice, this means the targets config maps the labels in your annotations
|
||||
to the detection and classification outputs used during training.
|
||||
|
||||
Use `example_data/targets.yaml` as a reference:
|
||||
|
||||
@ -107,16 +116,21 @@ classification_targets:
|
||||
value: Pipistrellus pipistrellus
|
||||
```
|
||||
|
||||
For your own project, update the matching rules and class definitions so they fit your labels.
|
||||
For your own project, update the matching rules and class definitions so they
|
||||
fit your labels.
|
||||
|
||||
In this example:
|
||||
|
||||
- `detection_target` says that echolocation calls should be treated as detections,
|
||||
- `detection_target` says that echolocation calls should be treated as
|
||||
detections,
|
||||
- `classification_targets` define the classes the model should predict,
|
||||
|
||||
It is worth taking a bit of time over this file, because your targets config decides what the model is actually being asked to learn.
|
||||
It is worth taking a bit of time over this file, because your targets config
|
||||
decides what the model is actually being asked to learn.
|
||||
|
||||
If you need help with that, see {doc}`../how_to/configure-target-definitions` and {doc}`../reference/targets-config-workflow`.
|
||||
If you need help with that, see
|
||||
{doc}`../how_to/data/configure-target-definitions` and
|
||||
{doc}`../reference/configs/data/targets-config-workflow`.
|
||||
|
||||
## 3. Run a first training command
|
||||
|
||||
@ -138,12 +152,16 @@ batdetect2 train \
|
||||
--targets example_data/targets.yaml
|
||||
```
|
||||
|
||||
This uses the same dataset for training and validation only to keep the example simple.
|
||||
For real training runs, you usually want separate training and validation datasets.
|
||||
This uses the same dataset for training and validation only to keep the example
|
||||
simple.
|
||||
For real training runs, you usually want separate training and validation
|
||||
datasets.
|
||||
|
||||
This uses the built-in default model and training settings.
|
||||
If you want to change the model architecture later, see {doc}`../reference/model-config`.
|
||||
If you want to change optimiser settings, batch size, epochs, or checkpoint behaviour, see {doc}`../reference/training-config`.
|
||||
If you want to change the model architecture later, see
|
||||
{doc}`../reference/configs/training/model-config`.
|
||||
If you want to change optimiser settings, batch size, epochs, or checkpoint
|
||||
behaviour, see {doc}`../reference/configs/training/training-config`.
|
||||
|
||||
## 4. Check the training outputs
|
||||
|
||||
@ -170,10 +188,13 @@ outputs/
|
||||
val_class_summary.csv
|
||||
```
|
||||
|
||||
The checkpoint is the trained model you can use later for inference, evaluation, or sharing with someone else.
|
||||
The checkpoint is the trained model you can use later for inference, evaluation,
|
||||
or sharing with someone else.
|
||||
|
||||
The files in `training_artifacts/` record which datasets and targets were used for the run.
|
||||
The `hparams.yaml` file records the full training setup, including the configs used for the model, training, and other parts of the run.
|
||||
The files in `training_artifacts/` record which datasets and targets were used
|
||||
for the run.
|
||||
The `hparams.yaml` file records the full training setup, including the configs
|
||||
used for the model, training, and other parts of the run.
|
||||
|
||||
The `metrics.csv` file stores one row per validation epoch.
|
||||
It includes training losses as well as validation losses and metrics such as:
|
||||
@ -190,19 +211,29 @@ You may also see class-specific metrics in extra columns.
|
||||
The more detailed metrics are computed from the validation set.
|
||||
If you do not provide `--val-dataset`, those validation metrics will not appear.
|
||||
|
||||
Other logger backends are also supported, including TensorBoard, MLflow, and DVCLive.
|
||||
See {doc}`../reference/logging-config` if you want to change that.
|
||||
Other logger backends are also supported, including TensorBoard, MLflow, and
|
||||
DVCLive.
|
||||
See {doc}`../reference/configs/training/logging-config` if you want to change
|
||||
that.
|
||||
|
||||
## Use the trained model
|
||||
|
||||
You can now use the trained checkpoint in BatDetect2, or share it with someone else to use in their own runs.
|
||||
If you want to load it for inference or evaluation, see {doc}`../how_to/choose-a-model`.
|
||||
You can now use the trained checkpoint in BatDetect2, or share it with someone
|
||||
else to use in their own runs.
|
||||
If you want to load it for inference or evaluation, see
|
||||
{doc}`../how_to/inference/choose-a-model`.
|
||||
|
||||
## Common next steps
|
||||
|
||||
- Evaluate the trained checkpoint: {doc}`evaluate-on-a-test-set`
|
||||
- Fine-tune from a checkpoint: {doc}`../how_to/fine-tune-from-a-checkpoint`
|
||||
- Configure targets in more detail: {doc}`../how_to/configure-target-definitions`
|
||||
- Configure audio preprocessing: {doc}`../how_to/configure-audio-preprocessing`
|
||||
- Configure spectrogram preprocessing: {doc}`../how_to/configure-spectrogram-preprocessing`
|
||||
- Check full train options: {doc}`../reference/cli/train`
|
||||
- Evaluate the trained checkpoint:
|
||||
{doc}`evaluate-on-a-test-set`
|
||||
- Fine-tune from a checkpoint:
|
||||
{doc}`../how_to/training/fine-tune-from-a-checkpoint`
|
||||
- Configure targets in more detail:
|
||||
{doc}`../how_to/data/configure-target-definitions`
|
||||
- Configure audio preprocessing:
|
||||
{doc}`../how_to/data/configure-audio-preprocessing`
|
||||
- Configure spectrogram preprocessing:
|
||||
{doc}`../how_to/data/configure-spectrogram-preprocessing`
|
||||
- Check full train options:
|
||||
{doc}`../reference/cli/train`
|
||||
|
||||
@ -2,9 +2,7 @@ import torch
|
||||
from loguru import logger
|
||||
|
||||
from batdetect2.models.types import ModelOutput
|
||||
from batdetect2.postprocess.config import (
|
||||
PostprocessConfig,
|
||||
)
|
||||
from batdetect2.postprocess.config import PostprocessConfig
|
||||
from batdetect2.postprocess.extraction import extract_detection_peaks
|
||||
from batdetect2.postprocess.nms import NMS_KERNEL_SIZE, non_max_suppression
|
||||
from batdetect2.postprocess.types import (
|
||||
|
||||
Loading…
Reference in New Issue
Block a user