mirror of
https://github.com/macaodha/batdetect2.git
synced 2026-07-07 21:00:10 +02:00
Update structure
Some checks failed
CI / Checks (push) Has been cancelled
Docs Pages / Build Docs (push) Has been cancelled
CI / Tests (Python ${{ matrix.python-version }}) (3.10) (push) Has been cancelled
CI / Tests (Python ${{ matrix.python-version }}) (3.11) (push) Has been cancelled
CI / Tests (Python ${{ matrix.python-version }}) (3.12) (push) Has been cancelled
Docs Pages / Deploy Docs (push) Has been cancelled
Some checks failed
CI / Checks (push) Has been cancelled
Docs Pages / Build Docs (push) Has been cancelled
CI / Tests (Python ${{ matrix.python-version }}) (3.10) (push) Has been cancelled
CI / Tests (Python ${{ matrix.python-version }}) (3.11) (push) Has been cancelled
CI / Tests (Python ${{ matrix.python-version }}) (3.12) (push) Has been cancelled
Docs Pages / Deploy Docs (push) Has been cancelled
This commit is contained in:
parent
f4cb7d3fc1
commit
67aee0b79c
@ -2,7 +2,8 @@
|
|||||||
|
|
||||||
Evaluation is not just "run predictions and compute one number".
|
Evaluation is not just "run predictions and compute one number".
|
||||||
|
|
||||||
The reported metric depends on the evaluation task, the matching rule, and the treatment of clip boundaries and generic labels.
|
The reported metric depends on the evaluation task, the matching rule, and the
|
||||||
|
treatment of clip boundaries and generic labels.
|
||||||
|
|
||||||
## Task families answer different questions
|
## Task families answer different questions
|
||||||
|
|
||||||
@ -18,7 +19,8 @@ Choose the task that matches the scientific or engineering question.
|
|||||||
|
|
||||||
## Matching matters
|
## Matching matters
|
||||||
|
|
||||||
For sound-event-style tasks, predictions and annotations are matched using an affinity function.
|
For sound-event-style tasks, predictions and annotations are matched using an
|
||||||
|
affinity function.
|
||||||
|
|
||||||
Important controls include:
|
Important controls include:
|
||||||
|
|
||||||
@ -27,22 +29,28 @@ Important controls include:
|
|||||||
- `strict_match`,
|
- `strict_match`,
|
||||||
- `ignore_start_end`.
|
- `ignore_start_end`.
|
||||||
|
|
||||||
Small changes here can change the reported metric without changing the underlying predictions.
|
Small changes here can change the reported metric without changing the
|
||||||
|
underlying predictions.
|
||||||
|
|
||||||
## Boundary handling matters
|
## Boundary handling matters
|
||||||
|
|
||||||
The evaluation base task can exclude events near clip boundaries through `ignore_start_end`.
|
The evaluation base task can exclude events near clip boundaries through
|
||||||
|
`ignore_start_end`.
|
||||||
|
|
||||||
This is useful when clip boundaries make matches ambiguous.
|
This is useful when clip boundaries make matches ambiguous.
|
||||||
|
|
||||||
## Generic labels can matter in classification
|
## Generic labels can matter in classification
|
||||||
|
|
||||||
Classification tasks can include or exclude generic targets depending on configuration.
|
Classification tasks can include or exclude generic targets depending on
|
||||||
|
configuration.
|
||||||
|
|
||||||
That affects what counts as a valid class-level comparison.
|
That affects what counts as a valid class-level comparison.
|
||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Evaluate on a test set: {doc}`../tutorials/evaluate-on-a-test-set`
|
- Evaluate on a test set:
|
||||||
- Evaluation config reference: {doc}`../reference/evaluation-config`
|
{doc}`../tutorials/evaluate-on-a-test-set`
|
||||||
- Model output and validation: {doc}`model-output-and-validation`
|
- Evaluation config reference:
|
||||||
|
{doc}`../reference/configs/evaluation/evaluation-config`
|
||||||
|
- Model output and validation:
|
||||||
|
{doc}`model-output-and-validation`
|
||||||
|
|||||||
@ -38,6 +38,6 @@ These are related ideas, but not necessarily one-to-one replacements.
|
|||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Inspect detection features in Python:
|
- Inspect detection features in Python:
|
||||||
{doc}`../how_to/inspect-detection-features-in-python`
|
{doc}`../how_to/analysis/inspect-detection-features-in-python`
|
||||||
- Legacy migration guide:
|
- Legacy migration guide:
|
||||||
{doc}`../legacy/migration-guide`
|
{doc}`../legacy/migration-guide`
|
||||||
|
|||||||
@ -2,21 +2,25 @@
|
|||||||
|
|
||||||
BatDetect2 can write predictions in several output formats.
|
BatDetect2 can write predictions in several output formats.
|
||||||
|
|
||||||
Those formats are different views of the same underlying detections, not different model behaviors.
|
Those formats are different views of the same underlying detections, not
|
||||||
|
different model behaviors.
|
||||||
|
|
||||||
## Separate the underlying detection from the serialized file
|
## Separate the underlying detection from the serialized file
|
||||||
|
|
||||||
Internally, the current stack works with clip-level detections containing geometry, detection score, class scores, and features.
|
Internally, the current stack works with clip-level detections containing
|
||||||
|
geometry, detection score, class scores, and features.
|
||||||
|
|
||||||
Output formatters then serialize those detections in different ways.
|
Output formatters then serialize those detections in different ways.
|
||||||
|
|
||||||
## Raw outputs are richest
|
## Raw outputs are richest
|
||||||
|
|
||||||
The `raw` format preserves the broadest structured view of detections and is a good default when you want to inspect or reload predictions later.
|
The `raw` format preserves the broadest structured view of detections and is a
|
||||||
|
good default when you want to inspect or reload predictions later.
|
||||||
|
|
||||||
## Tabular outputs are for analysis convenience
|
## Tabular outputs are for analysis convenience
|
||||||
|
|
||||||
The `parquet` format is convenient for data analysis workflows, but the tabular representation is only one projection of the underlying detection object.
|
The `parquet` format is convenient for data analysis workflows, but the tabular
|
||||||
|
representation is only one projection of the underlying detection object.
|
||||||
|
|
||||||
## Legacy-shaped outputs are mainly for compatibility
|
## Legacy-shaped outputs are mainly for compatibility
|
||||||
|
|
||||||
@ -26,11 +30,14 @@ Use it when you need compatibility with older downstream tools or workflows.
|
|||||||
|
|
||||||
## The meaning does not come from the file extension
|
## The meaning does not come from the file extension
|
||||||
|
|
||||||
Do not assume that a `.json`, `.parquet`, or `.nc` file changes what the model predicted.
|
Do not assume that a `.json`, `.parquet`, or `.nc` file changes what the model
|
||||||
|
predicted.
|
||||||
|
|
||||||
It changes how the prediction is packaged and how much detail is retained.
|
It changes how the prediction is packaged and how much detail is retained.
|
||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Output formats reference: {doc}`../reference/output-formats`
|
- Output formats reference:
|
||||||
- Outputs config reference: {doc}`../reference/outputs-config`
|
{doc}`../reference/configs/outputs/output-formats`
|
||||||
|
- Outputs config reference:
|
||||||
|
{doc}`../reference/configs/outputs/outputs-config`
|
||||||
|
|||||||
@ -1,8 +1,8 @@
|
|||||||
# Model output and validation
|
# Model output and validation
|
||||||
|
|
||||||
BatDetect2 outputs model predictions, not ground truth. The same configuration
|
BatDetect2 outputs model predictions, not ground truth.
|
||||||
can behave differently across recording conditions, species compositions, and
|
The same configuration can behave differently across recording conditions,
|
||||||
acoustic environments.
|
species compositions, and acoustic environments.
|
||||||
|
|
||||||
## Why threshold choice matters
|
## Why threshold choice matters
|
||||||
|
|
||||||
@ -10,8 +10,9 @@ acoustic environments.
|
|||||||
positives.
|
positives.
|
||||||
- Higher thresholds reduce false positives but can miss faint calls.
|
- Higher thresholds reduce false positives but can miss faint calls.
|
||||||
|
|
||||||
No threshold is universally correct. The right setting depends on your survey
|
No threshold is universally correct.
|
||||||
objectives and tolerance for false positives versus missed detections.
|
The right setting depends on your survey objectives and tolerance for false
|
||||||
|
positives versus missed detections.
|
||||||
|
|
||||||
## Why local validation is required
|
## Why local validation is required
|
||||||
|
|
||||||
@ -26,4 +27,4 @@ Recommended validation checks:
|
|||||||
3. Repeat checks across sites, seasons, and recorder setups.
|
3. Repeat checks across sites, seasons, and recorder setups.
|
||||||
|
|
||||||
For practical threshold workflows, see
|
For practical threshold workflows, see
|
||||||
{doc}`../how_to/tune-detection-threshold`.
|
{doc}`../how_to/inference/tune-detection-threshold`.
|
||||||
|
|||||||
@ -1,8 +1,9 @@
|
|||||||
# Postprocessing and thresholds
|
# Postprocessing and thresholds
|
||||||
|
|
||||||
After the detector runs on a spectrogram, the model output is still a set of
|
After the detector runs on a spectrogram, the model output is still a set of
|
||||||
dense prediction tensors. Postprocessing turns that into a final list of call
|
dense prediction tensors.
|
||||||
detections with positions, sizes, and class scores.
|
Postprocessing turns that into a final list of call detections with positions,
|
||||||
|
sizes, and class scores.
|
||||||
|
|
||||||
## What postprocessing does
|
## What postprocessing does
|
||||||
|
|
||||||
@ -27,9 +28,10 @@ You can tune this behavior per run without retraining the model.
|
|||||||
|
|
||||||
## Two common threshold controls
|
## Two common threshold controls
|
||||||
|
|
||||||
- `detection_threshold`: minimum score required to keep a detection.
|
- `detection_threshold`:
|
||||||
- `classification_threshold`: minimum class score used when assigning class
|
minimum score required to keep a detection.
|
||||||
labels.
|
- `classification_threshold`:
|
||||||
|
minimum class score used when assigning class labels.
|
||||||
|
|
||||||
Both settings shape the final output and should be validated on reviewed local
|
Both settings shape the final output and should be validated on reviewed local
|
||||||
data.
|
data.
|
||||||
@ -39,5 +41,7 @@ data.
|
|||||||
Tune thresholds on a representative subset first, then lock settings for the
|
Tune thresholds on a representative subset first, then lock settings for the
|
||||||
full analysis run.
|
full analysis run.
|
||||||
|
|
||||||
- How-to: {doc}`../how_to/tune-detection-threshold`
|
- How-to:
|
||||||
- CLI reference: {doc}`../reference/cli/predict`
|
{doc}`../how_to/inference/tune-detection-threshold`
|
||||||
|
- CLI reference:
|
||||||
|
{doc}`../reference/cli/predict`
|
||||||
|
|||||||
@ -6,8 +6,9 @@ performance.
|
|||||||
## Why consistency matters
|
## Why consistency matters
|
||||||
|
|
||||||
The detector is trained on spectrograms produced by a specific preprocessing
|
The detector is trained on spectrograms produced by a specific preprocessing
|
||||||
pipeline. If inference uses different settings, the model can see a shifted
|
pipeline.
|
||||||
input distribution and performance may drop.
|
If inference uses different settings, the model can see a shifted input
|
||||||
|
distribution and performance may drop.
|
||||||
|
|
||||||
Typical mismatch sources:
|
Typical mismatch sources:
|
||||||
|
|
||||||
@ -30,7 +31,8 @@ re-validate on reviewed local data.
|
|||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Configure audio preprocessing:
|
- Configure audio preprocessing:
|
||||||
{doc}`../how_to/configure-audio-preprocessing`
|
{doc}`../how_to/data/configure-audio-preprocessing`
|
||||||
- Configure spectrogram preprocessing:
|
- Configure spectrogram preprocessing:
|
||||||
{doc}`../how_to/configure-spectrogram-preprocessing`
|
{doc}`../how_to/data/configure-spectrogram-preprocessing`
|
||||||
- Preprocessing config reference: {doc}`../reference/preprocessing-config`
|
- Preprocessing config reference:
|
||||||
|
{doc}`../reference/configs/data/preprocessing-config`
|
||||||
|
|||||||
@ -11,8 +11,8 @@ At training time, the target system:
|
|||||||
2. assigns a classification label (or none for non-specific class matches),
|
2. assigns a classification label (or none for non-specific class matches),
|
||||||
3. maps event geometry into position and size targets.
|
3. maps event geometry into position and size targets.
|
||||||
|
|
||||||
This behaviour is configured through `TargetConfig`,
|
This behaviour is configured through `TargetConfig`, `TargetClassConfig`, and
|
||||||
`TargetClassConfig`, and ROI mapper settings.
|
ROI mapper settings.
|
||||||
|
|
||||||
## Decoding path (model outputs -> tags and geometry)
|
## Decoding path (model outputs -> tags and geometry)
|
||||||
|
|
||||||
@ -24,7 +24,8 @@ annotations.
|
|||||||
|
|
||||||
## Why this matters
|
## Why this matters
|
||||||
|
|
||||||
Target definitions are not just metadata. They directly shape:
|
Target definitions are not just metadata.
|
||||||
|
They directly shape:
|
||||||
|
|
||||||
- what events are treated as positive examples,
|
- what events are treated as positive examples,
|
||||||
- which class names the model learns,
|
- which class names the model learns,
|
||||||
@ -34,7 +35,11 @@ Small changes here can alter both training outcomes and prediction semantics.
|
|||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Configure detection target logic: {doc}`../how_to/configure-target-definitions`
|
- Configure detection target logic:
|
||||||
- Configure class mapping: {doc}`../how_to/define-target-classes`
|
{doc}`../how_to/data/configure-target-definitions`
|
||||||
- Configure ROI mapping: {doc}`../how_to/configure-roi-mapping`
|
- Configure class mapping:
|
||||||
- Target config reference: {doc}`../reference/targets-config-workflow`
|
{doc}`../how_to/data/define-target-classes`
|
||||||
|
- Configure ROI mapping:
|
||||||
|
{doc}`../how_to/data/configure-roi-mapping`
|
||||||
|
- Target config reference:
|
||||||
|
{doc}`../reference/configs/data/targets-config-workflow`
|
||||||
|
|||||||
@ -5,50 +5,64 @@
|
|||||||
### Do I need Python knowledge to use batdetect2?
|
### Do I need Python knowledge to use batdetect2?
|
||||||
|
|
||||||
Not much.
|
Not much.
|
||||||
If you only want to run the model on your own recordings, you can use the CLI and follow the steps in {doc}`getting_started`.
|
If you only want to run the model on your own recordings, you can use the CLI
|
||||||
|
and follow the steps in {doc}`getting_started`.
|
||||||
|
|
||||||
Some command-line familiarity helps, but you do not need to write Python code for standard inference workflows.
|
Some command-line familiarity helps, but you do not need to write Python code
|
||||||
|
for standard inference workflows.
|
||||||
|
|
||||||
### Are there plans for an R version?
|
### Are there plans for an R version?
|
||||||
|
|
||||||
Not currently.
|
Not currently.
|
||||||
Output files are plain formats (for example CSV/JSON), so you can read and analyze them in R or other environments.
|
Output files are plain formats (for example CSV/JSON), so you can read and
|
||||||
|
analyze them in R or other environments.
|
||||||
|
|
||||||
### I cannot get installation working. What should I do?
|
### I cannot get installation working. What should I do?
|
||||||
|
|
||||||
First, re-check {doc}`getting_started` and confirm your environment is active.
|
First, re-check {doc}`getting_started` and confirm your environment is active.
|
||||||
If it still fails, open an issue with your OS, install method, and full error output: [GitHub Issues](https://github.com/macaodha/batdetect2/issues).
|
If it still fails, open an issue with your OS, install method, and full error
|
||||||
|
output:
|
||||||
|
[GitHub Issues](https://github.com/macaodha/batdetect2/issues).
|
||||||
|
|
||||||
## Model behavior and performance
|
## Model behavior and performance
|
||||||
|
|
||||||
### The model does not perform well on my data
|
### The model does not perform well on my data
|
||||||
|
|
||||||
This usually means your data distribution differs from training data.
|
This usually means your data distribution differs from training data.
|
||||||
The best next step is to validate on reviewed local data and then fine-tune/train on your own annotations if needed.
|
The best next step is to validate on reviewed local data and then
|
||||||
|
fine-tune/train on your own annotations if needed.
|
||||||
|
|
||||||
### The model confuses insects/noise with bats
|
### The model confuses insects/noise with bats
|
||||||
|
|
||||||
This can happen, especially when recording conditions differ from training conditions.
|
This can happen, especially when recording conditions differ from training
|
||||||
|
conditions.
|
||||||
Threshold tuning and training with local annotations can improve results.
|
Threshold tuning and training with local annotations can improve results.
|
||||||
|
|
||||||
See {doc}`how_to/tune-detection-threshold`.
|
See {doc}`how_to/inference/tune-detection-threshold`.
|
||||||
|
|
||||||
### The model struggles with feeding buzzes or social calls
|
### The model struggles with feeding buzzes or social calls
|
||||||
|
|
||||||
This is a known limitation of available training data in some settings.
|
This is a known limitation of available training data in some settings.
|
||||||
If you have high-quality annotated examples, they are valuable for improving models.
|
If you have high-quality annotated examples, they are valuable for improving
|
||||||
|
models.
|
||||||
|
|
||||||
### Calls in the same sequence are predicted as different species
|
### Calls in the same sequence are predicted as different species
|
||||||
|
|
||||||
Currently we do not do any sophisticated post processing on the results output by the model.
|
Currently we do not do any sophisticated post processing on the results output
|
||||||
|
by the model.
|
||||||
We return a probability associated with each species for each call.
|
We return a probability associated with each species for each call.
|
||||||
You can use these predictions to clean up the noisy predictions for sequences of calls.
|
You can use these predictions to clean up the noisy predictions for sequences of
|
||||||
|
calls.
|
||||||
|
|
||||||
### Can I trust model outputs for biodiversity conclusions?
|
### Can I trust model outputs for biodiversity conclusions?
|
||||||
|
|
||||||
The models developed and shared as part of this repository should be used with caution.
|
The models developed and shared as part of this repository should be used with
|
||||||
While they have been evaluated on held out audio data, great care should be taken when using the model outputs for any form of biodiversity assessment.
|
caution.
|
||||||
Your data may differ, and as a result it is very strongly recommended that you validate the model first using data with known species to ensure that the outputs can be trusted.
|
While they have been evaluated on held out audio data, great care should be
|
||||||
|
taken when using the model outputs for any form of biodiversity assessment.
|
||||||
|
Your data may differ, and as a result it is very strongly recommended that you
|
||||||
|
validate the model first using data with known species to ensure that the
|
||||||
|
outputs can be trusted.
|
||||||
|
|
||||||
### The pipeline is slow
|
### The pipeline is slow
|
||||||
|
|
||||||
@ -65,7 +79,8 @@ You can train/fine-tune with your own annotated data and species labels.
|
|||||||
### Does this work on frequency-division or zero-crossing recordings?
|
### Does this work on frequency-division or zero-crossing recordings?
|
||||||
|
|
||||||
Not directly.
|
Not directly.
|
||||||
The workflow assumes audio can be converted to spectrograms from the raw waveform.
|
The workflow assumes audio can be converted to spectrograms from the raw
|
||||||
|
waveform.
|
||||||
|
|
||||||
### Can this be used for non-bat bioacoustics (for example insects or birds)?
|
### Can this be used for non-bat bioacoustics (for example insects or birds)?
|
||||||
|
|
||||||
|
|||||||
11
docs/source/how_to/analysis-in-python.md
Normal file
11
docs/source/how_to/analysis-in-python.md
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
# Analysis in Python
|
||||||
|
|
||||||
|
Use this section when you want to inspect model outputs more closely from
|
||||||
|
Python, beyond the default saved predictions.
|
||||||
|
|
||||||
|
```{toctree}
|
||||||
|
:maxdepth: 1
|
||||||
|
|
||||||
|
analysis/inspect-class-scores-in-python
|
||||||
|
analysis/inspect-detection-features-in-python
|
||||||
|
```
|
||||||
@ -1,6 +1,7 @@
|
|||||||
# How to inspect detection features in Python
|
# How to inspect detection features in Python
|
||||||
|
|
||||||
Use this guide when you want the per-detection feature vectors exposed by the current API.
|
Use this guide when you want the per-detection feature vectors exposed by the
|
||||||
|
current API.
|
||||||
|
|
||||||
## Get the feature vector for one detection
|
## Get the feature vector for one detection
|
||||||
|
|
||||||
@ -34,16 +35,21 @@ They can be useful for:
|
|||||||
|
|
||||||
They do not replace validation.
|
They do not replace validation.
|
||||||
|
|
||||||
They also do not automatically have a one-to-one interpretation as ecological variables.
|
They also do not automatically have a one-to-one interpretation as ecological
|
||||||
|
variables.
|
||||||
|
|
||||||
## Save predictions with features included
|
## Save predictions with features included
|
||||||
|
|
||||||
If you need features on disk, use an output format that supports them, such as `raw` or `parquet`, and keep feature inclusion enabled.
|
If you need features on disk, use an output format that supports them, such as
|
||||||
|
`raw` or `parquet`, and keep feature inclusion enabled.
|
||||||
|
|
||||||
See {doc}`save-predictions-in-different-output-formats`.
|
See {doc}`../inference/save-predictions-in-different-output-formats`.
|
||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Understanding features and embeddings: {doc}`../explanation/extracted-features-and-embeddings`
|
- Understanding features and embeddings:
|
||||||
- Output formats reference: {doc}`../reference/output-formats`
|
{doc}`../../explanation/extracted-features-and-embeddings`
|
||||||
- API reference: {doc}`../reference/api`
|
- Output formats reference:
|
||||||
|
{doc}`../../reference/configs/outputs/output-formats`
|
||||||
|
- API reference:
|
||||||
|
{doc}`../../reference/api`
|
||||||
16
docs/source/how_to/data-and-targets.md
Normal file
16
docs/source/how_to/data-and-targets.md
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
# Data and Targets
|
||||||
|
|
||||||
|
Use this section when you are preparing datasets, preprocessing audio, or
|
||||||
|
defining the targets used for training and evaluation.
|
||||||
|
|
||||||
|
```{toctree}
|
||||||
|
:maxdepth: 1
|
||||||
|
|
||||||
|
data/configure-aoef-dataset
|
||||||
|
data/import-legacy-batdetect2-annotations
|
||||||
|
data/configure-audio-preprocessing
|
||||||
|
data/configure-spectrogram-preprocessing
|
||||||
|
data/configure-target-definitions
|
||||||
|
data/define-target-classes
|
||||||
|
data/configure-roi-mapping
|
||||||
|
```
|
||||||
@ -5,7 +5,8 @@ including exports from Whombat.
|
|||||||
|
|
||||||
## 1) Add an AOEF source entry
|
## 1) Add an AOEF source entry
|
||||||
|
|
||||||
In your dataset config, add a source with `format: aoef`.
|
In your dataset config, add a source with `format:
|
||||||
|
aoef`.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
sources:
|
sources:
|
||||||
@ -49,5 +50,7 @@ batdetect2 data summary path/to/dataset.yaml
|
|||||||
|
|
||||||
## 4) Continue to training or evaluation
|
## 4) Continue to training or evaluation
|
||||||
|
|
||||||
- For training: {doc}`../tutorials/train-a-custom-model`
|
- For training:
|
||||||
- For field-level reference: {doc}`../reference/data-sources`
|
{doc}`../../tutorials/train-a-custom-model`
|
||||||
|
- For field-level reference:
|
||||||
|
{doc}`../../reference/configs/data/data-sources`
|
||||||
@ -63,4 +63,4 @@ before full-batch runs.
|
|||||||
- Spectrogram settings:
|
- Spectrogram settings:
|
||||||
{doc}`configure-spectrogram-preprocessing`
|
{doc}`configure-spectrogram-preprocessing`
|
||||||
- Preprocessing config reference:
|
- Preprocessing config reference:
|
||||||
{doc}`../reference/preprocessing-config`
|
{doc}`../../reference/configs/data/preprocessing-config`
|
||||||
@ -52,6 +52,9 @@ roi:
|
|||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Target definitions: {doc}`configure-target-definitions`
|
- Target definitions:
|
||||||
- Class definitions: {doc}`define-target-classes`
|
{doc}`configure-target-definitions`
|
||||||
- Target encoding overview: {doc}`../explanation/target-encoding-and-decoding`
|
- Class definitions:
|
||||||
|
{doc}`define-target-classes`
|
||||||
|
- Target encoding overview:
|
||||||
|
{doc}`../../explanation/target-encoding-and-decoding`
|
||||||
@ -55,5 +55,7 @@ Large mismatches can degrade model performance.
|
|||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Why consistency matters: {doc}`../explanation/preprocessing-consistency`
|
- Why consistency matters:
|
||||||
- Preprocessing config reference: {doc}`../reference/preprocessing-config`
|
{doc}`../../explanation/preprocessing-consistency`
|
||||||
|
- Preprocessing config reference:
|
||||||
|
{doc}`../../reference/configs/data/preprocessing-config`
|
||||||
@ -53,6 +53,9 @@ selection logic keeps the events you expect.
|
|||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Class mapping: {doc}`define-target-classes`
|
- Class mapping:
|
||||||
- ROI mapping: {doc}`configure-roi-mapping`
|
{doc}`define-target-classes`
|
||||||
- Targets reference: {doc}`../reference/targets-config-workflow`
|
- ROI mapping:
|
||||||
|
{doc}`configure-roi-mapping`
|
||||||
|
- Targets reference:
|
||||||
|
{doc}`../../reference/configs/data/targets-config-workflow`
|
||||||
@ -1,7 +1,6 @@
|
|||||||
# How to define target classes
|
# How to define target classes
|
||||||
|
|
||||||
Use this guide to map annotations to classification labels used during
|
Use this guide to map annotations to classification labels used during training.
|
||||||
training.
|
|
||||||
|
|
||||||
## 1) Add classification target entries
|
## 1) Add classification target entries
|
||||||
|
|
||||||
@ -54,6 +53,9 @@ classification_targets:
|
|||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Detection-target filtering: {doc}`configure-target-definitions`
|
- Detection-target filtering:
|
||||||
- ROI mapping: {doc}`configure-roi-mapping`
|
{doc}`configure-target-definitions`
|
||||||
- Targets config reference: {doc}`../reference/targets-config-workflow`
|
- ROI mapping:
|
||||||
|
{doc}`configure-roi-mapping`
|
||||||
|
- Targets config reference:
|
||||||
|
{doc}`../../reference/configs/data/targets-config-workflow`
|
||||||
@ -4,12 +4,15 @@ Use this guide if your annotations are in older batdetect2 JSON formats.
|
|||||||
|
|
||||||
Two legacy formats are supported:
|
Two legacy formats are supported:
|
||||||
|
|
||||||
- `batdetect2`: one annotation JSON file per recording
|
- `batdetect2`:
|
||||||
- `batdetect2_file`: one merged JSON file for many recordings
|
one annotation JSON file per recording
|
||||||
|
- `batdetect2_file`:
|
||||||
|
one merged JSON file for many recordings
|
||||||
|
|
||||||
## 1) Choose the correct source format
|
## 1) Choose the correct source format
|
||||||
|
|
||||||
Directory-based annotations (`format: batdetect2`):
|
Directory-based annotations (`format:
|
||||||
|
batdetect2`):
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
sources:
|
sources:
|
||||||
@ -19,7 +22,8 @@ sources:
|
|||||||
annotations_dir: /path/to/annotation_json_dir
|
annotations_dir: /path/to/annotation_json_dir
|
||||||
```
|
```
|
||||||
|
|
||||||
Merged annotation file (`format: batdetect2_file`):
|
Merged annotation file (`format:
|
||||||
|
batdetect2_file`):
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
sources:
|
sources:
|
||||||
@ -61,6 +65,9 @@ batdetect2 data convert path/to/dataset.yaml --output path/to/output.json
|
|||||||
|
|
||||||
## 4) Continue with current workflows
|
## 4) Continue with current workflows
|
||||||
|
|
||||||
- Run predictions: {doc}`run-batch-predictions`
|
- Run predictions:
|
||||||
- Train on imported data: {doc}`../tutorials/train-a-custom-model`
|
{doc}`../inference/run-batch-predictions`
|
||||||
- Field-level reference: {doc}`../reference/data-sources`
|
- Train on imported data:
|
||||||
|
{doc}`../../tutorials/train-a-custom-model`
|
||||||
|
- Field-level reference:
|
||||||
|
{doc}`../../reference/configs/data/data-sources`
|
||||||
11
docs/source/how_to/evaluation.md
Normal file
11
docs/source/how_to/evaluation.md
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
# Evaluation
|
||||||
|
|
||||||
|
Use this section when you want to choose evaluation tasks or understand the
|
||||||
|
artifacts produced by an evaluation run.
|
||||||
|
|
||||||
|
```{toctree}
|
||||||
|
:maxdepth: 1
|
||||||
|
|
||||||
|
evaluation/choose-and-configure-evaluation-tasks
|
||||||
|
evaluation/interpret-evaluation-outputs
|
||||||
|
```
|
||||||
@ -6,25 +6,15 @@ tutorial.
|
|||||||
Use this section when you already know the basic workflow and want help with one
|
Use this section when you already know the basic workflow and want help with one
|
||||||
specific task.
|
specific task.
|
||||||
|
|
||||||
|
The guides are grouped by topic so you can jump straight to the part of the
|
||||||
|
workflow you are working on.
|
||||||
|
|
||||||
```{toctree}
|
```{toctree}
|
||||||
:maxdepth: 1
|
:maxdepth: 1
|
||||||
|
|
||||||
choose-a-model
|
inference-and-outputs
|
||||||
choose-an-inference-input-mode
|
analysis-in-python
|
||||||
run-batch-predictions
|
training-and-fine-tuning
|
||||||
tune-inference-clipping
|
evaluation
|
||||||
tune-detection-threshold
|
data-and-targets
|
||||||
inspect-class-scores-in-python
|
|
||||||
inspect-detection-features-in-python
|
|
||||||
save-predictions-in-different-output-formats
|
|
||||||
fine-tune-from-a-checkpoint
|
|
||||||
choose-and-configure-evaluation-tasks
|
|
||||||
interpret-evaluation-outputs
|
|
||||||
configure-aoef-dataset
|
|
||||||
import-legacy-batdetect2-annotations
|
|
||||||
configure-audio-preprocessing
|
|
||||||
configure-spectrogram-preprocessing
|
|
||||||
configure-target-definitions
|
|
||||||
define-target-classes
|
|
||||||
configure-roi-mapping
|
|
||||||
```
|
```
|
||||||
|
|||||||
15
docs/source/how_to/inference-and-outputs.md
Normal file
15
docs/source/how_to/inference-and-outputs.md
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
# Inference and Outputs
|
||||||
|
|
||||||
|
Use this section when you want to run predictions, choose how inference runs, or
|
||||||
|
control what gets written to disk.
|
||||||
|
|
||||||
|
```{toctree}
|
||||||
|
:maxdepth: 1
|
||||||
|
|
||||||
|
inference/choose-a-model
|
||||||
|
inference/choose-an-inference-input-mode
|
||||||
|
inference/run-batch-predictions
|
||||||
|
inference/tune-inference-clipping
|
||||||
|
inference/tune-detection-threshold
|
||||||
|
inference/save-predictions-in-different-output-formats
|
||||||
|
```
|
||||||
@ -101,12 +101,12 @@ api = BatDetect2API.from_checkpoint(
|
|||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Run inference on a folder:
|
- Run inference on a folder:
|
||||||
{doc}`../tutorials/run-inference-on-folder`
|
{doc}`../../tutorials/run-inference-on-folder`
|
||||||
- `BatDetect2API` reference:
|
- `BatDetect2API` reference:
|
||||||
{doc}`../reference/api`
|
{doc}`../../reference/api`
|
||||||
- Process command reference:
|
- Process command reference:
|
||||||
{doc}`../reference/cli/predict`
|
{doc}`../../reference/cli/predict`
|
||||||
- Train a custom model:
|
- Train a custom model:
|
||||||
{doc}`../tutorials/train-a-custom-model`
|
{doc}`../../tutorials/train-a-custom-model`
|
||||||
- Fine-tune from a checkpoint:
|
- Fine-tune from a checkpoint:
|
||||||
{doc}`fine-tune-from-a-checkpoint`
|
{doc}`../training/fine-tune-from-a-checkpoint`
|
||||||
@ -68,4 +68,4 @@ recording paths before inference.
|
|||||||
- Tune inference clipping:
|
- Tune inference clipping:
|
||||||
{doc}`tune-inference-clipping`
|
{doc}`tune-inference-clipping`
|
||||||
- Process command reference:
|
- Process command reference:
|
||||||
{doc}`../reference/cli/predict`
|
{doc}`../../reference/cli/predict`
|
||||||
@ -66,8 +66,8 @@ Check a reviewed subset before applying clipping changes to a full project.
|
|||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Inference config reference:
|
- Inference config reference:
|
||||||
{doc}`../reference/inference-config`
|
{doc}`../../reference/configs/inference/inference-config`
|
||||||
- Run batch predictions:
|
- Run batch predictions:
|
||||||
{doc}`run-batch-predictions`
|
{doc}`run-batch-predictions`
|
||||||
- Understanding the pipeline:
|
- Understanding the pipeline:
|
||||||
{doc}`../explanation/pipeline-overview`
|
{doc}`../../explanation/pipeline-overview`
|
||||||
10
docs/source/how_to/training-and-fine-tuning.md
Normal file
10
docs/source/how_to/training-and-fine-tuning.md
Normal file
@ -0,0 +1,10 @@
|
|||||||
|
# Training and Fine-Tuning
|
||||||
|
|
||||||
|
Use this section when you already have a training workflow and want help with a
|
||||||
|
specific fine-tuning step.
|
||||||
|
|
||||||
|
```{toctree}
|
||||||
|
:maxdepth: 1
|
||||||
|
|
||||||
|
training/fine-tune-from-a-checkpoint
|
||||||
|
```
|
||||||
@ -25,8 +25,8 @@ For more detail on the underlying approach, see the pre-print:
|
|||||||
[Towards a General Approach for Bat Echolocation Detection and Classification](https://www.biorxiv.org/content/10.1101/2022.12.14.520490v1)
|
[Towards a General Approach for Bat Echolocation Detection and Classification](https://www.biorxiv.org/content/10.1101/2022.12.14.520490v1)
|
||||||
|
|
||||||
```{warning}
|
```{warning}
|
||||||
Treat outputs as model predictions, not ground truth.
|
Model outputs are predictions, not ground truth, and may be incorrect in subtle or significant ways.
|
||||||
Always validate on reviewed local data before using results for ecological inference.
|
We strongly encourage validating results on reviewed local data before using them for downstream analyses.
|
||||||
```
|
```
|
||||||
|
|
||||||
## What can I do with it?
|
## What can I do with it?
|
||||||
|
|||||||
@ -34,6 +34,6 @@ Defined in `batdetect2.api_v2`.
|
|||||||
- Python tutorial:
|
- Python tutorial:
|
||||||
{doc}`../tutorials/integrate-with-a-python-pipeline`
|
{doc}`../tutorials/integrate-with-a-python-pipeline`
|
||||||
- Outputs config reference:
|
- Outputs config reference:
|
||||||
{doc}`outputs-config`
|
{doc}`configs/outputs/outputs-config`
|
||||||
- Output formats reference:
|
- Output formats reference:
|
||||||
{doc}`output-formats`
|
{doc}`configs/outputs/output-formats`
|
||||||
|
|||||||
@ -33,8 +33,8 @@ Prefer `batdetect2 process directory` for new workflows.
|
|||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- {doc}`../../tutorials/run-inference-on-folder`
|
- {doc}`../../tutorials/run-inference-on-folder`
|
||||||
- {doc}`../../how_to/run-batch-predictions`
|
- {doc}`../../how_to/inference/run-batch-predictions`
|
||||||
- {doc}`../../how_to/tune-detection-threshold`
|
- {doc}`../../how_to/inference/tune-detection-threshold`
|
||||||
- {doc}`../configs`
|
- {doc}`../configs`
|
||||||
|
|
||||||
```{toctree}
|
```{toctree}
|
||||||
|
|||||||
@ -3,16 +3,23 @@ Config reference
|
|||||||
|
|
||||||
BatDetect2 uses separate config objects for different workflow surfaces.
|
BatDetect2 uses separate config objects for different workflow surfaces.
|
||||||
|
|
||||||
Use the dedicated reference pages for each config family:
|
Use this section when you need exact config fields for training, inference,
|
||||||
|
evaluation, outputs, preprocessing, postprocessing, or target definitions.
|
||||||
|
|
||||||
- model config
|
.. toctree::
|
||||||
- training config
|
:maxdepth: 1
|
||||||
- logging config
|
|
||||||
- inference config
|
configs/training/model-config
|
||||||
- evaluation config
|
configs/training/training-config
|
||||||
- outputs config
|
configs/training/logging-config
|
||||||
- preprocessing config
|
configs/inference/inference-config
|
||||||
- postprocess config
|
configs/evaluation/evaluation-config
|
||||||
- targets config workflow
|
configs/outputs/outputs-config
|
||||||
|
configs/outputs/output-formats
|
||||||
|
configs/outputs/output-transforms
|
||||||
|
configs/data/data-sources
|
||||||
|
configs/data/preprocessing-config
|
||||||
|
configs/data/postprocess-config
|
||||||
|
configs/data/targets-config-workflow
|
||||||
|
|
||||||
Example config files live under `example_data/configs/`.
|
Example config files live under `example_data/configs/`.
|
||||||
|
|||||||
@ -24,16 +24,19 @@ Optional fields:
|
|||||||
- `description`
|
- `description`
|
||||||
- `filter`
|
- `filter`
|
||||||
|
|
||||||
`filter` is only used when `annotations_path` points to an
|
`filter` is only used when `annotations_path` points to an `AnnotationProject`.
|
||||||
`AnnotationProject`.
|
|
||||||
|
|
||||||
AOEF filter options:
|
AOEF filter options:
|
||||||
|
|
||||||
- `only_completed` (default: `true`)
|
- `only_completed` (default:
|
||||||
- `only_verified` (default: `false`)
|
`true`)
|
||||||
- `exclude_issues` (default: `true`)
|
- `only_verified` (default:
|
||||||
|
`false`)
|
||||||
|
- `exclude_issues` (default:
|
||||||
|
`true`)
|
||||||
|
|
||||||
Use `filter: null` to disable project filtering.
|
Use `filter:
|
||||||
|
null` to disable project filtering.
|
||||||
|
|
||||||
## Legacy per-file (`format: batdetect2`)
|
## Legacy per-file (`format: batdetect2`)
|
||||||
|
|
||||||
@ -65,12 +68,15 @@ Optional fields:
|
|||||||
|
|
||||||
Legacy filter options:
|
Legacy filter options:
|
||||||
|
|
||||||
- `only_annotated` (default: `true`)
|
- `only_annotated` (default:
|
||||||
- `exclude_issues` (default: `true`)
|
`true`)
|
||||||
|
- `exclude_issues` (default:
|
||||||
|
`true`)
|
||||||
|
|
||||||
Use `filter: null` to disable filtering.
|
Use `filter:
|
||||||
|
null` to disable filtering.
|
||||||
|
|
||||||
## Related guides
|
## Related guides
|
||||||
|
|
||||||
- {doc}`../how_to/configure-aoef-dataset`
|
- {doc}`../../../how_to/data/configure-aoef-dataset`
|
||||||
- {doc}`../how_to/import-legacy-batdetect2-annotations`
|
- {doc}`../../../how_to/data/import-legacy-batdetect2-annotations`
|
||||||
@ -18,14 +18,20 @@ Defined in `batdetect2.postprocess.config`.
|
|||||||
|
|
||||||
## Defaults
|
## Defaults
|
||||||
|
|
||||||
- `detection_threshold`: `0.01`
|
- `detection_threshold`:
|
||||||
- `classification_threshold`: `0.1`
|
`0.01`
|
||||||
- `top_k_per_sec`: `100`
|
- `classification_threshold`:
|
||||||
|
`0.1`
|
||||||
|
- `top_k_per_sec`:
|
||||||
|
`100`
|
||||||
|
|
||||||
`nms_kernel_size` defaults to the library constant used by the NMS module.
|
`nms_kernel_size` defaults to the library constant used by the NMS module.
|
||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Threshold behaviour: {doc}`../explanation/postprocessing-and-thresholds`
|
- Threshold behaviour:
|
||||||
- Threshold tuning workflow: {doc}`../how_to/tune-detection-threshold`
|
{doc}`../../../explanation/postprocessing-and-thresholds`
|
||||||
- CLI predict options: {doc}`cli/predict`
|
- Threshold tuning workflow:
|
||||||
|
{doc}`../../../how_to/inference/tune-detection-threshold`
|
||||||
|
- CLI predict options:
|
||||||
|
{doc}`../../cli/predict`
|
||||||
@ -8,9 +8,12 @@ Defined in `batdetect2.audio.loader`.
|
|||||||
|
|
||||||
Fields:
|
Fields:
|
||||||
|
|
||||||
- `samplerate` (int): target audio sample rate in Hz.
|
- `samplerate` (int):
|
||||||
- `resample.enabled` (bool): whether to resample loaded audio.
|
target audio sample rate in Hz.
|
||||||
- `resample.method` (`poly` or `fourier`): resampling method.
|
- `resample.enabled` (bool):
|
||||||
|
whether to resample loaded audio.
|
||||||
|
- `resample.method` (`poly` or `fourier`):
|
||||||
|
resampling method.
|
||||||
|
|
||||||
## Model preprocessing config (`PreprocessingConfig`)
|
## Model preprocessing config (`PreprocessingConfig`)
|
||||||
|
|
||||||
@ -18,11 +21,16 @@ Defined in `batdetect2.preprocess.config`.
|
|||||||
|
|
||||||
Top-level fields:
|
Top-level fields:
|
||||||
|
|
||||||
- `audio_transforms`: ordered waveform transforms.
|
- `audio_transforms`:
|
||||||
- `stft`: STFT parameters.
|
ordered waveform transforms.
|
||||||
- `frequencies`: spectrogram frequency range.
|
- `stft`:
|
||||||
- `spectrogram_transforms`: ordered spectrogram transforms.
|
STFT parameters.
|
||||||
- `size`: final resize settings.
|
- `frequencies`:
|
||||||
|
spectrogram frequency range.
|
||||||
|
- `spectrogram_transforms`:
|
||||||
|
ordered spectrogram transforms.
|
||||||
|
- `size`:
|
||||||
|
final resize settings.
|
||||||
|
|
||||||
### `audio_transforms` built-ins
|
### `audio_transforms` built-ins
|
||||||
|
|
||||||
@ -44,7 +52,8 @@ Top-level fields:
|
|||||||
### `spectrogram_transforms` built-ins
|
### `spectrogram_transforms` built-ins
|
||||||
|
|
||||||
- `pcen`
|
- `pcen`
|
||||||
- `scale_amplitude` (`scale: db|power`)
|
- `scale_amplitude` (`scale:
|
||||||
|
db|power`)
|
||||||
- `spectral_mean_subtraction`
|
- `spectral_mean_subtraction`
|
||||||
- `peak_normalize`
|
- `peak_normalize`
|
||||||
|
|
||||||
@ -55,7 +64,9 @@ Top-level fields:
|
|||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Audio preprocessing how-to: {doc}`../how_to/configure-audio-preprocessing`
|
- Audio preprocessing how-to:
|
||||||
|
{doc}`../../../how_to/data/configure-audio-preprocessing`
|
||||||
- Spectrogram preprocessing how-to:
|
- Spectrogram preprocessing how-to:
|
||||||
{doc}`../how_to/configure-spectrogram-preprocessing`
|
{doc}`../../../how_to/data/configure-spectrogram-preprocessing`
|
||||||
- Why consistency matters: {doc}`../explanation/preprocessing-consistency`
|
- Why consistency matters:
|
||||||
|
{doc}`../../../explanation/preprocessing-consistency`
|
||||||
@ -0,0 +1,78 @@
|
|||||||
|
# Targets config workflow reference
|
||||||
|
|
||||||
|
This page summarizes the target-definition configuration used by batdetect2.
|
||||||
|
|
||||||
|
## `TargetConfig`
|
||||||
|
|
||||||
|
Defined in `batdetect2.targets.config`.
|
||||||
|
|
||||||
|
Fields:
|
||||||
|
|
||||||
|
- `detection_target`:
|
||||||
|
one `TargetClassConfig` defining detection eligibility.
|
||||||
|
- `classification_targets`:
|
||||||
|
list of `TargetClassConfig` entries for class encoding/decoding.
|
||||||
|
- `roi`:
|
||||||
|
ROI mapping config with `default` mapper and optional per-class `overrides`.
|
||||||
|
|
||||||
|
## `TargetClassConfig`
|
||||||
|
|
||||||
|
Defined in `batdetect2.targets.classes`.
|
||||||
|
|
||||||
|
Fields:
|
||||||
|
|
||||||
|
- `name`:
|
||||||
|
class label name.
|
||||||
|
- `tags`:
|
||||||
|
tag list used for matching (shortcut for `match_if`).
|
||||||
|
- `match_if`:
|
||||||
|
explicit condition config (`match_if` is accepted as alias).
|
||||||
|
- `assign_tags`:
|
||||||
|
tags used when decoding this class.
|
||||||
|
|
||||||
|
`tags` and `match_if` are mutually exclusive.
|
||||||
|
|
||||||
|
## Supported condition config types
|
||||||
|
|
||||||
|
Built from `batdetect2.data.conditions`.
|
||||||
|
|
||||||
|
- `has_tag`
|
||||||
|
- `has_all_tags`
|
||||||
|
- `has_any_tag`
|
||||||
|
- `duration`
|
||||||
|
- `frequency`
|
||||||
|
- `all_of`
|
||||||
|
- `any_of`
|
||||||
|
- `not`
|
||||||
|
|
||||||
|
## ROI mapper config
|
||||||
|
|
||||||
|
`roi.default` and each `roi.overrides.<class_name>` entry support built-in
|
||||||
|
mappers including:
|
||||||
|
|
||||||
|
- `anchor_bbox`
|
||||||
|
- `peak_energy_bbox`
|
||||||
|
|
||||||
|
Key `anchor_bbox` fields:
|
||||||
|
|
||||||
|
- `anchor`
|
||||||
|
- `time_scale`
|
||||||
|
- `frequency_scale`
|
||||||
|
|
||||||
|
Top-level ROI mapping shape:
|
||||||
|
|
||||||
|
- `default`:
|
||||||
|
fallback mapper used for all classes.
|
||||||
|
- `overrides`:
|
||||||
|
optional mapping from class name to mapper config.
|
||||||
|
|
||||||
|
## Related pages
|
||||||
|
|
||||||
|
- Detection target setup:
|
||||||
|
{doc}`../../../how_to/data/configure-target-definitions`
|
||||||
|
- Class setup:
|
||||||
|
{doc}`../../../how_to/data/define-target-classes`
|
||||||
|
- ROI setup:
|
||||||
|
{doc}`../../../how_to/data/configure-roi-mapping`
|
||||||
|
- Concept overview:
|
||||||
|
{doc}`../../../explanation/target-encoding-and-decoding`
|
||||||
@ -1,6 +1,7 @@
|
|||||||
# Evaluation config reference
|
# Evaluation config reference
|
||||||
|
|
||||||
`EvaluationConfig` defines which evaluation tasks run and which plots they generate.
|
`EvaluationConfig` defines which evaluation tasks run and which plots they
|
||||||
|
generate.
|
||||||
|
|
||||||
Defined in `batdetect2.evaluate.config`.
|
Defined in `batdetect2.evaluate.config`.
|
||||||
|
|
||||||
@ -41,6 +42,9 @@ The default evaluation config starts with:
|
|||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Choose and configure evaluation tasks: {doc}`../how_to/choose-and-configure-evaluation-tasks`
|
- Choose and configure evaluation tasks:
|
||||||
- Evaluation concepts: {doc}`../explanation/evaluation-concepts-and-matching`
|
{doc}`../../../how_to/evaluation/choose-and-configure-evaluation-tasks`
|
||||||
- Evaluate CLI reference: {doc}`cli/evaluate`
|
- Evaluation concepts:
|
||||||
|
{doc}`../../../explanation/evaluation-concepts-and-matching`
|
||||||
|
- Evaluate CLI reference:
|
||||||
|
{doc}`../../cli/evaluate`
|
||||||
@ -1,6 +1,7 @@
|
|||||||
# Inference config reference
|
# Inference config reference
|
||||||
|
|
||||||
`InferenceConfig` controls how files are clipped and batched during prediction-time workflows.
|
`InferenceConfig` controls how files are clipped and batched during
|
||||||
|
prediction-time workflows.
|
||||||
|
|
||||||
Defined in `batdetect2.inference.config`.
|
Defined in `batdetect2.inference.config`.
|
||||||
|
|
||||||
@ -37,5 +38,7 @@ Override `InferenceConfig` when:
|
|||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Tune inference clipping: {doc}`../how_to/tune-inference-clipping`
|
- Tune inference clipping:
|
||||||
- Predict CLI reference: {doc}`cli/predict`
|
{doc}`../../../how_to/inference/tune-inference-clipping`
|
||||||
|
- Predict CLI reference:
|
||||||
|
{doc}`../../cli/predict`
|
||||||
@ -70,6 +70,6 @@ It can also write legacy `_cnn_features.csv` sidecars when
|
|||||||
- Outputs config:
|
- Outputs config:
|
||||||
{doc}`outputs-config`
|
{doc}`outputs-config`
|
||||||
- Save predictions in different output formats:
|
- Save predictions in different output formats:
|
||||||
{doc}`../how_to/save-predictions-in-different-output-formats`
|
{doc}`../../../how_to/inference/save-predictions-in-different-output-formats`
|
||||||
- Understanding formatted outputs:
|
- Understanding formatted outputs:
|
||||||
{doc}`../explanation/interpreting-formatted-outputs`
|
{doc}`../../../explanation/interpreting-formatted-outputs`
|
||||||
@ -33,5 +33,7 @@ Built-in examples include:
|
|||||||
|
|
||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Outputs config: {doc}`outputs-config`
|
- Outputs config:
|
||||||
- Understanding outputs: {doc}`../explanation/interpreting-formatted-outputs`
|
{doc}`outputs-config`
|
||||||
|
- Understanding outputs:
|
||||||
|
{doc}`../../../explanation/interpreting-formatted-outputs`
|
||||||
@ -38,4 +38,4 @@ If no outputs config is provided, the CLI still uses its command defaults.
|
|||||||
- Output transforms:
|
- Output transforms:
|
||||||
{doc}`output-transforms`
|
{doc}`output-transforms`
|
||||||
- Save predictions in different output formats:
|
- Save predictions in different output formats:
|
||||||
{doc}`../how_to/save-predictions-in-different-output-formats`
|
{doc}`../../../how_to/inference/save-predictions-in-different-output-formats`
|
||||||
@ -30,8 +30,8 @@ Example files live under `example_data/configs/`, including
|
|||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Preprocessing config:
|
- Preprocessing config:
|
||||||
{doc}`preprocessing-config`
|
{doc}`../data/preprocessing-config`
|
||||||
- Postprocess config:
|
- Postprocess config:
|
||||||
{doc}`postprocess-config`
|
{doc}`../data/postprocess-config`
|
||||||
- Train command reference:
|
- Train command reference:
|
||||||
{doc}`cli/train`
|
{doc}`../../cli/train`
|
||||||
@ -43,8 +43,8 @@ Example files live under `example_data/configs/`, including
|
|||||||
## Related pages
|
## Related pages
|
||||||
|
|
||||||
- Evaluation config:
|
- Evaluation config:
|
||||||
{doc}`evaluation-config`
|
{doc}`../evaluation/evaluation-config`
|
||||||
- Train command reference:
|
- Train command reference:
|
||||||
{doc}`cli/train`
|
{doc}`../../cli/train`
|
||||||
- Fine-tune from a checkpoint:
|
- Fine-tune from a checkpoint:
|
||||||
{doc}`../how_to/fine-tune-from-a-checkpoint`
|
{doc}`../../../how_to/training/fine-tune-from-a-checkpoint`
|
||||||
@ -11,18 +11,6 @@ details, or Python API entries.
|
|||||||
cli/index
|
cli/index
|
||||||
api
|
api
|
||||||
detections
|
detections
|
||||||
model-config
|
|
||||||
training-config
|
|
||||||
logging-config
|
|
||||||
inference-config
|
|
||||||
evaluation-config
|
|
||||||
outputs-config
|
|
||||||
output-formats
|
|
||||||
output-transforms
|
|
||||||
data-sources
|
|
||||||
preprocessing-config
|
|
||||||
postprocess-config
|
|
||||||
targets-config-workflow
|
|
||||||
configs
|
configs
|
||||||
targets
|
targets
|
||||||
```
|
```
|
||||||
|
|||||||
@ -1,67 +0,0 @@
|
|||||||
# Targets config workflow reference
|
|
||||||
|
|
||||||
This page summarizes the target-definition configuration used by batdetect2.
|
|
||||||
|
|
||||||
## `TargetConfig`
|
|
||||||
|
|
||||||
Defined in `batdetect2.targets.config`.
|
|
||||||
|
|
||||||
Fields:
|
|
||||||
|
|
||||||
- `detection_target`: one `TargetClassConfig` defining detection eligibility.
|
|
||||||
- `classification_targets`: list of `TargetClassConfig` entries for class
|
|
||||||
encoding/decoding.
|
|
||||||
- `roi`: ROI mapping config with `default` mapper and optional per-class
|
|
||||||
`overrides`.
|
|
||||||
|
|
||||||
## `TargetClassConfig`
|
|
||||||
|
|
||||||
Defined in `batdetect2.targets.classes`.
|
|
||||||
|
|
||||||
Fields:
|
|
||||||
|
|
||||||
- `name`: class label name.
|
|
||||||
- `tags`: tag list used for matching (shortcut for `match_if`).
|
|
||||||
- `match_if`: explicit condition config (`match_if` is accepted as alias).
|
|
||||||
- `assign_tags`: tags used when decoding this class.
|
|
||||||
|
|
||||||
`tags` and `match_if` are mutually exclusive.
|
|
||||||
|
|
||||||
## Supported condition config types
|
|
||||||
|
|
||||||
Built from `batdetect2.data.conditions`.
|
|
||||||
|
|
||||||
- `has_tag`
|
|
||||||
- `has_all_tags`
|
|
||||||
- `has_any_tag`
|
|
||||||
- `duration`
|
|
||||||
- `frequency`
|
|
||||||
- `all_of`
|
|
||||||
- `any_of`
|
|
||||||
- `not`
|
|
||||||
|
|
||||||
## ROI mapper config
|
|
||||||
|
|
||||||
`roi.default` and each `roi.overrides.<class_name>` entry support built-in
|
|
||||||
mappers including:
|
|
||||||
|
|
||||||
- `anchor_bbox`
|
|
||||||
- `peak_energy_bbox`
|
|
||||||
|
|
||||||
Key `anchor_bbox` fields:
|
|
||||||
|
|
||||||
- `anchor`
|
|
||||||
- `time_scale`
|
|
||||||
- `frequency_scale`
|
|
||||||
|
|
||||||
Top-level ROI mapping shape:
|
|
||||||
|
|
||||||
- `default`: fallback mapper used for all classes.
|
|
||||||
- `overrides`: optional mapping from class name to mapper config.
|
|
||||||
|
|
||||||
## Related pages
|
|
||||||
|
|
||||||
- Detection target setup: {doc}`../how_to/configure-target-definitions`
|
|
||||||
- Class setup: {doc}`../how_to/define-target-classes`
|
|
||||||
- ROI setup: {doc}`../how_to/configure-roi-mapping`
|
|
||||||
- Concept overview: {doc}`../explanation/target-encoding-and-decoding`
|
|
||||||
@ -41,7 +41,8 @@ For an example, see `example_data/dataset.yaml`.
|
|||||||
|
|
||||||
If you need help creating the dataset config, follow the dataset section in
|
If you need help creating the dataset config, follow the dataset section in
|
||||||
{doc}`train-a-custom-model`.
|
{doc}`train-a-custom-model`.
|
||||||
For more detail on dataset source formats, see {doc}`../reference/data-sources`.
|
For more detail on dataset source formats, see
|
||||||
|
{doc}`../reference/configs/data/data-sources`.
|
||||||
|
|
||||||
Use a dataset that was not used for training or tuning.
|
Use a dataset that was not used for training or tuning.
|
||||||
|
|
||||||
@ -56,7 +57,7 @@ batdetect2 evaluate \
|
|||||||
|
|
||||||
If you do not pass `--model`, BatDetect2 uses the built-in default UK model.
|
If you do not pass `--model`, BatDetect2 uses the built-in default UK model.
|
||||||
If you want to choose a different checkpoint, alias, or Hugging Face model, see
|
If you want to choose a different checkpoint, alias, or Hugging Face model, see
|
||||||
{doc}`../how_to/choose-a-model`.
|
{doc}`../how_to/inference/choose-a-model`.
|
||||||
|
|
||||||
If you want to save the results somewhere else, add `--output-dir`:
|
If you want to save the results somewhere else, add `--output-dir`:
|
||||||
|
|
||||||
@ -121,17 +122,17 @@ So, depending on your evaluation config, you may see files such as:
|
|||||||
- saved prediction files.
|
- saved prediction files.
|
||||||
|
|
||||||
If you want to control which tasks run and which plots are generated, see
|
If you want to control which tasks run and which plots are generated, see
|
||||||
{doc}`../reference/evaluation-config` and
|
{doc}`../reference/configs/evaluation/evaluation-config` and
|
||||||
{doc}`../how_to/choose-and-configure-evaluation-tasks`.
|
{doc}`../how_to/evaluation/choose-and-configure-evaluation-tasks`.
|
||||||
|
|
||||||
## Common next steps
|
## Common next steps
|
||||||
|
|
||||||
- Choose a different model:
|
- Choose a different model:
|
||||||
{doc}`../how_to/choose-a-model`
|
{doc}`../how_to/inference/choose-a-model`
|
||||||
- Configure evaluation tasks:
|
- Configure evaluation tasks:
|
||||||
{doc}`../how_to/choose-and-configure-evaluation-tasks`
|
{doc}`../how_to/evaluation/choose-and-configure-evaluation-tasks`
|
||||||
- Interpret evaluation artifacts:
|
- Interpret evaluation artifacts:
|
||||||
{doc}`../how_to/interpret-evaluation-outputs`
|
{doc}`../how_to/evaluation/interpret-evaluation-outputs`
|
||||||
- Learn the evaluation concepts:
|
- Learn the evaluation concepts:
|
||||||
{doc}`../explanation/evaluation-concepts-and-matching`
|
{doc}`../explanation/evaluation-concepts-and-matching`
|
||||||
- Check full evaluate options:
|
- Check full evaluate options:
|
||||||
|
|||||||
@ -1,10 +1,12 @@
|
|||||||
# Integrate with a Python pipeline
|
# Integrate with a Python pipeline
|
||||||
|
|
||||||
This tutorial shows a simple Python workflow for loading audio, running BatDetect2, and inspecting the detections.
|
This tutorial shows a simple Python workflow for loading audio, running
|
||||||
|
BatDetect2, and inspecting the detections.
|
||||||
|
|
||||||
Use it when you want to work directly in Python rather than through the CLI.
|
Use it when you want to work directly in Python rather than through the CLI.
|
||||||
|
|
||||||
If you mainly want to run the model on recordings, start with {doc}`run-inference-on-folder` instead.
|
If you mainly want to run the model on recordings, start with
|
||||||
|
{doc}`run-inference-on-folder` instead.
|
||||||
|
|
||||||
## Before you start
|
## Before you start
|
||||||
|
|
||||||
@ -21,7 +23,8 @@ By the end of this tutorial you will have:
|
|||||||
- run inference on one file,
|
- run inference on one file,
|
||||||
- inspected detections, scores, and features,
|
- inspected detections, scores, and features,
|
||||||
- used lower-level audio and spectrogram methods for more control,
|
- used lower-level audio and spectrogram methods for more control,
|
||||||
- identified the next API workflows for batch processing, training, fine-tuning, and evaluation.
|
- identified the next API workflows for batch processing, training, fine-tuning,
|
||||||
|
and evaluation.
|
||||||
|
|
||||||
## 1. Create the API instance
|
## 1. Create the API instance
|
||||||
|
|
||||||
@ -34,11 +37,13 @@ from batdetect2 import BatDetect2API
|
|||||||
api = BatDetect2API.from_checkpoint()
|
api = BatDetect2API.from_checkpoint()
|
||||||
```
|
```
|
||||||
|
|
||||||
If you want to use a different checkpoint later, see {doc}`../how_to/choose-a-model`.
|
If you want to use a different checkpoint later, see
|
||||||
|
{doc}`../how_to/inference/choose-a-model`.
|
||||||
|
|
||||||
## 2. Run inference on one file
|
## 2. Run inference on one file
|
||||||
|
|
||||||
`process_file` is the simplest Python entry point when you want one prediction object per recording.
|
`process_file` is the simplest Python entry point when you want one prediction
|
||||||
|
object per recording.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from batdetect2 import BatDetect2API
|
from batdetect2 import BatDetect2API
|
||||||
@ -57,7 +62,8 @@ for detection in prediction.detections:
|
|||||||
`prediction` is a `ClipDetections` object.
|
`prediction` is a `ClipDetections` object.
|
||||||
See {doc}`../reference/detections` for the full reference.
|
See {doc}`../reference/detections` for the full reference.
|
||||||
|
|
||||||
Very briefly, `ClipDetections` represents all detections for one processed clip or recording.
|
Very briefly, `ClipDetections` represents all detections for one processed clip
|
||||||
|
or recording.
|
||||||
It includes:
|
It includes:
|
||||||
|
|
||||||
- the clip metadata,
|
- the clip metadata,
|
||||||
@ -76,10 +82,13 @@ Each `Detection` includes:
|
|||||||
|
|
||||||
The detection score and the class scores answer different questions.
|
The detection score and the class scores answer different questions.
|
||||||
|
|
||||||
- `detection_score` is about whether the model thinks there is a call at that time-frequency location.
|
- `detection_score` is about whether the model thinks there is a call at that
|
||||||
- `class_scores` are about which class the model prefers for that detected event.
|
time-frequency location.
|
||||||
|
- `class_scores` are about which class the model prefers for that detected
|
||||||
|
event.
|
||||||
|
|
||||||
So a detection can have a fairly strong detection score, but still have a more uncertain class ranking.
|
So a detection can have a fairly strong detection score, but still have a more
|
||||||
|
uncertain class ranking.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
for detection in prediction.detections:
|
for detection in prediction.detections:
|
||||||
@ -90,7 +99,8 @@ for detection in prediction.detections:
|
|||||||
print(f" {class_name}: {score:.3f}")
|
print(f" {class_name}: {score:.3f}")
|
||||||
```
|
```
|
||||||
|
|
||||||
If you want more detail on class-score inspection, see {doc}`../how_to/inspect-class-scores-in-python`.
|
If you want more detail on class-score inspection, see
|
||||||
|
{doc}`../how_to/analysis/inspect-class-scores-in-python`.
|
||||||
|
|
||||||
## 5. Inspect the detection features
|
## 5. Inspect the detection features
|
||||||
|
|
||||||
@ -104,15 +114,20 @@ They can be useful for things like:
|
|||||||
- comparing detections across files,
|
- comparing detections across files,
|
||||||
- building downstream analysis pipelines.
|
- building downstream analysis pipelines.
|
||||||
|
|
||||||
They are useful descriptors, but they are not direct ecological labels by themselves.
|
They are useful descriptors, but they are not direct ecological labels by
|
||||||
|
themselves.
|
||||||
|
|
||||||
For more detail, see {doc}`../how_to/inspect-detection-features-in-python` and {doc}`../explanation/extracted-features-and-embeddings`.
|
For more detail, see
|
||||||
|
{doc}`../how_to/analysis/inspect-detection-features-in-python` and
|
||||||
|
{doc}`../explanation/extracted-features-and-embeddings`.
|
||||||
|
|
||||||
## 6. Use lower-level audio and spectrogram methods for more control
|
## 6. Use lower-level audio and spectrogram methods for more control
|
||||||
|
|
||||||
If you want finer control over what gets processed and when, the API also lets you work step by step.
|
If you want finer control over what gets processed and when, the API also lets
|
||||||
|
you work step by step.
|
||||||
|
|
||||||
For example, you can load the audio yourself, inspect the waveform length, generate the spectrogram, and then run detection on that spectrogram:
|
For example, you can load the audio yourself, inspect the waveform length,
|
||||||
|
generate the spectrogram, and then run detection on that spectrogram:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from batdetect2 import BatDetect2API
|
from batdetect2 import BatDetect2API
|
||||||
@ -136,12 +151,14 @@ This is helpful when you want to:
|
|||||||
- control which audio segment is processed,
|
- control which audio segment is processed,
|
||||||
- run only part of the pipeline in custom code.
|
- run only part of the pipeline in custom code.
|
||||||
|
|
||||||
You can also call `process_audio(audio)` directly if you already have the waveform array in memory.
|
You can also call `process_audio(audio)` directly if you already have the
|
||||||
|
waveform array in memory.
|
||||||
|
|
||||||
## 7. Use the wider API workflows
|
## 7. Use the wider API workflows
|
||||||
|
|
||||||
The Python API is not only for single-file inference.
|
The Python API is not only for single-file inference.
|
||||||
It also exposes methods for batch processing, training, evaluation, and fine-tuning.
|
It also exposes methods for batch processing, training, evaluation, and
|
||||||
|
fine-tuning.
|
||||||
|
|
||||||
Examples:
|
Examples:
|
||||||
|
|
||||||
@ -152,9 +169,15 @@ Examples:
|
|||||||
|
|
||||||
Useful next pages:
|
Useful next pages:
|
||||||
|
|
||||||
- Choose a different model: {doc}`../how_to/choose-a-model`
|
- Choose a different model:
|
||||||
- Run batch predictions: {doc}`../how_to/run-batch-predictions`
|
{doc}`../how_to/inference/choose-a-model`
|
||||||
- Train a custom model: {doc}`train-a-custom-model`
|
- Run batch predictions:
|
||||||
- Evaluate on a test set: {doc}`evaluate-on-a-test-set`
|
{doc}`../how_to/inference/run-batch-predictions`
|
||||||
- Fine-tune from a checkpoint: {doc}`../how_to/fine-tune-from-a-checkpoint`
|
- Train a custom model:
|
||||||
- API reference: {doc}`../reference/api`
|
{doc}`train-a-custom-model`
|
||||||
|
- Evaluate on a test set:
|
||||||
|
{doc}`evaluate-on-a-test-set`
|
||||||
|
- Fine-tune from a checkpoint:
|
||||||
|
{doc}`../how_to/training/fine-tune-from-a-checkpoint`
|
||||||
|
- API reference:
|
||||||
|
{doc}`../reference/api`
|
||||||
|
|||||||
@ -1,10 +1,13 @@
|
|||||||
# Run BatDetect2 on a folder of audio files
|
# Run BatDetect2 on a folder of audio files
|
||||||
|
|
||||||
This tutorial shows how to run BatDetect2 on a folder of recordings from the command line.
|
This tutorial shows how to run BatDetect2 on a folder of recordings from the
|
||||||
|
command line.
|
||||||
|
|
||||||
Use it when you want a first pass over a folder of audio recordings and want to see what BatDetect2 finds.
|
Use it when you want a first pass over a folder of audio recordings and want to
|
||||||
|
see what BatDetect2 finds.
|
||||||
|
|
||||||
If you want to follow the tutorial exactly, you can use the example recordings that come with the repository.
|
If you want to follow the tutorial exactly, you can use the example recordings
|
||||||
|
that come with the repository.
|
||||||
|
|
||||||
## Before you start
|
## Before you start
|
||||||
|
|
||||||
@ -18,7 +21,8 @@ If you have not installed BatDetect2 yet, start with {doc}`../getting_started`.
|
|||||||
|
|
||||||
## Optional: use the repository example files
|
## Optional: use the repository example files
|
||||||
|
|
||||||
If you want to follow the steps with the same paths shown here, clone the repository and move into it:
|
If you want to follow the steps with the same paths shown here, clone the
|
||||||
|
repository and move into it:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/macaodha/batdetect2.git
|
git clone https://github.com/macaodha/batdetect2.git
|
||||||
@ -57,7 +61,8 @@ project/
|
|||||||
If `outputs/` does not exist yet, that is fine.
|
If `outputs/` does not exist yet, that is fine.
|
||||||
BatDetect2 can create it.
|
BatDetect2 can create it.
|
||||||
|
|
||||||
If you are using the repository example files, your layout already looks like this:
|
If you are using the repository example files, your layout already looks like
|
||||||
|
this:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
batdetect2/
|
batdetect2/
|
||||||
@ -95,7 +100,8 @@ What this does:
|
|||||||
You do not need to choose a model for this first run.
|
You do not need to choose a model for this first run.
|
||||||
If you do nothing, BatDetect2 uses the built-in default UK model.
|
If you do nothing, BatDetect2 uses the built-in default UK model.
|
||||||
|
|
||||||
If you want to use a different model later, see {doc}`../how_to/choose-a-model`.
|
If you want to use a different model later, see
|
||||||
|
{doc}`../how_to/inference/choose-a-model`.
|
||||||
|
|
||||||
## 3. Check the output files
|
## 3. Check the output files
|
||||||
|
|
||||||
@ -104,7 +110,8 @@ After the command finishes, look in your output folder.
|
|||||||
By default, the CLI writes predictions in the `batdetect2` output format.
|
By default, the CLI writes predictions in the `batdetect2` output format.
|
||||||
This is a JSON-based format used for BatDetect2-style outputs.
|
This is a JSON-based format used for BatDetect2-style outputs.
|
||||||
|
|
||||||
With the default settings, you will usually see one `.json` file and one `_detections.csv` file per recording.
|
With the default settings, you will usually see one `.json` file and one
|
||||||
|
`_detections.csv` file per recording.
|
||||||
|
|
||||||
For the repository example run, that means files like:
|
For the repository example run, that means files like:
|
||||||
|
|
||||||
@ -141,16 +148,20 @@ One of the JSON files will look roughly like this:
|
|||||||
|
|
||||||
Very briefly:
|
Very briefly:
|
||||||
|
|
||||||
- `annotated: false` means this is a prediction file, not a reviewed annotation file.
|
- `annotated:
|
||||||
|
false` means this is a prediction file, not a reviewed annotation file.
|
||||||
- `annotation` holds the list of detections.
|
- `annotation` holds the list of detections.
|
||||||
- Each detection includes a predicted class, detection score, class score, time bounds, and frequency bounds.
|
- Each detection includes a predicted class, detection score, class score, time
|
||||||
|
bounds, and frequency bounds.
|
||||||
|
|
||||||
For more detail, see {doc}`../explanation/interpreting-formatted-outputs`.
|
For more detail, see {doc}`../explanation/interpreting-formatted-outputs`.
|
||||||
If you want to save results in another format, see {doc}`../how_to/save-predictions-in-different-output-formats`.
|
If you want to save results in another format, see
|
||||||
|
{doc}`../how_to/inference/save-predictions-in-different-output-formats`.
|
||||||
|
|
||||||
## 4. Run the same folder with a higher threshold
|
## 4. Run the same folder with a higher threshold
|
||||||
|
|
||||||
If you want, you can also run the same folder again with a higher detection threshold and save that run in a separate output folder.
|
If you want, you can also run the same folder again with a higher detection
|
||||||
|
threshold and save that run in a separate output folder.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
batdetect2 process directory \
|
batdetect2 process directory \
|
||||||
@ -206,12 +217,18 @@ batdetect2 process file_list \
|
|||||||
example_outputs/selected_outputs
|
example_outputs/selected_outputs
|
||||||
```
|
```
|
||||||
|
|
||||||
This is useful when your recordings are spread across folders, or when you only want to run a chosen subset.
|
This is useful when your recordings are spread across folders, or when you only
|
||||||
|
want to run a chosen subset.
|
||||||
|
|
||||||
## Common next steps
|
## Common next steps
|
||||||
|
|
||||||
- If your recordings are not all in one folder, or you want to compare input modes, see {doc}`../how_to/choose-an-inference-input-mode`.
|
- If your recordings are not all in one folder, or you want to compare input
|
||||||
- If you want to save results in another format, see {doc}`../how_to/save-predictions-in-different-output-formats`.
|
modes, see {doc}`../how_to/inference/choose-an-inference-input-mode`.
|
||||||
- If you want to choose a different model, see {doc}`../how_to/choose-a-model`.
|
- If you want to save results in another format, see
|
||||||
- If you already write code and want more control from Python, see {doc}`integrate-with-a-python-pipeline`.
|
{doc}`../how_to/inference/save-predictions-in-different-output-formats`.
|
||||||
- If you want the full command reference, including `--model`, see {doc}`../reference/cli/predict`.
|
- If you want to choose a different model, see
|
||||||
|
{doc}`../how_to/inference/choose-a-model`.
|
||||||
|
- If you already write code and want more control from Python, see
|
||||||
|
{doc}`integrate-with-a-python-pipeline`.
|
||||||
|
- If you want the full command reference, including `--model`, see
|
||||||
|
{doc}`../reference/cli/predict`.
|
||||||
|
|||||||
@ -1,8 +1,10 @@
|
|||||||
# Train a custom model
|
# Train a custom model
|
||||||
|
|
||||||
This tutorial walks through a first custom training run using your own annotations.
|
This tutorial walks through a first custom training run using your own
|
||||||
|
annotations.
|
||||||
|
|
||||||
Use it when you already have labelled recordings and want to train a model for your own data.
|
Use it when you already have labelled recordings and want to train a model for
|
||||||
|
your own data.
|
||||||
|
|
||||||
## Before you start
|
## Before you start
|
||||||
|
|
||||||
@ -19,7 +21,8 @@ Use {doc}`run-inference-on-folder` for that.
|
|||||||
|
|
||||||
## Optional: use the repository example files
|
## Optional: use the repository example files
|
||||||
|
|
||||||
If you want to follow the steps with the same files shown here, clone the repository and move into it:
|
If you want to follow the steps with the same files shown here, clone the
|
||||||
|
repository and move into it:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/macaodha/batdetect2.git
|
git clone https://github.com/macaodha/batdetect2.git
|
||||||
@ -40,10 +43,12 @@ By the end of this tutorial you will have:
|
|||||||
|
|
||||||
The dataset config explicitly declares what data you want to use for training.
|
The dataset config explicitly declares what data you want to use for training.
|
||||||
It is a YAML file.
|
It is a YAML file.
|
||||||
If YAML is new to you, see [Learn YAML in Y Minutes](https://learnxinyminutes.com/yaml/).
|
If YAML is new to you, see
|
||||||
|
[Learn YAML in Y Minutes](https://learnxinyminutes.com/yaml/).
|
||||||
|
|
||||||
In the dataset config, you list one or more data sources.
|
In the dataset config, you list one or more data sources.
|
||||||
Each source tells `batdetect2` where the audio recordings live and where the matching annotations are stored.
|
Each source tells `batdetect2` where the audio recordings live and where the
|
||||||
|
matching annotations are stored.
|
||||||
|
|
||||||
BatDetect2 can read annotations from different source formats.
|
BatDetect2 can read annotations from different source formats.
|
||||||
In this example, we use the example data in the `batdetect2` format.
|
In this example, we use the example data in the `batdetect2` format.
|
||||||
@ -62,21 +67,25 @@ sources:
|
|||||||
```
|
```
|
||||||
|
|
||||||
For your own project, the main thing to change is the file paths.
|
For your own project, the main thing to change is the file paths.
|
||||||
If you have several collections of recordings, you can add more than one source to the same dataset config.
|
If you have several collections of recordings, you can add more than one source
|
||||||
|
to the same dataset config.
|
||||||
That lets you describe the full training data you want to use in one place.
|
That lets you describe the full training data you want to use in one place.
|
||||||
|
|
||||||
If you need more detail on dataset source formats, see {doc}`../reference/data-sources`.
|
If you need more detail on dataset source formats, see
|
||||||
|
{doc}`../reference/configs/data/data-sources`.
|
||||||
|
|
||||||
## 2. Define a targets config
|
## 2. Define a targets config
|
||||||
|
|
||||||
The targets config tells BatDetect2 how to turn your annotations into training targets.
|
The targets config tells BatDetect2 how to turn your annotations into training
|
||||||
|
targets.
|
||||||
|
|
||||||
It defines two main things:
|
It defines two main things:
|
||||||
|
|
||||||
- what should count as a detection,
|
- what should count as a detection,
|
||||||
- which classes the model should learn to predict.
|
- which classes the model should learn to predict.
|
||||||
|
|
||||||
In practice, this means the targets config maps the labels in your annotations to the detection and classification outputs used during training.
|
In practice, this means the targets config maps the labels in your annotations
|
||||||
|
to the detection and classification outputs used during training.
|
||||||
|
|
||||||
Use `example_data/targets.yaml` as a reference:
|
Use `example_data/targets.yaml` as a reference:
|
||||||
|
|
||||||
@ -107,16 +116,21 @@ classification_targets:
|
|||||||
value: Pipistrellus pipistrellus
|
value: Pipistrellus pipistrellus
|
||||||
```
|
```
|
||||||
|
|
||||||
For your own project, update the matching rules and class definitions so they fit your labels.
|
For your own project, update the matching rules and class definitions so they
|
||||||
|
fit your labels.
|
||||||
|
|
||||||
In this example:
|
In this example:
|
||||||
|
|
||||||
- `detection_target` says that echolocation calls should be treated as detections,
|
- `detection_target` says that echolocation calls should be treated as
|
||||||
|
detections,
|
||||||
- `classification_targets` define the classes the model should predict,
|
- `classification_targets` define the classes the model should predict,
|
||||||
|
|
||||||
It is worth taking a bit of time over this file, because your targets config decides what the model is actually being asked to learn.
|
It is worth taking a bit of time over this file, because your targets config
|
||||||
|
decides what the model is actually being asked to learn.
|
||||||
|
|
||||||
If you need help with that, see {doc}`../how_to/configure-target-definitions` and {doc}`../reference/targets-config-workflow`.
|
If you need help with that, see
|
||||||
|
{doc}`../how_to/data/configure-target-definitions` and
|
||||||
|
{doc}`../reference/configs/data/targets-config-workflow`.
|
||||||
|
|
||||||
## 3. Run a first training command
|
## 3. Run a first training command
|
||||||
|
|
||||||
@ -138,12 +152,16 @@ batdetect2 train \
|
|||||||
--targets example_data/targets.yaml
|
--targets example_data/targets.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
This uses the same dataset for training and validation only to keep the example simple.
|
This uses the same dataset for training and validation only to keep the example
|
||||||
For real training runs, you usually want separate training and validation datasets.
|
simple.
|
||||||
|
For real training runs, you usually want separate training and validation
|
||||||
|
datasets.
|
||||||
|
|
||||||
This uses the built-in default model and training settings.
|
This uses the built-in default model and training settings.
|
||||||
If you want to change the model architecture later, see {doc}`../reference/model-config`.
|
If you want to change the model architecture later, see
|
||||||
If you want to change optimiser settings, batch size, epochs, or checkpoint behaviour, see {doc}`../reference/training-config`.
|
{doc}`../reference/configs/training/model-config`.
|
||||||
|
If you want to change optimiser settings, batch size, epochs, or checkpoint
|
||||||
|
behaviour, see {doc}`../reference/configs/training/training-config`.
|
||||||
|
|
||||||
## 4. Check the training outputs
|
## 4. Check the training outputs
|
||||||
|
|
||||||
@ -170,10 +188,13 @@ outputs/
|
|||||||
val_class_summary.csv
|
val_class_summary.csv
|
||||||
```
|
```
|
||||||
|
|
||||||
The checkpoint is the trained model you can use later for inference, evaluation, or sharing with someone else.
|
The checkpoint is the trained model you can use later for inference, evaluation,
|
||||||
|
or sharing with someone else.
|
||||||
|
|
||||||
The files in `training_artifacts/` record which datasets and targets were used for the run.
|
The files in `training_artifacts/` record which datasets and targets were used
|
||||||
The `hparams.yaml` file records the full training setup, including the configs used for the model, training, and other parts of the run.
|
for the run.
|
||||||
|
The `hparams.yaml` file records the full training setup, including the configs
|
||||||
|
used for the model, training, and other parts of the run.
|
||||||
|
|
||||||
The `metrics.csv` file stores one row per validation epoch.
|
The `metrics.csv` file stores one row per validation epoch.
|
||||||
It includes training losses as well as validation losses and metrics such as:
|
It includes training losses as well as validation losses and metrics such as:
|
||||||
@ -190,19 +211,29 @@ You may also see class-specific metrics in extra columns.
|
|||||||
The more detailed metrics are computed from the validation set.
|
The more detailed metrics are computed from the validation set.
|
||||||
If you do not provide `--val-dataset`, those validation metrics will not appear.
|
If you do not provide `--val-dataset`, those validation metrics will not appear.
|
||||||
|
|
||||||
Other logger backends are also supported, including TensorBoard, MLflow, and DVCLive.
|
Other logger backends are also supported, including TensorBoard, MLflow, and
|
||||||
See {doc}`../reference/logging-config` if you want to change that.
|
DVCLive.
|
||||||
|
See {doc}`../reference/configs/training/logging-config` if you want to change
|
||||||
|
that.
|
||||||
|
|
||||||
## Use the trained model
|
## Use the trained model
|
||||||
|
|
||||||
You can now use the trained checkpoint in BatDetect2, or share it with someone else to use in their own runs.
|
You can now use the trained checkpoint in BatDetect2, or share it with someone
|
||||||
If you want to load it for inference or evaluation, see {doc}`../how_to/choose-a-model`.
|
else to use in their own runs.
|
||||||
|
If you want to load it for inference or evaluation, see
|
||||||
|
{doc}`../how_to/inference/choose-a-model`.
|
||||||
|
|
||||||
## Common next steps
|
## Common next steps
|
||||||
|
|
||||||
- Evaluate the trained checkpoint: {doc}`evaluate-on-a-test-set`
|
- Evaluate the trained checkpoint:
|
||||||
- Fine-tune from a checkpoint: {doc}`../how_to/fine-tune-from-a-checkpoint`
|
{doc}`evaluate-on-a-test-set`
|
||||||
- Configure targets in more detail: {doc}`../how_to/configure-target-definitions`
|
- Fine-tune from a checkpoint:
|
||||||
- Configure audio preprocessing: {doc}`../how_to/configure-audio-preprocessing`
|
{doc}`../how_to/training/fine-tune-from-a-checkpoint`
|
||||||
- Configure spectrogram preprocessing: {doc}`../how_to/configure-spectrogram-preprocessing`
|
- Configure targets in more detail:
|
||||||
- Check full train options: {doc}`../reference/cli/train`
|
{doc}`../how_to/data/configure-target-definitions`
|
||||||
|
- Configure audio preprocessing:
|
||||||
|
{doc}`../how_to/data/configure-audio-preprocessing`
|
||||||
|
- Configure spectrogram preprocessing:
|
||||||
|
{doc}`../how_to/data/configure-spectrogram-preprocessing`
|
||||||
|
- Check full train options:
|
||||||
|
{doc}`../reference/cli/train`
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user