mirror of
https://github.com/macaodha/batdetect2.git
synced 2026-05-22 22:32:18 +02:00
docs: expand python workflow tutorial
This commit is contained in:
parent
7a0f5c4b5a
commit
f909da5589
42
docs/source/reference/detections.md
Normal file
42
docs/source/reference/detections.md
Normal file
@ -0,0 +1,42 @@
|
|||||||
|
# Detections reference
|
||||||
|
|
||||||
|
These are the main prediction objects returned by BatDetect2 inference methods.
|
||||||
|
|
||||||
|
Defined in `batdetect2.postprocess.types`.
|
||||||
|
|
||||||
|
## `ClipDetections`
|
||||||
|
|
||||||
|
`ClipDetections` represents the predictions for one clip or one full recording.
|
||||||
|
|
||||||
|
Fields:
|
||||||
|
|
||||||
|
- `clip`
|
||||||
|
- the `soundevent` clip metadata for the processed audio.
|
||||||
|
- `detections`
|
||||||
|
- list of `Detection` objects for that clip.
|
||||||
|
|
||||||
|
## `Detection`
|
||||||
|
|
||||||
|
`Detection` represents one detected event.
|
||||||
|
|
||||||
|
Fields:
|
||||||
|
|
||||||
|
- `geometry`
|
||||||
|
- time-frequency geometry for the detected event.
|
||||||
|
- `detection_score`
|
||||||
|
- confidence that there is an event at this location.
|
||||||
|
- `class_scores`
|
||||||
|
- class ranking scores for the detected event.
|
||||||
|
- `features`
|
||||||
|
- per-detection feature vector from the model.
|
||||||
|
|
||||||
|
## Related pages
|
||||||
|
|
||||||
|
- Python tutorial:
|
||||||
|
{doc}`../tutorials/integrate-with-a-python-pipeline`
|
||||||
|
- API reference:
|
||||||
|
{doc}`api`
|
||||||
|
- What BatDetect2 predicts:
|
||||||
|
{doc}`../explanation/what-batdetect2-predicts`
|
||||||
|
- Features and embeddings:
|
||||||
|
{doc}`../explanation/extracted-features-and-embeddings`
|
||||||
@ -10,6 +10,7 @@ details, or Python API entries.
|
|||||||
|
|
||||||
cli/index
|
cli/index
|
||||||
api
|
api
|
||||||
|
detections
|
||||||
model-config
|
model-config
|
||||||
training-config
|
training-config
|
||||||
logging-config
|
logging-config
|
||||||
|
|||||||
@ -1,62 +1,50 @@
|
|||||||
# Tutorial: Integrate with a Python pipeline
|
# Integrate with a Python pipeline
|
||||||
|
|
||||||
This tutorial shows a minimal Python workflow for loading audio, running
|
This tutorial shows a simple Python workflow for loading audio, running BatDetect2, and inspecting the detections.
|
||||||
batdetect2, and collecting detections for downstream analysis.
|
|
||||||
|
|
||||||
This tutorial is for people who already want to work in Python.
|
Use it when you want to work directly in Python rather than through the CLI.
|
||||||
|
|
||||||
If you mainly want to run the model on recordings,
|
If you mainly want to run the model on recordings, start with {doc}`run-inference-on-folder` instead.
|
||||||
start with {doc}`run-inference-on-folder` instead.
|
|
||||||
|
|
||||||
## Before you start
|
## Before you start
|
||||||
|
|
||||||
- BatDetect2 installed in your Python environment.
|
You need:
|
||||||
- A model checkpoint.
|
|
||||||
- At least one input audio file.
|
|
||||||
|
|
||||||
```{note}
|
- BatDetect2 installed in your Python environment,
|
||||||
This page is more technical than the standard first-run tutorial.
|
- at least one input audio file.
|
||||||
You do not need this page for a normal first use of BatDetect2.
|
|
||||||
```
|
|
||||||
|
|
||||||
If you are working from this repository checkout, you can start with:
|
## What you will do
|
||||||
|
|
||||||
```text
|
|
||||||
src/batdetect2/models/checkpoints/Net2DFast_UK_same.pth.tar
|
|
||||||
```
|
|
||||||
|
|
||||||
## Outcome
|
|
||||||
|
|
||||||
By the end of this tutorial you will have:
|
By the end of this tutorial you will have:
|
||||||
|
|
||||||
- created a `BatDetect2API` object,
|
- created a `BatDetect2API` object,
|
||||||
- run inference on one file,
|
- run inference on one file,
|
||||||
- inspected the top class, class-score list, and detection score,
|
- inspected detections, scores, and features,
|
||||||
- identified where to go next for feature extraction, saving predictions, and batch workflows.
|
- used lower-level audio and spectrogram methods for more control,
|
||||||
|
- identified the next API workflows for batch processing, training, fine-tuning, and evaluation.
|
||||||
|
|
||||||
## 1. Create the API instance
|
## 1. Create the API instance
|
||||||
|
|
||||||
Load the checkpoint once and reuse the API object for multiple files.
|
For a first run, use the built-in default UK model:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from pathlib import Path
|
from batdetect2 import BatDetect2API
|
||||||
|
|
||||||
from batdetect2.api_v2 import BatDetect2API
|
# If you don't specify a checkpoint the default model will be loaded
|
||||||
|
api = BatDetect2API.from_checkpoint()
|
||||||
api = BatDetect2API.from_checkpoint(Path("path/to/model.ckpt"))
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If you want to use a different checkpoint later, see {doc}`../how_to/choose-a-model`.
|
||||||
|
|
||||||
## 2. Run inference on one file
|
## 2. Run inference on one file
|
||||||
|
|
||||||
`process_file` is the simplest Python entry point when you want one prediction object per recording.
|
`process_file` is the simplest Python entry point when you want one prediction object per recording.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from pathlib import Path
|
from batdetect2 import BatDetect2API
|
||||||
|
|
||||||
from batdetect2.api_v2 import BatDetect2API
|
api = BatDetect2API.from_checkpoint()
|
||||||
|
prediction = api.process_file("path/to/audio.wav")
|
||||||
api = BatDetect2API.from_checkpoint(Path("path/to/model.ckpt"))
|
|
||||||
prediction = api.process_file(Path("path/to/audio.wav"))
|
|
||||||
|
|
||||||
for detection in prediction.detections:
|
for detection in prediction.detections:
|
||||||
top_class = api.get_top_class_name(detection)
|
top_class = api.get_top_class_name(detection)
|
||||||
@ -64,21 +52,34 @@ for detection in prediction.detections:
|
|||||||
print(top_class, score)
|
print(top_class, score)
|
||||||
```
|
```
|
||||||
|
|
||||||
`prediction` is a `ClipDetections` object.
|
## 3. Understand the prediction objects
|
||||||
|
|
||||||
It contains:
|
`prediction` is a `ClipDetections` object.
|
||||||
|
See {doc}`../reference/detections` for the full reference.
|
||||||
|
|
||||||
|
Very briefly, `ClipDetections` represents all detections for one processed clip or recording.
|
||||||
|
It includes:
|
||||||
|
|
||||||
- the clip metadata,
|
- the clip metadata,
|
||||||
- a list of detections,
|
- the list of detections for that clip.
|
||||||
- a box for each detected event,
|
|
||||||
- one detection score per event,
|
|
||||||
- a full list of class scores per event,
|
|
||||||
- a feature vector per event.
|
|
||||||
|
|
||||||
## 3. Inspect class scores, not just the top class
|
Each item in `prediction.detections` is a `Detection` object.
|
||||||
|
|
||||||
If you are exploring results,
|
Each `Detection` includes:
|
||||||
it is often useful to inspect the full ranked class-score list.
|
|
||||||
|
- the time-frequency geometry of the event,
|
||||||
|
- a detection score,
|
||||||
|
- the class scores,
|
||||||
|
- a feature vector.
|
||||||
|
|
||||||
|
## 4. Inspect detection score and class scores
|
||||||
|
|
||||||
|
The detection score and the class scores answer different questions.
|
||||||
|
|
||||||
|
- `detection_score` is about whether the model thinks there is a call at that time-frequency location.
|
||||||
|
- `class_scores` are about which class the model prefers for that detected event.
|
||||||
|
|
||||||
|
So a detection can have a fairly strong detection score, but still have a more uncertain class ranking.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
for detection in prediction.detections:
|
for detection in prediction.detections:
|
||||||
@ -89,30 +90,71 @@ for detection in prediction.detections:
|
|||||||
print(f" {class_name}: {score:.3f}")
|
print(f" {class_name}: {score:.3f}")
|
||||||
```
|
```
|
||||||
|
|
||||||
This helps separate two different questions:
|
If you want more detail on class-score inspection, see {doc}`../how_to/inspect-class-scores-in-python`.
|
||||||
|
|
||||||
- "Did the model think there was a call here?"
|
## 5. Inspect the detection features
|
||||||
- "If there was a call, which class did it score highest?"
|
|
||||||
|
|
||||||
## 4. Keep the first workflow small
|
Each detection also carries a `features` vector.
|
||||||
|
|
||||||
Before scaling up, run the API on a few representative files and inspect the results manually.
|
These are internal model features attached to the detection.
|
||||||
|
They can be useful for things like:
|
||||||
|
|
||||||
This catches path issues and obviously implausible outputs early.
|
- exploratory visualisation,
|
||||||
|
- clustering similar detections,
|
||||||
|
- comparing detections across files,
|
||||||
|
- building downstream analysis pipelines.
|
||||||
|
|
||||||
## 5. Move to the right next workflow
|
They are useful descriptors, but they are not direct ecological labels by themselves.
|
||||||
|
|
||||||
Once the single-file path is working, choose the next page based on what you need:
|
For more detail, see {doc}`../how_to/inspect-detection-features-in-python` and {doc}`../explanation/extracted-features-and-embeddings`.
|
||||||
|
|
||||||
- save predictions to disk,
|
## 6. Use lower-level audio and spectrogram methods for more control
|
||||||
- inspect class scores more carefully,
|
|
||||||
- inspect detection features,
|
|
||||||
- process many files in one run.
|
|
||||||
|
|
||||||
## What to do next
|
If you want finer control over what gets processed and when, the API also lets you work step by step.
|
||||||
|
|
||||||
|
For example, you can load the audio yourself, inspect the waveform length, generate the spectrogram, and then run detection on that spectrogram:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from batdetect2 import BatDetect2API
|
||||||
|
|
||||||
|
api = BatDetect2API.from_checkpoint()
|
||||||
|
|
||||||
|
audio = api.load_audio("path/to/audio.wav")
|
||||||
|
print(audio.shape)
|
||||||
|
|
||||||
|
spec = api.generate_spectrogram(audio)
|
||||||
|
print(spec.shape)
|
||||||
|
|
||||||
|
detections = api.process_spectrogram(spec)
|
||||||
|
print(len(detections))
|
||||||
|
```
|
||||||
|
|
||||||
|
This is helpful when you want to:
|
||||||
|
|
||||||
|
- inspect the loaded audio before inference,
|
||||||
|
- inspect the generated spectrogram,
|
||||||
|
- control which audio segment is processed,
|
||||||
|
- run only part of the pipeline in custom code.
|
||||||
|
|
||||||
|
You can also call `process_audio(audio)` directly if you already have the waveform array in memory.
|
||||||
|
|
||||||
|
## 7. Use the wider API workflows
|
||||||
|
|
||||||
|
The Python API is not only for single-file inference.
|
||||||
|
It also exposes methods for batch processing, training, evaluation, and fine-tuning.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
- `process_files(...)` for batch processing from Python,
|
||||||
|
- `train(...)` for training,
|
||||||
|
- `evaluate(...)` for evaluation,
|
||||||
|
- `finetune(...)` for fine-tuning.
|
||||||
|
|
||||||
|
Useful next pages:
|
||||||
|
|
||||||
|
- Choose a different model: {doc}`../how_to/choose-a-model`
|
||||||
|
- Run batch predictions: {doc}`../how_to/run-batch-predictions`
|
||||||
|
- Train a custom model: {doc}`train-a-custom-model`
|
||||||
|
- Evaluate on a test set: {doc}`evaluate-on-a-test-set`
|
||||||
|
- Fine-tune from a checkpoint: {doc}`../how_to/fine-tune-from-a-checkpoint`
|
||||||
- API reference: {doc}`../reference/api`
|
- API reference: {doc}`../reference/api`
|
||||||
- Inspect ranked class scores: {doc}`../how_to/inspect-class-scores-in-python`
|
|
||||||
- Inspect detection features: {doc}`../how_to/inspect-detection-features-in-python`
|
|
||||||
- Save predictions to disk: {doc}`../how_to/save-predictions-in-different-output-formats`
|
|
||||||
- Learn the CLI happy path: {doc}`run-inference-on-folder`
|
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user