{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c539b3c0-19bf-47d7-9efb-9859d4b29686",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:43:39.142523Z",
     "iopub.status.busy": "2024-11-18T22:43:39.141003Z",
     "iopub.status.idle": "2024-11-18T22:43:39.191666Z",
     "shell.execute_reply": "2024-11-18T22:43:39.189282Z",
     "shell.execute_reply.started": "2024-11-18T22:43:39.142437Z"
    }
   },
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ef860e06-f18d-411c-9f3e-eb8422775fb6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:43:39.194627Z",
     "iopub.status.busy": "2024-11-18T22:43:39.193862Z",
     "iopub.status.idle": "2024-11-18T22:43:45.330723Z",
     "shell.execute_reply": "2024-11-18T22:43:45.329682Z",
     "shell.execute_reply.started": "2024-11-18T22:43:39.194560Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/santiago/Software/bat_detectors/batdetect2/.venv/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "from pathlib import Path\n",
    "from typing import List, Optional\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import xarray as xr\n",
    "from batdetect2.compat.data import load_annotation_project\n",
    "from batdetect2.train.preprocess import TrainPreprocessingConfig, preprocess_annotations\n",
    "from matplotlib.patches import Rectangle\n",
    "from soundevent import arrays, data"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "fc92a1fc-f7ca-480c-8aae-6a60c194c5a7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-04-24T16:17:15.039908Z",
     "iopub.status.busy": "2024-04-24T16:17:15.037455Z",
     "iopub.status.idle": "2024-04-24T16:17:15.095113Z",
     "shell.execute_reply": "2024-04-24T16:17:15.094292Z",
     "shell.execute_reply.started": "2024-04-24T16:17:15.039801Z"
    }
   },
   "source": [
    "# Audio Data Preprocessing for Bat Call Detection with batdetect2\n",
    "\n",
    "In this Notebook, we will guide you through the steps of preparing your audio data for training a bat call detection model using `batdetect2`. This process involves transforming raw audio recordings into a structured dataset that a machine learning model can learn from. \n",
    "\n",
    "Here's a breakdown of the key sections we'll cover:\n",
    "\n",
    "1. **Loading Annotations:**\n",
    "   - Importing existing bat call annotations, either in the standard `soundevent` format or from older batdetect2 versions.\n",
    "\n",
    "2. **Annotation Mapper:**\n",
    "   - Creating a custom `Mapper` class to assign meaningful labels (e.g., species or \"social call\") to each detected bat call.\n",
    "\n",
    "3. **Configuring Preprocessing:**\n",
    "   - Understanding the `PreprocessingConfig` object and its parameters for customizing how the audio data is transformed.\n",
    "   - Saving the configuration for reproducibility.\n",
    "\n",
    "4. **Preprocessing Audio Data:**\n",
    "   - Applying the preprocessing pipeline to all annotated audio clips.\n",
    "\n",
    "5. **Understanding the Outputs:**\n",
    "   - Exploring the structure of the output NetCDF files, which contain spectrograms, detection arrays, class arrays, size arrays, and metadata attributes.\n",
    "   - Visualizing the spectrograms, detections, class probabilities, and sizes of the bat calls."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c664c34f-bc0c-453a-b3e0-3c0554d863af",
   "metadata": {},
   "source": [
    "## Loading Annotations\n",
    "\n",
    "In this section, we'll focus on loading annotations that identify the presence of bat calls within your audio recordings. These annotations are crucial for training your machine learning model, as they provide the ground truth labels for the bat calls.\n",
    "\n",
    "### Understanding Annotation Formats\n",
    "\n",
    "* **Soundevent `ClipAnnotation` Objects:** The preferred way to work with annotations is using `ClipAnnotation` objects from the `soundevent` package. These objects can be directly exported from tools like `whombat` and provide a structured way to represent sound events in audio clips.\n",
    "\n",
    "* **Legacy batdetect2 Format:** If you have annotations from older versions of `batdetect2`, we've got you covered! The `batdetect2.data.compat` module provides tools to load and convert annotations from the previous batdetect2 format into the modern `ClipAnnotation` format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cdd3f5aa-d083-4c08-b70a-8ced4850586b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:43:45.332273Z",
     "iopub.status.busy": "2024-11-18T22:43:45.331605Z",
     "iopub.status.idle": "2024-11-18T22:43:45.380786Z",
     "shell.execute_reply": "2024-11-18T22:43:45.379638Z",
     "shell.execute_reply.started": "2024-11-18T22:43:45.332243Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded 3 annotated clips from /home/santiago/Software/bat_detectors/batdetect2/example_data\n"
     ]
    }
   ],
   "source": [
    "# Data Directory Setup\n",
    "data_dir = Path.cwd().parent / \"example_data\"  # Path to the example data directory\n",
    "\n",
    "proj = load_annotation_project(\n",
    "    data_dir / \"anns\",  # Specify the folder containing annotation files (json files)\n",
    "    audio_dir=data_dir / \"audio\",  # Specify the folder containing audio recordings\n",
    ")\n",
    "\n",
    "# Verify the number of loaded clips and the data directory\n",
    "print(f\"Loaded {len(proj.clip_annotations)} annotated clips from {data_dir}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0094d6c-31a2-4f2e-850c-1b71a3d8929a",
   "metadata": {},
   "source": [
    "## Annotation Mapper: Assigning Classes to Bat Calls\n",
    "\n",
    "In this section, we'll create a Mapper object that translates between bat call annotations (which can be complex) and simple class labels that your machine learning model can understand. This mapper defines which bat calls are relevant, how to assign them a class (like species), and how to decode model predictions back into the original annotation format."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1e13c56d-75ef-445a-95fe-53f825c5a606",
   "metadata": {},
   "source": [
    "## Configuring Preprocessing\n",
    "\n",
    "To fine-tune how your audio data is prepared for the model, you can create a PreprocessingConfig object. This object controls various aspects of the preprocessing pipeline, including the sample rate, frequency range, spectrogram settings, and more."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a86bcb98-b135-489e-b6da-678dd15c2846",
   "metadata": {},
   "source": [
    "### Using Default Configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "347c7137-b06b-4725-b4b8-3087ea8068f0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:43:45.382790Z",
     "iopub.status.busy": "2024-11-18T22:43:45.382222Z",
     "iopub.status.idle": "2024-11-18T22:43:45.433072Z",
     "shell.execute_reply": "2024-11-18T22:43:45.431617Z",
     "shell.execute_reply.started": "2024-11-18T22:43:45.382740Z"
    }
   },
   "outputs": [],
   "source": [
    "# Create a configuration object with default values\n",
    "config = TrainPreprocessingConfig()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a117d303-707b-4854-a9fe-75781e9577f9",
   "metadata": {},
   "source": [
    "### Customizing the Configuration\n",
    "\n",
    "If you have specific requirements or want to experiment with different settings, you can modify the configuration. Below is a list of the available fields and their default values for reference. You can change them as needed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "d11ed67a-c796-412f-8931-f94eeb737868",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:44:21.911286Z",
     "iopub.status.busy": "2024-11-18T22:44:21.910523Z",
     "iopub.status.idle": "2024-11-18T22:44:21.995499Z",
     "shell.execute_reply": "2024-11-18T22:44:21.993127Z",
     "shell.execute_reply.started": "2024-11-18T22:44:21.911218Z"
    }
   },
   "outputs": [],
   "source": [
    "from batdetect2.preprocess import (\n",
    "    AudioConfig,\n",
    "    FFTConfig,\n",
    "    FrequencyConfig,\n",
    "    PcenScaleConfig,\n",
    "    PreprocessingConfig,\n",
    "    ResampleConfig,\n",
    "    SpecSizeConfig,\n",
    "    SpectrogramConfig,\n",
    ")\n",
    "from batdetect2.terms import TagInfo, TermInfo\n",
    "from batdetect2.train.targets import TargetConfig, build_class_mapper\n",
    "\n",
    "preprocess_config = PreprocessingConfig(\n",
    "    audio=AudioConfig(\n",
    "        resample=ResampleConfig(\n",
    "            samplerate=256_000,  # Target sample rate for resampling audio\n",
    "            mode=\"poly\",\n",
    "        ),\n",
    "        center=True,  # Whether to remove the mean from the raw audio.\n",
    "        scale=False,  # Whether to scale the raw audio values to -1,-1 range\n",
    "        duration=None,  # Duration (in seconds) of each training example\n",
    "    ),\n",
    "    spectrogram=SpectrogramConfig(\n",
    "        fft=FFTConfig(\n",
    "            window_duration=512 / 256_000,  # Window length for FFT in seconds\n",
    "            window_overlap=0.75,  # Overlap between FFT windows\n",
    "            window_fn=\"hann\",\n",
    "        ),\n",
    "        frequencies=FrequencyConfig(\n",
    "            max_freq=120_000,  # Maximum frequency to include in spectrogram\n",
    "            min_freq=10_000,  # Minimum frequency to include in spectrogram\n",
    "        ),\n",
    "        scale=PcenScaleConfig(),  # Spectrogram scaling method (\"pcen\", \"db\", or None)\n",
    "        denoise=True,  # Whether to apply spectral denoising\n",
    "        max_scale=False,  # Whether to max-scale the spectrogram\n",
    "        size=SpecSizeConfig(\n",
    "            height=256,  # Height of the spectrogram (number of frequency bins)\n",
    "            resize_factor=0.5,  # Time resolution of the spectrogram (seconds per pixel)\n",
    "        ),\n",
    "    ),\n",
    ")\n",
    "\n",
    "targets = TargetConfig(\n",
    "    classes=[\n",
    "        TagInfo(\n",
    "            key=\"class\",\n",
    "            label=\"Eptesicus serotinus\",\n",
    "            value=\"Eptesicus serotinus\",\n",
    "        ),\n",
    "        TagInfo(\n",
    "            key=\"class\",\n",
    "            value=\"Myotis mystacinus\",\n",
    "            label=\"Myotis mystacinus\",\n",
    "        ),\n",
    "        TagInfo(\n",
    "            key=\"class\",\n",
    "            value=\"Pipistrellus pipistrellus\",\n",
    "            label=\"Pipistrellus pipistrellus\",\n",
    "        ),\n",
    "        TagInfo(\n",
    "            key=\"class\",\n",
    "            value=\"Rhinolophus ferrumequinum\",\n",
    "            label=\"Rhinolophus ferrumequinum\",\n",
    "        ),\n",
    "    ]\n",
    ")\n",
    "\n",
    "\n",
    "config = TrainPreprocessingConfig(\n",
    "    preprocessing=preprocess_config,\n",
    "    target=targets,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df5df21e-beeb-47d6-b151-2cbb44c8a0ad",
   "metadata": {},
   "source": [
    "## Preprocessing\n",
    "\n",
    "With our annotation mapper and configuration defined, we're ready to preprocess the audio data to prepare it for training our machine learning model. This crucial step involves:\n",
    "\n",
    "* Converting annotations to a model-friendly format: We'll transform the raw annotations (using our custom Mapper) into a format that our model can directly use for learning.\n",
    "* Saving preprocessed data: We'll store the prepared data in a structured way for easy access during the training process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "84de78ca-10b6-464f-9799-15f18afb1f97",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:44:22.605005Z",
     "iopub.status.busy": "2024-11-18T22:44:22.604317Z",
     "iopub.status.idle": "2024-11-18T22:44:23.397984Z",
     "shell.execute_reply": "2024-11-18T22:44:23.396877Z",
     "shell.execute_reply.started": "2024-11-18T22:44:22.604971Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00,  5.00it/s]\n"
     ]
    }
   ],
   "source": [
    "preprocess_annotations(\n",
    "    proj.clip_annotations,  # Provide a list of clip annotations\n",
    "    data_dir / \"preprocessed\",  # Output directory\n",
    "    config=config,  # Preprocessing configuration (either default or custom)\n",
    "    replace=True,  # Replace any existing preprocessed data\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2881ed78-162f-4138-91c8-7404c8863d1e",
   "metadata": {},
   "source": [
    "## Understanding the Outputs\n",
    "\n",
    "After preprocessing, each annotated audio clip is transformed into a self-contained NetCDF file. These NetCDF files store multiple arrays of data (like spectrograms and labels) along with metadata about their coordinates (time and frequency). This combination offers several advantages for training your machine learning model:\n",
    "\n",
    "* **Structured Data with Coordinates**: Instead of dealing with raw multi-dimensional arrays, NetCDF provides a clear structure with time and frequency information attached to each data point. This simplifies the process of extracting specific regions or slices of data for analysis.\n",
    "* **Efficient Storage and Access**: NetCDF files are optimized for storing array data, resulting in smaller file sizes compared to raw formats. They also support fast random access, allowing you to quickly read only the portions of data you need during training, which is crucial for handling large datasets efficiently.\n",
    "* **Bundled Training Artifacts**: All the essential components for training your model on a single audio clip are conveniently packaged together in one NetCDF file. This includes the spectrogram (a visual representation of the audio's frequencies), the corresponding labels for each bat call, and the shared coordinate information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c63b7694-3f6c-409d-8c72-5df8fd001025",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:44:23.399970Z",
     "iopub.status.busy": "2024-11-18T22:44:23.399617Z",
     "iopub.status.idle": "2024-11-18T22:44:23.456867Z",
     "shell.execute_reply": "2024-11-18T22:44:23.455876Z",
     "shell.execute_reply.started": "2024-11-18T22:44:23.399936Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 3 processed NetCDF files.\n"
     ]
    }
   ],
   "source": [
    "# Gather All Processed NetCDF Files\n",
    "output_files = list((data_dir / \"preprocessed\").glob(\"*.nc\"))\n",
    "print(f\"Found {len(output_files)} processed NetCDF files.\")\n",
    "\n",
    "# Load an Example NetCDF File\n",
    "example = xr.open_dataset(output_files[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00c06ec6-6df8-4491-9c02-e2b1b3481e51",
   "metadata": {},
   "source": [
    "Now that we have a NetCDF file loaded as an `xarray.Dataset`, let's explore its contents to see how the preprocessed data is organized. This will give us insight into the information available for training our machine learning model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "2344eb37-fc72-4143-ad9c-b01c56754768",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:44:24.347231Z",
     "iopub.status.busy": "2024-11-18T22:44:24.346393Z",
     "iopub.status.idle": "2024-11-18T22:44:24.428203Z",
     "shell.execute_reply": "2024-11-18T22:44:24.425454Z",
     "shell.execute_reply.started": "2024-11-18T22:44:24.347161Z"
    }
   },
   "outputs": [],
   "source": [
    "# The example data is an xr.Dataset with four data variables and additional helpful attributes\n",
    "\n",
    "# Data variables\n",
    "spectrogram = example[\"spectrogram\"]  # The preprocessed spectrogram data\n",
    "detection_array = example[\"detection\"]  # Array indicating if a detection was made\n",
    "class_array = example[\"class\"]  # Array indicating the class of the detection\n",
    "size_array = example[\"size\"]  # Array with the size of each detection\n",
    "\n",
    "# Metadata Attributes\n",
    "config_attr = example.attrs[\"config\"]  # train preprocessing config\n",
    "clip_annotation_attr = example.attrs[\"clip_annotation\"]  # Original annotation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "c115b239-4605-4572-8ab9-7e6e3c26ff90",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:44:24.923915Z",
     "iopub.status.busy": "2024-11-18T22:44:24.921933Z",
     "iopub.status.idle": "2024-11-18T22:44:24.999326Z",
     "shell.execute_reply": "2024-11-18T22:44:24.997152Z",
     "shell.execute_reply.started": "2024-11-18T22:44:24.923834Z"
    }
   },
   "outputs": [],
   "source": [
    "### Spectrogram"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "6572902e-bd1f-4a4d-b5e4-0d84664c3c9d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:44:25.674041Z",
     "iopub.status.busy": "2024-11-18T22:44:25.671879Z",
     "iopub.status.idle": "2024-11-18T22:44:26.193576Z",
     "shell.execute_reply": "2024-11-18T22:44:26.192794Z",
     "shell.execute_reply.started": "2024-11-18T22:44:25.673960Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Spectrogram Dimensions: ('frequency', 'time')\n",
      "Spectrogram Shape: (128, 512)\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 127220 characters]",
      "text/plain": [
       "<Figure size 1500x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Display Spectrogram Dimensions and Shape\n",
    "print(f\"Spectrogram Dimensions: {spectrogram.dims}\")\n",
    "print(f\"Spectrogram Shape: {spectrogram.shape}\")\n",
    "\n",
    "# Plot the Spectrogram\n",
    "spectrogram.plot(figsize=(15, 4), cmap=\"Greys\");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4e334d5-97bc-4707-9441-be9bd87746c0",
   "metadata": {},
   "source": [
    "### Detections Array\n",
    "\n",
    "Each bat call identified in the audio is represented in the `detection_array`, which has the same dimensions as the spectrogram.  This array helps us visually pinpoint the start and frequency range of each detected bat call:\n",
    "\n",
    "* Gaussian Point Marking: A small Gaussian point (a concentrated dot) is placed at the lower-left corner of each bat call's location on the spectrogram.\n",
    "* Start Time and Frequency: The Gaussian point marks the exact time the bat call begins and the call's lowest frequency.\n",
    "\n",
    "Let's visualize the locations of these bat calls on the spectrogram to better understand how the the call location is represented."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "6c658b7c-c7de-49cc-b2bf-ecb15c37449c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:44:27.866159Z",
     "iopub.status.busy": "2024-11-18T22:44:27.865327Z",
     "iopub.status.idle": "2024-11-18T22:44:28.542364Z",
     "shell.execute_reply": "2024-11-18T22:44:28.541315Z",
     "shell.execute_reply.started": "2024-11-18T22:44:27.866087Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Detection Array Dimensions: ('frequency', 'time')\n",
      "Detection Array Shape: (128, 512)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.QuadMesh at 0x71d269cf8380>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 134540 characters]",
      "text/plain": [
       "<Figure size 1500x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Display Detection Array Dimensions and Shape\n",
    "print(f\"Detection Array Dimensions: {detection_array.dims}\")\n",
    "print(f\"Detection Array Shape: {detection_array.shape}\")\n",
    "\n",
    "# Plot the grayscale spectrogram on the axes\n",
    "_, ax = plt.subplots(figsize=(15, 4))\n",
    "spectrogram.plot(ax=ax, cmap=\"Greys\")\n",
    "\n",
    "# Overlay the detections on top of the spectrogram\n",
    "detection_array.where(detection_array > 0.1).plot(\n",
    "    ax=ax,  # Plot on the same axes as the spectrogram\n",
    "    alpha=0.5,  # Set transparency of the detection overlays\n",
    "    vmin=0,  # Minimum value for the colormap\n",
    "    vmax=1,  # Maximum value for the colormap\n",
    "    add_colorbar=False,  # Don't add a separate colorbar for detections\n",
    "    cmap=\"Reds\",  # Use a red colormap for the detections\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "004f30bf-0771-4420-bb03-dab817e590d8",
   "metadata": {},
   "source": [
    "### Predicted Class Probabilities\n",
    "\n",
    "The `class_array` also has the same dimensions as the spectrogram, but with an additional dimension to represent the probability of each location belonging to each bat call class (species or social call). This allows us to visualize where the model is most confident in its predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "7b6fd8ec-de4f-41c0-a6e0-564d144fcbcc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:44:38.061435Z",
     "iopub.status.busy": "2024-11-18T22:44:38.060644Z",
     "iopub.status.idle": "2024-11-18T22:44:39.060157Z",
     "shell.execute_reply": "2024-11-18T22:44:39.059343Z",
     "shell.execute_reply.started": "2024-11-18T22:44:38.061379Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Class Array Dimensions: ('category', 'frequency', 'time')\n",
      "Class Array Shape: (4, 128, 512)\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 134604 characters]",
      "text/plain": [
       "<Figure size 1500x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Display Class Array Dimensions and Shape\n",
    "print(f\"Class Array Dimensions: {class_array.dims}\")\n",
    "print(f\"Class Array Shape: {class_array.shape}\")\n",
    "\n",
    "# Plot the grayscale spectrogram as the base layer\n",
    "_, ax = plt.subplots(figsize=(15, 4))\n",
    "spectrogram.plot(ax=ax, cmap=\"Greys\")\n",
    "\n",
    "# Define colormaps for each class\n",
    "cmaps = {\n",
    "    \"Eptesicus serotinus\": \"Reds\",\n",
    "    \"Myotis mystacinus\": \"Blues\",\n",
    "    \"Pipistrellus pipistrellus\": \"Greens\",\n",
    "    \"Rhinolophus ferrumequinum\": \"Purples\",\n",
    "    \"social\": \"Greys\",\n",
    "    \"Bat\": \"Oranges\",\n",
    "}\n",
    "\n",
    "# Iterate through each class label\n",
    "for class_label in class_array.category.values:\n",
    "\n",
    "    # Select the 2D probability array for the current class\n",
    "    array = class_array.sel(category=class_label)\n",
    "\n",
    "    # Overlay the class probabilities on the spectrogram, highlighting areas above a threshold\n",
    "    array.where(array > 0.1).plot(\n",
    "        vmin=0,  # Minimum probability for the colormap\n",
    "        vmax=1,  # Maximum probability for the colormap\n",
    "        alpha=0.5,  # Set transparency to see the spectrogram underneath\n",
    "        cmap=cmaps[class_label],  # Use the appropriate colormap for the class\n",
    "        add_colorbar=False,  # Don't add separate colorbars for each class\n",
    "        add_labels=False,  # Don't add axis labels (they're already on the plot)\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2050e8a1-dbef-458e-a5bc-8d7256f6137e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-15T23:57:59.752201Z",
     "iopub.status.busy": "2024-07-15T23:57:59.751403Z",
     "iopub.status.idle": "2024-07-15T23:57:59.828876Z",
     "shell.execute_reply": "2024-07-15T23:57:59.827115Z",
     "shell.execute_reply.started": "2024-07-15T23:57:59.752132Z"
    }
   },
   "source": [
    "### Bat Call Sizes\n",
    "\n",
    "The `size_array` provides information about the size of each detected bat call in terms of its duration (length in time) and bandwidth (range of frequencies).\n",
    "\n",
    "**Note**: The size values in the `size_array` are stored relative to the resolution of the spectrogram. This means they represent proportions of the spectrogram's time and frequency dimensions rather than absolute values in seconds and Hertz. This relative scaling is beneficial for machine learning, as it keeps the values within a manageable range for the model to learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "0e5f8311-8ffb-4c29-af47-dd3349eab147",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:45:49.616702Z",
     "iopub.status.busy": "2024-11-18T22:45:49.615898Z",
     "iopub.status.idle": "2024-11-18T22:45:50.383648Z",
     "shell.execute_reply": "2024-11-18T22:45:50.383000Z",
     "shell.execute_reply.started": "2024-11-18T22:45:49.616634Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size Array Dimensions: ('dimension', 'frequency', 'time')\n",
      "Size Array Shape: (2, 128, 512)\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 132556 characters]",
      "text/plain": [
       "<Figure size 1500x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Display Size Array Dimensions and Shape\n",
    "print(f\"Size Array Dimensions: {size_array.dims}\")\n",
    "print(f\"Size Array Shape: {size_array.shape}\")\n",
    "\n",
    "# Plot the grayscale spectrogram as the base layer\n",
    "_, ax = plt.subplots(figsize=(15, 4))\n",
    "spectrogram.plot(ax=ax, cmap=\"Greys\")\n",
    "\n",
    "# Find non-zero size locations\n",
    "indices = np.argwhere((size_array != 0).any(dim=\"dimension\").values)\n",
    "\n",
    "for freq_idx, time_idx in indices:\n",
    "    # Get size at this location\n",
    "    point = size_array.isel(time=time_idx, frequency=freq_idx)\n",
    "\n",
    "    # Calculate width in seconds\n",
    "    width = point.sel(dimension=\"width\") / config.heatmaps.time_scale\n",
    "\n",
    "    # Calculate height in Hz\n",
    "    height = point.sel(dimension=\"height\") / config.heatmaps.frequency_scale\n",
    "\n",
    "    # Draw rectangle\n",
    "    ax.add_artist(\n",
    "        Rectangle((point.time, point.frequency), width, height, ec=\"red\", fc=\"none\")\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac88601d-f743-4110-94de-7dc6836c69a2",
   "metadata": {},
   "source": [
    "### Metadata Attributes\n",
    "\n",
    "In addition to the data arrays, each NetCDF file contains important metadata attributes that provide context and traceability. These attributes include the preprocessing configuration and the original clip annotation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "64c5b7b9-abf6-4b29-baee-711237cdb527",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-18T22:44:41.482135Z",
     "iopub.status.busy": "2024-11-18T22:44:41.481206Z",
     "iopub.status.idle": "2024-11-18T22:44:41.573920Z",
     "shell.execute_reply": "2024-11-18T22:44:41.571905Z",
     "shell.execute_reply.started": "2024-11-18T22:44:41.482065Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'soundevent.data.clip_annotations.ClipAnnotation'>\n"
     ]
    }
   ],
   "source": [
    "# Get config as JSON string\n",
    "config_attr = example.attrs[\"config\"]\n",
    "\n",
    "# Load and validate configuration from JSON string\n",
    "loaded_config = TrainPreprocessingConfig.model_validate_json(config_attr)\n",
    "\n",
    "# Ensure the loaded config matches the one used\n",
    "assert config == loaded_config\n",
    "\n",
    "# Get annotation as JSON string\n",
    "clip_annotation_json = example.attrs[\"clip_annotation\"]\n",
    "\n",
    "# Load and validate clip annotation from JSON string\n",
    "clip_annotation = data.ClipAnnotation.model_validate_json(clip_annotation_json)\n",
    "\n",
    "# Display the annotation type\n",
    "print(type(clip_annotation))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4bc0625a-2441-488a-8a6c-15709547a09e",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "batdetect2-dev",
   "language": "python",
   "name": "batdetect2-dev"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
