betaFeedback

WorksComputer Science

A Paired RGB-Sonar Dataset for Fish Monitoring with Consumer Sonar

Abstract

Imaging sonar enables underwater observation in conditions where optical imaging is ineffective due to darkness and turbid water. However, imaging-sonar systems previously used for ecological monitoring have typically been expensive, limiting their accessibility. Consumer forward-looking sonar has the potential to make fish monitoring more accessible, but its noisy imagery makes annotation and the development of automated methods challenging. We introduce the Huntington and Caltech Koi RGB and Sonar dataset (HaCKRS), which pairs Garmin LiveScope imaging-sonar recordings with top-down RGB video of koi carp. The dataset contains more than 105,000 temporally matched frame pairs, spanning approximately 3.3 hours of recordings across four range and orientation configurations. We provide frame-wise metadata, sonar geometry and coordinate information, synchronization data for all four configurations, and planar registration for the three horizontal configurations. The release focuses on paired observations and calibration rather than task-specific fish annotations. We intend HaCKRS to support annotation and the development of fish detection, tracking, and counting methods, and investigations into the potential of consumer sonar for ecological monitoring.

Introduction

Monitoring underwater environments is important for ecological science and fisheries management. Video has clear utility for tasks such as fish detection, tracking, counting, and behavioral monitoring.

However, optical video is limited in many aquatic settings. Its performance degrades in low-light or turbid conditions, restricting its use in natural settings.

Active sonar observes its environment by emitting acoustic signals and measuring their reflections off objects in the water, allowing it to operate in darkness, turbid water, poor weather, and at depth. Imaging sonar measurements also contain metric range information and can, in principle, be used to estimate properties such as fish length and tail beat frequency.

These properties have motivated substantial work on fish detection, tracking, counting, and measurement using imaging sonar [9, 18, 11]. However, much of this work has focused on DIDSON and ARIS sonars, specialized and expensive scientific instruments. These systems are often effective, but their cost significantly limits their accessibility and the scale at which they can be deployed. Similar methods that could utilize lower-cost consumer sonar designed for recreational fishing could increase the scale at which environmental monitoring could be undertaken.

Developing computer-vision methods for these systems requires suitable training and evaluation data, yet sonar imagery can be difficult to interpret and annotate. Fish returns may have poorly defined boundaries, overlap with environmental structure or other fish, and vary substantially with range, orientation, and scene geometry.

Synchronized optical imagery can provide complementary information when sonar returns are difficult to interpret. A fish that is difficult to identify confidently in a sonar frame may be considerably easier to locate in a synchronized RGB observation. Paired optical imagery can therefore assist annotation, model development, and diagnostic evaluation without requiring the optical sensor to remain available when a sonar-based method is ultimately deployed.

We recognize that planar transformations between RGB and sonar observations, as well as differences between the capture modalities, mean that perfect pixelwise correspondence cannot be expected. However, the complementary information provided by the RGB imagery can still be valuable for guiding annotation and model development.

We introduce the Huntington and Caltech Koi RGB and Sonar (HaCKRS) dataset, consisting of paired imaging-sonar and top-down RGB recordings of koi carp at the Chinese Garden of The Huntington Library, Art Museum, and Botanical Gardens. Sonar data was recorded using a Garmin Panoptix LiveScope LVS32 and GLS 10 system, while the RGB imagery was captured using a smartphone. The released paired imagery spans approximately 3.3 hours in four configurations: three horizontal sonar configurations at nominal ranges of 15, 40, and 100 feet, and one vertical configuration at 20 feet. An example paired sonar and RGB observation is shown in Figure 1.

Paired sonar and RGB frames of koi carp at the Huntington Gardens

Figure 1. HaCKRS contains paired sonar and RGB frames of koi carp at the Huntington Gardens. Colored lines correspond to matched boundaries between modalities. The white circle denotes the sonar location. Crosshatching denotes areas that were not captured by the other modality.

We temporally match the sonar and RGB frames for all four runs and estimate planar homographies between the two imaging modalities for the three horizontal configurations. We also provide supporting information including frame-wise metadata, sonar coordinate transforms, and synchronization and calibration data; the dataset contents and statistics are summarized in section 3.

HaCKRS is intended to support research on fish detection, tracking, and counting using accessible consumer imaging sonar. Specifically, HaCKRS provides:

  • more than 105,000 temporally matched Garmin LiveScope sonar and top-down RGB frame pairs spanning approximately 3.3 hours

  • four acquisition configurations spanning different sonar ranges and imaging-plane orientations

  • frame-level timestamps, pairing information, and modality-specific acquisition metadata

  • sonar geometry and coordinate information sufficient to reconstruct polar-to-Cartesian representations

  • synchronization and calibration observations, including estimated planar RGB-sonar homographies for the three horizontal configurations.

The current release does not provide task-specific fish detections, identities, tracks, or count annotations. Instead, it provides paired observations and geometric information from which such annotations and cross-modal methods can be developed. In addition to providing the dataset itself, we document the practical synchronization, calibration, and acquisition issues encountered during collection; practical lessons from these are summarized in section 7. We hope that these details make the resource easier to use and provide useful guidance for future multimodal sonar data collections.

Related work

Imaging sonar for fish monitoring

Imaging sonar has been widely used for observing fish in environments where optical cameras are limited by turbidity, low light, or occlusion. Systems such as DIDSON and ARIS provide high-resolution acoustic imagery and have been applied to fish detection, counting, tracking, and behavioral monitoring [9, 18, 11]. These systems have also motivated work on automated analysis of sonar imagery, including methods for detecting and counting individual fish [2, 5, 13]. The Caltech Fish Counting Dataset (CFC22) provides a large-scale benchmark for fish detection, tracking, and counting in imaging-sonar video [7], while CFC26 extends this work toward evaluations relevant to deployment [17]. SAVeD further demonstrates how self-supervised denoising of low-SNR video can improve downstream detection, tracking, and counting performance on sonar imagery [15]. However, scientific imaging-sonar systems can require specialized hardware and deployment procedures, motivating interest in more accessible sonar platforms.

More recently, consumer forward-looking sonar systems have begun to be explored for ecological monitoring. Dickson et al. [4] adapted a Garmin Panoptix LiveScope system for ecological monitoring in the Dutch Wadden Sea. They mounted the system on a customized floating platform and used it to measure pelagic fish abundance around experimental tree-reef and paired control sites. Their study demonstrates the potential of consumer fish-finding sonar for non-invasive ecological monitoring in shallow, turbid environments where conventional optical surveys can be difficult.

This growing use of consumer imaging sonar motivates the development of datasets and analysis tools tailored to these systems. Whereas prior ecological work has demonstrated that LiveScope can provide useful measurements of fish abundance [4], our work focuses on the data and methodological infrastructure needed to support automated interpretation of its imagery. Specifically, HaCKRS provides temporally paired LiveScope and RGB observations, together with sonar geometry, synchronization information, and approximate spatial registration, enabling the study of fish observations across the two sensing modalities.

Joint optical and acoustic observation of fish. Optical imagery has been combined with imaging sonar to validate acoustic observations and provide complementary biological information. In the Ocqueoc River dataset, McCann et al. used synchronized underwater video to identify fish observed by DIDSON and produce high-confidence labeled clips [10]. Simultaneous camera and sonar recordings have also been used for cross-modal image generation in aquaculture and for direct comparisons of fish detectability and density estimates between acoustic and optical systems [16, 14]. More recently, the HODOR system combines forward-looking imaging sonar with low-light stereo cameras for continuous ecological observation and provides hundreds of hours of synchronized sonar and optical recordings [6, 19]. Collectively, these studies demonstrate the value of complementary acoustic and optical observations. They also motivate paired datasets in which the more readily interpretable optical modality can assist the analysis of sonar imagery.

Paired sonar–visual datasets and accessible sensing. Paired sonar and visual data has also been developed for broader underwater perception tasks. RGBS50 contains temporally aligned RGB-sonar sequences for single-object tracking [8]. Dahn et al. provide more than 74,000 matched ARIS and GoPro observations of underwater unexploded ordnance together with calibration and pose information [3]. Other AUV and ROV datasets combine forward-looking sonar with camera imagery and navigation measurements. SOLAQUA provides camera, sonar, and navigation data collected at an operational fish farm [12], while SOVIS provides more than 76,000 paired sonar–visual frames from open-water ROV deployments and demonstrates cross-modal fish detection [1]. These datasets establish temporal pairing and multimodal acoustic–visual sensing in scientific and robotic settings.

In parallel, consumer forward-facing sonar systems such as Garmin LiveScope have begun to be evaluated for ecological monitoring, suggesting that sensors originally developed for recreational fishing may also be useful for scientific observation [4]. HaCKRS occupies a complementary setting: it pairs observations of fish from a consumer LiveScope system with a top-down consumer RGB camera. All four acquisition configurations provide temporally matched RGB-sonar observations and explicit sonar geometry, while the three horizontal configurations additionally provide planar RGB-sonar mappings. This combination is intended to support the development and evaluation of automated fish-monitoring methods specifically for accessible consumer imaging sonar.

A structured comparison of HaCKRS with these related datasets and studies is provided in Table 5 in the Appendix.

ResourceFish-focusedPaired opticsPublic dataSpatial calib.Sonar classCross-modal ML intent
McCann 2018 [10]●●●◐Research◐
Terayama 2019 [16]●●○/◐◐Research●
RGBS50 [8]○/◐●●◐Research●
DFKI UXO [3]○●●●Research●
SOLAQUA [12]●/◐●●◐Research◐
HODOR [19, 6]●●●◐Research●
SOVIS [1]●/◐●●◐Research●
LiveScope ecological study [4]●○○/◐○Recreational○
HaCKRS●●●● 3/4Recreational●

Table 5. Comparison of relevant multimodal and imaging-sonar datasets and studies. ● indicates that the feature is a central, explicitly documented part of the resource; ◐ indicates that it is present but secondary, partial, or differently configured; and ○ indicates that it is absent or not central. “Paired optics” denotes synchronized optical imagery. “Cross-modal ML intent” indicates that the resource is explicitly framed for fusion, cross-modal learning, translation, or using one modality to inform another, rather than merely deploying two sensors simultaneously.

Dataset Overview

HaCKRS contains paired sonar and top-down RGB observations of koi carp recorded at a single pond in the Chinese Garden of The Huntington Library, Art Museum, and Botanical Gardens. The sonar and camera remained in fixed acquisition configurations during each run while fish moved freely through the observed region.

The dataset contains four configurations, denoted 15H, 40H, 100H, and 20V. In 15H, 40H, and 100H, the sonar imaging plane was oriented approximately horizontally, producing a plan-like view that can be related to the top-down RGB camera. These configurations used nominal sonar ranges of 15 feet (∼4.5\sim4.5m), 40 feet (∼12\sim12m), and 100 feet (∼30\sim30m), respectively. In 20V, the sonar was rotated 90 degrees about its centerline and placed near the pond floor, producing a vertical profile through the water column at a nominal range of 20 feet (∼6\sim6m). The horizontal and vertical sonar orientations are illustrated in Figure 2.

Horizontal and vertical sonar viewpoints, showing fish, bridge, water surface, and pond floor

Figure 2. Horizontal and vertical viewpoints. The horizontal sonar approximates a top down view. The vertical run takes a thin profile slice through the water column. In the vertical setting, the sensor is near the pond floor. Significant reflections can be seen from both the water surface and the hard pond floor.

The selected release intervals exclude frames containing people, the calibration object, and the manual synchronization actions.

Dataset contents

The release is intended to preserve both the observations themselves and the information required to interpret their temporal and spatial relationship. RGB frames are decoded from MOV files, while sonar frames are extracted as timestamped PNGs whose identifiers may contain gaps.

For each paired observation, we provide the corresponding sonar and RGB frame identifiers and timestamps together with their temporal difference. Sonar metadata includes the recorded frame identifier, wall-clock timestamp, source clip, polar image dimensions, range information, and supplied angular coordinates. RGB metadata includes the corresponding frame identifier, timestamps, source capture, image dimensions, and available camera metadata. The framewise metadata fields are detailed in section D.

For the sonar modality, we retain the geometry needed to interpret the recorded polar representation and construct Cartesian representations. The Cartesian transformation used in this work is described in § 4.4, but the release retains the underlying coordinate information so that users are not restricted to our particular rasterization or interpolation procedure.

The release also includes the observations used to establish and assess correspondence between the modalities. Manual synchronization events, in which a physical object was rapidly introduced into the scene and observed in both modalities, are retained separately from the matched analysis footage. Frames containing the geometric calibration object are similarly retained as supporting calibration data.

For the three horizontal configurations, we provide the estimated planar homography between the Cartesian sonar representation and the corresponding RGB view together with the calibration correspondences used to estimate it. The corresponding temporal-alignment and spatial-calibration procedures are described in § 4.3 and § 4.5. No planar homography is reported for the vertical 20V configuration because its geometry and available correspondences did not support a defensible planar mapping.

The release therefore distinguishes between three types of information: the primary paired observations, acquisition and pairing metadata describing those observations, and supporting synchronization and calibration data documenting how the correspondence was obtained.

Dataset statistics

Table 1 summarizes the selected recording intervals and matched frame counts for the four configurations. Together, these intervals span approximately 3.3 hours and contain 105,362 matched RGB-sonar pair records. Each pair contains a unique sonar frame. Four RGB frames are each associated with two sonar frames near recording boundaries, resulting in 105,358 unique RGB frames in the paired release; these cases are listed in Table 2. Matched RGB-sonar image pairs have less than 100 ms temporal discrepancy, with the median difference below 8.5 ms. Per-run temporal differences are summarized in Table 3. Sonar frame-rate statistics are summarized in Table 4, with frame-rate variation over time and its distribution shown in Figure 8 and Figure 9.

RunRangeOrientationDurationTotal FramesMatched Pairs
15H15ft / 4.57mHorizontal1:03:08.01836,02835,984
40H40ft /12.19mHorizontal1:09:11.83132,89832,878
100H100ft /30.48mHorizontal0:47:25.25520,64120,635
20V20ft / 6.10mVertical0:18:18.50715,86915,865

Table 1. Recording durations and matched frame counts by run. Sonar frames were matched with the nearest RGB frame. Frames without an RGB observation within 100 ms were excluded from the paired release. Duration is in HH:mm:ss.fff.

RunRGB frame #Sonar frame #Δt\Delta t (ms)
15H50,96050,657+82.63
15H50,96050,658+27.27
40H231,20825,589+9.08
40H231,20825,590−67.29
40H231,23025,599+73.08
40H231,23025,600+3.35
20V438,245133,025−13.23
20V438,245133,026−76.86
100HAll matches are one-to-one—

Table 2. Cases where the same RGB frame was matched to two sonar frames. These occur around RGB capture boundaries where fewer RGB frames were captured due to file saving. Overall, HaCKRS contains 105,362 pair records for the 105,362 unique sonar frames linked to 105,358 unique RGB frames. A positive difference means the corrected RGB timestamp is later than the sonar timestamp.

RunMedian95th%Max≤33\le33 ms
15H8.30 ms15.86 ms82.63 ms99.989%
40H8.67 ms15.86 ms91.91 ms99.982%
100H8.20 ms15.88 ms63.12 ms99.990%
20V8.05 ms15.85 ms76.86 ms99.987%

Table 3. Temporal difference between the matched RGB and sonar frames for each run. Frames with >100 ms difference were not matched. 95th% refers to the 95th percentile.

Run5th%MeanMedian95th%
15H6.29.508.615.2
40H5.07.926.413.4
100H5.47.257.68.6
20V14.014.4414.415.0

Table 4. Frames per second of the sonar frames. 5th% and 95th% refer to the 5th and 95th percentiles. See Figure 8 for a graph of how the frame rate changes over the duration of each run, and see Figure 9 for a visualization of the frame rate distribution.

Sonar frame rate over time for 15H

(a) (a) 15H usable sonar frames 39,700-111,182

Sonar frame rate over time for 40H

(b) (b) 40H usable sonar frames 19,038-75,238

Sonar frame rate over time for 100H

(c) (c) 100H usable sonar frames 165,958-191,253

Sonar frame rate over time for 20V

(d) (d) 20V usable sonar frames 128,380-148,999

Figure 8. Sonar frame rate over time for each recording configuration. Solid blue vertical lines mark time-synchronization events, when a pipe was rapidly inserted into the water to produce a recognizable event in both sonar and RGB video. Dashed pink lines mark frames containing the cross-shaped pipe calibration object used to identify correspondences for homography estimation. These annotations refer to physical objects and actions recorded in the videos, rather than electronic synchronization signals. Gray shading indicates intervals outside the selected usable footage; orange crosses mark sonar frames without an RGB match within 100ms.

Distributions of sonar frame rates estimated over rolling 5 s windows for the selected usable footage in each recording configuration

Figure 9. Distributions of sonar frame rates estimated over rolling 5 s windows for the selected usable footage in each recording configuration. Violin width represents the density of frame-rate estimates; white bars mark the medians, and red diamonds mark the overall mean frame rates. The number of estimates is shown below each run label. The horizontal runs exhibit broader, multimodal distributions, whereas 20V has a narrower distribution.

Intended use

HaCKRS is intended primarily as a data resource for studying automated analysis of recreational imaging sonar. Potential uses include fish detection, tracking, and counting; development of annotation procedures for difficult sonar imagery; analysis of range-dependent sonar appearance; and cross-modal methods in which optical observations provide additional supervision during development.

The paired RGB imagery should not be interpreted as exhaustive ground truth for the sonar modality. Instead, it provides an additional observation of the same scene with different geometry and different failure modes. Methods using HaCKRS should account explicitly for incomplete field-of-view overlap, depth-dependent geometric disagreement, temporal uncertainty, and cases in which an object is visible in only one modality. These limitations are discussed in more detail in § 6.

Data Collection and Alignment

We recorded sonar and top-down RGB imagery for a total of 3.3 hours. Recordings were conducted at a single location in the Chinese Garden at The Huntington Library, Art Museum, and Botanical Gardens.

Recording specifications

RGB imagery was captured as video using the “Ultra Wide” lens on an iPhone 17 at a resolution of 1920×10801920 \times1080 pixels and 30 frames per second.

Sonar imagery was recorded using a Panoptix LiveScope LVS32 transducer and GLS 10 sonar module. The imagery is composed of 511 angular beams and 512 range bins. The extracted metadata contains 511 angular coordinates spanning approximately 150°. This differs from Garmin’s specified 135° front-to-back field of view. We report the recorded coordinate values here without assuming that their full numerical span corresponds directly to the transducer’s effective physical field of view. The recorded angular spacing is characterized in section G.

The recorded polar imagery is incomplete at the largest ranges: the first 345–351 range bins span the full angular range (see Figure 7 for the variation across runs). Beyond these bins, the angular coverage is incomplete. When transformed into Cartesian coordinates, this produces the flat upper edge shown in Figure 3. The straight horizontal edge has a constant yy-coordinate in the Cartesian representation at the specified range. When we refer to the range of a video, we mean the range at the centerline. The advertised beam height for the sensor is 20 degrees, though we were not able to confirmed this. The returned values appear to be a combination of all the reflections in the 20 degree beam. This can increase the apparent size of objects as they can intersect multiple range bins in the same beam.

Raw polar sonar frame

(a) Raw polar sonar frame

Cartesian-mapped sonar frame

(b) Cartesian-mapped sonar frame

Figure 3. Example imaging-sonar frame before and after cartesian mapping. The same 3 ft (91 cm) range lines and 30∘30^\circ angular reference lines are shown in both.

Sonar polar range-line position measured from the bottom of the image to the center-column trough
RunMin (px)Median (px)Mean (px)Max (px)Std. Dev. (px)Std. Dev. (%)
15H345345345.063510.370.11
40H349349349.173510.450.13
100H346346346.973511.480.43
20V345347347.223510.900.26

Figure 7. Sonar polar range-line position measured from the bottom of the image to the center-column trough (illustrated right). Each test contains 1,000 randomly sampled frames (4,000 frames total). Std. Dev. (%) is relative to the median value.

Both modalities were saved in clips at 15 minute intervals. Clip-level sonar and RGB inventories are provided in tables 6a and 6b. After recording, they were combined into a complete corpus of frames.

Sonar orientation

Three of the runs (15H, 40H, and 100H) were conducted with the sonar imaging plane oriented horizontally, approximating a top-down view. For these runs, the sensor was positioned less than 15 cm (approximately 6 inches) below the water surface. In the vertical run (20V), the sonar was rotated 90 degrees about its centerline, giving a vertical or profile view, and placed in contact with the pond floor, approximately 60 cm (2 feet) below the surface.

Temporal alignment

We temporally aligned the frames using each capture mode’s internal frame-wise metadata. To check our temporal alignment, we performed an action that was visible in both modalities. We quickly placed a pipe vertically into the water. This produced a suddenly appearing bright area in the sonar imagery.

RGB timestamps were converted to a common wall-clock timeline and aligned with the sonar timestamps. At some file-save boundaries, the RGB stream briefly froze or repeated frames, creating small discontinuities in the timeline. Separate timing corrections were applied on either side of these discontinuities to preserve alignment with the continuously recorded sonar stream. Additional details of the timestamp correction and synchronization quality are provided in § 17.

Frames that included the manual time-sync actions are not included in the released temporally matched dataset, but are included as additional data.

The sonar frame rate sometimes decreased over the course of a recording, for reasons we have not determined. These variations are documented in section H. Temporal matching therefore uses the recorded frame timestamps rather than assuming a constant sonar frame rate.

Frames were paired by finding the temporally closest RGB frame to each sonar frame. If no RGB frame was found within 100 ms, no match was made and the sonar frame was excluded from the dataset.

Of the 105,436 selected sonar frames before temporal matching, 105,362 were paired with an RGB frame within 100 ms. The remaining 74 were excluded. All 105,362 matches contain a unique sonar image. Four RGB frames are each paired with two sonar frames near recording boundaries, so the released dataset includes 105,358 unique RGB frames. The duplicate boundary matches and per-run temporal differences are reported in Table 2 and Table 3.

Sonar polar-to-Cartesian transformation

To support geometric comparison between the two modalities, we transform the recorded sonar measurements from their polar representation into a Cartesian coordinate system, see Figure 3. The Cartesian representation also provides a uniform spatial coordinate system in the sonar imaging plane, making geometric comparisons more convenient, i.e., objects of the same size should appear the same size. For this transformation, we used the beam angles and bin distances defined in the recorded sonar metadata.

For the visualization in this paper we used inverse mapping to convert each Cartesian output pixel into its corresponding range and beam-angle coordinates in the polar image. We then used bilinear interpolation between the neighboring polar samples to determine its intensity. Pixels outside the valid recorded range and angular extent were assigned a value of zero. The dataset includes the original polar imagery and the underlying sonar coordinate information so that alternative representations can be constructed.

RGB-sonar planar registration

To estimate spatial relationships between the two imaging modalities we performed planar registration. To aid in finding homographic correspondences we used a calibration object. The calibration object is a cross with pipes extending downwards; see the third column of Figure 5. The vertical pipes that descend into the water column are approximately 1.5 m center-to-center and 48 mm in diameter. The geometric layout of the reference object provided multiple points per frame and known straight lines. The regular geometry, as opposed to multiple free-floating buoys, also aided manual localization of the object in the noisy sonar imagery.

Figure 5. (a) 15H: 15-foot (4.57m) setup. (b) 40H: 40-foot (12.19m) setup. (c) 100H: 100-foot (30.48) setup. Figure 5: Example sonar-RGB correspondences used for homography estimation. From left to right: full cartesian sonar frame, cropped sonar region containing the calibration object with homography points, cropped RGB region containing the calibration object with homography points, and full RGB frame.

The tops of the pipes were used as reference points for the RGB imagery and the center of the cross-section of the pipes was used as reference points in the sonar imagery. This introduces parallax error as the fish are below the surface and the RGB reference points are above the water (<25mm<25\mathrm{mm}).

Corresponding points were manually localized in temporally-matched cartesian-transformed sonar and RGB frames, see Table 8 for the number of points per run.

Run# Frames# PointsOutliersInlier rateRGB (pixels)RGB (pixels)Sonar (cm)Sonar (cm)Sonar (cm)
Run# Frames# PointsOutliersInlier rateMedianRMSEMedianRMSEInlier RMSE
15H728292.9%3.926.281.702.512.15
40H832487.5%5.697.603.064.113.57
100H624291.7%6.666.904.935.955.89

Table 8. Shared-homography agreement. # Frames and # Points refer to the number of homography frames and points used. RGB (pixels) and Sonar (cm) represent the error between the sonar and RGB points projected into the specified image space. Errors include all points, including outliers, in all columns other than Inlier RMSE. Outliers are detailed in table 7. The corrected sonar images have a resolution of 989 × 512 pixels for all three runs. The RGB calibration images for 15H and 100H have a resolution of 1920 × 1080 pixels. For 40H, the RGB image is rotated 90°, resulting in a calibration-image resolution of 1080 × 1920 pixels.

Homographic mappings were estimated using RANSAC with a 12-RGB-pixel threshold and refined using the consensus points. The inlier rates were 92.9%, 87.5%, and 91.7% for 15H, 40H, and 100H, respectively. The inlier RMSE was 2.15 cm, 3.57 cm, and 5.89 cm for 15H, 40H, and 100H, respectively (see Table 8 for details). Zoomed-in examples after alignment are shown in Figure 4.

15H homographically aligned imagery

(a) 15H: 15-foot (4.57m) homographically aligned imagery (cropped).

40H homographically aligned imagery

(b) 40H: 40-foot (12.19m) homographically aligned imagery (cropped).

100H homographically aligned imagery

(c) 100H: 100-foot (30.48m) homographically aligned imagery (cropped).

Figure 4. After homographic warping, the two modalities are approximately aligned. The RGB frame is homographically warped (right) to align with the sonar frame (left). As discussed in § 4.5, a planar mapping cannot fully capture depth-dependent displacement.

We recognize that there are some errors in this pipeline. Depth-dependent parallax is one likely source of residual misalignment, which methods using this dataset may need to account for.

Details, including the homography matrices, are provided in section J; evaluation of the homography accuracy is provided in section K.

Frames that included the homography calibration object are not included in the released temporally matched dataset, but are included as additional data.

Median cm and RMSE cm values are measured in the sonar plane and represent the physical error between the RGB points projected onto the corresponding sonar points, after converting the resulting sonar-pixel distances to centimeters using the sonar range-bin scale. These values are specified in Table 8. The resulting spatial overlap between modalities for the horizontal configurations is shown in Figure 6.

Spatial overlap between sonar and RGB imagery for the three horizontal configurations

Figure 6. (a) 15H sonar, 84.3% RGB coverage (b) 40H sonar, 32.3% RGB coverage (c) 100H sonar, 59.7% RGB coverage (d) 15H RGB, 51.5% Sonar Coverage (e) 40H RGB, 81.2% Sonar Coverage (f) 100H RGB, 73.6% Sonar Coverage Figure 6: Spatial overlap between sonar and RGB imagery for the three horizontal configurations (15H, 40H, and 100H, left to right), estimated using the calibrated homographies. Top: RGB coverage in cartesian sonar coordinates, retaining the full recorded sonar footprint. Bottom: sonar coverage in RGB coordinates. Percentages indicate the fraction of each reference image covered by the other modality.

Data Availability and Release

The HaCKRS dataset is publicly available on Hugging Face. The dataset comprises approximately 229 GB of data and is distributed under the MIT License.

Documentation describing the dataset organization, metadata fields, coordinate conventions, and example procedures for loading and aligning paired observations is provided alongside the dataset. Code and tools for working with the dataset are available on GitHub.

Limitations

Planar geometry and calibration. RGB and sonar have different imaging geometries, and the homographies for 15H, 40H, and 100H approximate the relationship between the modalities using a single plane. Fish, vegetation, and background structures occupy different depths, and the RGB and sonar calibration reference points are not physically coincident. Refraction at the air–water interface and

the finite vertical extent of the sonar beam introduce additional differences between the modalities. Consequently, planar registration does not recover three-dimensional fish positions, and agreement on the calibration object does not imply equivalent alignment for fish at other depths. The reported calibration residuals are also evaluated on correspondences involved in model estimation and therefore measure internal calibration agreement rather than independent localization accuracy. A future collection could improve this evaluation using held-out calibration targets and reference points distributed across multiple positions and depths. The registration procedure and calibration residuals are described in sections 4.5 and K.

Temporal alignment. The camera and sonar were not synchronized by a shared hardware trigger. Pairing instead uses corrected device timestamps, supported by physical synchronization events visible in both modalities. RGB timestamps were mapped to wall-clock time using robust affine fits to the integer-second transitions in the Mac receipt timestamps, then adjusted using median sonar-RGB offsets from manually matched pipe events; the 15-ft recording required separate offsets before and after a timestamp discontinuity. Capture starts and ends were checked against the frame counts within partial wall-clock seconds, and elapsed recording durations and inter-capture gaps were evaluated using the recorded timestamps rather than an assumed constant frame rate. At the fitted synchronization anchors, the residual timestamp RMSE was 6.0, 6.8, 6.1, and 16.9 ms for the 15-ft, 40-ft, 100-ft, and vertical-plane runs, respectively, with maximum absolute residuals of 6.3, 17.5, 11.0, and 37.3 ms across 4, 8, 6, and 13 annotated pairs. These are residuals at the events used to estimate the corrections, rather than independent measurements of exposure-time synchronization or guarantees of alignment throughout each recording. The 100 ms matching threshold bounds the difference between the assigned timestamps, not necessarily the physical acquisition times. Residual transport latency, dropped or repeated RGB frames, capture-boundary discontinuities, and variable sonar frame rates may remain. Motion-sensitive analyses should therefore use the supplied timestamps rather than assume uniformly spaced or exposure-synchronized frame pairs. A future acquisition would benefit from a common hardware timing signal or an independently recorded synchronization source where supported by the devices. Further details of the timing correction and synchronization quality are provided in § 4.3 and § 17.

Incomplete overlap and unequal visibility. The two modalities have different fields of view and visibility failure modes, observe only partially overlapping regions, and geometric overlap does not imply equal visibility. Fish may lie outside the sonar beam’s vertical extent while remaining visible to the top-down camera, or may be obscured in RGB by water color, turbidity, or surface reflections. Conversely, sonar returns may be ambiguous or overlap with environmental structure. RGB observations should therefore be treated as complementary evidence rather than exhaustive ground truth for individual sonar returns. Absence from one modality should not automatically be interpreted as absence from the scene. Cross-modal annotation and evaluation procedures should distinguish between verified negatives and regions or objects that are unobserved or ambiguous. The measured spatial overlap for the three horizontal configurations is shown in Figure 6.

Range and spatial representation. The physical sampling represented by a sonar pixel depends on acquisition geometry and range. Transforming the measurements into Cartesian coordinates provides a convenient spatial representation but does not increase the information contained in the original sonar observations. Apparent object size, boundary precision, and separation between nearby fish should therefore be interpreted with respect to the native range and angular sampling rather than the displayed Cartesian raster alone. The differences among the 15H, 40H, and 100H recordings provide examples of these range-dependent effects, but because the runs also differ in scene content and calibration observations they should not be interpreted as a controlled experiment isolating range alone.

Dataset scope and downstream evaluation. HaCKIRs contains approximately 3.3 hours of selected footage from a single pond, one fish type, one sonar system, and one RGB camera. It therefore does not establish generalization to other species, sites, sensors, or environmental conditions, including the highly turbid or dark environments that motivate many applications of imaging sonar. The current release also does not provide task-specific reference detections, identities, tracks, or counts. Such annotations and appropriate evaluation protocols are required before drawing conclusions about downstream task performance. Because neighboring video frames are highly correlated, random frame-level train/test splits could also place nearly identical observations, and potentially the same individual fish, in both sets. Future benchmarks should use temporally separated evaluation intervals and, where possible, independent sites or populations.

Practical Lessons for Future Collections

One purpose of releasing the supporting synchronization and calibration information is to make the collection procedure itself inspectable. Several aspects of the present dataset suggest straightforward improvements for future RGB-sonar collections.

The most important improvement would be stronger synchronization. A shared hardware trigger or independently recorded timing signal would reduce uncertainty introduced by separate device clocks, file boundaries, and transport latency. When hardware synchronization is unavailable, repeated physical synchronization events throughout a recording can help reveal timing drift or discontinuities rather than providing only a single alignment check. The temporal-alignment procedure and its remaining uncertainties are described in § 4.3 and § 17.

Spatial calibration would similarly benefit from being designed around independent validation rather than only model estimation. Calibration targets could be observed at multiple known depths and positions, with separate observations reserved for evaluating registration accuracy. For top-down optical imaging, explicitly measuring camera position, orientation, height, and the water-surface geometry would also make it possible to investigate models that account for refraction and three-dimensional scene structure rather than relying entirely on a planar homography. The planar-registration procedure and its evaluation are described in § 4.5 and K.

Finally, field-of-view overlap should ideally be evaluated before the primary recordings begin. The substantially different footprints produced by different sonar ranges mean that a camera placement appropriate for one configuration may provide limited coverage in another. A short calibration acquisition at each intended range can be used to verify both geometric overlap and whether fish-sized objects remain sufficiently resolved in each modality. The resulting spatial overlap for the three horizontal configurations is shown in Figure 6.

These observations are not intended to imply that the present alignment provides exact cross-modal ground truth. Rather, documenting where uncertainty enters the acquisition pipeline is part of the purpose of the dataset. We expect that future collections can use these experiences to obtain stronger temporal synchronization, more complete spatial calibration, and greater environmental diversity.

Conclusion

We introduce HaCKRS, a paired dataset of consumer imaging-sonar and top-down RGB recordings of koi carp at The Huntington Library, Art Museum, and Botanical Gardens. HaCKRS includes more than 105,000 temporally matched pairs from four range/orientation configurations spanning 3.3 hours. In addition, we provide frame-wise metadata, sonar polar-to-Cartesian transforms, supporting synchronization and calibration frames, and derived planar homographies for each of the three horizontal runs.

The paired RGB imagery offers complementary evidence when sonar observations are difficult to interpret and can support annotation, method development, and diagnostic evaluation of sonar-based fish monitoring methods. However, differences in visibility, depth, and acquisition timing, as well as calibration inaccuracies, mean that the RGB-sonar mapping is not perfectly one-to-one. Methods utilizing this data should account for these differences in their design.

Within this scope, HaCKRS provides a resource for investigating consumer imaging sonar for fish monitoring and for studying how paired optical observations can assist the interpretation and development of sonar-based methods. Extending the dataset to additional sites, species, and environmental conditions, together with task-specific annotations, would enable more comprehensive evaluation of detection, tracking, and counting methods.

Comparison of Related Datasets and Studies

The main text discusses prior work on imaging sonar for fish monitoring, paired optical–acoustic observations, and multimodal underwater datasets. Here, we provide a structured comparison of the most closely related datasets and studies along several dimensions relevant to HaCKRS. Table 5 summarizes whether each resource focuses on fish, includes paired or synchronized optical observations, makes data publicly available, provides spatial calibration between modalities, uses research-grade or consumer imaging sonar, and is intended to support cross-modal learning. The comparison is not intended to be exhaustive, rather, it highlights how existing resources combine these characteristics and provides additional context for the design and intended use of HaCKRS.

Sonar frame inventory

Table 6a lists the capture times, durations, and extracted frame counts for each sonar clip. The 17 listed clips contain 134,220 frames before usable-footage selection and temporal matching: 42,307 in 15H, 39,895 in 40H, 25,830 in 100H, and 26,188 in 20V. Most clips last approximately 15 minutes, with shorter final clips in the horizontal runs.

RGB video inventory

Table 6b summarizes the capture times, durations, and frame counts for the listed RGB clips. These 19 clips contain 479,661 frames, with approximately 27,000 frames in each full 15-minute clip. The RGB and sonar recording intervals do not coincide exactly, in particular, the RGB recording for 40H extends beyond the sonar recording.

Framewise metadata

For each sonar-RGB frame pair we have the associated metadata:

Sonar:

  • Sonar frame ID: Identifier of the sonar frame within the recorded dataset.

  • Sonar capture timestamp: Absolute wall-clock acquisition time, including the date and time-zone offset.

  • Sonar clip: Identifier of the source sonar recording segment containing the frame.

  • Polar bins and angles: Number of radial range samples and angular beam samples in the polar sonar representation. Each sample records sonar intensity at a particular range and bearing.

  • Range (m): Maximum radial distance represented by the sonar frame, expressed in meters.

  • θmin⁡\theta_{\min} and θmax⁡\theta_{\max}: Minimum and maximum beam bearings, expressed in degrees relative to the sonar’s angular reference axis. Together, they specify the angular extent of the sonar frame.

RGB:

  • RGB frame ID: Identifier of the RGB frame within the recorded dataset.

  • RGB frame timestamp: Frame time expressed as elapsed seconds relative to the RGB capture’s time origin, and as an estimated absolute wall-clock time obtained through clock calibration. The additional offset used to synchronize RGB and sonar is recorded separately.

  • RGB capture number: Identifier of the source RGB recording containing the frame.

  • fxf_x, fyf_y, cxc_x, cyc_y: Camera intrinsic parameters for the native, unrotated RGB image. The focal lengths fxf_x and fyf_y are expressed in horizontal and vertical pixel units, respectively; (cx,cy)(c_x,c_y) gives the principal-point location in image coordinates.

  • Width and height: Dimensions of the RGB image in pixels.

  • Lens position: Camera-reported lens focus position at acquisition, rather than a physical distance to the subject.

  • Time received on Mac: Timestamp recorded when the receiving Mac received the frame data; this is distinct from the acquisition timestamp.

  • Focus, exposure, and white-balance locks: Boolean indicators specifying whether each camera control was locked during acquisition. A locked control remains fixed rather than being adjusted automatically.

Sonar polar range-line position

Figure 7 summarizes the position of the center-column trough that marks where the polar image ceases to cover the full angular range. Median positions range from 345 to 349 pixels across the four runs, with standard deviations below 1.5 pixels (0.43% or less). The small within-run variation indicates that this boundary is nearly constant over time.

Human-free usable footage

The following intervals were selected as footage without people visible in either modality. Counts below are counts of files/metadata rows actually present between the supplied endpoints; they are not endpoint subtraction alone.

Based on sonar wall-clock timestamps, these intervals span approximately 63.13 minutes (15H), 69.20 minutes (40H), and 47.42 minutes (100H). The usable sonar subsets contain approximately 85.2%, 82.5%, and 79.9% of each test’s extracted sonar frames. The corresponding RGB subsets contain approximately 88.8%, 63.8%, and 82.9% of the recorded RGB frames. The lower RGB fraction in 40H is principally a consequence of the deliberately selected human-free end-point, not evidence of file corruption.

Sonar angular spacing

Angular beam spacing is the difference in angle between adjacent sonar beams. The recorded angle tables were identical across all sampled frames and all four runs. Individual angular separations ranged from 0.294114∘0.294114^\circ to 0.294122∘0.294122^\circ, with an average of 0.294117643∘0.294117643^\circ and a maximum deviation from the average spacing of approximately 4.36×10−64.36 \times10^{-6} degrees. These small differences are consistent with numerical precision and rounding, indicating effectively uniform angular spacing. No frame-to-frame variation in angular sampling was observed in the inspected frames. This finding describes the recorded angular coordinates.

Frame rate variation

The sonar frame rate varies both across configurations and over time, as shown in Figure 8. The horizontal runs show sustained decreases during recording, whereas 20V remains comparatively stable. The distributions in Figure 9 are correspondingly broad and multimodal for the horizontal runs, but concentrated near 14–15 frames per second for 20V. The cause of the decreases has not been determined.

Time synchronization quality

RGB wall-clock time was recovered by robust affine fitting of precise camera timestamps to transitions in the integer-second Mac receipt clock. Capture

RunClip #Capture startCapture endDuration (s)Frames
15H00109:21:23.4709:36:23.9915:00.5213,643
15H00209:36:24.0509:51:23.9014:59.8510,007
15H00309:51:24.0310:06:23.8714:59.847,715
15H00410:06:24.0210:21:23.9014:59.876,375
15H00510:21:24.0310:33:36.9312:12.904,567
40H00111:01:43.3011:16:43.9815:00.699,911
40H00211:16:44.0511:31:43.7814:59.7310,538
40H00311:31:43.9611:46:43.9615:00.006,261
40H00411:46:44.0812:01:43.9214:59.845,297
40H00512:01:44.0612:16:43.8914:59.844,867
40H00612:16:44.1212:26:59.1910:15.073,021
100H00113:47:00.2714:02:00.9315:00.667,625
100H00214:02:01.1714:17:00.9914:59.827,322
100H00314:17:01.2214:32:00.7314:59.526,414
100H00414:32:00.9514:44:56.7112:55.764,469
20V00112:56:51.0713:11:51.9915:00.9213,272
20V00213:11:52.0613:26:51.9314:59.8812,916

Table 6. Recording inventory for the paired sonar and RGB data. Each table lists the capture interval, duration, and number of extracted frames for each clip and recording configuration. For RGB, duration is calculated from the final precise timestamp rather than from the nominal frame count divided by 30.

RunClip #Capture startCapture endDuration (s)Frames
15H00109:21:35.2909:36:35.1914:59.8926,936
15H00209:36:36.0109:51:35.2414:59.2126,977
15H00309:51:36.0510:06:35.2814:59.2426,978
15H00410:06:36.0610:21:35.2714:59.2726,979
15H00510:21:36.0210:32:41.6811:05.6719,971
40H00111:01:45.1211:16:44.3814:59.2726,979
40H00211:16:45.1111:31:44.4414:59.3026,980
40H00311:31:45.1211:46:44.4514:59.3426,981
40H00411:46:45.1212:01:44.4814:59.3726,982
40H00512:01:45.1212:16:44.5214:59.4026,983
40H00612:16:45.1112:31:44.5914:59.4426,979
40H00712:31:45.1512:46:44.6214:59.4726,985
40H00812:46:45.1512:50:06.9803:21.836,056
100H00113:46:55.1814:01:54.7514:59.5726,988
100H00214:01:55.1614:16:54.8614:59.6026,989
100H00314:16:55.2514:31:54.8814:59.6426,990
100H00414:31:55.2914:44:07.0712:11.8121,955
20V00112:56:46.1513:11:45.6914:59.5026,986
20V00213:11:46.1813:26:45.7214:59.5426,987

Table 6. Recording inventory for the paired sonar and RGB data. Each table lists the capture interval, duration, and number of extracted frames for each clip and recording configuration. For RGB, duration is calculated from the final precise timestamp rather than from the nominal frame count divided by 30.

starts and ends were checked using the number of frames present in their partial wall-clock seconds. Actual timestamp rows, rather than a fixed 30-fps assumption, are used by the viewers.

  • 15H: one internal RGB timestamp discontinuity occurs at RGB 28,279–28,280 (2.084 s). The adopted viewer model shifts the pre-discontinuity timeline while preserving the post-discontinuity alignment. All MOVs still decode exactly the number of frames listed in metadata.

  • 40H: capture 6 contains a documented five-frame drop at RGB 355,191–355,197. Its timestamp advances by 0.200 s, so alignment after the event is preserved. No other internal timestamp discontinuity was found.

  • 100H: no internal timestamp or frame-number discontinuity was found. Capture-boundary gaps are approximately 0.411, 0.392, and 0.407 s and do not accumulate synchronization error because each capture is independently fitted.

  • 20V: no internal timestamp discontinuity was found. The first RGB recording contains only 22 frames (0.700 s), so it cannot independently observe enough wall-clock-second transitions for an affine fit. Its start time was recovered by linking its continuous camera PTS clock to the immediately following capture. Capture-boundary gaps are approximately 1.300, 0.490, and 0.467 s. Thirteen existing sonar-RGB matched pairs span the two principal captures; their median corrections are −1.153 s and −1.179 s, with within-capture spreads of 53.2 ms and 18.3 ms. The dedicated viewer presents raw-polar or warped-XY sonar beside RGB and permits per-capture visual refinement.

The timing models locate the nearest recorded RGB timestamp typically within one-half RGB frame (about 16.7 ms), apart from intervals where a modality did not record a new frame. This timestamp proximity is not, by itself, proof of exposure-level synchronization: the devices lacked a shared hardware trigger, and constant transport latency or rolling-shutter effects may remain.

Shared homography method

For each distance, four point correspondences were selected on each of several synchronized warped-sonar/RGB pairs. The original model used RANSAC with a 12-RGB-pixel threshold and was refined using its consensus points. For this report, each annotated sonar point was mapped exactly from its former raster coordinates into its frame’s full-warp coordinates. The original inlier membership was preserved, and the shared matrix was then refitted on those migrated consensus points without new manual annotation. RGB was rotated 180 degrees for 15H, 90 degrees clockwise for 40H, and not rotated for 100H.

The vertical-plane test is intentionally absent from the matrices and error table below. Its perspective and available correspondences did not support a consistent planar homography, so reporting a pixel or centimeter homography error for it would imply a model that the data does not justify. Its synchronization remains useful independently of geometric overlay.

For sonar homogeneous point s=(x,y,1)T\mathbf{s}=(x,y,1)^{T}, the predicted RGB point is

r^=π(Hs),π([a,b,c]T)=[a/c,b/c]T.(1)\hat{\mathbf{r}}=\pi(H\mathbf{s}), \qquad\pi([a,b,c]^{T})=[a/c,b/c]^{T}. \tag*{(1)}

The reported RGB residual is ∥r^−r∥2\lVert\hat{\mathbf{r}}-\mathbf{r}\rVert_{2}.

Homography matrices

15H, RGB rotated 180 degrees

H15=[6.902864260.339648326−1483.032670.9449398184.64554217−540.1740850.00254177561−0.0001977454571]H_{15}= \begin{bmatrix} 6.90286426 & 0.339648326 & -1483.03267\\ 0.944939818 & 4.64554217 & -540.174085\\ 0.00254177561 & -0.000197745457 & 1 \end{bmatrix}

40H, RGB rotated 90 degrees clockwise

H40=[8.91419579−5.86796112−446.0099248.591302385.35463179−3894.989680.00383600753−0.002103838501]H_{40}= \begin{bmatrix} 8.91419579 & -5.86796112 & -446.009924\\ 8.59130238 & 5.35463179 & -3894.98968\\ 0.00383600753 & -0.00210383850 & 1 \end{bmatrix}

100H, RGB unrotated

H100=[1.95051635−1.73585783−2.811007610.09238781900.09318565910.3308168020.000124706315−0.001898968241]H_{100}= \begin{bmatrix} 1.95051635 & -1.73585783 & -2.81100761\\ 0.0923878190 & 0.0931856591 & 0.330816802\\ 0.000124706315 & -0.00189896824 & 1 \end{bmatrix}

Homography accuracy

Table 8 summarizes agreement on the annotated calibration correspondences for the three horizontal runs. Median RGB reprojection errors range from 3.92 to 6.66 pixels, while median sonar-plane residuals increase from 1.70 cm in 15H to 4.93 cm in 100H. These values describe agreement on the calibration points used for fitting, rather than independent localization accuracy.

RANSAC-rejected control points

The stored RANSAC consensus rejected eight of the 84 annotated correspondences: two in 15H, four in 40H, and two in 100H. Table 7 identifies these points and reports their residuals after coordinate migration and refitting. Two rejected points have final RGB residuals below the original 12-pixel threshold, illustrating that the preserved consensus labels do not follow directly from the final residuals.

RunSonar frameRGB framePoint IDSonar residual (px)RGB residual (px)
15H38,79732,10735.1711.36
15H39,20632,75746.4916.62
40H14,609206,88322.6014.24
40H76,014342,97932.3415.13
40H76,014342,97942.3615.28
40H77,530346,33723.8513.34
100H160,880501,98410.957.99
100H162,212506,47111.2212.78

Table 7. Points rejected by the stored RANSAC consensus. A rejected point may reflect a mis-click, parallax/depth, imperfect timing, or violation of the planar model. Consensus membership was preserved during coordinate migration; it is not equivalent to thresholding the final refined residual alone.

The centimeter-equivalent residual is computed in the sonar plane, not by assigning a constant physical size to an RGB pixel. For annotated RGB point r\mathbf{r} and its paired sonar point s\mathbf{s},

ecm=100q∥π(H−1r)−s∥2.(2)e_{\mathrm{cm}}=100q\left\lVert\pi\left(H^{-1}\mathbf{r}\right)-\mathbf{s}\right\rVert_{2}. \tag*{(2)}

where qq is the sonar range-bin scale: 9.0 mm/pixel, 23.6 mm/pixel, and 59.6 mm/pixel for the 15-, 40-, and 100H settings. Because a projective transform has position-dependent scale, this inverse mapping is preferable to multiplying the RGB residual by one global RGB-pixel scale. The original RANSAC fit used a 12-RGB-pixel reprojection threshold.

For each candidate homography, a sonar point was projected into the RGB image. Its reprojection error was calculated as:

eRGB=∥π(Hs)−r∥2.e_{\mathrm{RGB}}=\lVert\pi(H\mathbf{s})-\mathbf{r}\rVert_{2}.

A correspondence was considered an inlier when this error was within approximately 12 RGB pixels under the RANSAC candidate model; otherwise, it was rejected from that model’s consensus set.

Importantly, the table retains the original RANSAC classifications after migrating the sonar points into the corrected warp and refitting the homography. Therefore, the final displayed residual alone does not determine whether a point is listed as rejected. A preserved outlier can now have a final residual below 12 pixels.

References

  1. [1]Weitung Chen, Phil Tinn, Per Gunnar Auran, Martin Ludvigsen, and Peter Halland Haro. A sonar-visual dataset for cross-modal underwater robot perception, 2026.arxiv.org/abs/2606.01398
  2. [2]R. M. Connolly, K. I. Jinks, A. Shand, M. D. Taylor, T. F. Gaston, A. Becker, and E. L. Jinks. Out of the shadows: automatic fish detection from acoustic cameras. Aquatic Ecology, 2023.DOI
  3. [3]Nikolas Dahn, Miguel Bande Firvida, Proneet Kumar Sharma, Leif Christensen, Oliver Geisler, Jochen Mohrmann, Torsten Frey, Prithvi Sanghamreddy, and Frank Kirchner. An acoustic and optical dataset for the perception of underwater unexploded ordnance (UXO). In OCEANS 2024. IEEE, 2024.DOI
  4. [4]Jon Dickson, Maryann Watson, Bass Dye, Madelon Neefjes, Roos van Dorp, Anne Jasker, Oscar Franken, Tjeerd Bouma, Laura Govers, Han Olff, et al. Sonar, so good: Tree-reefs drive net gain in fish size, abundance, and diversity. Marine Ecology Progress Series, 2026.DOI
  5. [5]Yunhong Duan, Shubin Zhang, Yang Liu, Jincun Liu, Dong An, and Yaoguang Wei. Boosting fish counting in sonar images with global attention and point supervision. Engineering Applications of Artificial Intelligence, 2023.DOI
  6. [6]Joachim P. Gröger, Boris Cisewski, Sabah Badri-Hoeher, Gordon Böer, Karin Boos, Catriona Clemmesen, Ala Cojocaru, Verena Dauben, Peter A. Hoeher, Andreas Lehmann, Sebastian Matz, Hela Mehrtens, Felix Mittermayer, Helge Renkewitz, Hauke Schramm, Tobias Strickmann, Jonni Westphalen, Thomas Wilts, Julian Winkler, Dennis Wolf, and Oliver Zenk. Development and operation of a novel non-invasive opto-acoustic underwater fish observatory in kiel bight, southwestern baltic sea. Frontiers in Marine Science, 2024.
  7. [7]Justin Kay, Peter Kulits, Suzanne Stathatos, Siqi Deng, Erik Young, Sara Beery, Grant Van Horn, and Pietro Perona. The catch fish counting dataset: A benchmark for multiple-object tracking and counting. In Proceedings of the European Conference on Computer Vision (ECCV), 2022.arxiv.org/abs/2207.09295
  8. [8]Yunfeng Li, Bo Wang, Jiuran Sun, Xueyi Wu, and Ye Li. RGB-Sonar tracking benchmark and spatial cross-attention transformer tracker. IEEE Transactions on Circuits and Systems for Video Technology, 2025.arxiv.org/abs/2406.07189
  9. [9]François Martignac, Aurélie Daroux, Jean-Luc Baglinière, Dominique Ombredane, and Jean Guillard. The use of acoustic cameras in shallow waters: new hydroacoustic tools for monitoring migratory fish population. a review of DIDSON technology. Fish and Fisheries, 2014.
  10. [10]Erin L. McCann, Liling Li, Kevin Pangle, Nicholas S. Johnson, and Jesse Eickholt. An underwater observation dataset for fish classification and fishery assessment. Scientific Data, 2018.DOI
  11. [11]Ryan T. Munnelly, Jose C. Castillo, Nils Olav Handegard, Matthew E. Kimball, Kevin M. Boswell, and Guillaume Rieucau. Applications and analytical approaches using imaging sonar for quantifying behavioural interactions among aquatic organisms and their environment. ICES Journal of Marine Science, 2024.DOI
  12. [12]Sveinung Johan Ohrem, Bent Haugaløkken, and Eleni Kelasidi. SOLAQUA: SINTEF ocean large aquaculture robotics dataset, 2025.arxiv.org/abs/2504.01790
  13. [13]Wei Shen, Mengqi Liu, Quanshui Lu, Zhaowei Yin, and Jin Zhang. A fish target identification and counting method based on DIDSON sonar and YOLOv5 model. Fishes, 2024.DOI
  14. [14]Edward C. P. Sibley, Travis S. Elsdon, Michael J. Marnane, Alethea S. Madgett, Euan S. Harvey, Thomas Cornulier, Damon Driessen, and Paul G. Fernandes. Sound sees more: A comparison of imaging sonars and optical cameras for estimating fish densities at artificial reefs. Fisheries Research, 2023.DOI
  15. [15]Suzanne Stathatos, Michael Hobley, Pietro Perona, and Markus Marks. SAVeD: Learning to denoise low-snr video for improved downstream performance. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2026.arxiv.org/abs/2504.00161
  16. [16]Kei Terayama, Kento Shin, Katsunori Mizuno, and Koji Tsuda. Integration of sonar and optical camera images using deep neural network for fish monitoring. Aquacultural Engineering, 2019.
  17. [17]Madison Van Horn, Suzanne Stathatos, Sevan Brodjian, Justin Kay, Kai Van Brunt, Erik Young, Sara Beery, Pietro Perona, and Michael Hobley. CFC26: Building evaluations for deployment in sonar-based fish counting. In Advances in Neural Information Processing Systems (NeurIPS), Evaluations & Datasets Track, 2026.arxiv.org/abs/2610.22549
  18. [18]Yaoguang Wei, Yunhong Duan, and Dong An. Monitoring fish using imaging sonar: Capacity, challenges and future perspective. Fish and Fisheries, 2022.DOI
  19. [19]Thomas Wilts, Gordon Böer, Julian Winkler, Boris Cisewski, Hauke Schramm, and Sabah Badri-Hoeher. Descriptor: Hydroacoustic and optical dataset for oceanic research (HODOR). IEEE Data Descriptions, 2025. Associated dataset DOI: 10.1594/PANGAEA.980000.DOI

Paper details

vlnc:W_eyex55gq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 4

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents