betaFeedback

WorksPhysics and Astronomy

CBTV: Colour Bean Tracking Velocimetry for Dense Granular Flows

Abstract

Conventional particle tracking velocimetry (PTV) has been the standard diagnostic tool for studying fluid and granular flows. However, in dense granular systems, the average distance moved by particles between frames can approach or exceed inter-particle spacing, demanding ultra-high frame rates. Furthermore, neighbouring particles may move together, and common velocity-based prediction fails after energetic collisions. Existing methods struggle to track granular particles without losing identities or swapping. We present Colour Bean Tracking Velocimetry (CBTV), a training-free multi-pass method leveraging multicoloured particles to achieve enhanced tracking. CBTV first matches constellations of neighbouring coloured particles as single units, guided by a cross-correlation motion prior, and only then assigns the remaining particles individually. We compare it with 12 baseline tracker configurations, from classical linkers to graph-neural-network and transformer models, on 59 cases from 15 simulated discrete-element granular flows with exact ground truth (0.006-9 diameters/frame). CBTV matches or exceeds every tracker tested on an impulsive collision at every displacement. Its advantage lies in dense flows in which particles move between about half and twice their spacing per frame, where links between frames are ambiguous. On agitated shear flow, a colour relaxation tracker, which also uses neighbour context, matches or narrowly exceeds CBTV. Where links are unambiguous, CBTV is close to the best tracker, and beyond about two to three particle diameters per frame, the range of its motion prior, trackers that compensate for bulk motion are more accurate. For experimentalists, dense granular flows of coloured particles can be tracked at displacements per frame where trackers without neighbour context lose identities, easing the demand for ultra-high frame rates without training data.

Introduction

Granular materials are among the most widely handled substances in industry, comprising nearly 80% of raw materials in the chemical and pharmaceutical sectors ([17]). Despite their ubiquity, the physics governing dense granular flows remains poorly understood compared to that of Newtonian fluids. A major barrier to progress is the difficulty of obtaining detailed particle level trajectories inside dense, rapidly deforming granular assemblies. X-ray tomography and magnetic resonance imaging can probe opaque three-dimensional packings [28], but these methods are slow, expensive, and impractical for systems requiring high temporal resolution, so most measurements use optical methods on quasi-two-dimensional geometries.

Particle Image Velocimetry (PIV), originally developed for fluid flows [1], has been adapted for granular systems by correlating successive images of a granular bed viewed through transparent boundaries [23] (White et al. 2003). It yields Eulerian velocity fields, which are interrogation window averaged and cannot follow individual tracer particles.

Particle Tracking Velocimetry (PTV) identifies particles by local brightness and links them across frames by nearest-neighbour or probabilistic assignment [32, 11]. Its performance degrades sharply when inter-frame displacements become comparable to the inter-particle spacing: Malik et al. [26] showed that reliable nearest-neighbour linking requires a spacing-to-displacement ratio of roughly two or more, and cross-correlation with relaxation [6], higher-order predictive search [9], and Voronoï-based methods developed for granular imagery [8] relax but do not remove this bound. Published granular experiments keep the displacement per frame small by raising the frame rate: reported values range from 0.02 to 0.3 diameters per frame in vibrated monolayers [31], discharging silos [15], granular gases [37], impact [10], and shear cells [39] (computed from the reported speeds and frame rates where not stated). Where the frame rate cannot keep up, automated tracking has been replaced by following coloured particles by hand [12]. In a dense packing, mean surface separation is very small relative to particle diameter, so at inter-frame particle displacements (Δx\Delta x) approaching one particle diameter (Δx/d∼1\Delta x/d \sim1), per-particle linking becomes ambiguous unless the motion is predictable.

Gollin et al. [16] assess the accuracy of PIV and PTV for granular flow measurement. The strongest PTV methods for fluids are predictive and multi-pass: Shake-the-Box [35] predicts each particle’s position in the next frame from its track history and then finds new particles in the residual images left after the tracked ones are removed, which lets it reach high seeding densities in time-resolved data. History-based prediction assumes trajectories that are smooth on the time scale of the frame interval; in a granular flow, collisions can reverse a particle’s velocity between two frames. Granular flows pose both difficulties at once: in the same flow some particles collide and rebound violently, so their motion cannot be extrapolated from history, while some clusters move together with almost no relative motion, which removes the velocity contrast and spatial separation that helps disambiguate trajectories in dilute systems. Current methods often struggle to successfully link particle trajectories across frames in both situations.

A distinct PTV family links particles by neighbour consistency: the spring model of Okamoto et al. [30] and the match-probability relaxation methods of Baek and Lee [4] and Ohmi and Li [29] iteratively reinforce candidate links whose neighbours move alike, assuming that small groups of particles move nearly rigidly between frames.

Colour has been explored as a discriminant between otherwise identical particles. Bendicks et al. [5] used dyed tracers to reduce stereo-matching ambiguity in 3D PTV, where rainbow methods encoded depth into a wavelength for volumetric velocimetry (Xiong et al., 2017), an approach recently reviewed by Wang et al. [40]. In granular systems, coloured particles have labelled populations, for example to measure segregation or to separate species in binary mixtures [16], and machine learning has been applied to colour-marked rod tracking in dilute granular gases [34]. To this point, colour remains a population or depth label rather than a tool to track individual particle identities of a single species. Jiang et al. [18] developed a colour PTV that tracks colour-marked particles in a pseudo-two-dimensional fluidised bed by combining Voronoï tracking with relaxation-probability tracking, verified against CFD–DEM; colour there identifies and labels the particles in each image, and the links between frames are made from positions alone. This work is the closest to our proposed method; however, it does not use the rich information provided by colour variation in clusters. In a dense system the arrangement of colours across a group of neighbouring particles encodes structure at a higher level than the colour of any individual particle, so matching that arrangement can resolve links that positions alone leave ambiguous.

Machine-learning-based tracking methods represent the main thrust of recent work. Deep-learning particle trackers such as DeepTrack ([27]) address detection and localisation. Graph neural networks have set the state of the art in fluid PTV: GotFlow3D ([22]) learns 3D flow motion from consecutive particle sets via recurrent graph optimal transport, and GOTrack (Wang et al., 2026) builds a complete PTV framework around it (CNN detector, GNN displacement predictor, optimal-transport linker). These models assume and are trained on tracer particles following incompressible fluid flow, and their transfer to dense granular flows exhibiting correlated, intermittent, and frictional motion has not previously been evaluated. For granular flow specifically, HU-FlowNet (Xu et al., 2025) links particles in a dense drum. The learned optical-flow prior provides only the context of local mean flow, and primary linking uses a Hungarian algorithm. As generalised experimental tools, learned trackers have a further limitation: when trained on simulated flows, they implicitly assume the correctness of the very models that granular experiments are meant to test, and real training data is scarce across the wide parameter ranges those experiments span. This motivates a tracker that needs no training.

We present Colour Bean Tracking Velocimetry (CBTV), a particle tracking method for dense granular flows at displacements approaching the inter-particle spacing, where motion is unpredictable, crowded, or both. CBTV uses particles of at least three distinct colours recorded with a colour camera, and combines position and colour when matching particles between frames. Its core benefit has two parts. First, neighbouring particles in a dense flow move coherently, that is, neither rigidly nor independently of one another, so the arrangement of a small group of particles carries information about where each of its members is, and this larger-scale structure can be used rather than each particle alone. Second, colour makes these group patterns distinguishable from one another, so they remain recognisable at larger deformation between frames than patterns of positions alone, which extends the range of deformation and displacement over which identities can be kept. We call these recognisable patterns of coloured particles, present in successive frames, constellations.

Like conventional PTV, CBTV links detected particles from frame to frame, but it tracks whole particles several pixels across, whose colour is read over their area and matched as patterns among neighbours, rather than point-like tracers located by their brightness. Similar to the Shake-the-Box method, CBTV is multi-pass, but its first pass is different. It matches whole colour constellations as single units, and only the remaining particles require per-particle assignment (§ 2). Committing whole constellations first settles most of the particles in one step and leaves far fewer candidates for the per-particle pass, which is where identities are otherwise lost. Its motion prior is measured from the images by cross-correlation (§ 2.5), and a term built on the spatial pattern of colours around each particle helps to assign the remaining particles. Matching the colour pattern of a whole neighbourhood as one unit allows CBTV to keep identities where per-particle linking becomes ambiguous. Because CBTV tracks whole coloured particles rather than tracers, its name uses bean rather than particle.

The present research makes three contributions: (i) CBTV, a multi-pass, colour-enhanced tracking method for accurate particle tracking in dense granular flow, which matches colour constellations jointly before assigning the remaining particles (§ 2); (ii) a benchmark on 15 discrete-element simulation-based granular flows rendered to video with exact ground truth, on which CBTV and 12 baseline tracker configurations (ten methods) are compared across 59 flow–displacement cases covering 0.006–9.0 diameters/frame, with detection treated as a separate step so that only the linking of particle identities from frame to frame is compared (§ 3); it shows that CBTV’s advantage lies in a band of assignment ambiguity, measured from the ground truth as the displacement per frame relative to the spacing between neighbouring particles, and that the displacement at which learned trackers overtake classical ones depends on the flow (§ 4); and (iii) a characterisation of scaling with particle count and image resolution.

Method

CBTV has two stages: detection of the particles and their colours in each frame, then linking of the detections into trajectories. Linking proceeds in two passes. A motion prior, that is, a prediction of each particle’s displacement between the two frames (§ 2.5), gives the starting point. The first pass matches colour constellations as single units and commits most of the particles at once. The second pass assigns the remaining particles one by one. Colour is measured in CIELAB space throughout, because equal numerical differences in that space are close to equal perceived differences. § 4.6 reports what each component contributes, measured by removing it.

Colour palette design

Because CBTV identifies particles partly by their colour, the colour of every particle must be classified reliably from the image, despite camera noise and variations in lighting. The palette is chosen with this in mind. For a given number of colours NcN_c (Nc=3N_c = 3 in this study), the colours are placed as far apart as possible in CIELAB space, so that the smallest difference between any two of them is well above the colour noise of the camera and lighting. Three colours are a practical compromise. They are far enough apart to be read reliably on each particle, and the number of possible colour arrangements among nn neighbours grows as NcnN_c^n, so even a small neighbourhood carries a colour pattern that is unlikely to recur nearby.

Particle detection

Detection is separate from linking particle identities from frame to frame. In every comparison of § 4, every tracker receives the same detections, so only the linking of particle identities from frame to frame is compared. Because a particle’s colour is read from the pixels it covers, CBTV needs particles that are several pixels across rather than point-like tracers; the smallest particles in the corpus are 11.5 px in diameter (§ 3.1). For the benchmark we use Cellpose-SAM [33], a general-purpose segmentation model, without any training on our videos. Its detection F1F_1, the harmonic mean of precision and recall when detections are matched one-to-one to ground-truth particles within 5 px (10 px on the doubled-resolution large chute, whose particles are twice as large), is 0.9998 on the collision flow and ≈1.00 across the corpus. Each particle’s colour is read from the CIELAB values of the pixels inside its detected mask. Because detection is a separate step and every tracker receives the same detections, the choice of detector does not change the order of the trackers. Two lighter detectors, a Hough transform with a Bayesian colour classifier and a multi-channel watershed, are available; they detect fewer particles (lower F1F_1) and give the same order of trackers.

Colour-neighbourhood matching

CBTV identifies a particle by the arrangement of the particles around it, that is, by their positions relative to it and by their colours. Two implementations were built. The first is an image-space kernel, a three-channel CIELAB template that combines an edge and intensity map with colour-weighted neighbour blobs, matched by normalised cross-correlation. The second, the structural matcher, compares the detected neighbour positions and colours directly as point sets, without rendering a template. With accurate detections the two are equally accurate, and the structural matcher has no template to build, so it is the default; the kernel is kept as a documented alternative. Two particles of the same colour are distinguished by the different colour arrangements of their neighbours. In the final pipeline this matcher is a secondary term in the assignment cost (weight wk=0.1w_k = 0.1, § 2.6). The constellation matching of the next subsection is the primary mechanism.

Colour constellations

The main step of linking is the joint matching of colour constellations, small, compact groups of neighbouring particles (typically 5–12) whose colour pattern is recognisable from one frame to the next. A neighbour graph over the detections proposes the groups. Each group is matched to the next frame as a single unit. Starting from the per-particle motion predictions of § 2.5, candidate local affine transforms are scored on how well they carry the members’ positions and colours onto detections in the next frame. A match that clears a confidence threshold commits all members at once and removes them from the assignment that follows. Colour contributes most at this step. A constellation is matched as one unit, under a local affine transform, on the colours of its members together with their relative positions. A candidate match must therefore reproduce both the colour sequence and the spatial arrangement, which a wrong group of nearby particles is far less likely to do than to pass per-particle colour checks one at a time, and one match settles many particles at once. Because the members are committed together to distinct detections, a committed group cannot swap members among the detections it was given. Identities within a group are exchanged only if the whole match is wrong, which is rare. Like the neighbour-consistency methods of § 1, this assumes that neighbouring particles move coherently between frames. Because each match allows a local affine transform, a group may stretch and shear and still be matched. The neighbour-consistency methods relax per-particle match probabilities on grey-scale positions, whereas CBTV commits colour-labelled groups as single units. The number of groups proposed per frame grows with the particle count NN as min⁡(2048,max⁡(256,⌈0.324 N⌉))\min(2048,\max(256,\lceil0.324\,N\rceil)). A fixed budget that was too small for large scenes was the largest single loss of accuracy found during development (§ 4.6). With the motion prior of § 2.5, constellations commit about 94% of the particles per frame on the collision flow at moderate displacement, and 99.99% of those commits are correct; on the large smooth chutes they commit 63–77%, and the rest go to the per-particle assignment. The constellation stage involves no training.

Displacement-gated dense motion prior

Each track gets a prediction of where its particle will be in the next frame. The prediction starts the constellation search and centres the search gate of the per-particle assignment, the radius within which detections are considered as candidates. The prediction comes from one of two sources, chosen per track by its observed speed. Slow tracks use a PIV-style cross-correlation field computed from the raw pixels of consecutive frames. In the flows studied here, the slow tracks are the particles of the dense, slowly creeping regions, such as the bulk of a chute flow or of a dense block, and the correlation field measures their collective motion directly in the current pair of frames rather than extrapolating it from the past. Fast tracks use a constant-velocity extrapolation of their own history, which is better than image correlation once a particle has moved far enough that the local image pattern no longer matches. The threshold between slow and fast tracks is a displacement of about 0.5 diameters per frame (Δx/d≈0.5\Delta x/d \approx0.5), chosen by a parameter sweep. Three more elaborate rules for choosing between the two sources, based on the sharpness of the correlation peak and on the agreement between a track’s extrapolation and the motion of its neighbours, were tested against pre-registered criteria and did not improve on the fixed threshold. The prior uses only the images, and no ground truth enters it.

Residual assignment

Particles that no constellation has committed are assigned one by one. A cost matrix over the remaining track–detection pairs is solved by the Hungarian algorithm [20]. The cost of pairing track ii with detection jj is

Cij=wdd^ij+wcΔEab∗(i,j)+wk(1−Sij),(1)C_{ij} = w_d \hat{d}_{ij} + w_c \Delta E^{*}_{ab}(i,j) + w_k (1-S_{ij}), \tag*{(1)}

with wd=0.8w_d = 0.8, wc=0.1w_c = 0.1, wk=0.1w_k = 0.1, where d^ij\hat{d}_{ij} is the distance between the predicted position of track ii and detection jj divided by the search gate, ΔEab∗(i,j)\Delta E^{*}_{ab}(i,j) is the CIE76 colour difference in CIELAB space between the colour of track ii and that of detection jj, and SijS_{ij} is the structural matcher score of § 2.3. The constellation search gate is set in particle diameters, and the remaining parameters in pixels; pixel units were found to be adequate at the image scales studied (§ 4.7).

Track lifecycle: coasting, exits, and colour memory

A track that misses its particle for one or more frames must be carried across the gap without losing its identity, and three mechanisms handle this. First, a track that has lost its particle is moved each missed frame by the motion prior of § 2.5 instead of staying at its last position (we call this coasting), so that it stays close to the particle and re-acquires it at the next detection. Second, a coasting track that leaves the field of view is ended, rather than left waiting at the border, where it would otherwise capture the next particle to enter. Third, the colour term of Equation (1) compares each detection with the colour the track stored at its last match, rather than with the current frame at the track’s previous position, which is mostly background once a particle moves more than about half a radius per frame. All three mechanisms were added after the configuration first benchmarked had been frozen. We call the configuration with them the released configuration (BT in the tables and appendices) and the earlier one the frozen configuration (v4 ship); everywhere else, “CBTV” means the released configuration, and the two are compared in § 4.8 and § 4.9. The pre-registered selection of the released configuration, a fourth mechanism that was tested and rejected, and the sensitivity of the released configuration to the parameters of these mechanisms are given in Appendices J and I.

Validation methodology

Synthetic granular flow corpus with exact ground truth

Quantitative evaluation of tracking in dense flows requires per-particle ground truth, which manual annotation cannot provide at scale. We therefore validate on videos rendered from discrete-element simulations: the flows are simulated in LAMMPS [36], and the spheres are rendered to video, each with a fixed colour from the three-colour palette, on controlled backgrounds, and with controlled noise. The corpus has fifteen flows, chosen to cover mechanically different situations: an impulsive two-body collision; sheared chutes at $25^\circ/35^\circ/50^\circ; a draining wedge hopper; a chute flowing over a bump; four large-canvas smooth chutes spanning particle count (N = 594 to 2373) and image scale (11.5 to 24.3 px/diameter); a dense block translating through a narrow horizontal camera strip, with particles entering and leaving the window (the strip view); a coherent swirl, gravity flow past a cylinder; and a rotating drum (Froude number 0.68), simulated at two rotation rates (two flows) with gravity scaled to hold the Froude number fixed. The displacement per frame is controlled by subsampling the frames; its median over the particles ranges from 0.006 to 9.0 diameters per processed frame. Each case is one flow at one sampling step, and every case is labelled by its measured displacement, not by its nominal step. Figure 1 sketches the geometry of every flow, and Figure 2 shows four of them in motion.

Image not extracted

Figure 1. Geometry of the flows in the corpus, traced from the wall particles and the moving particles of the simulations. Arrows show the main direction of motion and gg the direction of gravity. (a) Impulsive collision; (b) inclined chutes at 25∘25^\circ, 35∘35^\circ, and 50∘50^\circ; (c) chute over a bump; (d) wedge hopper; (e) large smooth chutes; (f) dense block translating through a narrow camera strip; (g) swirl; (h) gravity flow past a cylinder; (i) rotating drum. Dashed rectangles mark the camera window where it is smaller than the simulated domain. Short thin arrows in (b) and (d) indicate velocity fluctuations, and the profile in (b) is a schematic of the shear across the layer. The identifiers under each panel are those used in the tables and in Appendix § 9.

Image not extracted

Figure 2. The four flows of the fine displacement scan (Figure 3), drawn from the ground truth at four frames each. Every particle is drawn in its colour label; arrows give the ground-truth velocity (finite difference over one frame) of a subset of particles, coloured by speed in particle diameters per frame (dd/frame) on one logarithmic scale shared by all rows. Arrow length is scaled per row (key arrow in the first panel of each row) and is not comparable between rows. The chute rows show a 30-diameter window; every flow of the corpus is shown in Appendix § 9. Top row, the impulsive collision: a dense block moves into a second block at rest (frame 120), the two meet (180), merge into one compacted block (210), and eject particles (280); at contact the velocities reverse, so a constant-velocity extrapolation predicts the wrong direction.

To score the trackers, the ground-truth positions from each simulation are mapped onto the pixels of its video by a fitted transform. We check each transform by comparing every detection with its matched ground-truth position: the mean offset must be close to zero and the spread small. We check the offsets, and not only the detection F1F_1, because a constant offset would lower every tracker’s score by the same amount and would therefore not show up in the rankings. With this check, four flows that an earlier calibration had set aside as limited by detection were found to have detections within a quarter of a pixel of the ground truth (median residual; detection F1F_1 of 1.0000 at an 8 px match radius), and they were kept in the corpus. The first drum flow failed the check under the earlier calibration because the fit searched too narrow a range of image scales and settled on a wrong transform. With the range widened, it passes (detection F1F_1 of 1.0000 at 8 px, mean offset 0.27 px), and both drums are included.

Metrics and protocol

We report the particle tracking ratio (PTR) and HOTA [24], the standard higher-order tracking accuracy that combines detection and association. PTR is defined over every ground-truth particle and every pair of consecutive frames as the fraction of transitions in which the particle keeps the same track identity. A transition fails if the particle’s track changes (a swap), if its track ends (lost), or if the particle has no track at all (missing), so PTR penalises lost and untracked particles as well as swaps. It corresponds to the yield, or recovery ratio, used in PTV, the number of correct links divided by the number of ground-truth links [4]. PTR is the main measure in the figures and in the claims made across the corpus; HOTA is given for individual cases, and the corpus-wide comparison in HOTA is in Appendix § 10. Identity-swap counts are reported where they say something about the mechanism. Four protocol rules are applied throughout. First, detection is a separate step. Every tracker receives the same detections from the same detector, so any detection error affects all trackers alike, and only the linking of particle identities from frame to frame is compared. By default, CBTV refines the centre of each detection it receives by fitting a circle to the particle’s edge; setting its output positions back to the shared detections changes its HOTA and PTR by at most 0.005 on every case, so the tables report CBTV as run. Second, one configuration per tracker. Every tracker, CBTV included, was run with a single configuration on every case, with no per-video or per-step setting. The only quantities that change between cases are rules computed from the input (CBTV’s cluster budget, radius window, and motion prior, and the search radius of the relaxation tracker) and one image-scale rule that doubles the pixel gates of every tracker on the doubled-resolution chute. The baselines were tuned on the collision flow at the smallest displacement, and the classical linkers keep the fixed search radii chosen there, 13 px (about one particle diameter on that flow) for trackpy and LapTrack and 26 px (about two diameters) for btrack. CBTV’s configuration was tuned on more of the corpus: its motion-prior gate on the collision flow at steps 1, 4, and 8, checked on two chute cases, and its cluster budget and weights (§ 4.6) on three large-chute cases, which are labelled as tuning cases wherever they are quoted. The released configuration (§ 2.7) was selected on the corpus and confirmed on

two flows simulated afterwards (§ 4.9). Third, cases are ordered by measured displacement. Fourth, deterministic trackers are run once (for a given set of detections, CBTV gives identical tracks on CPU and GPU; see Reproducibility and code availability), stochastic trackers are run at least three times, and every failure is reported.

The baseline tracking methods

CBTV is compared with nine baseline tracker configurations from four families (eight methods, one of them also in a colour-augmented variant), each run as a linker only, on the shared detections: classical proximity and predictive PTV (trackpy nearest-neighbour linking and its nearest-velocity predictor, trackpy-NVP ([11, 2], LapTrack ([13])); Bayesian tracklet linking (btrack ([38], run with and without a per-particle colour feature); video multi-object trackers (ByteTrack (Zhang et al., 2022), OC-SORT ([7]), and the learned-appearance Deep-OC-SORT ([25]), fed circle-to-box conversions); and the graph-neural-network PTV linker GOTrack ([22] (Wang et al. 2026)), applied zero-shot, that is, as pretrained on tracer particles in fluid flow and not fine-tuned on granular flow; where a frame held fewer detections than the 512 neighbours its correlation step expects, that number was set to the frame’s detection count. Each family relies on a different cue (proximity, velocity extrapolation, or learned appearance), so the comparison shows which cue works in which flow. Two further learned trackers complete the set of baselines under the same protocol. Trackastra ([14]), a transformer linker for given detections, is applied zero-shot on all 59 cases2. The point tracker CoTracker3 ([19]) must be seeded with the detections of frame 0, so it runs only on the 53 cases in which no particle enters later (Appendix § 15). The neighbour-consistency family is represented by our own implementation of relaxation PTV from the papers ([4, 29]), since no public code exists. It is run without colour and with the same three-bin colour label that btrack receives; colour either gates the candidate links or down-weights colour mismatches. The colour PTV of Jiang et al. [18] combines relaxation with Voronoï tracking and uses colour only to detect and label particles; its relaxation linker corresponds to the variant run here without colour, while the colour variant adds colour to the linking. The method itself has no public code or public validation images, so it is not compared directly. The headline colour mode (gate) was fixed on the tuning case (collision, step 1) by a pre-registered rule before the corpus was run; the two modes tied exactly on HOTA and PTR there, and the pre-registered order decided. Tuning on that case was already at the ceiling, so the defaults were kept for every mode. All 59 cases completed in every mode (mean HOTA 0.533 without colour, 0.565 with the gate, 0.564 with the weight). Colour enters explicitly only for btrack and relaxation PTV, in their colour variants. Deep-OC-SORT and CoTracker3 read the colour images and so see colour implicitly; Trackastra reads grey-level frames; the proximity, motion-only video, and graph-neural-network linkers use position alone. The 12 baseline tracker configurations counted throughout are these ten methods, two of them also in a colour variant; CoTracker3, which runs only on the cases in which no particle enters, is reported separately. No single tracker is the most accurate on every flow, and an average over the corpus would depend on the mix of flows chosen; the claims below are therefore made per regime, measured as described next.

Measured regimes of assignment ambiguity

Each case is assigned to a regime by two quantities measured from the ground truth alone. For a particle at position xp\mathbf{x}_p in processed frame pp, whose nearest neighbour is at distance sps_p, the crowding is the displacement per processed frame in units of that spacing,

ϕ=∣xp+1−xp∣sp,(2)\phi= \frac{\lvert\mathbf{x}_{p+1} - \mathbf{x}_p \rvert}{s_p}, \tag*{(2)}

and the unpredictability is the error of a constant-velocity prediction in the same units,

ε=∣xp+1−(2xp−xp−1)∣sp.(3)\varepsilon= \frac{\lvert\mathbf{x}_{p+1} - (2\mathbf{x}_p - \mathbf{x}_{p-1}) \rvert}{s_p}. \tag*{(3)}

For each case we take the median and the 90th percentile of each quantity over the particles and sampled frames. The threshold on both is 0.5, because once the true partner is displaced by more than half the nearest-neighbour spacing from the previous position or from the prediction, a wrong neighbour can be as close as the true one; this is the spacing-to-displacement ratio of about two found by Malik et al. [26]. A case is unpredictable if the median of ε\varepsilon exceeds the threshold. Otherwise, it is crowded if the median of ϕ\phi exceeds it (partly unpredictable if the 90th percentile of ε\varepsilon does), marginal if only the 90th percentile of ϕ\phi does, and unambiguous if neither does. Cases on which no tracker other than CBTV reaches a HOTA of 0.15 are reported separately as floor cases. Because the quantities are medians over a whole case, they do not register short impulsive events such as the few contact frames of the collision, which the errors per transition do capture (§ 4.2). This rule was fixed from the ground truth before the tracker scores were read for this analysis.3 The descriptive flow groups used in the tables and figures (collision, agitated and smooth chutes, strip view, swirl and cylinder flow, drums) were assigned by flow identity after the results were known. They are kept because they explain mechanisms, and the corpus-wide claims rest on the measured regimes.

Results

The crossover displacement depends on the flow

The displacement per frame does not by itself determine how well a tracker performs, and the crossover displacement above which learned methods overtake classical ones depends on the flow. First, at matched ≈1\approx1 diameter/frame, outcomes differ by up to ∼0.7\sim0.7 HOTA depending on the flow, and the leading tracker changes (Table 1; Figure 4 shows the PTR of every case). Along the chute series the inclination increases, and with it the agitation. The granular temperature (the fluctuating part of the velocity ([21])), measured from the ground truth, rises from 0.0031 to 0.0043 to 0.0053 d2/frame2d^2/\mathrm{frame}^2. Along this series the HOTA of the velocity-extrapolating tracker (trackpy-NVP, called the extrapolator below) falls sharply while that of CBTV changes little. On the smooth large-canvas chutes at the same displacement the extrapolator performs well, and the two are within a few hundredths (0.741/0.757 and 0.778/0.750). Second, a fine displacement scan (seven steps per video on four flows, against all 12 baseline trackers; Figure 3 shows that where the crossover lies depends on the flow, not on the pair of trackers. On the impulsive collision CBTV ties at the first 4 steps and leads from ≈0.7 d\approx0.7\,d, with a margin that grows to +0.14. On the agitated chute CBTV leads from ≈1.2 dd to ≈1.9 dd. On the smooth and structured flows the extrapolator leads or ties at low displacement, and CBTV overtakes it at the largest steps (from ≈0.7 dd on the bump chute, at ≈1.2 dd on the smooth large chute). The agitated chute has the lowest HOTA of the four flows for every tracker, already at its smallest step (Figure 3). Identity retention there is nevertheless high (PTR 0.989 for CBTV and 0.992 for trackpy-NVP, Table 1), so the low HOTA is not caused by linking errors.

Image not extracted

Figure 3. Fine displacement scan. Each panel plots HOTA (higher-order tracking accuracy) against the measured displacement per frame, Δx/d\Delta x/d, on a linear axis whose range differs between panels, for one flow at seven subsampling steps: the impulsive collision, the agitated 25° chute, the structured bump chute and the smooth large chute. Lines: CBTV (green, bold), the five strongest other trackers (trackpy-NVP, GOTrack (GNN), Deep-OC-SORT, Trackastra, Relax-PTV with colour), and the best of all 12 baseline trackers at each step (dotted grey); leads and ties in the text are measured against all of them. Where the crossover lies depends on the flow: on the collision CBTV ties at the first 4 steps and leads from ≈0.7 dd; on the agitated chute it leads from ≈1.2 to ≈1.9 dd; on the smooth and structured chutes the extrapolator leads or ties at small steps, and CBTV leads at the largest.

Flow (Δx/d\Delta x/d)charactertrackpy-NVPCBTVCBTV − best other tracker [95% CI]
Chute 25° (0.62)agitated0.541 (0.992)0.540 (0.989)−0.0014 [−0.0022, −0.0008]
Chute 35° (0.84)more agitated0.359 (0.720)0.466 (0.982)−0.0030 [−0.0049, −0.0019]
Chute 50° (1.13)most agitated0.295 (0.656)0.405 (0.980)−0.0016 [−0.0028, −0.0009]
Large chute (1.15)smooth0.757 (0.820)0.741 (0.785)−0.0158 [−0.0373, +0.0060]
Large chute (1.16)smooth0.750 (0.833)0.778 (0.894)+0.0285 [−0.0016, +0.0365]

Table 1. HOTA (PTR) at ≈1 diameter/frame for trackpy-NVP and CBTV along the chute series, in order of increasing inclination and agitation, and on the smooth large chutes. Along the chute series the HOTA of the extrapolator falls sharply while that of CBTV changes little; on the smooth large chutes the two are within a few hundredths. Last column: HOTA of CBTV minus that of the best other tracker in that case, with a paired frame-block bootstrap 95% CI (Appendix § 20).

Image not extracted

Figure 4. Particle tracking ratio (PTR) against the measured median displacement per processed frame, Δx/d\Delta x/d (logarithmic axis; its range differs between panels), for CBTV and five other trackers (trackpy-NVP, GOTrack (GNN), Deep-OC-SORT, Trackastra, Relax-PTV with colour; one marker per tracker and case), across all 59 cases of the 15-flow corpus. One panel per flow class: the impulsive collision (7 cases), the agitated chutes and hopper (16), the smooth and structured chutes (23), the strip view, swirl and flow past a cylinder (7), and the rotating drums (6). Displacement alone does not order the outcomes, and the leader changes with the character of the flow. The same comparison in HOTA is in Figure 10.

The outcomes are instead organised by assignment ambiguity, that is, by whether more than one

detection is compatible with a track when it is matched, which the measured regimes of § 3.4 quantify. Ambiguity arises in two ways, from unpredictable motion and from crowding, and the corpus contains a test of each.

Impulsive collision and agitated shear flows

The first source of ambiguity is failure of motion prediction. The clearest example is the impulsive two-body collision, in which a dense block of particles is driven into a second block at rest; the two meet, compact, and eject particles (Figure 2, top row). At impact a velocity extrapolation points the wrong way, because particles reverse and scatter, so every Kalman- and velocity-based tracker degrades. CBTV’s cross-correlation prior measures the motion after impact, and its colour constellations resolve the ejected particles. CBTV leads or ties every other tracker at all seven displacements of the fine scan and leads from ≈0.7\approx0.7 diameters/frame (Table 2, Figure 3). By paired 95% confidence intervals it is ahead at all seven steps, though by less than 0.005 HOTA at the smallest ones. On one case (step 6, 0.81 diameters/frame) the lead comes from association rather than from per-frame velocity accuracy. After impact the best other tracker’s per-frame velocity errors are smaller than CBTV’s, while CBTV keeps identities linked better over the whole sequence (Appendix § 11). Because the contact lasts only a few frames, the case medians of ε\varepsilon stay below the threshold of § 3.4, and the measured regimes place the collision cases by crowding alone, from unambiguous at the smallest step to crowded at the largest.

trackerΔx/d=0.14\Delta x/d = 0.140.581.22
CBTV1.000 (1.000)0.995 (1.000)0.904 (0.974)
GNN0.992 (0.999)0.877 (0.987)0.610 (0.895)
Deep-OC0.994 (1.000)0.772 (0.906)0.388 (0.507)
trackpy-NVP0.999 (1.000)0.787 (0.755)0.654 (0.641)
ByteTrack0.999 (1.000)0.652 (0.655)0.363 (0.424)
Trackastra0.999 (1.000)0.992 (0.999)0.767 (0.773)
Relax-PTV0.999 (1.000)0.966 (0.990)0.604 (0.613)
Relax-PTV + colour0.999 (1.000)0.978 (0.991)0.639 (0.635)
CBTV − best other tracker, HOTA [95% CI]+0.0005 [+0.0002, +0.0007]+0.0030 [+0.0009, +0.0036]+0.1368 [+0.0057, +0.2354]

Table 2. Impulsive collision benchmark, detection-matched, best of each family shown. Each entry: HOTA (PTR). Trackastra is the zero-shot transformer linker; Relax-PTV is our relaxation-PTV implementation (colour: gate mode); the point tracker CoTracker3 is reported with the other cases in which no particle enters (Appendix § 15). CBTV is run once (bit-reproducible on a given platform; Appendix § 16).

At 1.22 diameters/frame CBTV keeps 97.4% (95% CI 95.4–98.7%) of the ground-truth identities per transition, the highest of all trackers (GNN 0.895, velocity extrapolation 0.641, learned appearance 0.507, Trackastra 0.773). Its HOTA margin over the best other tracker, the zero-shot transformer linker Trackastra, is +0.137 (+0.294 over the GNN). The measured regimes place the agitated shear cases (≈1\approx1 diameter/frame on the 35°/50° chutes and the draining hopper) with the crowded, predictable cases, since their constant-velocity prediction error is close to zero, so their ambiguity comes from crowding. On HOTA the colour relaxation tracker is marginally ahead of CBTV on the 35° and 50° chutes (CBTV minus relaxation −0.0030 [−0.0049, −0.0019] and −0.0016 [−0.0028, −0.0009], significant by paired 95% CIs). On the hopper CBTV is marginally ahead on HOTA (+0.0031) but keeps fewer identities per transition (PTR −0.0132). Both are above every other tracker (CBTV 0.466/0.405/0.440), and the relaxation tracker keeps more identities per transition (0.985–0.989 against CBTV’s 0.974–0.982; extrapolator 0.656–0.777).

Ambiguity from crowding (dense flows)

The second source of ambiguity is crowding, that is, a displacement per frame that is a sizeable fraction of the spacing between neighbouring particles, so that a per-particle link can reach a neighbour even when the motion is predictable. Apart from the floor cases, all 11 cases on which CBTV keeps more identities than the best other tracker lie in this band, at a median ϕ\phi of 0.57 to 1.71 (Figure 5). They are the collision at step 8, the large smooth chutes at steps 5 to 8, the bump chute at step 8, the 25° chute at steps 2 and 3, the faster drum at step 1 and the slowed drum at step 4. In these cases a single link is ambiguous, but a colour constellation is matched and committed as a group, so a wrong match would have to reproduce the colour arrangement of the whole group, and its members keep their identities together. Per ground-truth transition, pooled over the 12 cases with per-link outcomes, CBTV keeps 0.96 of the links at ϕ\phi between 0.5 and 1.2, against 0.90 for the extrapolator and 0.80 for the GNN (Appendix § 12). On the strip view at ≈0.56\approx0.56 diameters/frame, where the particles move at nearly constant velocity, the velocity-extrapolating tracker makes 47 identity swaps, because it estimates each velocity from its own earlier links and a wrong link carries a wrong velocity into the links that follow. CBTV makes 0 swaps there and is level with colour relaxation (−0.001-0.001 HOTA), and neither of the two depends on its own velocity history.

Image not extracted

Figure 5. Measured regimes of the corpus. Each marker is one case, placed by the median crowding ϕ\phi, the displacement per processed frame in units of the nearest-neighbour spacing (horizontal axis), and by the 90th percentile of the unpredictability ε\varepsilon, the constant-velocity prediction error in the same units (vertical axis), both on logarithmic axes. Colour gives the PTR of CBTV minus that of the best other tracker in that case, clipped at ±0.15\pm0.15. Marker shape gives the flow group; hollow markers are floor cases, on which no tracker other than CBTV reaches a HOTA of 0.15. The dashed lines mark the threshold of 0.5 spacings on both axes (§ 3.4); cases whose prediction error is zero (rigid translation) are drawn on the dotted line at the bottom. Apart from the floor cases, CBTV’s leads lie at ϕ\phi between 0.57 and 1.71.

Both sources of ambiguity lead to the same conclusion. Trackers that use neighbour context give the most accurate tracking, or close to it, where per-particle assignment is ambiguous, because motion is unpredictable or because the displacement approaches the inter-particle spacing, up to about 1.71 spacings per frame. Where assignment is unambiguous, other cues perform as well or better. Colour relaxation’s margin over CBTV tends to grow with local motion rigidity (Appendix § 19), and CBTV’s constellations lead where local motion is least rigid, as in the impulsive collision at large steps. Assignment ambiguity thus covers both the classical spacing-to-displacement criterion of Malik et al. [26] (crowding) and flow predictability, and the cases on which joint colour-constellation assignment keeps more identities than the per-particle linkers lie within that range. Figure 5 places every case in the measured regimes of § 3.4, coloured by the difference in PTR; the same comparison in HOTA is in Appendix § 10. Apart from the floor cases, CBTV keeps more identities than the best other tracker on 11 cases, with a PTR up to 0.162 higher, and all of them lie at a median displacement of 0.57 to 1.71 nearest-neighbour spacings per frame, where links are ambiguous but the motion is still largely predictable. On the 20 unambiguous and marginal cases, CBTV’s PTR is at most 0.034 below that of the best other tracker. On the 7 inclined-chute cases at 2.3 spacings per frame and more, the motion is close to plug flow and predictable, yet CBTV’s PTR is up to 0.70 lower, and on 4 of them it keeps no identities at all (§ 4.4). A difference within ±0.005 counts as a tie, and because the best other tracker is the maximum over all other tracker configurations in each case, chosen after the fact, CBTV’s deficits are upper bounds.

Where CBTV is not the best tracker

In three regimes other methods are more accurate than CBTV; we report each with its mechanism. Smooth flow at low displacement is tracked best by velocity extrapolation. On the smooth and structured chutes, CBTV trails by at most 0.025 at low displacement (below 1.15 diameters/frame, mostly to Kalman and Bayesian linkers at the finest sampling), and from 0.62 diameters/frame it is ahead on 6 of the chute cases by up to +0.102, significantly so on 2 (paired 95% CIs). The reason is that CBTV now coasts an unmatched track along the measured motion (§ 2.7), so it carries extrapolation inside it. The external extrapolator keeps its advantage only where every gate contains a single candidate and nothing needs to coast; it also runs at 11–244 ms per frame. At extreme displacement (≳2.5\gtrsim2.5 diameters/frame) CBTV loses identities, and trackers that compensate for the bulk motion are more accurate. Deep-OC-SORT (0.28–0.43 on the 2.5–5 dd chute cases; beyond that it too falls to the floor value) leads every such case above the floor, while the geometric linkers are at ≤0.14\le0.14 (CBTV at most 0.28 above 3 dd), and their identity retention per transition there exceeds the geometric methods’ by up to 0.7. At ≈1.9\approx1.9 diameters/frame CBTV leads the learned-appearance tracker on the agitated chute (+0.048 HOTA), and at 2.5 it trails it (−0.011), so the boundary lies between 2 and 2.5 diameters/frame. CBTV’s deficit there comes from its motion prior. The cross-correlation field measures displacements up to half its correlation window, 35 px or about 3 particle diameters on these chutes. Where the bulk displacement per frame is larger, the prediction for the first frame pair falls to zero motion, no constellation commits, and each track stays at its old position and links to the particle that flows into it. Widening the search gate does not help, because the gate stays centred on that old position. Deep-OC-SORT keeps identities on these cases because its camera-motion compensation estimates the bulk shift of the bed and so centres its gate correctly. GOTrack keeps identities only within its own capture range. CBTV has a whole-frame bulk-motion pass, which is off in the released configuration. Tested on two of these cases, it corrects the first prediction, but identities are still lost within about ten frames, and it lowers accuracy on other cases, so it stays off. Extending the range of the motion prior remains future work. Dense coherent rotation is a weakness. In the dense early phase of the swirl CBTV changes track ownership often

(2,114 identity changes against the extrapolator’s 29) even though the aggregate scores stay close. One further result is unfavourable to CBTV. On the flow past a cylinder, which partly occludes the particles, the learned-appearance tracker scores higher at fine sampling (0.952 vs 0.941).

Identity swaps and flow statistics

Because identity errors bias the statistics computed from the tracks, we computed the local mean velocity, granular temperature, strain rate, and vorticity from each tracker’s output on the shared detections (CBTV’s positions set back to the detections, § 3), with a Gaussian kernel of width 1.5 diameters on 12 cases, and compared each field with the same field computed from the ground truth (Figure 6). For the trackers that keep most identities below one spacing per frame, the local temperature error is 2 to 234 times the mean-velocity error of the same tracks (median 14), since the temperature is a second moment of the velocity that a few wrong vectors inflate, and on the chutes at step 1, where the true temperature is below 6% of the kinetic energy of the mean flow, even the detections linked by their true identities overstate it by 9 to 475%. In the 3 cases in which CBTV keeps more identities than every other tracker (the collision at step 4, the drum at step 1, and the slowed drum at step 4, ϕ\phi from 0.38 to 0.97), its temperature, strain-rate, and vorticity errors are the smallest, except for the temperature on the slowed drum at step 4, where the GNN’s error is as low (0.92 against 0.98, overlapping intervals) while it keeps 0.30 of the identities against CBTV’s 0.71; against the extrapolator, CBTV’s temperature error is lower by a factor of 2 to 80. On the 5 chute cases within the range of its motion prior, where its PTR is within 0.030 of the best tracker’s and ties it on 3 of them, CBTV’s temperature error is 3 to 31 times that of the best tracker (the extrapolator on the 25∘25^\circ and bump chutes, Deep-OC-SORT on the large chute). That excess comes from a few long links made where CBTV re-acquires a particle after coasting (§ 2.7): 1 to 29 links per case of about two diameters (median 1.7 to 2.3 dd, against a true step whose 99.9th percentile is at most 1.4 dd), all within 20 diameters of the inflow or outflow, against at most 1 for the extrapolator. That band holds 37 to 40% of the field but 92 to 99.9% of CBTV’s squared temperature error on these cases, and in the interior its error is within a factor of 1.2 of the detection floor on four of the five cases and 3.6 on the bumpchutesatstep4. Beyond two spacings per frame no tracker recovers the temperature (local errors of 64% or more), and within that range CBTV improves the measured temperature, strain rate, and vorticity on the flows where it keeps more identities than the alternatives; where another tracker keeps as many, its coasting re-acquisitions at the chute ends make its temperature less accurate.

Image not extracted

Figure 6. Flow statistics as an experiment would obtain them. For each of the 12 cases, ordered by crowding ϕ\phi (median displacement over nearest-neighbour spacing; the shaded band is the crowded regime of § 3.4, and the cases right of the dashed line lie beyond the range of CBTV’s motion prior): relative RMS error of the local granular temperature (top) and of the local mean velocity (middle), over frames and grid points holding at least three ground-truth particles within the kernel of width 1.5 diameters, and the PTR of the same tracks (bottom). Marker shape gives the tracker; CBTV’s positions are set back to the shared detections. Hollow markers are trackers whose PTR is more than 0.05 below the highest in that case or whose coverage of the ground-truth grid is below 98%, so a low error obtained by dropping particles is marked. The grey bar is the error of the shared detections linked by their true identities, the localisation floor; whiskers are 95% block-bootstrap intervals. Relax-PTV was not run on the drums.

Mechanism attribution

Each component was ablated on the collision benchmark. Constellation matching is the largest single contributor (+0.19 HOTA at 0.58 dd/frame over the same pipeline without it). That ablated pipeline (a dense motion prior followed by Hungarian assignment) is structurally analogous to HU-FlowNet ((Xu et al., 2025)), which links particles in a dense granular drum with an optical-flow prior and the Hungarian algorithm and uses no colour for identity; HU-FlowNet itself was not run, and its learned flow field differs from the cross-correlation field used here. Colour is essential to the constellation matching. A colour-blind constellation matcher drops the commit accuracy from 0.964 to 0.681 at 1.22 dd/frame. A baseline result also indicates that colour helps as a pattern over neighbours rather than as a label per particle. Adding a plain per-object colour feature to the Bayesian tracker destabilised its optimiser on every full-length case attempted. Where prediction fails, the measured-motion prior has a large effect. In an ablation on the code of that development stage, it lifts collision HOTA from 0.735 to 0.894 at 0.58 dd/frame, and replacing it by velocity

extrapolation in the released configuration costs 0.04 to 0.15 HOTA at 0.58 dd and 0.10 to 0.24 at 1.22 dd, depending on the matcher. Across the corpus the measured prior is better or equal on 16–17 of 18 cases within the ±0.005 tie band; the exceptions are ≤0.009 on floor cases (v4 HOTA < 0.15; Appendix § 18). Inside CBTV, extrapolation fails through feedback. One swap corrupts the shared prior, so the two mechanisms do not combine well in one pipeline, even though extrapolation performs well as a stand-alone tracker on smooth flow. The largest gain during development came from the number of constellations proposed per frame. A fixed per-frame cluster budget silently left up to a third of the particles in larger scenes to the per-particle matcher. Raising and reweighting it recovered 0.07–0.15 HOTA and improved or tied 27 of 30 original cases; scaling it with the particle count (§ 2.4) added a further 0.21–0.25 on the largest scenes. This removed what had earlier appeared to be an intrinsic decline of CBTV’s advantage with particle count, which we therefore retract as an artefact of the budget.

Scaling, resolution and compute

With the scaled cluster budget, CBTV’s accuracy holds as the particle count grows: at N=2373N = 2373 it matches the best other tracker on the smooth chute at moderate displacement (+0.0160 [−0.0075, +0.0290], not significant) and scores above the zero-shot GNN on all 16 large-canvas cases (the four large chutes at every tested step). Robustness to image resolution was tested from both directions. Expressing the pixel-space parameters in particle diameters gives identical results at the scale they were tuned at and neutral or worse results elsewhere, so pixel units are not what limits CBTV at the scales studied. On the doubled-resolution chute at step 8, CBTV scores 0.741 against 0.757 for the best other tracker (§ 4.8). CBTV is slower than the baselines. The tracking-only cost is about 0.9–1.3 ms per assigned particle, against 0.02–0.1 for the classical extrapolator and about 0.3 for the GNN, and it grows with NN. CBTV’s advantage where assignment is ambiguous is therefore in accuracy rather than in speed.

Ablation of the released configuration

Table 3 and Figure 7 separate the two parts of the released configuration (§ 2.7) over the 53 cases without the drums, which were not part of the ablation. Adding coasting and exit handling to the frozen configuration (configuration B in the table) raises mean HOTA from 0.6515 to 0.6736; adding the stored track colour (BT, the released configuration) raises it to 0.6967. BT improves 29 cases by more than 0.005 HOTA and is worse by more than 0.002 on 0 cases. The gains concentrate at large displacement on crowded or smooth flow, for example the 25° chute at step 4 (0.246 → 0.381 → 0.431). A coasting track that follows the motion prior is still near its particle when it is re-detected, and the stored colour no longer depends on whether the old position still overlaps the particle. The released configuration does not change where the two sources of ambiguity apply. It raises CBTV inside them (collision at 1.22 dd/frame: 0.715 → 0.904) and narrows the unambiguous regime in which extrapolation is more accurate (§ 4.4). The one doubled-resolution case on which the frozen configuration scored far below the best other tracker (HOTA 0.336 vs 0.757) is largely recovered by the released configuration (0.741). In the ablation runs on that case (the doubled-resolution large chute at step 8; run on the frozen code, so the values differ slightly from the release-tree numbers above), coasting and exit handling raise HOTA from 0.320 to 0.439 (swaps 18,250 → 16,014) and track-memory colour raises it further to 0.740 (swaps 5,868), the larger part of the gain. An audit of every true track–detection pair in the cost matrices of both runs shows where the gain comes from. Between 0.5 and 1.5 diameters/frame the frozen configuration’s colour gate removed 23.8% (0.5–1 dd) and 47.5% (1–1.5 dd) of the true pairs that lay well inside the distance gate, against 0.3% and 0.4% for the released configuration, and the share of true pairs assigned rose from 70% to 98% and from 27% to 96%. This is consistent with the frozen configuration reading the current frame’s colour at the track’s previous position, which at 24.3 px/diameter and these displacements is mostly background, and with the ablation, in which track-memory colour carries most of the gain. Above 1.5 diameters/frame the gain (true pairs assigned 6% → 59%) is not explained by the colour gate, because the distance gate binds in both configurations (87% and 86% of true pairs lie beyond it). We attribute it to flow-predicted coasting or to later matching stages, which the audit does not test. The same pattern holds across the smooth chutes at step 8 (B adds +0.02 to +0.12, track colour a further +0.07 to +0.30).

Image not extracted

Figure 7. Ablation of the released configuration on the 53 cases without the drums. Left: HOTA against the measured displacement per frame, Δx/d\Delta x/d (logarithmic axis), for the v4 ship configuration (hollow triangles), B = ship with flow-prior coasting and exit kill (purple squares), and BT = B with track-memory colour (green circles). Right: the HOTA gain of B and of BT over ship on the same cases. The gains concentrate at large displacement on crowded or smooth flow; BT is worse than ship by more than 0.002 HOTA on 0 cases (Table 3).

configurationmean HOTA
v4 ship (as originally frozen)0.6515
B = ship + flow-prior coasting + exit kill0.6736
BT = B + track-memory colour (release)0.6967

Table 3. Ablation of the released configuration over the 53 cases without the drums (mean HOTA). The three configurations were run together on the originally frozen code, so their CBTV values can differ from the main tables (release tree) by the tie-order effects of Appendix § 16.

Out-of-sample confirmation on held-out flows

The released configuration was found on the evaluation corpus, so its selection is post hoc. To test it out of sample, two new flows were simulated after the configuration was frozen and were never used in development: an oscillating-piston compaction and a silo discharging past an obstacle, each at steps 1, 4, and 8, with the corpus render settings, Cellpose detections, and the same calibration gate. The pre-registered rule (logged before the held-out videos were rendered) required BT to be within 0.002 of the frozen v4 configuration on both HOTA and PTR in at least five of the six cases, and to raise mean HOTA. The rule is confirmed, with 6 of 6 cases passing and a mean HOTA of 0.7467 against 0.6525 (Table 4). The other trackers were run on the same detections for context; they are not part of the rule. This pre-registered result is reported as run, on the frozen code. Re-applying the same rule with both configurations re-run on the release tree (Appendix § 16) gives the same verdict, confirmed: 6 of 6 cases pass, mean HOTA 0.7477 vs 0.6524.

caseCBTV (BT)CBTV (v4)best context
piston s10.867 (0.989)0.823 (0.985)0.827 (btrack)
piston s40.586 (0.826)0.495 (0.769)0.516 (LapTrack)
piston s80.493 (0.655)0.362 (0.555)0.389 (LapTrack)
silo s10.880 (0.996)0.880 (0.996)0.882 (trackpy)
silo s40.874 (0.985)0.870 (0.982)0.862 (trackpy-NVP)
silo s80.780 (0.931)0.484 (0.726)0.650 (trackpy-NVP)

Table 4. Held-out flows (pre-registered confirmation, as run on the frozen code). HOTA (PTR). “Best context” is the best of the 8 context baselines on the same detections, not part of the rule.

Planned experimental deployment

A quasi-two-dimensional vibrated cell has been constructed to deploy CBTV on real data; experimental tracking results are not reported in this paper. The cell confines a monolayer of painted glass beads (d=2d = 2 mm, Nc=3N_c = 3 colours) between glass plates (100×50100 \times50 mm), with corrugations along one wall rectifying oscillatory motion into directed transport. An electromagnetic shaker (20–200 Hz, 0.5–3 mm) drives the cell; a high-speed colour camera (1280 ×\times 1024, up to 2000 fps, ∼40\sim40 px/diameter) images the face. At the packing fractions and drive amplitudes of interest, the flow is vibration-driven, intermittent, and dense, so assignment is ambiguous from both sources of § 4; the frame rate can be set so that Δx/d\Delta x/d spans about 0.3 to 2. The apparatus follows the vibration-driven granular transport line of the VIP-Gran instrument (Aumaitre et al., [3]), and the method is intended for deployment aboard ESA parabolic-flight campaigns, where Lagrangian measurements of vibration-driven transport in microgravity are not available from interrogation window averaged methods such as PIV. Of the known risks in going from synthetic to real video, partial occlusion has been measured on a synthetic flow that stresses it (§ 4.4). Non-uniform illumination and frame-to-frame colour jitter are further risks that this paper does not test, and the experimental campaign will test all three directly.

Discussion

Benchmarked against 12 baseline trackers across 15 ground-truth flows, CBTV keeps the most identities on 11 cases outside the floor, all at a median displacement of 0.57 to 1.71 nearest-neighbour spacings per frame, and its PTR is within 0.034 of the best wherever links are unambiguous or marginal. It is the most accurate on the impulsive collision. On agitated shear and on the strip view, colour relaxation matches or narrowly exceeds it. It requires coloured particles and a colour camera, and no training. Across the benchmark, tracking difficulty is organised by assignment ambiguity, which history-based velocity prediction does not remove. On the strip view at 0.56 dd, where the particles move at nearly constant velocity, the velocity-extrapolating tracker makes 47 identity swaps. The three cues we tested divide the corpus at displacements that depend on the flow. Assignment that uses neighbour context (CBTV’s constellations and, on agitated and crowded flow, colour relaxation) is the most accurate where assignment is ambiguous, up to about 1.71 spacings per frame. At extreme displacement, beyond the range of CBTV’s motion prior, the tracker that compensates for the bulk motion of the bed (Deep-OC-SORT) is the most accurate. Velocity extrapolation is the most accurate where assignment is unambiguous, a regime now narrowed to smooth chutes at low displacement because CBTV’s coasting borrows extrapolation where it helps. The flow statistics largely follow the identity retention. On the 3 cases where CBTV keeps more identities than every other tracker, the granular temperature, strain rate, and vorticity computed from its tracks are the most accurate, and on the chutes, where its PTR ties the best tracker’s or trails it by at most 0.030, its temperature error is larger and comes from a few long links made where it re-acquires particles after coasting at the inflow and outflow (§ 4.5). CBTV’s decisions can also be inspected, since every commit can be checked against the colour and geometry constraint that produced it, which matters in physics applications where tracker errors bias statistics without warning.

Limitations

The method needs optical access to individually resolved particles, which restricts it to quasi-two-dimensional or surface-layer geometries, and it currently assumes near-monodisperse spheres. It needs particles that are coloured and large enough in the image for their colour to be measured. Across the 15 benchmark flows particles are 11.5 to 24.3 px in diameter (median 11.7; ground-truth diameter at the calibrated image scale), and the planned experiment images them at ∼40\sim40 px/diameter; smaller particles were not tested. It is not the best tracker on smooth chutes at low displacement, where velocity extrapolation or Kalman linking leads by up to 0.025 HOTA; beyond the range of its motion prior (about two to three diameters per frame), where a tracker that compensates for bulk motion keeps identities; or under dense coherent rotation, its measured weak spot. Tracking-only compute is one to two orders of magnitude above the classical extrapolator’s. All quantitative results are synthetic; real-data validation is the object of the experimental campaign in progress.

Future directions

Three lines follow from the benchmark. First, choosing per particle between the measured (cross-correlation) prior and velocity extrapolation according to the local ambiguity. Three switching rules were built and rejected against pre-registered criteria, so this choice appears to need information the tracker does not currently compute; the benchmark quantifies what a successful rule would recover. Second, extending the range of the motion prior beyond about two to three diameters per frame, since the whole-frame bulk-motion pass corrects only the first prediction (§ 4.4). Third, the experimental campaign (§ 5), including a systematic study of performance versus colour count NcN_c and microgravity deployment. Two further directions are untested. Constellation matching needs only a short descriptor per particle, so colour could be replaced by another descriptor, for example in tracking cells or crowds. And because CBTV needs no training, its tracks of real flows could supply training data for learned trackers that do not rely on simulated flows.

Conclusion

We have presented CBTV, a training-free colour bean tracking method for dense granular flows, and benchmarked it against 12 baseline trackers across 15 ground-truth flows and 59 flow–displacement cases, with detection treated as a separate step so that only the linking of particle identities from frame to frame was compared. The benchmark does not support the view that displacement sets tracking difficulty. Outcomes at matched displacement differ by up to ∼0.7\sim0.7 HOTA, the leader changes with the character of the flow, and the crossover displacement between classical and learned methods depends on the flow. The outcomes are organised by assignment ambiguity, which arises from unpredictable motion or from crowding. CBTV’s joint colour-constellation assignment makes it the most accurate tracker on the impulsive collision tested and keeps it close to the best on agitated shear flow and on the strip view, where colour relaxation keeps slightly more identities per transition (CBTV at most 0.003 HOTA behind). Measured from the ground truth, the cases on which it keeps the most identities lie at displacements of 0.57 to 1.71 nearest-neighbour spacings per frame, and on inclined chutes beyond the range of its motion prior, about two to three diameters per frame, it loses identities. It leads or ties at every tested displacement on an impulsive collision, leading from ≈0.7\approx0.7 diameters/frame (HOTA 1.000/0.995/0.904, with the highest identity retention of all trackers). On agitated shear flow, only a neighbour-consistency (relaxation) tracker given the same colour labels matches or narrowly exceeds it, and CBTV is ahead of every other method. On the strip view, where a velocity-extrapolating tracker swaps identities 47 times, CBTV makes no swaps and is level with that relaxation tracker. Colour accounts for a measured share of the accuracy, and it acts as a pattern over neighbours rather than as a label on each particle. Its released configuration was selected by a pre-registered rule and confirmed on two flows simulated afterwards. A vibrated-cell experiment now in progress will test the method on real, intermittently driven granular flow, where assignment is ambiguous from both sources, on the way to Lagrangian granular measurements in microgravity. For experiments on dense granular flows, these results mean that colour constellations can keep particle identities at displacements per frame where trackers without neighbour context lose them, without training data and with less need for ultra-high frame rates.

Reproducibility and code availability

CBTV will be released as open-source software under the BSD-3-Clause licence, together with the code that produced every table and figure in this paper. All CBTV results use the release default (BT, § 2.7); the configuration as originally frozen is available as -preset ship-v4. Both use constellation matching with the particle-count-scaled cluster budget of § 2.4, the displacement-gated dense motion prior of § 2.5, and the structural matcher with the weights of Equation (1). Every comparison follows the detection-matched protocol of § 3: the shared detection files are distributed with the release so that any linker can be re-run on exactly the inputs used here. The release carries a regression (“golden”) test: re-running the release reproduces the published tracker output byte-for-byte on seven reference cases. On Windows (x86-64), CPU and GPU runs of CBTV give byte-identical tracks on all 59 benchmark cases, including the drums (released configuration), and on the release’s regression fixtures, with every random draw seeded from a single --seed option. Linux CPU matches Windows byte for byte on the 10 cases checked. Every CBTV number in this paper comes from that release tree; its tie-order sensitivity against the originally frozen runs, and the effect of the two bug fixes made before release, are reported in Appendix § 16. Simulation inputs, rendered videos, detections, and ground truth will be archived with the release.

Acknowledgements

We thank Joseph Owen for help with the original LAMMPS simulations, Zeyan Kherani for help with the experiments, and Disha Tandon for help with rendering. This work is partially supported by an NSERC Discovery Grant (RGPIN-2020-04486).

Supplementary: the flows of the corpus

Figure 8 shows one ground-truth snapshot, drawn as in Figure 2, of every flow in the corpus except the two rotating drums, which are sketched in Figure 1(i). In the supplementary tables and lists, a case is written as the flow identifier of this figure followed by its subsampling step (for example, M02 s4 is flow M02 at step 4), and RES_B_4K is the doubled-resolution large chute.

Image not extracted

Figure 8. One ground-truth snapshot of every flow in the corpus, drawn as in Figure 2: each panel is labelled with the flow’s identifier in the data release and the frame shown, particles are drawn in their colour labels, and arrows give the ground-truth velocity of a subset of particles, coloured by speed in diameters per frame on one logarithmic scale, with a key arrow per panel.

Supplementary: the corpus-wide comparison in HOTA

Figure 9 and Figure 10 repeat Figure 5 and Figure 4 with HOTA in place of PTR. In HOTA, CBTV leads the best other tracker on 13 cases, by up to 0.14, at a median ϕ\phi of 0.25 to 1.7; it is within 0.025 of the best on the 20 unambiguous and marginal cases, and it trails by up to 0.25 on the inclined chutes at 2.3 spacings per frame and more.

Image not extracted

Figure 9. Measured regimes of the corpus, drawn as in Figure 5, with the colour giving the HOTA of CBTV minus that of the best other tracker in each case.

Image not extracted

Figure 10. HOTA against the measured median displacement per processed frame, drawn as in Figure 4.

Supplementary: velocity error on the collision

To see where CBTV’s lead on the impulsive collision comes from, Figure 11 compares per-particle velocity errors of CBTV (BT) and of the best other tracker on one case: the step closest to one diameter per frame (step 6, 0.81 diameters/frame), where the best other tracker by HOTA is Track30 astra (0.929 against 0.963 for CBTV). Tracks are matched to the ground truth as in the evaluator (within 5 px, per processed frame), and the error is the tracked minus the ground-truth displacement over one processed frame. Before impact both trackers’ errors are close to zero. After impact CBTV is worse on both per-frame measures. Over processed frames 30–55 its median error is larger (0.033–0.040 against 0.018–0.024 diameters/frame) and it makes more post-impact link swaps (212 against 144), defined as frame transitions within frames 30–55 in which a ground-truth particle’s matched track changes, under this figure’s per-frame matching. CBTV’s lead on this case therefore comes from association over the whole sequence rather than from more accurate velocities. With the same detections (detection F1F_1 0.9998 and 1.0000), its association accuracy is higher (AssA 0.928 against 0.863) and it makes fewer evaluator identity swaps (284 against 412), the swap count of the main tables, taken over all frames of the sequence. The two swap counts differ in window (a post-impact stretch against the whole sequence) and in matching, so they measure local and global identity errors respectively and do not contradict each other.

Image not extracted

Figure 11. Velocity error on the impulsive collision (step 6, 0.81 diameters/frame) at processed frame 35 (source frame 210, during compression): CBTV (BT, left) and Trackastra, the best other tracker by HOTA on this case (right), with each panel’s HOTA in its title. Arrows show each matched particle’s velocity error (tracked minus ground-truth displacement over one processed frame), coloured by magnitude on one scale for both panels and capped in length and colour at 0.098 diameters/frame; grey discs are unmatched particles; rings mark post-impact link swaps, that is, particles whose matched track changed since the previous frame. Median error at this frame: 0.038 (CBTV) and 0.020 (Trackastra) diameters/frame; link swaps: 11 and 6. At this stage of the collision CBTV has the larger velocity error and more link swaps; its higher HOTA on this case comes from association over the whole sequence, as described in the text.

Supplementary: where swaps begin

Figure 12 shows the outcome of every ground-truth particle transition of the 12 cases with per-link outcomes (1627 thousand transitions) against its crowding ϕ\phi, pooled over the cases. Between $\phi = 0.5 and 1.2, CBTV keeps 0.96 of the links, against 0.90 for the extrapolator, 0.80 for the GNN, and 0.96 for colour relaxation on the eight non-drum cases; Deep-OC-SORT swaps only 0.02 of the links there but loses or never tracks 0.27 of them, so its low swap rate comes with dropped links. CBTV’s swap fraction first exceeds 1% at ϕ≈0.6\phi\approx0.6, the extrapolator’s at 0.2, and the GNN’s at the smallest ϕ\phi sampled (0.04). Above ϕ≈1.2\phi\approx1.2 every tracker loses most links; the rise of every curve between ϕ=2\phi= 2 and 2.5 is the 25° chute at step 4, which dominates that bin and on which the motion is close to plug flow.

Image not extracted

Figure 12. Outcome of ground-truth transitions against crowding, pooled over the 12 cases with per-link outcomes: fraction of links kept (a), swapped (b), and lost or missing (c), in bins of ϕ\phi holding at least 300 transitions; (a) is the PTR per bin. The tables behind this figure keep the transitions that start inside the video frame and do not cross the periodic boundary of the chute simulations, so the chute swap counts here exclude the boundary wraps that the main-table swap counts include for every tracker alike. Relax-PTV (dashed) was not run on the drums. Dotted lines mark ϕ=0.5\phi= 0.5 and 2.

Supplementary: how the released configuration was chosen

The configuration benchmarked as “v4 ship” was frozen before the in/outflow work. The three BT options were developed afterwards for particles entering and leaving the field of view; their benefit on the main corpus was a post-hoc discovery on the evaluation corpus itself. To limit the resulting selection bias, the choice among four candidate configurations was made by a rule written down before the candidates were scored (eligible: no case more than 0.002 below v4 on HOTA or PTR; among eligible configurations, the highest mean HOTA; ties to the fewest flags; Appendix § 18). BT was the only eligible candidate. It was then confirmed out of sample on two flows simulated after it was frozen (§ 4.9). Two further pre-registered in/outflow questions returned negatives and are reported as such (Appendix § 18: the no-harm question failed on 6 of 71 cases, and the edge-benefit question was met on 0 of 6 registered cases). Both configurations are reported throughout, and the release keeps the frozen v4 configuration available as a preset that reproduces every v4 number exactly.

Supplementary: a pre-registered predictability test that failed

Before the two sources of ambiguity of § 4 were identified, we tested a simpler hypothesis, pre-registered before any number was computed: that CBTV’s advantage is explained by how predictable the motion is, measured from ground truth alone. The predictability metric was EextrapE_{\mathrm{extrap}}, the median error of a constant-velocity extrapolation of the ground-truth trajectories, in particle diameters, computed on 30 cases without running any tracker; a granular-temperature proxy TgranT_{\mathrm{gran}} (a nearest-neighbour velocity-fluctuation measure) was a secondary diagnostic. The pre-registered prediction was a positive correlation between CBTV’s HOTA margin over the best other tracker and EextrapE_{\mathrm{extrap}}.

The test failed. The correlation was r=−0.585r=-0.585 (p=6.8×10−4p=6.8 \times10^{-4}), the opposite sign to the prediction. All trackers’ HOTA was dominated by displacement, and once displacement was controlled EextrapE_{\mathrm{extrap}} added nothing for CBTV (partial r=−0.145r=-0.145, p=0.45p=0.45). Extrapolation error grows faster than linearly with the sampling interval, so a coarsely subsampled smooth case accumulates more error than one frame of agitated flow, and the metric tracks the interval rather than the agitation. Figure 13–Figure 15 show the failed test and the diagnostics.

Image not extracted

Figure 13. HOTA against the ground-truth constant-velocity extrapolation residual EextrapE_{\mathrm{extrap}} (median over particles and sampled frame triples, in diameters per processed frame; symmetric-logarithmic axis), one panel per tracker: CBTV (v4 ship), trackpy-NVP, Deep-OC-SORT, and GOTrack (GNN). Marker area grows with the measured displacement per processed frame; filled markers are the agitated cases (including the impulsive collision), open markers the smooth large chutes. EextrapE_{\mathrm{extrap}} follows the sampling interval rather than the agitation (Spearman +0.72 with dd/frame): the agitated single-step cases lie at E∼0.001E \sim0.001–0.020.02 and the smooth step-8 cases at E∼0.11E \sim0.11–0.130.13, so a one-step extrapolation residual does not separate the regimes at matched displacement.

Image not extracted

Figure 15. A post-hoc diagnostic, not part of the pre-registered test: HOTA of trackpy-NVP against the granular-temperature proxy TgranT_{\mathrm{gran}} (in diameters2^{2} per frame2^{2}, logarithmic axis, after removal of the periodic-box wrap from the ground-truth velocities); filled markers are the impulsive and agitated cases, open markers the smooth large chutes, and marker area grows with displacement. Spearman r=−0.76r=-0.76 (p=1.1×10−6p=1.1 \times10^{-6}). The relation is a correlation rather than a separation of the regimes. trackpy-NVP predicts each particle from a neighbour’s velocity, so it degrades as the velocities of neighbouring particles become less alike.

This negative result led to the explanation reported in the main text, that tracking difficulty is set by assignment ambiguity, which arises either from unpredictable motion or from crowding (§ 4.2 and § 4.3); a single monotone predictability score cannot capture the second source. No claim in this paper rests on EextrapE_{\mathrm{extrap}}, and TgranT_{\mathrm{gran}} correlations are post hoc.

Image not extracted

Figure 14. HOTA margin of CBTV (v4 ship, the configuration frozen at the time of the pre-registered test) over the best other tracker, per case, against EextrapE_{\mathrm{extrap}} (axes as in Figure 13); positive values mean that CBTV scores higher than every other tracker. Filled markers are the impulsive and agitated cases, open markers the smooth large chutes; marker area grows with displacement. The margin does not follow EextrapE_{\mathrm{extrap}} (Spearman −0.59-0.59, the opposite sign to the prediction): CBTV’s leads lie in the impulsive and agitated cases, which the interval-dominated EextrapE_{\mathrm{extrap}} places at the low end of its axis.

Supplementary: point-tracking baseline on cases in which no particle enters

CoTracker3 tracks the frame-0 detections as query points and cannot start new tracks, so it was run only on the 53 cases in which no ground-truth particle appears after the first frame (decided by script over the ground truth, never by hand): the collision, the four large-canvas chutes, the 25∘/35∘/50∘25^\circ/35^\circ/50^\circ chutes, the hopper, the bump chute, the swirl, and the two rotating drums. On 11 of these cases every query point lost visibility before the end of the video, so the run produced no tracks for the remaining frames; these runs are reported as failed and excluded from every mean. Over the 42 completed cases CoTracker3 averages HOTA 0.226, against 0.714 for CBTV (BT) on the same cases (Table 5). At the collision a sensitivity run with the model’s internal resolution set to the native 1024×10241024 \times1024 instead of its default 384×512384 \times512 raises it to 0.809/0.861 (steps 1/4; 0.457/0.490 at the default resolution). Trackastra averages HOTA 0.524 over all 59 cases, against 0.668 for CBTV (BT).

Flow groupcasesfailedCoTracker3TrackastraCBTV (BT)
Impulsive collision700.4830.9490.978
Agitated chutes and hopper, < 1.5 dd320.0970.3960.506
Smooth and structured chutes2300.2090.5220.818
Swirl01–––
Agitated chutes and hopper, ≥1.5d\ge1.5 d650.0980.1180.262
Rotating drums330.1460.4210.412
All such cases42110.2260.5190.714

Table 5. CoTracker3 on the cases in which no particle enters, by descriptive flow group: mean HOTA over the completed cases, with Trackastra and CBTV (BT) on the same cases. Failed runs (all query points lost before the end of the video) are counted but not averaged; their partial scores are in Table 6.

Table 5: CoTracker3 on the cases in which no particle enters, by descriptive flow group: mean HOTA over the completed cases, with Trackastra and CBTV (BT) on the same cases. Failed runs (all query points lost before the end of the video) are counted but not averaged; their partial scores are in Table 6.

CaseΔx/d\Delta x/dlast tracked frameHOTA (partial, lower bound)CBTV (BT)
M03 chute 35°, step 10.843950.1620.466
M03 chute 35°, step 43.38980.0870.085
M03 chute 35°, step 86.76450.1180.116
M04 chute 50°, step 11.132640.2760.405
M04 chute 50°, step 44.54730.1230.086
M04 chute 50°, step 89.03240.0490.117
M05 hopper, step 88.24480.0760.126
K05 swirl, step 10.603040.6630.954
M01 drum, step 10.533930.0730.640
Slowed drum, step 40.88730.1430.461
Slowed drum, step 81.44190.1810.168

Table 6. The 11 failed CoTracker3 runs. Partial run, lower bound: HOTA scored by the benchmark evaluator with only its completeness check bypassed, so every frame after the last tracked frame (index in the subsampled sequence) counts as missed. Not used in any main-text number.

Table 6: The 11 failed CoTracker3 runs. Partial run, lower bound: HOTA scored by the benchmark evaluator with only its completeness check bypassed, so every frame after the last tracked frame (index in the subsampled sequence) counts as missed. Not used in any main-text number.

Supplementary: tie-order sensitivity of the CBTV numbers

The originally frozen CBTV runs used numpy’s default (unstable) sort, whose tie-breaking depends on the CPU’s SIMD backend, so the same code could assign different track identities (and, through cascades, occasionally different links) on different machines. The release replaces every tie-sensitive sort with a stable sort, which makes the output identical across CPU/GPU and SIMD backends on the regression fixtures, and every CBTV number in this paper was regenerated on that tree. Table 7 compares the stable-sort values (before the two bug fixes described below) with the originally frozen ones over the 53 corpus cases without the drums.

configurationexactd≤0.002d \leq0.002d>0.002d > 0.002max⁡∣ΔHOTA∣\max|\Delta\mathrm{HOTA}|max⁡d\max d
v4284490.01920.0255
BT (released)355030.00850.0147

Table 7.

Table 7: Tie-order sensitivity: stable-sort tree (before the two release fixes) vs originally frozen runs, per case, d=max⁡(∣ΔHOTA∣,∣ΔPTR∣)d = \max(|\Delta\mathrm{HOTA}|, |\Delta\mathrm{PTR}|).

Cases with d>0.002d > 0.002 (ΔHOTA/ΔPTR\Delta\mathrm{HOTA}/\Delta\mathrm{PTR}, release minus frozen). v4: M02 s3 (+0.0152/+0.0227), M02 s4 (−0.0192/−0.0255), M02 s5 (+0.0026/+0.0068), M06 s5 (+0.0061/+0.0024), M06 s6 (−0.0100/−0.0130), M06 s8 (+0.0018/−0.0057), RES_A_1080 s4 (−0.0027/−0.0031), RES_A_1080 s8 (+0.0021/+0.0035), RES_B_4K s8 (−0.0023/−0.0036). BT: M02 s4 (−0.0085/−0.0115), M02 s5 (−0.0044/−0.0147), M06 s8 (−0.0019/−0.0023). All are chute cases at ≈0.6 diameters/frame or more.

Two bug fixes made before release. Every CBTV number in the paper comes from the release tree, which adds two fixes to the stable-sort tree. (i) Frame-0 radius estimate. CBTV sets its radius window from the particle radii at the first frame. It previously estimated them with its own Laplacian-of-Gaussian detector, whose CPU and GPU implementations differ, so CPU and GPU runs could choose different windows; it now uses the radii of the shared frame-0 detections (rescaled to the detector’s radius convention), so the decision no longer depends on the device. In the corpus this changes the window on one flow only, the doubled-resolution large chute (6–11 px → 5–10 px). (ii) Exit handling at the frame edge (BT only; the v4 configuration does not use it). Tracks were terminated when their predicted position fell outside the frame, which also killed particles resting at, or detected just past, the edge; only tracks that are leaving are now terminated. Against the stable-sort values, BT is unchanged (d<5×10−5d < 5 \times10^{-5}) on 41 of 53 cases and within 0.002 on 51 (beyond: RES_A_1080 s6 (+0.0005/+0.0021), RES_B_4K s8 (+0.0011/+0.0075)); v4 on 50 and 51 (beyond: RES_B_4K s4 (−0.0002/−0.0045), RES_B_4K s8 (+0.0177/+0.0046)). The held-out cases are unchanged (max d=0.0000d = 0.0000). No verdict, regime assignment, or ranking in the paper changes.

Selection rule re-checked. With both configurations from the release tree, BT is within 0.002 of v4 on both HOTA and PTR on every one of the 48 non-floor cases (v4 HOTA ≥0.15\geq0.15; 0 violations), so BT remains eligible under the pre-registered rule. Held-out rule re-checked. On the release tree the held-out rule again returns confirmed (6 of 6 cases; mean HOTA 0.7477 vs 0.6524).

Supplementary: sensitivity of BT to its own parameters

Varying each parameter of the released configuration (BT) one at a time by a factor of two (the coast-velocity weight, the exit margin, and the colour-memory weight) changes HOTA by at most 0.0029 and PTR by at most 0.0045 on average over 30 smooth and collision cases. The colour EMA is the most sensitive parameter; the largest change on any single case, 0.0408 HOTA, is on the doubled-resolution large chute at step 8. Each BT parameter that has a scale was changed alone, to half and to twice its default, and the released configuration was re-run on the 30 smooth and collision cases of the corpus. Table 8 gives the change against the unmodified released configuration run on the same machine. These runs are on Linux CPU, so the baseline is the same-machine re-run of BT, not the Windows GPU numbers of the main tables, and a difference measures the parameter alone. The coast-velocity weight was only halved: its range is (0, 1] and its default is 1, so doubling is not a valid value. The defaults were not changed by this study. The runs were made just before the frame-0 radius fix of Appendix § 16; the only cases that fix touches are on the doubled-resolution large chute, and those runs were repeated on the release tree with byte-identical tracks, so every number here holds for the release. “Standing” is the number of cases whose lead / tie / behind classification against the best other tracker (HOTA and PTR respectively, tie band 0.005) changes.

parametercoast EMA ×0.5exit margin ×0.5exit margin ×2colour EMA ×0.5colour EMA ×2
default → value1.0 → 0.51.0 → 0.51.0 → 2.00.3 → 0.150.3 → 0.6
mean Δ\DeltaHOTA−0.0002+0.0001+0.0001−0.0023+0.0011
mean Δ\DeltaPTR−0.00120.00000.0000−0.0038+0.0020
mean ∣ΔHOTA∣\lvert\Delta\mathrm{HOTA}\rvert0.00190.00040.00030.00290.0019
max ∣ΔHOTA∣\lvert\Delta\mathrm{HOTA}\rvert0.01460.00410.00220.04080.0271
caseRES_B_4K s8RES_A_1080 s6RES_B_1080 s4RES_B_4K s8RES_B_4K s8
max ∣ΔPTR∣\lvert\Delta\mathrm{PTR}\rvert0.01370.00790.00530.07130.0331
caseRES_B_1080 s8RES_A_1080 s8RES_B_4K s4RES_B_4K s8RES_B_4K s8
standing changes (HOTA/PTR)2/01/12/11/00/1

Table 8. BT parameter sensitivity, one parameter at a time, 30 cases. Mean over cases of Δ\Delta = variant − baseline; largest ∣Δ∣\lvert\Delta\rvert with its case; number of cases whose standing against the best other tracker changes (HOTA/PTR).

The mean absolute change is at most 0.0029 HOTA and 0.0045 PTR for every variant; excluding the colour-memory weight it is at most 0.0019 and 0.0023. The largest change on any case is on the doubled-resolution large chute at step 8, a case on which BT is close to failing on that machine (0.610 HOTA, 0.624 PTR in the baseline run, against 0.741 and 0.785 in the main tables; an unexplained difference seen only in one external environment outside the pinned environment): halving the colour-memory weight changes it by −0.0408 HOTA and −0.0713 PTR. Without that case no variant changes any single case by more than 0.0138 HOTA or 0.0227 PTR.

Supplementary: record of the pre-registered selection

The released configuration (BT) was chosen from candidate configurations by rules written down before the corresponding runs, and the in/outflow questions were pre-registered before any of their runs. The rules, the verdicts, the full per-case tables and the per-case values behind the numbers below will be archived with the data.

In/outflow questions (with Amendment 1). Question 1 (no harm): the in/outflow configuration must stay within 0.002 of v4 on HOTA and PTR on every one of the 71 cases. It failed on 6 cases, all on HOTA only, by 0.0025 to 0.0049, and 4 of them are floor cases (v4 HOTA < 0.15). Question 2 (edge benefit): a relative improvement of at least 25% in a primary edge metric (birth-latency 90th percentile, exit-capture swaps, edge fragmentation) on at least 3 of the 6 registered edge cases, each with at least 30 entry events. It was met on 0 of 6 cases. Exit-capture swaps were 0 on every corpus and crop case in every configuration and 1 on the synthetic in/outflow video, so the corpus does not show the failure the flags target. Both are reported as negatives.

Candidate screen. Rule fixed before the runs: a candidate is eligible if no case falls more than 0.002 below v4 on HOTA or PTR; among eligible candidates the one with the highest mean HOTA over the 71 cases is taken, ties to the fewest flags. Table 9 gives the outcome; BT is the only eligible candidate. CT has a higher mean HOTA than BT but is ineligible under the rule. CT adds a fourth mechanism, requiring two consecutive detections before a new track is confirmed; it lowered HOTA on the strip view (at step 1, from 0.9914 to 0.9911), raised the 90th-percentile delay before a new particle receives a track from 13.6 to 16.8 frames, and failed the selection rule on 3 cases. The selected configuration, BT, is labelled a post-hoc discovery on the evaluation corpus and was confirmed on new held-out flows (§ 4.9).

candidatecases below v4 −0.002mean HOTAeligible
B10.7530no
BT (released)00.7703yes
CT30.7710no
C60.7532no

Table 9. Candidate screen, all 71 cases. v4 mean HOTA 0.7365. B = v4 + flow-prior coasting + exit kill; BT = B + track-memory colour; CT = BT + confirm-hits; C = B + confirm-hits.

Table 9: Candidate screen, all 71 cases. v4 mean HOTA 0.7365. B = v4 + flow-prior coasting + exit kill; BT = B + track-memory colour; CT = BT + confirm-hits; C = B + confirm-hits.

Second adoption rule. Before any run, a variant of BT was fixed as the one candidate replacement (BTN): its motion prior is switched from the dense PIV field to a nearest-velocity prior in the style of trackpy’s NearestVelocityPredict (-mu-source nvp), in which each track’s predicted displacement is its own last velocity when known and otherwise the median velocity of its nearest track with known velocity. BTN was to replace BT only if it left no non-floor case below v4 −0.002 and raised the mean HOTA by at least 0.005. It left 14 non-floor cases below that line and its mean HOTA is 0.7675 against 0.7703 for BT, so BT stayed.

Supplementary: local motion rigidity (pre-registered test)

The rigidity explanation for where colour relaxation matches or exceeds CBTV was tested by a rule written down before any rigidity value was computed. For each case, at the case’s frame step and from ground truth only, RR is the fraction of pairs of particles within the relaxation tracker’s neighbour radius (3 diameters) whose displacements over one step differ by less than its consistency tolerance (0.35 diameters), averaged over frames. The outcome is the HOTA of the better relaxation variant minus that of CBTV (BT), over the 63 non-floor cases. Because RR falls with displacement (Spearman −0.81), the primary test is the partial Spearman correlation of RR with the outcome controlling for log⁡10\log_{10} displacement, one-sided.

Result. On its pre-registered set of 71 cases, the primary test passes: partial ρ=+0.56\rho= +0.56, one-sided p=9.1×10−7p = 9.1 \times10^{-7} (n=63n = 63), so the pre-registered verdict is supported. On the corpus presented here the correlation is weaker and not significant (partial ρ=+0.20\rho= +0.20, one-sided p=8.5×10−2p = 8.5 \times10^{-2}, n=48n = 48). The raw Spearman correlation is +0.74. The secondary prediction that all 7 collision cases fall in the lowest rigidity tercile is not met (4 of 7).

Caveats. Most cases sit near the rigidity ceiling (median R=0.996R = 0.996; R≥0.97R \ge0.97 on 45 of 63 cases), so the correlation is carried by the less rigid cases, which are mostly at high displacement. Relaxation scores below CBTV on 33 cases, above it on 24, and equal on 6. One implementation bug was fixed before any statistic was read: on the strip-view videos of the pre-registered set the first ground-truth frames contain no mobile particles, so the particle diameter used to scale RR was undefined and six cases had no value; the diameter is now taken from the first frame that contains mobile particles, with the definitions unchanged.

Supplementary: uncertainty of the benchmark numbers

Confidence intervals come from a moving-block bootstrap over processed frames (blocks of 10 frames, 1000 resamples, seed 20261005, 95% percentile intervals), applied to the stored tracks of every tracker and case on the benchmark branch and re-scored with the unmodified evaluator. Significance is judged on paired differences: in each resample the same frames are drawn for CBTV (BT) and for the best other tracker of that case, and the interval of the difference is reported. We report absolute intervals for PTR but not for HOTA. PTR is a ratio of sums over transitions and its bootstrap is unbiased (median ∣bias∣|\mathrm{bias}| 0.0010). HOTA is not: resampling frames with replacement leaves each particle covered by fewer distinct frames, which inflates its association term, so the bootstrap distribution of CBTV’s HOTA sits above the point estimate (median bias +0.023, maximum +0.061), and the point lies outside its own percentile interval in 18 of 59 cases. The bias cancels in the paired difference (median ∣bias∣|\mathrm{bias}| 0.0014), which is why only paired HOTA intervals are used. The best other tracker is the maximum over all other tracker configurations in each case, chosen after the fact; counts of cases where CBTV is behind it are therefore upper bounds.

flow groupHOTAPTR
aheadbehindspans 0aheadbehindspans 0
Impulsive collision700106
Agitated shear230140
Strip view and cylinder flow, step 8002011
Strip view, swirl and cylinder flow, other steps014032
Smooth and structured chutes26154136
Extreme displacement182191
Rotating drums222312
all cases142025103118

Table 10. CBTV (BT) minus the best other tracker, by descriptive flow group (§ 3.4): cases where the paired 95% CI lies above 0 (ahead), below 0 (behind), or spans 0.

References

  1. [1]Ronald J. Adrian and Jerry Westerweel. Particle Image Velocimetry. Cambridge Aerospace Series. Cambridge University Press, 2011.
  2. [2]Daniel B. Allan, Thomas Caswell, Nathan C. Keim, Casper M. van der Wel, and Ruben W. Verweij. soft-matter/trackpy: Trackpy v0.6.1. Zenodo, 2023. doi: 10.5281/zenodo.7670439. Software; implements the Crocker–Grier algorithm with predictive tracking.
  3. [3]S. Aumaitre, R. P. Behringer, A. Cazaubiel, E. Clément, J. Crassous, D. J. Durian, E. Falcon, S. Fauve, D. Fischer, A. Garcimartín, Y. Garrabos, M. Hou, X. Jia, C. Lecoutre, S. Luding, D. Maza, M. Nöhringme, E. Opsomer, F. Palencia, T. Pöschel, J. Schockmel, M. Sperl, R. Stannarius, N. Vandewalle, and P. Yu. An instrument for studying granular media in low-gravity environment. Review of Scientific Instruments, 89(7):075103, 2018. doi: 10.1063/1.5034061.DOI
  4. [4]S. J. Baek and S. J. Lee. A new two-frame particle tracking algorithm using match probability. Experiments in Fluids, 22(1):23–32, 1996. doi: 10.1007/BF01893303.DOI
  5. [5]Christian Bendicks, Dominique Tarlet, Christoph Roloff, Róbert Bordás, Bernd Wunderlich, Bernd Michaelis, and Dominique Thévenin. Improved 3-D particle tracking velocimetry with colored particles. Journal of Signal and Information Processing, 2(2):59–71, 2011. doi: 10.4236/jsip.2011.22009.DOI
  6. [6]W. Brevis, Y. Niño, and G. H. Jirka. Integrating cross-correlation and relaxation algorithms for particle tracking velocimetry. Experiments in Fluids, 50(1):135–147, 2011. doi: 10.1007/s00348-010-0907-z.DOI
  7. [7]Jinkun Cao, Jiangmiao Pang, Xinshuo Weng, Rawal Khirodkар, and Kris Kitani. Observation-centric SORT: Rethinking SORT for robust multi-object tracking. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9686–9696, 2023. doi: 10.1109/CVPR52729.2023.00934.DOI
  8. [8]H. Capart, D. L. Young, and Y. Zech. Voronoï imaging methods for the measurement of granular flows. Experiments in Fluids, 32(1):121–135, 2002. doi: 10.1007/s003480200013.DOI
  9. [9]Christian Cierpka, Benjamin Lütke, and Christian J. Kähler. Higher order multi-frame particle tracking velocimetry. Experiments in Fluids, 54(5):1533, 2013. doi: 10.1007/s00348-013-1533-3.DOI
  10. [10]Abram H. Clark, Lou Kondic, and Robert P. Behringer. Particle scale dynamics in granular impact. Physical Review Letters, 109(23):238302, 2012. doi: 10.1103/PhysRevLett.109.238302.DOI
  11. [11]John C. Crocker and David G. Grier. Methods of digital video microscopy for colloidal studies. Journal of Colloid and Interface Science, 179(1):298–310, 1996. doi: 10.1006/jcis.1996.0217.DOI
  12. [12]G. Félix, V. Falk, and U. D’Ortona. Granular flows in a rotating drum: the scaling law between velocity and thickness of the flow. The European Physical Journal E, 22(1):25–31, 2007. doi: 10.1140/epje/e2007-00002-5.DOI
  13. [13]Yohsuke T. Fukai and Kyogo Kawaguchi. LapTrack: linear assignment particle tracking with tunable metrics. Bioinformatics, 39(1):btac799, 2023. doi: 10.1093/bioinformatics/btac799.DOI
  14. [14]Benjamin Gallusser and Martin Weigert. Trackastra: Transformer-based cell tracking for live-cell microscopy. In European Conference on Computer Vision (ECCV), 2024. arXiv:2405.15700.arxiv.org/abs/2405.15700
  15. [15]Angel Garcimartín, Iker Zuriguel, Alvaro Janda, and Diego Maza. Fluctuations of grains inside a discharging two-dimensional silo. Physical Review E, 84(3):031309, 2011. doi: 10.1103/PhysRevE.84.031309.DOI
  16. [16]D. Gollin, W. Brevis, E. T. Bowman, and P. Shepley. Performance of PIV and PTV for granular flow measurements. Granular Matter, 19(3):42, 2017. doi: 10.1007/s10035-017-0730-9.DOI
  17. [17]Heinrich M. Jaeger, Sidney R. Nagel, and Robert P. Behringer. Granular solids, liquids, and gases. Reviews of Modern Physics, 68(4):1259–1273, 1996. doi: 10.1103/RevModPhys.68.1259.DOI
  18. [18]Zhaochen Jiang, Thomas Hagemeier, Andreas Bück, and Evangelos Tsotsas. Color-PTV measurement and CFD-DEM simulation of the dynamics of poly-disperse particle systems in a pseudo-2D fluidized bed. Chemical Engineering Science, 179:115–132, 2018. doi: 10.1016/j.ces.2018.01.013.DOI
  19. [19]Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. CoTracker3: Simpler and better point tracking by pseudo-labelling real videos. arXiv preprint arXiv:2410.11831, 2024.arxiv.org/abs/2410.11831
  20. [20]Harold W. Kuhn. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1–2):83–97, 1955. doi: 10.1002/nav.3800020109.DOI
  21. [21]Yan Li, Wei Hu, Qiang Xu, Runqiu Huang, ChingShung Chang, Jianye Chen, and Yujie Wang. Velocity profile geometries and granular temperature distributions in very dense granular flows. Geophysical Research Letters, 51(2):e2023GL104410, 2024. doi: 10.1029/2023GL104410.DOI
  22. [22]Jiaming Liang, Chao Xu, and Shengze Cai. Recurrent graph optimal transport for learning 3D flow motion in particle tracking. Nature Machine Intelligence, 5:505–517, 2023. doi: 10.1038/s42256-023-00648-y.DOI
  23. [23]Richard M. Lueptow, Alp Akonur, and Troy Shinbrot. PIV for granular flows. Experiments in Fluids, 28(2):183–186, 2000. doi: 10.1007/s003480050023.DOI
  24. [24]Jonathan Luiten, Aljoša Ošep, Patrick Dendorfer, Philip Torr, Andreas Geiger, Laura Leal-Taixé, and Bastian Leibe. HOTA: A higher order metric for evaluating multi-object tracking. International Journal of Computer Vision, 129:548–578, 2021. doi: 10.1007/s11263-020-01375-2.DOI
  25. [25]Gerard Maggiolino, Adnan Ahmad, Jinkun Cao, and Kris Kitani. Deep OC-SORT: Multi-Pedestrian tracking by adaptive re-identification. In IEEE International Conference on Image Processing (ICIP), pages 3025–3029, 2023. doi: 10.1109/ICIP49359.2023.10222576.DOI
  26. [26]N. A. Malik, Th. Dracos, and D. A. Papantoniou. Particle tracking velocimetry in three-dimensional flows. Part II: Particle tracking. Experiments in Fluids, 15(4–5):279–294, 1993. doi: 10.1007/BF00223406.
  27. [27]Benjamin Midtvedt, Saga Helgadottir, Aykut Argun, Jesús Pineda, Daniel Midtvedt, and Giovanni Volpe. Quantitative digital microscopy with deep learning. Applied Physics Reviews, 8(1):011310, 2021. doi: 10.1063/5.0034891.DOI
  28. [28]Daniel M. Mueth, Georges F. Debregeas, Greg S. Karczmar, Peter J. Eng, Sidney R. Nagel, and Heinrich M. Jaeger. Signatures of granular microstructure in dense shear flows. Nature, 406:385–389, 2000. doi: 10.1038/35019032.DOI
  29. [29]Kazuo Ohmi and Hang-Yu Li. Particle-tracking velocimetry with new algorithms. Measurement Science and Technology, 11(6):603–616, 2000. doi: 10.1088/0957-0233/11/6/303.DOI
  30. [30]K. Okamoto, Y. A. Hassan, and W. D. Schmidl. New tracking algorithm for particle image velocimetry. Experiments in Fluids, 19(5):342–347, 1995. doi: 10.1007/BF00203419.DOI
  31. [31]J. S. Olafsen and J. S. Urbach. Clustering, order, and collapse in a driven granular monolayer. Physical Review Letters, 81(20):4369–4372, 1998. doi: 10.1103/PhysRevLett.81.4369.DOI
  32. [32]Nicholas T. Ouellette, Haitao Xu, and Eberhard Bodenschatz. A quantitative study of three-dimensional Lagrangian particle tracking algorithms. Experiments in Fluids, 40(2):301–313, 2006. doi: 10.1007/s00348-005-0068-7.DOI
  33. [33]Marius Pachitariu, Michael Rariden, and Carsen Stringer. Cellpose-SAM: superhuman generalization for cellular segmentation. bioRxiv, 2025. doi: 10.1101/2025.04.28.651001. Preprint; update on journal publication.
  34. [34]Dmitry Puzyrev, Kirsten Harth, Torsten Trittel, and Ralf Stannarius. Machine learning for 3D particle tracking in granular gases. Microgravity Science and Technology, 32:897–906, 2020. doi: 10.1007/s12217-020-09800-4.DOI
  35. [35]Daniel Schanz, Sebastian Gesemann, and Andreas Schröder. Shake-The-Box: Lagrangian particle tracking at high particle image densities. Experiments in Fluids, 57(5):70, 2016. doi: 10.1007/s00348-016-2157-1.DOI
  36. [36]Aidan P. Thompson, H. Metin Aktulga, Richard Berger, Dan S. Bolintineanu, W. Michael Brown, Paul S. Crozier, Pieter J. in ’t Veld, Axel Kohlmeyer, Stan G. Moore, Trung Dac Nguyen, Ray Shan, Mark J. Stevens, Julien Tranchida, Christian Trott, and Steven J. Plimpton. LAMMPS – a flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales. Computer Physics Communications, 271:108171, 2022. doi: 10.1016/j.cpc.2021.108171.
  37. [37]Torsten Trittel, Dmitry Puzyrev, Kirsten Harth, and Ralf Stannarius. Rotational and translational motions in a homogeneously cooling granular gas. npj Microgravity, 10(1):81, 2024. doi: 10.1038/s41526-024-00420-5.DOI
  38. [38]Kristina Ulicna, Giulia Vallardi, Guillaume Charras, and Alan R. Lowe. Automated deep lineage tree analysis using a Bayesian single cell tracking approach. Frontiers in Computer Science, 3:734559, 2021. doi: 10.3389/fcomp.2021.734559.
  39. [39]Brian Utter and R. P. Behringer. Self-diffusion in dense granular shear flows. Physical Review E, 69 (3):031308, 2004. doi: 10.1103/PhysRevE.69.031308.DOI
  40. [40]Yizhu Wang, Xiaoming He, Yuan Tian, Chang Liu, and Depeng Wang. Particle imaging velocimetry with color-encoded illumination: A review. Sensors, 25(16):4981, 2025. doi: 10.3390/s25164981.DOI

Paper details

vlnc:W_d28pudmf
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents