SPDMX: Large-Scale Open Stem Audio From Symbolic Music

Index Terms— Public-domain music, audio datasets, multitrack stems, symbolic-to-audio, source separation

Introduction

Massive public datasets have driven much of the progress in machine learning over the past decade, from text [1, 2] and images [3, 4] to speech and general audio [5, 6]. Music cannot yet follow the same path, as copyright blocks redistribution of recordings that would form such corpora, while modern models need individual stems for source separation, accompaniment generation, and stem-conditioned synthesis. Legally redistributable multitrack supervision remains scarce.

Existing resources trade off realism, scale, and redistributability (Table 1). Recorded sets such as MUSDB18 [7], MedleyDB [8, 9], and MoisesDB [10] are realistic but small or restricted, while Slakh2100 [11] uses non-redistributable commercial instruments. PDMX [12, 13] offers symbolic scale, but naive General MIDI rendering is dry and acoustically weak. No prior resource jointly offers score-collection scale and an open render stack.

Dataset	$N$	$h$	Type	D	P
MUSDB18 [7]	150	10	R	×	—
MedleyDB‡ [8, 9]	196	13.8	R	×	—
MoisesDB [10]	240	14.4	R	×	—
Slakh2100† [11]	2,100	145	S	×	×
CocoChorales [16]	240,000	1,411	S	✓	✓
EnsembleSet [17]	80	6.15	S	✓	×
SPDMX (All)	253,982	5,469	S	✓	✓
SPDMX (Multitrack)	88,403	2,734	S	✓	✓

To address this gap, we propose SPDMX, a rendering of PDMX comprising 253,982 songs, 490,159 mono stems, and 5,469 song-hours at 44.1 kHz under CC BY 4.0. Its 88,403 multitrack songs total 2,734 song-hours, exceeding Slakh2100’s full inventory, while singles support generation and transcription. Songs pair MIDI with PDMX identifiers. We clean the MIDI and select category-specific FluidSynth or MIDI-DDSP [18] recipes through a small listening pilot, producing a reusable open pipeline. Matched-budget separation and transcription checks on the project page confirm usable in-domain audio–label alignment; demos, schema details, and further experimental notes live there as well.¹ The release supports separation, stem-conditioned synthesis, MIDI-to-audio, transcription, and joint symbolic–audio research.

Related work

Recorded multitracks. MUSDB18 [7], MedleyDB [8, 9], and MoisesDB [10] provide high-quality studio stems, while chamber sets such as URMP [19] offer realistic ensembles at still smaller scale. All remain limited in hours and/or access. SPDMX instead prioritizes open scale, with thousands of song-hours of redistributable stems with aligned scores, achieved through synthesis rather than expensive stu dio recording.

Synthetic multitracks. Slakh2100 [11] synthesizes Lakh MIDI [20] into $\sim145$ song-hours of multitrack audio but depends on proprietary sample libraries that cannot be redistributed. Neural synthesizers such as MIDI-DDSP [18] (trained on URMP [19]) improve realism for selected monophonic instruments and underlie generative releases like CocoChorales [16]; EnsembleSet [17] and SynthSOD [21] similarly target chamber or orchestral stems with commercial libraries. SPDMX instead renders real public-domain PDMX repertoire with an open FluidSynth/MIDI-DDSP stack, delivering stems, mixtures, and aligned scores at a scale those resources do not jointly provide. Among the resources surveyed in Table 1, its 2,734-hour multitrack subset is the largest openly-licensed synthetic collection by song-hours.

SPDMX Dataset

SPDMX ships 253,982 songs (5,469 song-hours) as 490,159 mono FLAC stems plus paired MIDI, every file joinable to PDMX on song identity. Song-hours are mixture durations counted once per song (not multiplied by stem count). Of these, 88,403 songs (34.8%; 2,734 song-hours, 50.0% of hours) have $\ge2$ stems (mean 1.93 stems per song, max 66). Figure 1 compares multitrack song-hours by stem count against Slakh2100: even excluding SPDMX’s 165,579 singles (65.2%), the multitrack subset alone already exceeds Slakh2100’s full 145 h inventory. A 632-song BDGP-eligible subset (bass, drums, guitar, and piano present) supports matched Slakh2100 comparisons. Rendering uses only open synthesizers, and audio is released under CC BY 4.0.

We build the release in three stages: clean PDMX scores, select a renderer family for each instrument category, and assemble the full release from those stems.

Initial data cleaning

PDMX MIDI programs often disagree with track names (e.g., a named violin encoded as piano). We correct GM programs from track name aliases and drop tracks with no note events. Figure 2 shows the corrected inventory by stem count and non-silent hours. For each rendered stem, we sum 0.25 s windows whose RMS amplitude is at least 0.01 ($\approx-40$ dBFS), excluding near-silent gaps between notes and trailing padding. Much of SPDMX concentrates in piano, voice, drums, and common orchestral families. Piano dominates stem count (57%) more than non-silent hours (41%), so many piano tracks are sparse or short.

Vector (PDF)

Although cleaning makes the symbolic source of SPDMX more reliable, it does not resolve the acoustic limitations of using a single GM bank to render the whole dataset. Naive GM rendering remains dry and synthetic, and short sound-font notes poorly match acoustic sustain for winds, brass, and strings. That motivates listening among open FluidSynth banks and selective MIDI-DDSP.

Listening study

We use a small listening pilot to select a render recipe for each instrument category. Every track becomes a mono stem stored at 44.1 kHz. The basic soundfont strategy uses one GM bank (SGM–V2.01). The varied strategy samples from a shortlist of redistributable soundfonts per category, including FluidR3_GM, GeneralUser GS, Arachno, and AirFont, plus a light reverb FX profile, for Slakh2100-like timbral variety without Slakh2100’s proprietary libraries. For eligible monophonic strings, brass, and winds, MIDI-DDSP [18] can replace the soundfont. However, we fall back to FluidSynth for polyphonic parts, instruments outside the MIDI-DDSP (URMP) inventory, and all other GM programs. MIDI-DDSP synthesizes at 16 kHz; we upsample to 44.1 kHz so all stems share one sample rate and mix together easily, though those stems remain band-limited relative to native 44.1 kHz audio. Voice-class tracks use choir or voice soundfonts (lyric singing synthesis is out of scope).

We ask listeners to compare four families on perceived realism: basic, varied, MIDI-DDSP over basic fallbacks, and MIDI-DDSP over varied fallbacks. The basic soundfont is the open reference, repeated among shuffled unlabeled conditions. Each trial scores a 10-second stem excerpt for perceived realism (0–100). Two stems represent each of 10 categories. $N=8$ listeners completed the study (Figure 3).

Vector (PDF)

We treat ratings as design guidance. We choose the highest mean by default, but when basic and varied are close, we retain varied to preserve timbral diversity. This heuristic is not a significance rule. SPDMX uses basic for drums, organ, and guitar, and varied for piano, voice, mallet, and miscellaneous classes. Strings and brass use MIDI-DDSP over basic, and wind uses MIDI-DDSP over varied (⋆ in Figure 3).

Final assembly

We apply the frozen recipe to every cleaned song. Because independent per-stem loudness matching would erase MIDI velocity balance, we loudness-normalize each stem toward −23 LUFS [22] with peak limiting, after which we scale each stem by the ratio of that track’s maximum note-on velocity to the song maximum. We uniformly apply a single anti-clip peak gain on the stem sum in order to keep stems linearly summable (mixture equals stem sum) [11].

Release layout

The public release is sharded into downloadable chunks. Each song ships mono FLAC stems and paired MIDI, while multitrack songs also include a mono mix equal to the linear stem sum. Single-stem songs need no separate mix. Songs share PDMX identifiers, so downloading PDMX alongside SPDMX makes scores and other symbolic fields easy to use with the audio. Release metadata also marks convenient subsets such as multitrack songs and BDGP-eligible four-stem packs. Schema details and download instructions are on the project page.

Several limitations follow from this construction. About 65% of songs are single-stem, and piano dominates the inventory. MIDI comes from engraved scores with quantized timing and dynamics. All stems are mono. The small, synthetic-reference listening study does not establish corpus-wide absolute realism. MIDI-DDSP stems remain band-limited after upsampling from 16 kHz.

Conclusion

SPDMX brings public-domain PDMX scores into the audio domain as a large CC BY 4.0 stem corpus: 253,982 songs, 490,159 mono stems, and 5,469 song-hours at 44.1 kHz, with paired MIDI and identifiers linking to PDMX metadata. 88,403 songs (34.8%; 2,734 song-hours) supply multitrack supervision beyond Slakh2100’s full inventory, while a substantial single-stem subset still supports synthesis, transcription, and analysis. The listening-guided, open FluidSynth/MIDI-DDSP pipeline makes the release reproducible and extensible to other symbolic collections. Matched-budget checks on the project page indicate usable in-domain audio–label alignment for separation and transcription. We hope SPDMX will support source separation, stem-conditioned synthesis, MIDI-to-audio, transcription, and joint symbolic–audio research.

Compliance with Ethical Standards

SPDMX is derived from public-domain PDMX scores. The rendered audio, MIDI, and release tables are distributed under CC BY 4.0. Rendering uses soundfonts selected for redistribution-compatible terms and open MIDI-DDSP weights (Apache 2.0; trained on URMP), not closed commercial VST sample libraries. The listening study involved adult volunteers rating synthetic audio excerpts.

References