betaFeedback

WorksComputer Science

Organizational Learning in Materials Discovery Campaigns: Which Databases, Calculations, and Agent Teams Pay Off

Abstract

More databases can make a materials campaign worse. Through an organizational lens, we trace each source's contribution. One band-gap campaign ranks seven anion families from archived measured gaps, one vote per source. In the first half of its budget, all seven pooled rank worse (rank agreement 0.737; 1 = measured order) than the best pool of three (0.851). As a stability check, choosing the pool on half the runs and testing on the rest kept the gap. Splitting Materials Project afterwards points to its transition-metal oxides and fluorides. Two databases from one pipeline add nothing over one, but pooling gives their shared ordering two votes. New calculations repay their cost early only when two or three campaigns reuse them. Flat language-model agent teams trailed a random-pick baseline (mean 0.876, sd 0.096, 20 searches). An adaptive Al-Mg-Si team, mostly interatomic potentials, was scored on unseen HSE06 calculations. In its first quarter of compute it found 49% of the ten lowest-formation-energy compounds, against at most 40% for fixed teams and 73% for MACE-MP-0 alone. Merging members whose orderings agreed carried most of its gain. This suggests pooling a few sources that fit the measurements, counting each pipeline once and charging for computation.

Introduction

A group screening wide-gap semiconductors can consult several computed band-gap databases before it measures anything. The Materials Project [1], JARVIS [2], and SNUMAT [3] hold gaps computed with the generalized gradient approximation (GGA) [4], meta-GGA (r2r^{2}SCAN) [5], modified Becke–Johnson (TB-mBJ) [6], and hybrid functionals (HSE06) [7, 8, 9], and each level departs from measurement by different amounts in different chemistries [10]. Multi-fidelity learning fuses such levels into one predictor [11, 12, 13], and multi-source Bayesian optimization draws on several sources within the search [14, 15], in one variant with a cost-aware stopping rule [16]; here each level is kept as a separate … genome approach [17, 18, 19] invites a group to use all of them. Databases are pooled as wholes, however, and whether each added database helps, repeats another, or misleads the campaign is rarely known before the campaign is run.

Organization science has long asked whether reused knowledge helps. Organizations learn from their own and others’ experience [20, 21]; transferred knowledge can raise performance or, when it does not fit the receiving context, lower it [22], and transfer is hard to achieve [23, 24]; redundant ties carry little new information [25, 26]; and effort is divided between exploiting what is known and exploring what is not [27]. A materials campaign can be read, by analogy, as such an organization: its members are databases, calculations, interatomic potentials, acquisition rules, and language-model agents; its organizational memory [28] is the archive; and its structure is who shares results with whom. Self-driving laboratories coordinate such members against shared budgets [29, 30, 31], and teams of language-model agents now work on research and software tasks together [32, 33, 34, 35, 36, 37].

Benchmarks of these systems report the acceleration of the whole loop [38, 39, 40]. A whole-loop number cannot say which member carried the campaign’s knowledge forward and which held it back, and a leaderboard of members run one at a time cannot either, because members interact: two databases computed with one pipeline repeat each other, and a misaligned ordering can pull a pooled prior away from a good one. Recent multi-agent language-model systems make some organization-science constructs executable as team configurations [41], but not as values read from what a campaign found. Here every subset of the knowledge sources is run as a campaign of its own, replayed from a version-locked archive or from calculations made once, so the outcome of “the campaign without source X” is the result of a run rather than a counterfactual. Cooperative-game methods [42, 43, 44, 45] trace from those outcomes how far each source advances the campaign, and how that changes when members meet, are coordinated, or learn whom to rely on. Shapley values already divide the gains of federated and collaborative learning among data holders that train one model [46, 47, 48]; here the players are the sources of a campaign that buys measurements.

Two campaigns are studied: ranking wide-gap semiconductor candidates by measured band gap, using a version-locked archive of band-gap records from eight sources and HSE06 gaps computed on demand, and finding the Al–Mg–Si compounds with the lowest formation energy per atom, where every query triggers a density functional theory (DFT) or molecular-dynamics (MD) calculation with a measured cost. Four questions of a materials group are each read through an organization-science construct that the campaigns illustrate rather than test, and each is answered as a rule: pool a few well-aligned databases rather than all of them (knowledge integration, negative transfer); count databases from one pipeline once (redundancy); compute an ordering no database holds only when the calculations will be reused (exploration); and, when the best member is not known in advance, learn from the record whom to rely on, which beats fixed structures but does not replace the best member (communication structure, trust).

Results

The organizational terms used below (learning, trust, coordination, teams) are an interpretive lens on the campaign records; the organizational constructs themselves are not tested.

How a campaign draws on each knowledge source

A worked example fixes the terms. The band-gap campaign puts seven anion families (borides, oxides, halides, nitrides, chalcogenides other than oxides, pnictides other than nitrides, and others) in order of band gap. Each step buys the values of a tray of four candidates from a receiver source (the source whose values the campaign buys), one archived record per candidate, none drawn twice. The orderings of the donor sources form a prior with a trust weight, fixed or learned from the receiver values acquired so far, and each purchase corrects it. Quality is the Kendall τ\tau [49] between the fitted ranking and the receiver’s own fixed order of family median gaps (τ=1\tau= 1 when the two orders agree, 0 when they are unrelated); its uncertainty is not propagated into the intervals. For a set SS of donors, v(S)v(S) is the mean τ\tau over the first half of the budget (the early window) of a campaign whose prior uses only the donors in SS, averaged over matched seeds; v(∅)v(\varnothing) is the campaign without donors. All 2n2^n subsets are run.

From the measured table vv, the Shapley value ϕi\phi_i [42] is the average change that source ii makes when it joins the others in every order, and the values sum to v(N)−v(∅)v(N)-v(\varnothing); leave-one-out, v(N)−v(N∖{i})v(N)-v(N\setminus\{i\}), is what removing one source at a time from the full pool shows; and the Harsanyi dividend d(S)d(S) [43] is what a set of sources creates beyond all its subsets (negative for overlap, positive for complementarity). Communication structure restricts which sources can combine (Methods; Figure 1).

Figure 1. How a campaign draws on each knowledge source. The members of a campaign are its knowledge sources: stored knowledge (band-gap databases), created knowledge (Perdew–Burke–Ernzerhof (PBE) and HSE06 calculations, interatomic potentials), and members who act (language-model agents, an active learner, a scout). Every subset SS of members is run as a campaign of its own in the wide-gap semiconductor or Al–Mg–Si alloy system, giving the measured table v(S)v(S) over all 2n2^n subsets, which contribution measures divide among the members. Meetings (dashed) restrict which members combine. The measures also use Owen levels, nested groups (a share is divided first between groups, then within each), and Beta(16,1) weights, which count small pools most.

Pooling band-gap databases: more stored knowledge does not make a better campaign

A group that combines public band-gap databases to advise a measured-gap campaign has to decide which databases to keep. Seven databases advised the campaign on the measured gaps of the compilation of Zhuo et al. [50]: the Materials Project (MP) GGA records [1], the MP GGA records of experimentally synthesized compounds identified by ICSD provenance [51], MP r2r^2SCAN, JARVIS OptB88vdW and TB-mBJ [2, 52], and SNUMAT GGA and HSE06 [3]. All 128 subsets of this seven-source game were run with 10 matched seeds (Supporting Information Section S2).

Early in the budget, integrating every database held the campaign back. The campaign without donors reached τ=0.695\tau= 0.695 and the full pool of seven 0.737, but the best pool of three (MP r2r^2SCAN and both SNUMAT sources) reached 0.851, and from three to seven donors the best attainable value fell with each added source. Chosen on five seeds and scored on the other five, the best pool of three was the same in both halves, so this check shows that the choice is stable rather than testing it on independent data: the held-out pool exceeded the full pool by 0.115 (95% interval from 0.037 to 0.193; the ten-seed interval, not adjusted for selection); in the final window the halves chose different pools and the difference was not resolved (+0.057, [−0.023, +0.137]; Supporting Information Section S2). The two SNUMAT databases contributed most (Shapley value +0.054 each) and MP GGA least (−0.029); the intervals of this game are read as descriptive (Methods). With seven families one exchange of two neighboring families moves τ\tau by 0.095; differences below about 0.03 are reported but not ranked.

A post-hoc eight-donor game repeats the pattern. After the seven-source game had been run, the MP GGA records were split at the Materials Project rule that applies GGA+U [53] into the transition-metal oxides and fluorides, MP GGA+U (TM-O/F), and MP GGA (no +U), and all 256 donor subsets were run with 10 matched seeds (Figure 2; Methods). This game is exploratory and serves only to locate the loss; the headline values are those of the seven-source game.

Every donor subset run and contribution of each database

Figure 2. In the post hoc eight-donor game, integrating all eight band-gap sources reaches a lower early ranking quality than the best pool of four. (a) All 256 donor subsets, each a campaign run with 10 matched seeds: dot, seed mean; line, 95% tt interval over seeds; orange, subsets that hold the MP GGA+U (TM-O/F) subset. Full pool τ=0.751\tau= 0.751, best pool of four 0.879, no donors 0.695. (b) Shapley contribution of each source with the 95% tt interval over seeds; colors mark the database family (Materials Project blue, its TM-O/F subset orange, JARVIS green, SNUMAT purple). Quality: Kendall τ\tau of the anion-family ranking against the measured gaps, mean over the first five of ten budget checkpoints.

The full pool reached 0.751, but the best pool of four (JARVIS TB-mBJ, MP GGA (no +U), and both SNUMAT sources) reached 0.879 (the in-sample best of the 256 subsets, not adjusted for selection), and from four to seven donors the best attainable value fell with each added source (all eight, 0.751; Supporting Information Section S3); chosen on five seeds and scored on the other five, the best pool still exceeded the full pool by 0.128 (95% interval from 0.061 to 0.195; Supporting Information Section S2). The two SNUMAT databases carried the largest contributions (+0.051 each). In this post-hoc split, which is confounded with chemistry and generates a hypothesis rather than testing one, the TM-O/F subset held the campaign back most (−0.067-0.067, interval [−0.091,−0.042][-0.091, -0.042]) while the rest of MP GGA advanced it: a block of records that does not fit the receiving context [22, 54], where the control does not resolve whether the loss comes from the +U records or from the chemistries they cover (Supporting Information Section S2).

The loss appears when many sources are integrated. The pooled prior acts as a vote whose weight each added source dilutes. The dividends (Figure 3) give two readings. First, pairs mostly help: 17 of the 28 pairs have a positive dividend in point estimate, though none has an interval above zero, and the only pair whose interval lies below zero is the two SNUMAT databases (−0.107-0.107, interval [−0.200,−0.014][-0.200, -0.014]), exactly minus one of them, because either one alone produces the same campaign as both. Second, in the post hoc eight-donor game the loss sits, in point estimate, in interactions of three or more sources: the eight sources alone add +0.219+0.219 over the campaign without donors and the pairs −0.031-0.031, but higher-order dividends of alternating sign net −0.132-0.132, so the full pool keeps only +0.056+0.056. Over ten seeds the interval of the pair sum includes zero, and the higher orders do not settle the location either: orders four to seven each have an interval that excludes zero, but they alternate in sign, and their net is a point value by subtraction with no interval of its own. This is a reading of the point estimates, not a resolved location. These high-order terms are what a saturating, mean-of-orderings prior produces and are the noisiest entries of the table, so they name no mechanism among particular databases. Past a few well-aligned sources, more stored knowledge, pooled with one vote per source, lowers what the campaign knows early.

Harsanyi dividends by subset size and pair dividends

Figure 3. In point estimate, the value of integrating databases is lost in large pools rather than between pairs (post hoc eight-donor game); the ten-seed intervals do not separate the two. (a) Harsanyi dividends of the eight-donor campaign summed by subset size: positive and negative mass, and the net with its 95% tt interval over 10 seeds; order sums can exceed the range of τ\tau because terms of opposite sign cancel. A dividend is what a set of sources adds beyond all its smaller subsets; orders three to eight carry large dividends of alternating sign. (b) Dividend of every pair (×100, Kendall τ\tau; color scale clipped at ±8, the outlined cell lies beyond it): 17 of 28 pairs have a positive point estimate, none with an interval above zero; the only pair whose interval lies below zero is the two SNUMAT databases.

Databases from one pipeline: redundant knowledge and what a removal screen misses

Two copies of one pipeline count once alone and twice in a pool. The two SNUMAT databases carry identical family orderings, the database analogue of the structurally equivalent contacts of Burt [26], so the second adds nothing to a pool that holds only the first. In 59 of the 63 pools of the eight-donor game that also held another donor, however, it added to the outcome (mean +0.040, up to +0.093; Figure 4a), because it doubles the weight of the SNUMAT ordering in the vote. With one vote per distinct ordering instead, the second copy adds exactly nothing to every pool that holds the first, the full pool falls to 0.707, and the contribution of each copy falls to +0.032 (Supporting Information Section S3). What redundancy does therefore depends on how the campaign pools its sources, not on the redundancy alone. The doubling belongs to per-source pooling, and the Shapley value, which pays both copies by symmetry, passes it on; replication-robust valuations of data sources were proposed for this reason [55, 56, 57].

Four panels showing redundant knowledge, source contributions, and seed agreement

Figure 4. Redundant knowledge counts once alone and twice in a pool, and the measures disagree on which source is worst. (a) Gain in early ranking quality from adding SNUMAT HSE06 to each pool that holds SNUMAT GGA, against the number of other donors in the pool: zero when SNUMAT GGA is alone, positive in 59 of the 63 pools with another donor (dots, pools; lines, means). (b) Contribution of each source under four measures on the same 256 measured campaigns; sources ordered by Shapley value, colors by family (MP GGA+U (TM-O/F) subset orange). Only the Shapley values sum to v(N)−v(∅)v(N) - v(\emptyset). (c) Agreement of each measure’s ranking between two seeds: dot, mean Spearman correlation over the 45 seed pairs; whisker, 95% bootstrap interval over seeds.

A removal screen misses what the subset runs show. A group that screens its subscriptions by removing one database at a time from the full pool computes leave-one-out values (Figure 4b), and every mean was positive, so the screen would keep all eight donors: at the full pool the TM-O/F subset and the ICSD subset, which still holds GGA records of the same chemistries, mask each other, as redundant ties do [25, 26]. Over the 10 seeds, leave-one-out singled out the source that Shapley ranked lowest in only two. Read from the measured values instead, dropping in turn the source whose removal raises the value most or lowers it least removes the TM-O/F subset, the ICSD subset, JARVIS OptB88vdW and MP r2SCANr^{2}\mathrm{SCAN} and reaches the best of all 256 pools after four steps (Supporting Information Section S3). The useful pool size decides which sources to keep. Weighting small pools, Beta(16, 1) gave JARVIS TB-mBJ a positive contribution where Shapley gave a negative one (+0.005 against −0.009) and the TM-O/F subset its most negative value (−0.109), so the measure should match the pool size a group intends to use. Across seeds the Banzhaf ranking, which averages each source’s gain uniformly over pools, was the most consistent (mean Spearman correlation between two seeds 0.89, against 0.85 for Shapley and 0.28 for leave-one-out; Figure 4c), although its margin over Shapley lies within the bootstrap interval of the difference (Supporting Information Section S3).

Computing what no database holds: progress once the compute is charged

The games above reuse stored knowledge. Exploration creates it, at a cost [27]. In the first exploring campaign, 171 semiconductors and ordered semiconductor alloys were relaxed with the Perdew–Burke–Ernzerhof (PBE) functional [4] in Quantum ESPRESSO [58] and their HSE06 gaps computed at the PBE geometry; 24 of the 32 alloys are held by no database. The goal was the ten widest HSE06 gaps among the 142 candidates without halogens, three of which are such alloys. Five sources advised: the computed PBE gaps and four archives (MP GGA, JARVIS TB-mBJ, SNUMAT HSE06, and the measured compilation), with v(S)v(S) the early share of the ten found, over 20 seeds and batches of one and four, all 32 subsets run (Figure 5a).

Three-panel bar charts showing contributions for band gaps and Al–Mg–Si

Figure 5. Created knowledge advances a campaign until its cost is charged. Charging the PBE relaxations of the whole pool turns the PBE ordering from the largest contribution to a negative one in both systems (hatched bars), the most negative for band gaps under both trust settings and for Al–Mg–Si under learned trust. Value: early share of the ten targets found; all subsets run (32, 16, and 32), each over 20 seeds and batches of one and four. Solid, fixed trust; light, learned trust; hatched, relaxations also charged. (a) Band gaps, HSE06 computed on demand. (b) Al–Mg–Si, PBE receiver, four low-cost sources. (c) Al–Mg–Si, HSE06 receiver, five sources. Whiskers: 90% percentile-bootstrap intervals over the 40 runs, conditional on the one fixed pool; in (b) and (c) every MD value is drawn from its measured sampling error.

The source that covered the unexplored candidates advanced the campaign most. Charging HSE06 compute only, the computed PBE ordering contributed +0.203 under fixed trust, against +0.018 to +0.040 for the archives, because it was the only source that held the alloys; three of the ten targets are such hypothetical ordered alloys. Charged its own cost, exploration held the campaign back. A campaign that borrows the PBE ordering must first relax all 142 candidates, 9.9% of that campaign’s combined PBE and HSE06 compute; charging that cost, the computed source contributed the most negative value (−0.15 fixed, −0.26 learned trust), and the best subsets held archived sources only. A group that reuses the relaxations in several campaigns charges each an equal share [59]. Under this accounting the PBE ordering contributed a positive value when charged less than 0.57 of the relaxations (fixed trust; 95% bootstrap interval [0.50, 0.63]) or 0.34 (learned trust; [0.28, 0.39]), that is, once the relaxations served about two or three campaigns, judged at a quarter of the receiver compute. Over the whole budget the relaxations pay after 1.2 to 2.2 campaigns, and with the alloys removed from the pool, or kept as candidates but barred as targets, the fixed-trust multiple becomes 2.1 or 2.4, so the break-even is a statement about the early window of this band-gap pool (Supporting Information Section S11).

The same reversal held for the Al–Mg–Si alloy, where every query was a calculation. The targets were the ten of 168 Al–Mg–Si compounds with the lowest formation energy per atom, 141 of them ordered fcc decorations that no archive holds, with PBE as the receiver and four low-cost sources: the MACE-MP-0 foundation potential [60] relaxed at 0 K, MD at 300 K with NEP89 [61] and with the modified embedded-atom method (MEAM) potential of Jelinek et al. [62], and the Materials Project archive. Under fixed trust MACE-MP-0 contributed +0.23 and the archive +0.15, while NEP89 MD and MEAM MD held the campaign back (−0.20, −0.11); the full pool (0.59) stayed below MACE-MP-0 alone (0.88), and learned trust made all four contributions non-negative. With HSE06 as receiver and PBE as a fifth source, PBE and MACE-MP-0 led (+0.17 each) until the PBE relaxations of the pool (9.3% of the Al–Mg–Si campaign’s combined compute) were charged; PBE then fell to −0.11 (fixed) and −0.20 (learned), and MACE-MP-0 alone was the best subset. The most accurate created knowledge is therefore not the most valuable once the exploration that produced it is paid for, and a statement of what a source contributed has to name the compute it charges.

Data providers, meetings, and coordinators: communication structure

Knowledge moves more readily between units that are tied to each other [63]; here the databases of one organization share a workflow and a release schedule, and this provenance declares meetings. A meeting is a set of members who share results, convened only when all of them are present (Methods). Meeting structure reordered the contributions (Figure 6). With organization meetings joined by a cross-organization beyond-GGA meeting, the tie between the SNUMAT databases broke in favor of SNUMAT GGA (+0.063 against +0.028), and JARVIS OptB88vdW rose from rank 4 to rank 3, though its sign is not resolved (positive in 7 of 10 seeds; Supporting Information Section S4). A screen of a group’s sources should therefore be read under the structure the group actually uses.

Meetings reorder the contributions of the databases

Figure 6. Meetings reorder the contributions of the databases. (a) The declared meetings: one per data provider (shaded) and a cross-organization beyond-GGA meeting (dashed) of JARVIS TB-mBJ, MP r2r^{2}SCAN, and SNUMAT HSE06; members combine only through a meeting they share. (b) Contribution of each database to the early ranking quality without structure (Shapley) and under the meetings, valued by the sum rule (a separated team is worth the sum of its groups) and by the best-group rule (worth its best group), with the equivalent pairwise links for comparison. In point estimate (one run), the meetings break the tie between the SNUMAT databases and move JARVIS OptB88vdW from fourth to third under both rules.

Figure 7 shows the team structures compared: the language-model agent teams (a), discussed here, and the seven-member Al–Mg–Si team (b). Five language-model agents (Qwen3-4B [64]) ranked the seven anion families of the measured compilation, each holding one database’s family ordering as private knowledge, a setting closer to opinion aggregation than to a hidden profile, in which the facts that decide the answer are split among members [65]; in each of 12 rounds every agent loaded one four-family tray. Flat teams loaded few distinct trays and trailed the random-tray search baseline: only 5–10% of their trays were distinct, and the five-agent flat team reached 0.756 against 0.876 for random trays at the same budget, a search baseline (sd 0.096 over 20 random searches of 240 evaluations each), a coordination loss against this baseline rather than against a nominal group (the same members working alone, results pooled), akin to process loss [66, 67, 68].

A coordinator raised the team, but no coordinated team beat the random-tray search baseline. Coordinated minus flat team τ\tau, paired by seed, was +0.096 [+0.029, +0.165] for five Qwen agents, +0.121 [+0.002, +0.229] for five Nemotron agents, a lower bound just above zero, and +0.052 [+0.006, +0.102] on a finer task of 13 anion-element groups, a lower bound narrowly clear of zero, with the effect uneven across seeds (10 of 20 seeds positive); that interval resamples whole seeds rather than treating the 100 coordinated-minus-flat differences as independent, because each seed’s flat run serves all five coordinator rotations; for ten Qwen agents the interval includes zero (+0.045 [−0.019, +0.105]), and in no agent set did a coordinated team beat the random-tray search baseline (0.876, sd 0.096, five agents, a spread rather than the coordinator effect; 0.929, sd 0.073, ten agents; Figure 8). As a process count, coordinated minus flat, paired by seed, the share of distinct trays rose in every set, by +1.75 [+0.42, +3.25] percentage points for five Qwen agents, +11.67 [+8.17, +15.33] for five Nemotron agents and +2.81 [+1.56, +3.85] for ten Qwen agents, and coordinated teams still repeated most of their trays. The agents were small models, decision order and the database the coordinator held remain confounded with its effect, and six of the twelve predictions made before the runs failed (Supporting Information Section S5). Replay valuation [69, 70] assigns the coordinator’s effect to the agents it directed.

Team ranking quality and coordination value

Figure 8. A coordinator raises the team; replay valuation assigns the gain to the agents it directs. (a) Team ranking quality of every recorded campaign of language-model agents, flat (open) and with a coordinator (filled), in three agent sets; bars, means; dashed, random trays at the same budget as a search baseline (mean of 20 random searches; sd 0.096 for five agents, 0.073 for ten), above every coordinated mean. (b) Interventional coordinator effect on team τ\tau (coordinated minus flat, paired by seed) in each set and on the finer task with the coordinator role rotated over the five databases, and the coordinator’s own share (role premium) under replay valuation; lines, 95% bootstrap intervals (for the finer task resampled by seed, since each flat run serves all five rotations); filled markers, intervals excluding zero; open markers, intervals including zero (not resolved).

The team structures compared

Figure 7. The team structures compared. (a) Language-model agent teams: flat, with all agents acting at once on a shared notebook, or led by a coordinator (agent 1) that receives the agents’ messages and assigns the trays. The coordinator team is compared with the flat team of the same seed in three agent sets and on a finer task with the coordinator role rotated over the five databases. Random trays at the same budget are the search baseline; replay valuation assigns the coordinator’s effect to the agents it directed. (b) The seven-member Al–Mg–Si team in the structures of Figure 9: members working alone, a flat team, a fixed supervisor, tiers by nominal fidelity, and the team ranked by record, which re-ranks members into tiers after every batch and merges agreeing members (ablations: Ranked by record, merging off; Merging only, no tiers; Supplementary Section S6). MACE-MP-0 alone is the reference.

An adaptive team in the Al–Mg–Si campaign: learned trust

Fixed structures are declared before a campaign. A group running many advisers at once can instead let its structure adapt while the alloy campaign runs. Seven members ran the Al–Mg–Si campaign together (168 compounds, batches of four, PBE receiver, with HSE06 and r2r^{2}SCAN held out): five low-cost orderings (MACE-MP-0 relaxation, NEP89 MD and relaxation, MEAM MD, and the archive), an active learner with no prior, and a random scout. Every sub-team of every organization was run in 100 campaigns, with seeds that differ between organizations, so intervals that compare organizations are unpaired (Figure 9).

Team value by structure and contribution change

Figure 9. A seven-member team ranked by its own record found more of the ten Al–Mg–Si compounds with the lowest formation energy per atom early than any fixed structure, on the receiver its settings were tuned on and on the held-out ones, but fewer than MACE-MP-0 alone. (a) Early share of the ten found (first quarter of the receiver compute), mean over 100 campaigns per structure with the 90% bootstrap interval, for the PBE receiver (on which learned settings were tuned) and the held-out HSE06 and r²SCAN receivers; dashed lines, MACE-MP-0 alone (0.808 on PBE in this seven-member setting, against 0.88 in the four-source game of § 2.4); shaded rows, ablations of the record-ranked team. (b) Shapley value of each member in the record-ranked team minus its value in the flat team; all 128 sub-teams run per structure. The differences are descriptive, not normalized, since the two structures have different totals.

A team that learns from its record whom to rely on finds low-formation-energy compounds earlier than any fixed structure, though not as early as its best member alone. After every batch each member was re-ranked into tiers by how well its scores had anticipated the results it could see, and members whose orderings agreed were merged for the rest of the campaign into one member that proposed once per batch. Scored on the held-out HSE06 receiver, this team, with its default settings, found 0.49 of the ten compounds with the lowest formation energy per atom in the first quarter of the compute, against at most 0.40 for fixed teams (a gap of 0.09, 90% bootstrap interval 0.07–0.11 over 100 campaigns per structure), but MACE-MP-0 alone found 0.73. On the PBE receiver, on which the learned settings were tuned, the order was the same (0.70 with default settings, against 0.64 for tiers assigned by nominal fidelity and 0.61 for a flat team). Merging carried most of the gain: once the potentials had lost trust in their own orderings they proposed from the same composition surrogate, and merging them returned their duplicated batch slots to the leader. This is a simple analogue of the credibility part of transactive memory [71, 72]; it is not strong synergy, since the team stayed below its best single member [73, 74].

In these campaigns, combining groups as records accumulated never lowered the outcome; separating them lowered it in most settings. When a coordinator could fuse, nest, split, or detach groups, combining raised the outcome above the fixed structure with the same start in 10 of 12 settings and separating lowered the mean in all 12; members allowed to move to the group whose ordering they agreed with raised the early value by +0.026 to +0.071 across the three receivers, and these sharing rules kept the results of NEP89 MD away from the groups. The settings share members and starts, so these counts are descriptive, exploratory, and hypothesis-generating (Methods; Supporting Information § 14–§ 17).

Discussion

The measured campaigns suggest four working rules for a materials group, each consistent with an organization-science construct that the campaigns illustrate rather than test. Each rule is a hypothesis from the receivers and budget windows measured here.

Pool a few well-aligned databases (knowledge integration has an optimum [75]; in data integration, adding sources can lower quality [76], and an ensemble of a few models can beat one of all [77]). In both band-gap games the full pool scored below smaller pools, and the loss arose when many sources were integrated. A campaign should integrate the few sources whose orderings fit the receiver, chosen on the receiver’s own results, rather than every source it can access. Because each pool is run as a campaign, its value is an outcome rather than a score of agreement, and a post-hoc split can locate a block of records that holds a loss, a hypothesis for the next campaign.

Treat databases from one pipeline as one source (redundancy depends on integration, as in groups pooling information [78], and copying between sources inflates their votes in truth discovery [79]). Two databases from one pipeline repeat each other alone and double each other’s weight in a pool, and a removal screen sees only the full pool, where one copy masks the other. A group should integrate them as one source unless it deliberately weights them: one vote per distinct ordering removes the double weight, whereas valuing them as one player or as one Owen union does not, because the second vote sits in the pooled prior itself.

Charge the compute of created knowledge (exploration must be paid for). Early in the budget, created knowledge that covers candidates no database holds advanced the campaign most, and the same knowledge held it back once the calculations that produced it were charged. A group deciding whether to compute its own low-level ordering should therefore do so when the ordering covers candidates the databases lack and the calculations will be reused across campaigns, because charged to one campaign they cost more early budget than they return; exploration that pays only over a horizon longer than one campaign is the trade-off March [27] describes.

When the best member is not known in advance, learn from the group’s own campaigns which members and sources to draw on (trust learned from the record). Meetings reordered the contributions of the databases, a coordinator raised agent teams, and a team that learned from its record whom to rely on found low-formation-energy compounds earlier than every team whose weights were set in advance. Neither baseline was beaten: no language-model team reached the random-tray search baseline (0.876, sd 0.096 over 20 random searches of 240 evaluations each), and MACE-MP-0 alone found more targets early than the adaptive team. Learning from the record does not replace finding the best member.

Cost of the method. Running every subset costs 2n2^n campaigns: a few core-hours for the 256-subset band-gap game, which replays archived records, and about 270 core-hours for the Al–Mg–Si PBE pool, which replays calculations made once (Supporting Information Section S11). A larger group can remove sources one at a time and keep the best pool seen, as in § 2.3, in about n2/2n^2/2 campaigns; estimate Shapley values from sampled orderings (Supporting Information § 9); or reweight its members by their record during the campaign.

Data valuation assigns Shapley-type values to training points and data sources [80, 81, 82, 83], and Beta Shapley and Data Banzhaf were proposed because Shapley rankings are noisy and weight large subsets heavily [84, 85]; the higher seed consistency of the Banzhaf ranking and the change of sign of JARVIS TB-mBJ under Beta(16, 1) agree with that motivation, here on run campaigns rather than retrained models. Negative transfer is widely reported [54]; here it appeared in large pools rather than in pairs. Benchmarks of self-driving laboratories report the acceleration of a whole loop [38, 39]; subset campaigns decompose it into members, at the cost of 2n2^n runs. Computational organization theory studies these constructs in stylized agent-based models [27, 86]; run materials campaigns add real knowledge sources whose every subset’s outcome is run rather than modeled.

Limitations. The band-gap campaigns replay archived records, and the structural results rest on one fully measured game (one measured receiver, an equal-vote prior, the early window); whether the optimum pool size and the TM-O/F negative transfer hold for another receiver, such as SNUMAT HSE06, is untested. The language-model teams were compared with a search baseline, not with the control organization science would use, a nominal group of the same members working alone with their results pooled afterwards. The eight-donor game is post hoc, since the GGA+U split and the lowered entry threshold came after the seven-source game had been run, and its split is confounded with chemistry. The computed source’s lead rests partly on hypothetical ordered alloys; the computed band-gap game has one HSE06 receiver without spin–orbit coupling, the Al–Mg–Si receivers are calculations, and formation energies at temperature omit entropy. The agents were small models (Qwen3-4B, and Nemotron 30B-A3B in one set), so duplication and the gap to the random-tray search baseline (sd 0.096 over 20 random searches) may reflect model capability as well as team structure, and the coordinator effect is not separated from turn order or from the database the coordinator held. The adaptive team had seven computational members and DFT receivers; it is a fixed re-ranking rule with set thresholds, and nothing in it closes the loop to an experiment. The constructs are read by analogy: the members are knowledge sources and agents without goals of their own, and people were not modeled.

Conclusions

When every subset of a materials campaign’s knowledge sources is run, what carries the campaign’s knowledge forward can be read from the campaign record rather than assumed. Early in one band-gap campaign, with one measured receiver and an equal-vote prior, integrating more databases helped up to a few well-aligned sources and then held the campaign back (Kendall τ\tau 0.737 for all seven databases against 0.851 for the best three). Databases from one pipeline counted once alone and twice in a pool. Created knowledge advanced the campaign until its cost was charged. A team that learned from its record whom to rely on did better than structures fixed in advance, but not better than its best member. Because the values are read from runs on the receiver’s own measurements, what carries to a new campaign is the rule (which kinds of source to pool, merge, or charge for), not the numbers.

Methods

Band-gap campaigns. The archive held 261,053 records from eight sources, the measured gaps of Zhuo et al. [50] among them, version-locked by a manifest hash. In the eight-donor game the MP GGA records were split by the Materials Project GGA+U rule (Co, Cr, Fe, Mn, Mo, Ni, V, or W with O or F) into the TM-O/F subset (31,828) and MP GGA (no +U); a donor entered the prior when it held at least two anion families of at least 30 records each (four in the seven-source game). Each archived record was drawn at most once per campaign. Rankings were fitted with the Plackett–Luce model [87, 88]; the donor prior, the mean of the coalition’s orderings with one vote per source, entered with trust weight λ\lambda, fixed or chosen by its prequential log-score [89] (Supporting Information Section S1).

Computed campaigns. The band-gap pool held 171 compounds, relaxed with PBE in Quantum ESPRESSO 7.4 [58] with HSE06 single points at the PBE geometry; its MgCaO and ZnBeO alloys are hypothetical. The Al–Mg–Si pool held 168 compounds. Each step computed four compounds chosen by blending the pooled donor ordering with a composition surrogate fitted to the receiver values so far. MD formation energies (NEP89 in GPUMD [90], MEAM in LAMMPS [91]) were time averages; in each of 20 runs every MD value was drawn from its measured standard error. Every subset paid the PBE relaxation of each compound it queried, and a subset that borrowed the PBE ordering paid that of the whole pool, so no relaxation was charged twice. v(S)v(S) was the mean share of the receiver’s ten best candidates found over the first quarter of its compute (Supporting Information Section S11).

Contribution measures. The players have no preferences or strategies, so no bargaining claim is made. Beside the Shapley value [42], leave-one-out [92] and Harsanyi dividends [43] defined in the Results, the Banzhaf value averages the same marginal changes uniformly over subsets [93] and Beta(α\alpha, β\beta) semivalues reweight subset sizes [84]. A communication structure restricts the game: with meetings HH, only meetings contained in SS convene, so a meeting of three is not three pairwise links, and the value of SS is the sum (sum rule [44, 45]) or the maximum (best-group rule) of the zero-normalized values v(C)−v(∅)v(C) - v(\varnothing) of the groups CC the convened meetings link, so the empty team is worth zero and, under the best-group rule, the efficient values sum to max⁡C[v(C)−v(∅)]\max_C[v(C) - v(\varnothing)]. Teams split into fixed groups use the Owen value [94]. All values were computed exactly by enumerating every subset (definitions and axioms in Supporting Information § 9).

Language-model teams. Agents ran Qwen3-4B-Instruct-2507 or Nemotron 3.5 Lightning 30B-A3B, served with vLLM 0.29.0 [95] at temperature 0.7; each held one database’s family ordering, and meetings were read from messages and notebook citations. Twelve predictions were recorded in time-stamped commits before the campaigns, and six held (Supporting Information § 13 and § 19).

Statistics. In the eight-donor game intervals are tt intervals over 10 seeds, with Holm-adjusted tests of a zero Shapley value [96]; elsewhere they are percentile bootstrap intervals [97] over seeds or campaigns (paired by seed where the text says so, resampled by seed where one flat run serves several coordinated runs, and unpaired between organizations), 95% for the seven-source and agent-team games and 90% for the computed and organization games. The organization games are uncorrected for multiplicity and exploratory; the seven-source intervals and those of agent sets S2 and S3 (8 to 10 seeds) are descriptive (sample sizes in Supporting Information § 9). All intervals are conditional on the fixed pool of sources, candidates and receivers: they describe variation over seeds or campaigns, not over other pools.

Supporting Information

Data availability statement

Code, campaign logs, subset values, the version-locked archive manifest, and the scripts that draw every figure are openly available on Zenodo as release v1.3.0 (https://doi.org/10.5281/zenodo.23413476; this DOI resolves to the latest version), with code under the MIT License and data under CC BY 4.0, following the FAIR principles [98]. The band-gap archive is identified by its build fingerprint: it is rebuilt with the kit in this release, and the rebuilt manifest has SHA-256 f2158ecdc78e022926f6b05a40433152f069d706ec4d2cb706977e99c7a531f3; no separate record of the archive is deposited.

Funding

Taylor D. Sparks and Kinston Ackölf acknowledge support from the Army Research Office Materials Design, under Contract No. W911NF-23-1-0333. Taylor D. Sparks acknowledges support from the National Science Foundation through the AURORA Programmable Cloud Lab (NSF Award No. 2607535).

Conflict of interest

The authors declare no conflict of interest.

Use of generative AI

The language-model agents studied in the agent-team campaigns are part of the research and are described in Methods.

Author contributions

Kinston Ackölf: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing–original draft, Writing–review & editing. Taylor D. Sparks: Funding acquisition, Resources, Supervision, Writing–review & editing.

How this Supporting Information is organized

Sections S1–S3 cover the band-gap campaigns: the ranking model and estimators (S1), the seven-source and eight-donor games with their selection checks and controls (S2), and the contribution measures compared (S3). Sections S4 and S5 cover communication structure and the language-model agent teams. Sections S6–S10 cover the adaptive Al–Mg–Si teams and their variants. Section S11 gives the extended Al–Mg–Si games, the cost-share sweep and the composition fits, and S12 lists code, data and result files.

Organization-science constructs and the quantities that stand for them

The main text reads each campaign question through one organization-science construct, by analogy. Table S1 lists, for each construct, the campaign quantity that stands for it and where it is reported.

ConstructQuantity in the campaignsWhere
Knowledge integration; negative transferbest value by pool size; full pool against the best pool; Shapley value of each database and of the MP GGA+U (TM-O/F)2.2; S2, S3
Redundancy; structural equivalenceincrement of the second SNUMAT copy; leave-one-out against Shapley; one vote per ordering2.3; S3
Exploration and its costcontribution of the computed PBE ordering under HSE06-only and charged cost; cost-share sweep2.4; S11
Communication structure (meetings)restricted-game values under declared and recorded meetings (sum and best-group rules)2.5; S4
Coordination; process losscoordinator effect paired by seed; share of distinct trays; random-tray search baseline with its spread2.5; S5
Organizational learning (trust learned from the record)early share of the record-ranked team against fixed tiers and a flat team; ablations; held-out receivers2.6; S6
Boundary spanningmembers that move between groups or sit in two2.6; S9

Table S1. Constructs and campaign quantities. Sections refer to the main text (2.x) and to this Supporting Information (S).

Ranking model, trust, and sampling

The Plackett–Luce fit and the anchored update

An observed event ee that ranks its kk participants r1≻r2≻⋯≻rkr_1 \succ r_2 \succ\cdots\succ r_k contributes the sequential-choice likelihood

P(e)=∏s=1k−1wrs∑l=skwrl,wi=exp⁡(θi)>0,(1)P(e)=\prod_{s=1}^{k-1}\frac{w_{r_s}}{\sum_{l=s}^{k}w_{r_l}},\qquad w_i=\exp(\theta_i)>0, \tag*{(1)}

fitted by Hunter’s MM iteration [99], which collects over all stages of all events the statistics

Wi=total weight of stages won by i,Di=∑stages with remaining set R∋iweighte∑j∈Rwj,(2)W_i = \text{total weight of stages won by }i,\qquad D_i = \sum_{\text{stages with remaining set }R\ni i}\frac{\text{weight}_e}{\sum_{j\in R}w_j}, \tag*{(2)}

and updates wi←(Wi+λ0)/(Di+λ0/wref)w_i \leftarrow(W_i+\lambda_0)/(D_i+\lambda_0/w_{\mathrm{ref}}) with a light damping prior λ0=0.01\lambda_0=0.01 anchored at wref=1w_{\mathrm{ref}}=1. For k=2k=2 the fixed point of the undamped update satisfies wa/wb=winsa/winsbw_a/w_b=\mathrm{wins}_a/\mathrm{wins}_b, exactly Bradley–Terry [100]. The anchored MAP fit used when prior and data are both present replaces the damping anchor by the mean-centered prior πi=exp⁡(θiprior)\pi_i=\exp(\theta_i^{\mathrm{prior}}) with trust weight λ≥0\lambda\ge0,

wi←Wi+λDi+λ/πi,(3)w_i \leftarrow\frac{W_i+\lambda}{D_i+\lambda/\pi_i}, \tag*{(3)}

The limiting regimes are as follows. With no data, the fixed point is w=πw=\pi. With a flat prior, the update reduces exactly to the unanchored model. As events accumulate, the prior’s influence decays as λ/nevents\lambda/n_{\mathrm{events}}. When the trust weight is learned, a grid of candidate λ\lambda values is maintained. Each incoming event is scored under the model fitted to all previous events at each λ\lambda (prequential scoring, [89]). The cumulative log-score selects λ\lambda, with ties broken toward smaller λ\lambda.

Monte-Carlo estimators and their observed convergence

The sampled-permutation Shapley estimator draws mm uniform permutations σ(1),…,σ(m)\sigma^{(1)},\ldots,\sigma^{(m)} of NN and averages each player’s marginal contribution at its arrival position,

ϕ^i=1m∑t=1m(v(pre⁡i(σ(t))∪{i})−v(pre⁡i(σ(t)))),(4)\widehat{\phi}_i=\frac{1}{m}\sum_{t=1}^{m}\left(v\left(\operatorname{pre}_i(\sigma^{(t)})\cup\{i\}\right)-v\left(\operatorname{pre}_i(\sigma^{(t)})\right)\right), \tag*{(4)}

where pre⁡i(σ)\operatorname{pre}_i(\sigma) is the set of players preceding ii in σ\sigma; the Banzhaf estimator averages marginal contributions over uniformly sampled subsets instead. On the fully measured seven-source game (MP GGA unsplit, Section S2), whose 128 coalition values are all cached, the median maximum absolute value error of the sampled Shapley estimate is 8.57×10−48.57\times10^{-4} at m=3,000m=3{,}000 permutations. By the symmetry axiom, two players whose exact values are equal (the two SNUMAT donors, whose class orderings over the archive are identical; the HSE06 field is a separate calculation, not a shift of the GGA value: over the 10,481 paired records it lies 1.08 eV above GGA on average with a standard deviation of 0.50 eV and no equal pair, record-level Spearman ρ=0.94\rho=0.94, yet the order of the anion-family medians is unchanged) cannot be strictly ordered by a sampled estimate at any mm, because the difference of their estimands is exactly zero. The truncated variant of Ghorbani and Zou [80] sets the marginal contribution of every later player in a permutation to zero once the value of the prefix is within a tolerance of v(N)v(N); its error on the same game is reported in Section S3.

Extended methods

Each step buys the gaps of a tray of four candidates; before the first purchase the databases’ own orderings of the families serve as a starting guess, and each purchase corrects that guess.

The reference ranking is the receiver’s own fixed order of family median gaps (oxides 3.33, nitrides 3.20, and halides 3.02 eV at the top); its uncertainty is not propagated into the intervals. A purchased evaluation returns the receiver’s order for a tray of four candidates, one archived record per candidate, and no record is drawn twice within a campaign: a campaign that can afford 240 evaluations draws 240 different archived measurements, none repeated.

Or it can estimate Shapley values from sampled orderings of its sources, running only the subsets those orderings visit; on the seven-source game, 1,000 sampled orderings gave a median maximum error of 0.0014 and no sign reversal in 50 repeats (Section S1, which also reports 3,000 orderings); with seven sources these orderings still visit nearly every one of the 128 subsets, so this checks the accuracy of the estimator, and the saving in campaigns appears only for larger groups.

Band-gap archive and campaigns. The archive held 261,053 records from eight sources, version-locked by a manifest hash: MP GGA/GGA+U (95,920 records), MP GGA restricted to ICSD-provenance compounds (27,347), MP r2r^{2}SCAN (14,672), JARVIS OptB88vdW (75,993) and TB-mBJ (19,805), SNUMAT GGA and HSE06 (10,481 each), and the measured compilation of Zhuo et al. [50] (6,354). In the eight-donor game the MP GGA records were split by the Materials Project GGA+U rule, a formula holding Co, Cr, Fe, Mn, Mo, Ni, V, or W together with O or F, into the MP GGA+U (TM-O/F) subset (31,828) and MP GGA (no +U; 64,092); the ICSD subset was not split, and 5,433 of its records fall under the rule.

Candidate identity was the reduced formula with space group; the measured compilation lists no space group, so its records were matched by reduced formula alone (973 of its 4,921 formulas have more than one computed space group in the archive), which leaves the anion family of every record unchanged; no source was designated ground truth.

Rankings were fitted with the Plackett–Luce model [87, 88] by minorization–maximization [99]; a donor prior, the mean of the coalition’s donor orderings with one vote per source (so two sources with one ordering carry it twice), entered as an anchor with trust weight λ\lambda, fixed or chosen after every event by its prequential log-score [89] (Section S1).

Contribution measures. Each game is a transferable-utility attribution game with v(∅)≠0v(\varnothing) \ne0. Its players are knowledge sources or members without preferences or strategies, so no bargaining, core or equilibrium claim is made, and each value is an allocation rule chosen for its axioms: the Shapley value is the one rule that is efficient, symmetric and additive and gives a member that changes no outcome zero [42].

Harsanyi dividends are d(S)=∑T⊆S(−1)∣S∣−∣T∣v(T)d(S) = \sum_{T \subseteq S}(-1)^{|S|-|T|}v(T), so that v(S)=∑T⊆Sd(T)v(S) = \sum_{T \subseteq S} d(T) and ϕi=∑S∋id(S)/∣S∣\phi_i = \sum_{S \ni i} d(S)/|S| [43, 101]. Pairs are compared by their dividend rather than by the Shapley interaction index [102], which sums the dividends of every coalition that holds the pair, each divided by its size less one, because the dividend isolates what the pair adds beyond its two members alone. A communication structure restricts the game: with meetings HH (sets of members), only meetings contained in SS convene, SS splits into groups linked by convened meetings, and the restricted value of SS is the sum (sum rule [44, 45]; a meeting is Myerson’s conference structure [45, 103]) or the maximum (best-group rule) of the zero-normalized group values v(C)−v(∅)v(C)-v(\varnothing), so the empty team is worth zero and under the best-group rule the values sum to max⁡C[v(C)−v(∅)]\max_C[v(C)-v(\varnothing)]; the contribution is the Shapley value of the restricted game.

Teams split into fixed groups use the Owen value [94], and groups that change during a campaign the level structure value [104], averaged over the structures in force; this average has no axiomatic characterization and is used as a robustness check.

With 8 to 10 seeds or campaigns a percentile interval tends to under-cover, so the seven-source intervals, used for the meetings, and the agent-team intervals of sets S2 and S3 are read as descriptive.

The eight-donor campaign in full

The eight donors of the follow-up game in main-text § 2.2 advised its measured-gap campaign; the split and the lower entry threshold were chosen after the seven-source game had been run, so this game is post hoc. They are the seven databases of the archive with the Materials Project GGA records split at the Materials Project GGA+U rule: the records whose formula holds Co, Cr, Fe, Mn, Mo, Ni, V, or W together with O or F (31,828 of the 95,920 MP GGA records) form the donor MP GGA+U (TM-O/F) subset, and the other 64,092 form MP GGA (no +U); the ICSD subset was not split. A donor enters the prior when it holds at least two anion families of at least 30 records. The seven-source game used four, which the TM-O/F subset, confined to the oxide and halide families, does not meet; on the unsplit archive both thresholds keep the same seven sources. Every one of the 256 donor subsets was run with 10 matched seeds (pairwise arm, early window), so the value table is a lookup into campaign logs. The campaign without donors reached v(∅)=0.6949v(\emptyset) = 0.6949 and the full pool v(N)=0.7509v(N) = 0.7509. Intervals in this section and the next are tt intervals over the 10 seeds (9 degrees of freedom), and the pp values for a zero Shapley value are Holm-adjusted over the eight donors. With seven anion families one exchange of two families in the fitted ranking moves τ\tau by 2/21=0.0952/21 = 0.095; smaller differences are averages of such steps over checkpoints and seeds.

The split leaves the rest of the game in place. The Shapley values of the two halves of MP GGA sum to −0.0292, against −0.0294 for MP GGA unsplit in the seven-source game, and the other donors keep their signs except JARVIS OptB88vdW, which moves from −0.005 to +0.002 (Table S2). The TM-O/F subset holds the campaign back wherever other donors can carry it: its mean marginal change is −0.116 alone, between −0.101 and −0.054 with one to five other donors, −0.022 with six, and +0.002 when it joins the other seven. At the full pool the TM-O/F subset and the ICSD subset, which still holds the GGA records of the TM-O/F chemistries with ICSD provenance, mask each other: removing either alone changes the value little (leave-one-out +0.002 and +0.006), but with the ICSD subset absent the leave-one-out value of the TM-O/F subset is −0.025, with the TM-O/F subset absent that of the ICSD subset is −0.021, and removing both raises the full pool by 0.019.

DatabaseSeven sourcesEight donors95% interval
JARVIS TB-mBJ−0.0134−0.0086[−0.020, +0.002]
JARVIS OptB88vdW−0.0051+0.0019[−0.008, +0.012]
MP GGA (unsplit)−0.0294
MP GGA (no +U)+0.0376[+0.023, +0.053]
MP GGA+U (TM-O/F)−0.0668[−0.091, −0.042]
MP GGA (ICSD)−0.0234−0.0194[−0.040, +0.001]
MP r2r^2SCAN+0.0054+0.0104[−0.001, +0.022]
SNUMAT GGA+0.0539+0.0505[+0.027, +0.074]
SNUMAT HSE06+0.0539+0.0505[+0.027, +0.074]

Table S2. Shapley values in the seven-source game (MP GGA unsplit, 128 subsets) and in the eight-donor game (256 subsets), with the 95% tt interval over the 10 seeds for the eight-donor values. Full pool: 0.7368 and 0.7509; v(∅)=0.6949v(\emptyset) = 0.6949 in both.

Selection check, seven-source game. The best pool of three in the seven-source game (MP r2r^2SCAN, SNUMAT GGA and SNUMAT HSE06; τ=0.851\tau= 0.851 against 0.737 for all seven, early window) was chosen again on five of the ten seeds and scored on the other five, in both directions. Both halves chose the same pool, so the held-out mean equals the ten-seed mean and the interval below is the in-sample interval, not adjusted for the selection, and the held-out pool exceeded the full pool by +0.115 (95% tt interval [+0.037, +0.193], 9 degrees of freedom). In 2000 bootstrap resamples the best pool had three sources in 71%, two in 22% and four in 7%. In the final window the two halves chose different pools (MP r2r^{2}SCAN or JARVIS OptB88vdW, each with both SNUMAT sources), and the held-out difference, +0.057 ([−0.023, +0.137]), is not resolved. Result file: ateki/results/stats_reanalysis/seven_source.json (keys early.crossfit and final.crossfit, size 3), written by ateki/experiments/22_stats_reanalysis.py.

SizeBest v(S)v(S)Pool
00.6949(no donors)
10.8019SNUMAT GGA
20.8686MP GGA (no +U), SNUMAT GGA
30.8533MP GGA (no +U), SNUMAT GGA, SNUMAT HSE06
40.8789JARVIS TB-mBJ, MP GGA (no +U), SNUMAT GGA, SNUMAT HSE06
50.8156JARVIS TB-mBJ, MP GGA (no +U), MP r2r^{2}SCAN, SNUMAT GGA, SNUMAT HSE06
60.7813MP GGA (no +U), MP GGA (ICSD), MP GGA+U (TM-O/F), MP r2r^{2}SCAN, SNUMAT GGA, SNUMAT HSE06
70.7490all except MP GGA+U (TM-O/F)
80.7509all eight

Table S3. Best attainable early ranking quality by pool size (eight-donor campaign, Kendall τ\tau against the measured gaps). Pools of two to four sources reach 0.853 to 0.879; pools of five or more reach at most 0.816 (0.749 to 0.781 for six to eight).

Control for the split. A control asks whether the value of the TM-O/F subset comes from its +U records or from the chemistries they cover. The records in the +U chemistries (Co, Cr, Fe, Mn, Mo, Ni, V, or W with O or F) were removed from one donor not computed with +U and the game was rerun. Removing the 1,471 such records of JARVIS TB-mBJ left every value unchanged; removing the 1,903 of SNUMAT HSE06 moved the TM-O/F subset from −0.067 to −0.058 and SNUMAT HSE06 from +0.051 to +0.060. The loss stays with the +U records when a beyond-GGA source no longer covers the same chemistries, but the shift (−0.067 to −0.058) lies inside both intervals, so the control does not separate the +U records from the chemistries. The runs are in ateki/results/split_controls/ (ateki/experiments/08d_split_controls.py).

Harsanyi dividends. The dividends d(S)=∑T⊆S(−1)∣S∣−∣T∣v(T)d(S) = \sum_{T \subseteq S} (-1)^{|S|-|T|}v(T) were obtained by Möbius inversion of the 256 values; they reproduce every v(S)v(S) and the Shapley values to 10−1210^{-12}. Table S4 gives the dividends summed by order and Table S5 every pairwise dividend. The singleton dividends are SNUMAT GGA and HSE06 +0.1070 each, MP GGA (no +U) +0.0968, MP r2r^{2}SCAN +0.0267, JARVIS OptB88vdW +0.0103, JARVIS TB-mBJ +0.0065, MP GGA (ICSD) −0.0190, and MP GGA+U (TM-O/F) subset −0.1162. Of the 28 pairs, 17 have a positive dividend, none has an interval above zero, and only the two SNUMAT databases have an interval below zero. The high-order dividends are large, noisy, and of alternating sign, so no single high-order coalition should be read on its own; their sums at orders four to seven have intervals that exclude zero.

OrderCoalitionsSum ddPositive / negativeSum of ∣d∣\lvert d\rvert95% interval of sum
18+0.21906 / 20.4895[−0.232, +0.670]
228−0.031217 / 110.7360[−1.787, +1.725]
356−2.167211 / 442.6305[−6.047, +1.712]
470+6.043065 / 56.3158[+0.242, +11.844]
556−7.69793 / 537.8396[−13.500, −1.896]
628+5.171428 / 05.1714[+1.470, +8.873]
78−1.64760 / 81.6476[−3.119, −0.177]
81+0.16651 / 00.1665[−0.182, +0.515]
all255+0.056024.9970

Table S4. Harsanyi dividends by coalition order (eight-donor campaign, early window). Interval: 95% tt interval of the order sum over the 10 seeds.

Pairdd95% interval
SNUMAT GGA + SNUMAT HSE06−0.1070[−0.200, −0.014]
JARVIS TB-mBJ + MP GGA (no +U)−0.0510[−0.119, +0.017]
MP GGA (no +U) + MP r2r^{2}SCAN−0.0438[−0.108, +0.020]
JARVIS OptB88vdW + MP GGA (no +U)−0.0350[−0.109, +0.039]
MP GGA (no +U) + SNUMAT GGA−0.0301[−0.133, +0.073]
MP GGA (no +U) + SNUMAT HSE06−0.0301[−0.133, +0.073]
MP GGA (no +U) + MP GGA (ICSD)−0.0293[−0.090, +0.031]
JARVIS OptB88vdW + SNUMAT GGA−0.0244[−0.106, +0.057]
JARVIS OptB88vdW + SNUMAT HSE06−0.0244[−0.106, +0.057]
JARVIS TB-mBJ + MP r2r^{2}SCAN−0.0065[−0.072, +0.059]
MP GGA (ICSD) + MP r2r^{2}SCAN−0.0019[−0.066, +0.062]
MP GGA (no +U) + MP GGA+U (TM-O/F)+0.0057[−0.084, +0.096]
JARVIS TB-mBJ + MP GGA (ICSD)+0.0107[−0.044, +0.065]
MP GGA+U (TM-O/F) + SNUMAT GGA+0.0114[−0.053, +0.076]
MP GGA+U (TM-O/F) + SNUMAT HSE06+0.0114[−0.053, +0.076]
JARVIS TB-mBJ + SNUMAT GGA+0.0130[−0.075, +0.101]
JARVIS TB-mBJ + SNUMAT HSE06+0.0130[−0.075, +0.101]
MP GGA (ICSD) + SNUMAT GGA+0.0156[−0.048, +0.079]
MP GGA (ICSD) + SNUMAT HSE06+0.0156[−0.048, +0.079]
MP r2r^{2}SCAN + SNUMAT GGA+0.0171[−0.056, +0.091]
MP r2r^{2}SCAN + SNUMAT HSE06+0.0171[−0.056, +0.091]
JARVIS OptB88vdW + MP r2r^{2}SCAN+0.0210[−0.048, +0.090]
JARVIS TB-mBJ + JARVIS OptB88vdW+0.0248[−0.064, +0.114]
MP GGA+U (TM-O/F) + MP r2r^{2}SCAN+0.0274[−0.058, +0.113]
JARVIS OptB88vdW + MP GGA (ICSD)+0.0286[−0.049, +0.107]
JARVIS TB-mBJ + MP GGA+U (TM-O/F)+0.0324[−0.055, +0.120]
MP GGA (ICSD) + MP GGA+U (TM-O/F)+0.0381[−0.031, +0.107]
JARVIS OptB88vdW + MP GGA+U (TM-O/F)+0.0495[−0.012, +0.111]

Table S5. All pairwise dividends of the eight-donor campaign, most negative first, with 95% tt intervals over the 10 seeds.

Further detail

With seven families one exchange of two neighboring families moves τ\tau by 0.095, so the shortfall amounts to between one and two such exchanges in the early ranking, averaged over seeds and checkpoints. The two SNUMAT databases carried a third of the positive contribution each (+0.051), MP GGA (no +U) contributed +0.038, MP r2r^{2}SCAN +0.010, and JARVIS TB-mBJ about zero (−0.009), and the TM-O/F subset held the campaign back most (−0.067, interval [−0.091, −0.042]), followed by the ICSD subset (−0.019), 5,433 of whose records fall under the +U rule.

Contribution measures compared

Table S6 gives every measure on the seed-averaged eight-donor game. Three arguments from data valuation were tested. Duplicates. Leave-one-out is expected to give exact duplicates zero [80]. The TM-O/F subset and the ICSD subset, which mask each other at the full pool, received two of the three smallest leave-one-out values (+0.002 and +0.006; JARVIS TB-mBJ ties the TM-O/F subset), and every leave-one-out value is positive, so a removal screen would keep all eight donors. The identical SNUMAT pair received the largest leave-one-out value (+0.044 each): the second copy adds nothing alone, but a mean +0.040 (range −0.019 to +0.093, positive in 59 of the 63 pools) to pools holding another donor. Small-pool weighting. Beta(16, 1) gave JARVIS TB-mBJ a positive value (+0.005, against −0.009 under Shapley) and the TM-O/F subset the most negative value (−0.109), because the harm of the TM-O/F subset is largest in small pools. Consistency across runs. Banzhaf had the highest mean seed-to-seed Spearman correlation (0.89, against 0.85 for Shapley and 0.28 for leave-one-out; Table S9), in line with the safety-margin argument of Wang and Jia [85], but its margin over Shapley lies within the bootstrap interval of the difference, [−0.011, +0.107]; leave-one-out named the Shapley-lowest donor in 2 of the 10 seeds. Truncated sampling. On the seven-source game, at the 3n=213n = 21 permutations often used for Monte-Carlo Data Shapley, 16 of 50 untruncated repeats reversed the sign of at least one database, and truncation at the seed noise of the full-pool gain (tolerance 0.0216) biased values by up to 0.019 (Table S10); games of a few members should be enumerated exactly.

DatabaseShapleyBanzhafLOOB(4, 1)B(16, 1)B(1, 4)B(1, 16)
JARVIS TB-mBJ−0.0086−0.0146+0.0019−0.0018+0.0053−0.0101−0.0044
JARVIS OptB88vdW+0.0019−0.0020+0.0084+0.0055+0.0099+0.0015+0.0051
MP GGA (no +U)+0.0376+0.0232+0.0343+0.0589+0.0832+0.0280+0.0318
MP GGA (ICSD)−0.0194−0.0258+0.0057−0.0208−0.0180−0.0127−0.0025
MP GGA+U (TM-O/F)−0.0668−0.0743+0.0019−0.0975−0.1089−0.0309−0.0083
MP r2r^{2}SCAN+0.0104+0.0001+0.0248+0.0154+0.0246+0.0129+0.0206
SNUMAT GGA+0.0505+0.0358+0.0438+0.0748+0.0976+0.0376+0.0409
SNUMAT HSE06+0.0505+0.0358+0.0438+0.0748+0.0976+0.0376+0.0409

Table S6. Semivalues of the eight donors (seed-averaged game, early window, Kendall-τ\tau gain over the campaign without donors, w(N)=+0.0560w(N) = +0.0560).

Table S7 gives the decision that the measured values support directly. The best pool of each size and the removal path, which drops from the full pool at each step the source whose removal raises the value most or lowers it least, are read from the 256 seed-averaged values; no contribution measure is involved. The removal path reaches the best of all 256 pools (JARVIS TB-mBJ, MP GGA (no +U), SNUMAT GGA, SNUMAT HSE06, τ=0.879\tau= 0.879) after four steps. Its first step drops the TM-O/F subset and lowers the value slightly (0.751 to 0.749; dropping JARVIS TB-mBJ instead gives exactly the same mean, and the script breaks such ties by a fixed donor order), because the ICSD subset masks them; the second step, dropping the ICSD subset, raises it to 0.770. From four to seven sources the best attainable value falls with each added source (0.749 at seven), and all eight reach 0.751. The best pools of two to four sources lie within 0.03 of each other (0.869, 0.853, 0.879): over 2000 bootstrap resamples of the seeds the best size was four in 1757, two in 214, and three in 29, and the pools chosen on seeds 1–5 and on seeds 6–10 coincide at sizes 1, 2, and 4. The full pool fell short of the best pool by 0.128 (95% tt interval [+0.061, +0.195]).

Table S7. Best pool of each size and the greedy removal path in the eight-donor game (seed-averaged early-window Kendall τ\tau over the 10 seeds). Removal path: at each step the source whose removal raises the value most, or lowers it least, is dropped.

SizeBest pool of this sizeτ\tauRemoval path: source droppedτ\tau after
8all eight0.751–0.751
7JARVIS TB-mBJ, JARVIS OptB88vdW, MP GGA (no +U), MP GGA (ICSD), MP r2r^{2}SCAN, SNUMAT GGA, SNUMAT HSE060.749MP GGA+U (TM-O/F)0.749
6MP GGA (no +U), MP GGA (ICSD), MP GGA+U (TM-O/F), MP r2r^{2}SCAN, SNUMAT GGA, SNUMAT HSE060.781MP GGA (ICSD)0.770
5JARVIS TB-mBJ, MP GGA (no +U), MP r2r^{2}SCAN, SNUMAT GGA, SNUMAT HSE060.816JARVIS OptB88vdW0.816
4JARVIS TB-mBJ, MP GGA (no +U), SNUMAT GGA, SNUMAT HSE060.879MP r2r^{2}SCAN0.879
3MP GGA (no +U), SNUMAT GGA, SNUMAT HSE060.853JARVIS TB-mBJ0.853
2MP GGA (no +U), SNUMAT GGA0.869SNUMAT HSE060.869
1SNUMAT GGA0.802MP GGA (no +U)0.802
0none0.695SNUMAT GGA0.695

Pooling per distinct ordering. The donor prior of a coalition is the mean of its donors’ orderings with one vote per source, so the two SNUMAT sources, whose orderings of the receiver’s families are identical, carry the SNUMAT ordering twice in every pool that holds both; the other six donors have distinct orderings. Under the alternative rule with one vote per distinct ordering, a pool holding both SNUMAT sources has the same prior as the pool without SNUMAT HSE06, and with matched seeds the same campaign, so the 256 values of this game follow from the measured ones without a new run (Table S8). The second copy then adds exactly zero to all 64 pools that hold the first, against a mean of +0.040 under per-source pooling; the full pool falls from 0.751 to 0.707, the two copies still receive equal values by symmetry, and their sum falls from +0.101 to +0.063.

Table S8. Shapley values of the eight donors under per-source pooling (as in the main text) and per-distinct-ordering pooling (seed-averaged early-window game; 95% tt interval over the 10 seeds for the per-ordering values). Full pool: 0.751 per source, 0.707 per ordering; v(∅)v(\emptyset) = 0.695.

DatabasePer sourcePer distinct ordering
JARVIS TB-mBJ−0.009−0.009 [−0.019, −0.000]
JARVIS OptB88vdW+0.002+0.005 [−0.006, +0.017]
MP GGA (no +U)+0.038+0.040 [+0.024, +0.056]
MP GGA (ICSD)−0.019−0.023 [−0.043, −0.003]
MP GGA+U (TM-O/F)−0.067−0.072 [−0.101, −0.043]
MP r2r^{2}SCAN+0.010+0.008 [−0.004, +0.021]
SNUMAT GGA+0.051+0.032 [+0.017, +0.047]
SNUMAT HSE06+0.051+0.032 [+0.017, +0.047]
MeasureMean pairwise95% intervalMinimumTo averaged ShapleyShapley-last namedSign flips
Shapley0.850[0.79, 0.96]0.5420.923100
Banzhaf0.890[0.80, 0.98]0.5420.947104
Leave-one-out0.280[0.11, 0.66]−0.6550.547210
Beta(4, 1)0.694–0.0360.83196
Beta(16, 1)0.619–−0.1330.78887
Beta(1, 4)0.718–0.2290.84367
Beta(1, 16)0.518–−0.2050.71858

Table S9. Consistency of each measure across the 10 seeds of the eight-donor game: mean Spearman correlation between the values of two seeds (45 pairs) with its 95% bootstrap interval over seeds, the minimum over the 45 pairs, mean correlation with the seed-averaged Shapley values, seeds in which the measure names the Shapley-last donor last, and seeds with a sign different from that seed’s own Shapley value for at least one donor.

TolerancePermutationsMedian max error90th pctSign reversalsLookups
0.00210.00990.013916/50147
0.0010000.00140.00250/507000
0.01210.01260.018323/50135
0.0110000.00390.00520/506425
0.05210.01200.019017/50131
0.0510000.00480.00660/506283
0.25210.01560.023618/50113
0.2510000.00890.01010/505323

Table S10. Truncated Monte-Carlo Shapley on the seven-source game (50 repeats per setting): tolerance as a fraction of w(N)w(N), permutations, median and 90th-percentile maximum error, repeats with a sign reversal, and value lookups.

The same checks on the computed Al–Mg–Si games. These games are unaffected by the archive rerun (every query is a calculation); Tables S11 and S12 give the measures for the PBE and HSE06 receivers and for the higher-temperature MD receivers.

Table S12 gives the consistency checks for the six higher-temperature MD games, with every MD value redrawn from its sampling error in each of the 20 runs.

Further detail

This resembles the redundancy of overlapping ties [25, 26], here as substitutes in a database portfolio.

The contribution measures serve a different purpose, dividing the gain among the sources, where their axioms matter.

Whether redundant knowledge is worthless therefore depends on how the campaign integrates its sources, not on the redundancy alone.

The TM-O/F subset does most harm as one of a few advisers, and a TB-mBJ ordering advances a small pool and holds back a large one.

Table S11: Semivalues of the donors in the computed Al–Mg–Si games (receivers QE PBE and QE HSE06, every coalition run; value: early share of the ten compounds with the lowest formation energy per atom, 40 runs). Fixed: constant weight on the pooled donor ordering; learned: weight re-estimated after every batch. In each run every MD formation energy is drawn from its measured sampling error (0 K, DFT and archive values fixed), so the 40 per-run games (k=1k = 1 and 4, 20 runs each) differ by MD sampling noise. The last four rows give the mean Spearman correlation between the values of two runs over the 780 run pairs of each game.

GameDonorShapleyBanzhafBeta(16,1)Beta(1,16)LOO
PBE, fixedMACE-MP-0 relaxation+0.226+0.193+0.326+0.216+0.224
MP archive+0.148+0.147+0.149+0.150+0.146
MEAM MD−0.109−0.109−0.242+0.024+0.037
NEP89 MD−0.197−0.214−0.179−0.166−0.159
PBE, learnedMP archive+0.114+0.091+0.265+0.024+0.016
MACE-MP-0 relaxation+0.105+0.071+0.234+0.068+0.072
NEP89 MD+0.043+0.017+0.140+0.019+0.022
MEAM MD+0.024−0.004+0.084+0.039+0.047
HSE06, fixedQE PBE+0.171+0.132+0.344+0.116+0.119
MACE-MP-0 relaxation+0.167+0.130+0.343+0.103+0.105
MP archive+0.120+0.107+0.167+0.118+0.122
MEAM MD−0.084−0.088−0.133−0.016−0.008
NEP89 MD−0.209−0.237−0.145−0.187−0.178
HSE06, learnedQE PBE+0.145+0.118+0.345+0.022+0.009
MACE-MP-0 relaxation+0.101+0.069+0.267+0.027+0.025
MP archive+0.064+0.034+0.215+0.000−0.006
MEAM MD−0.002+0.001−0.049+0.038+0.040
NEP89 MD−0.051−0.084+0.084−0.091−0.091
Run-pair Spearman, PBE fixed1.0001.0001.0001.0001.000
Run-pair Spearman, PBE learned0.1620.1950.6850.1690.373
Run-pair Spearman, HSE06 fixed0.9820.9620.9190.9140.902
Run-pair Spearman, HSE06 learned0.8400.8710.8970.2760.281

Table S11.

Table S12: Data-valuation checks on the higher-temperature MD games (receiver, trust mode; six donors, all 64 coalitions; 20 runs, each drawing every MD formation energy from its measured sampling error). Spearman: mean correlation between the values of two runs over 190 run pairs. Top: runs in which the same potential at 300 K received the largest value. Other potentials: combined value of the two other potentials at the target temperature in the run-averaged game, and the runs in which its Shapley value was negative. The NEP89 spreads may be underestimated.

Receiver, trustShapleyBanzhafBeta(16,1)LOOShapleyLOOruns < 0
MEAM 453 K, learned0.8460.8340.8750.3022011−0.075−0.06420
MEAM 453 K, fixed0.9050.8800.9700.7012020−0.103+0.01320
MEAM 803 K, learned0.8520.8850.6200.6372019−0.106−0.12820
MEAM 803 K, fixed0.9710.9530.9890.6072019−0.061+0.01820
NEP89 453 K, learned0.9320.9450.8550.4192018−0.043−0.02618
NEP89 453 K, fixed0.9100.8680.9350.7692020−0.154−0.00620
NEP89 803 K, learned0.7600.8200.7060.040204−0.036−0.01218
NEP89 803 K, fixed0.9770.9340.9730.6282020−0.164−0.01620
MACE-MP-0 453 K, learned0.8970.8200.9360.534208−0.010−0.08411
MACE-MP-0 453 K, fixed0.8840.8990.9700.882200−0.004+0.09410
MACE-MP-0 803 K, learned0.9020.8880.9360.609175−0.005+0.0579
MACE-MP-0 803 K, fixed0.9730.9270.9730.915209−0.095+0.03019

Table S12.

Communication structure

The meeting games of this section restrict the seven-source band-gap game (MP GGA unsplit, 128 subsets; Table S2); they were not rerun with the TM-O/F subset as a separate donor.

Weighting by team structure. A post-hoc option reweights unrestricted Shapley values by a centrality weight wi(G)w_i(G) of the team graph,

ϕiG=nwi(G) ϕi+1n(v(N)−∑jnwj(G) ϕj),(5)\phi_i^G = n w_i(G)\,\phi_i + \frac{1}{n}\left(v(N)-\sum_j n w_j(G)\,\phi_j\right), \tag*{(5)}

where the second term restores efficiency additively. The principled alternative restricts the game itself (Myerson value; main-text § 2.5).

The degree-centrality weight

The communication weight in Eq. (5) is the normalized degree centrality, wi(G)=deg⁡(i)/∑jdeg⁡(j)w_i(G)=\operatorname{deg}(i)/\sum_j\operatorname{deg}(j). On a star graph with nn bodies the hub has degree n−1n-1 and each leaf degree 1, so the hub receives (n−1)/(2(n−1))=0.5(n-1)/(2(n-1))=0.5 and each leaf 1/(2(n−1))1/(2(n-1)); for the n=7n=7 databases of Section S4 this is 6/12=0.56/12=0.5 for the hub and 1/121/12 for each leaf. After the mean normalization of Eq. (5), the hub’s Shapley value is multiplied by n/2=3.5n/2=3.5 and each leaf’s by n/(2(n−1))=7/12n/(2(n-1))=7/12 before the additive shift, so the leaf ordering is preserved. This is the graded correction quoted in Section S4. In contrast, betweenness centrality is exactly 1 for the hub and 0 for every leaf.

Two failure modes of centrality weighting

Both failure modes of centrality weighting return plausible values without any error signal:

  1. Star-graph betweenness fails silently, by construction. The natural choice of betweenness centrality [105] assigns exactly 1 to the hub and exactly 0 to every leaf on a star graph (one supervisor, n−1n-1 leaves), so betweenness weighting assigns the hub the entire (shifted) outcome regardless of any member’s contribution: winner-takes-all by construction, not by evidence. Every leaf receives the same weight (zero), so after the efficiency shift all leaves receive one identical value and the ranking carries no information about any leaf. We therefore weight by degree centrality (Section S4.1).

  1. Efficiency must be restored additively. Rescaling ϕiG\phi_i^G multiplicatively to match v(N)v(N) inverts every sign whenever the scale factor, the ratio of v(N)v(N) to the sum of the ϕiG\phi_i^G, is negative, as when v(N)<0v(N)<0 (which happens routinely when the value is a negated cost) while the weighted values sum to a positive number. The additive shift in Eq. (5) is sign-safe.

The post-hoc weighting is transparent but ad hoc; the principled alternative, the Myerson value, makes the structure part of the game itself.

The pairwise reduction check

When every hyperedge has exactly two members, containment of a hyperedge in SS is containment of a graph edge in the induced subgraph. In this case, vHv_H and every resulting attribution coincide exactly with the graph restriction. We verify this executably on a synthetic four-player superadditive game over all 16 coalitions and on the measured seven-source game: feeding the clique expansion of the size-three conferences through the hypergraph code as pure pairs reproduces the graph-restricted value on all 128 coalitions and the graph Myerson value with maximum absolute deviation 0.0 at tolerance 10−910^{-9}.

One database as hub of a star. Each database was placed in turn at the hub of a star (Table S13). Betweenness weighting gave every leaf the same value; with MP GGA as hub it gave the hub −0.1707 and every leaf +0.0354. Degree weighting kept the leaf ordering for every hub. The Myerson value placed every hub below its unrestricted value, because on a star every interaction between members arises only in subsets that contain the hub and on this game interactions of three or more sources sum to a loss in point estimate; the best-group rule roughly halved that charge.

HubUnrestrictedDegree-weightedMyerson, sumGraph value, best-group ruleRank (sum, best)
JARVIS TB-mBJ−0.0134−0.0389−0.0633−0.03497, 7
JARVIS OptB88vdW−0.0051−0.0131−0.0580−0.03147, 7
MP GGA−0.0294−0.0883−0.0933−0.04797, 7
MP GGA (ICSD)−0.0234−0.0696−0.0835−0.04327, 7
MP r2r^{2}SCAN+0.0054+0.0192−0.0442−0.02447, 7
SNUMAT GGA+0.0539+0.1687+0.0243+0.01742, 2
SNUMAT HSE06+0.0539+0.1687+0.0243+0.01742, 2

Table S13. Each database as the hub of a star (seven-source campaign, early window): unrestricted Shapley value, degree-weighted value, Myerson value under the sum rule, and graph value under the best-group rule; hub rank after restriction.

Declared meetings. Structure A holds one meeting per organization (MP GGA, MP GGA (ICSD), MP r2r^{2}SCAN; JARVIS TB-mBJ, OptB88vdW; SNUMAT GGA, HSE06) and a beyond-GGA meeting (JARVIS TB-mBJ, MP r2r^{2}SCAN, SNUMAT HSE06); Structure B adds a GGA-level meeting (JARVIS OptB88vdW, MP GGA, SNUMAT GGA). Each restricted value was compared with the Myerson value of the clique expansion of the same meetings (any two members of a meeting may combine without the rest). Both structures connect all seven databases, so every column sums to w(N)=0.0419w(N) = 0.0419 (Table S14).

DatabaseNoneStructure A: Hyper sumStructure A: Hyper bestStructure A: Clique sumStructure A: Clique bestStructure B: Hyper sumStructure B: Hyper bestStructure B: Clique sumStructure B: Clique best
SNUMAT GGA+.0539 (1)+.0633 (1)+.0579 (1)+.0682 (1)+.0658 (1)+.0545 (1)+.0482 (2)+.0521 (2)+.0501 (2)
SNUMAT HSE06+.0539 (1)+.0277 (2)+.0322 (2)+.0288 (2)+.0314 (2)+.0482 (2)+.0491 (1)+.0568 (1)+.0589
MP r2r^{2}SCAN+.0054 (3)+.0006 (4)−.0081 (4)−.0112 (5)−.0221 (7)+.0045 (3)−.0065 (4)+.0032 (3)−.0023 (3)
JARVIS OptB88vdW−.0051 (4)+.0110 (3)+.0032 (3)+.0040 (3)+.0012 (3)+.0023 (4)−.0065 (4)−.0054 (4)−.0064 (4)
JARVIS TB-mBJ−.0134 (5)−.0070 (5)−.0108 (5)−.0103 (4)−.0069 (4)−.0061 (5)−.0058 (3)−.0119 (5)−.0104 (5)
MP GGA (ICSD)−.0234 (6)−.0215 (6)−.0153 (6)−.0167 (6)−.0132 (5)−.0206 (6)−.0098 (6)−.0227 (6)−.0186 (6)
MP GGA−.0294 (7)−.0322 (7)−.0171 (7)−.0208 (7)−.0143 (6)−.0409 (7)−.0268 (7)−.0302 (7)−.0294 (7)

Table S14. Contributions of the seven databases under declared meetings (early window; rank in parentheses). Hyper: all-members-present meetings; Clique: pairwise links between meeting members. Sum and Best: rule for valuing a separated team.

Further detail

Knowledge moves more readily between units that are tied to each other, and weak ties help a unit search for knowledge but hinder the transfer of complex knowledge [63].

Language-model agent teams

Protocol. Each agent held the family ordering of one database as private knowledge and, in each of 12 rounds, loaded one four-family tray, for which the archive returned measured orders (each record drawn at most once per campaign). In the flat team all agents acted at once on a shared notebook and could message any agent; in the coordinator-led team agent 1 (an MP GGA holder) assigned the trays. Meetings were read from the logs (a message from ii to TT is the meeting {i}∪T\{i\} \cup T; a tray that cites other agents’ notebook entries is the meeting of its author and the cited authors; meetings recurring at least twice were kept), and v(S)v(S) was measured by replaying the trays of the members of SS as a Plackett–Luce ranking over all 2n2^n subsets (n=10n = 10: 1,024 subsets).

Prompts, parsing, and fallback. Every agent received the same system prompt with its own knowledge filled in: it is one of the five (or ten) agents of a materials screening team that must rank the families by measured optical band gap, widest first; each query names four different families, the laboratory measures one archived compound of each and returns their order, and one measurement may not represent its family; its private knowledge is its database’s family ordering by computed gap, which can disagree with measured gaps, and other agents hold other databases; it may write one notebook entry (observation, fact, failure, claim, or summary), send one message to chosen agents, and must list the notebook entries and query results its query builds on; it replies in JSON only. Each turn’s user message gave the round, the 30 most recent query results, the 20 most recent notebook entries, and the eight most recent messages addressed to the agent. In the coordinator-led team the coordinator was asked, each round, to assign one four-family query to every agent including itself, with a message of at most 40 words; the other agents saw their assignment and could follow or change it, and could message only the coordinator. Replies were sampled at temperature 0.7 (at most 900 tokens for agents and 1000 for the coordinator). A reply that did not parse or did not name four valid families was requested again with a new seed, up to three attempts; after three failures the agent’s tray was its assignment, or else four random families, and the failure was counted. Sets S1, S2, and S3 used 2,642, 1,340, and 2,017 model calls with no fallback; the finer task used 8,499 calls with 26 fallbacks. The templates are in ateki/experiments/11_recorded_meetings.py and 13_recorded_meetings_4b.py.

Team ranking quality and tray statistics. Table S15 gives the values plotted in the main-text agent-team figure.

TeamCampaignsTeam τ\tauDistinct traysCoordinator effect
flatcoordinatedflatcoordinated
Qwen3-4B, 5 agents20 + 200.7560.8529.6%11.3%+0.096 [+0.029, +0.165]
Nemotron 30B-A3B, 5 agents10 + 100.6700.7905.3%17.0%+0.121 [+0.002, +0.229]
Qwen3-4B, 10 agents8 + 80.8120.8575.6%8.4%+0.045 [−0.019, +0.105]

Table S15. Coordination in language-model agent teams ranking seven anion families by measured gap (12 rounds of four-family trays). Team τ\tau: Kendall τ\tau against the measured ranking, mean over campaigns. Distinct trays: share of distinct trays among all trays a team loaded. Random trays at the same budget, a search baseline, reach 0.876, sd 0.096 (5 agents, 240 evaluations) and 0.929, sd 0.073 (10 agents, 480 evaluations), each over 20 random searches; the five-agent sd of 0.096 is the spread of the random trays and is not the coordinator effect of +0.096 below. Coordinator effect: coordinator-led minus flat, paired by seed, with 95% bootstrap interval.

Sequential turns. To test whether the flat team duplicates trays because its agents choose at the same time from the same snapshot, a third S1 arm let the five Qwen3-4B agents choose in a fixed order that rotated each round, each agent seeing the trays already claimed that round but not their results; results, notes and messages were applied at the end of the round as in the flat arm, with the same seeds (0–19), prompts, budget and archive draws (1200 calls, no fallback replies). The sequential team reached team τ\tau 0.771 and a distinct-tray share of 11.9%, against 0.756 and 9.6% flat and 0.852 and 11.3% coordinated. Paired by seed (95% percentile bootstrap, 5000 resamples), sequential minus flat was +0.015 [−0.027, +0.062] in team τ\tau and +0.023 [+0.014, +0.033] in distinct share; sequential minus coordinated was −0.081 [−0.152, −0.010] in team τ\tau and +0.006 [−0.007, +0.018] in distinct share. Taking turns removed about as much duplication as the coordinator did but not the coordinator’s gain in ranking, and the sequential team stayed below the random-tray search baseline (0.876, sd 0.096).

Shares by database. Table S16 gives each database’s share of team τ\tau as the unrestricted Shapley value of the replayed team, summed over the agents holding the database; in a coordinated team the MP GGA column is the coordinator’s unrestricted share. The last two columns come from a different game, the team restricted to the recorded meetings: the coordinator’s Myerson value (sum rule) and its value under the best-group rule. The columns therefore have no common total and are not compared with each other. Because team τ\tau is bounded, the sum rule adds the values of disconnected groups as if each could raise τ\tau on its own, so it overstates the value of a fragmented team; the best-group rule does not.

SetTeamMP GGAMP r2r^{2}SCANOptB88vdWTB-mBJSNUMATHSE06Coord. sumCoord. best
S1 Qwen 5flat0.0990.2010.2190.1490.332
S1 Qwen 5coordinated0.1290.2550.1380.2130.265−0.825+0.061
S2 Nemotron 5flat0.1040.1930.2240.0770.403
S2 Nemotron 5coordinated0.1850.1670.2450.1990.203−0.792+0.144
S3 Qwen 10flat0.0770.2160.1600.2120.335
S3 Qwen 10coordinated0.1670.2070.1390.1970.290−2.963−0.064

Table S16. Share of team τ\tau by the database an agent holds (unrestricted Shapley value of the replayed team, mean over campaigns), and the coordinator’s own share in the game restricted to the recorded meetings, under the sum rule (Myerson value) and the best-group rule. The two groups of columns come from different games and are not compared.

Registered predictions (sets S1–S3). Recorded on 4 Oct 2026, before any campaign of the extension, in time-stamped commits of the project repository (files PREDICTION.md in the code release). P1 (S1: coordinator-led ranks better, interval excluding zero; flat distinct trays below 20%): pass (+0.096, [+0.029, +0.165]; 9.6%). P2 (S2: flat distinct trays below 30%; coordinator-led at least as good): pass (5.3%; 0.790 against 0.670). P3 (S3: flat distinct trays at 10 agents no higher than at 5; coordinator-led at least as good): pass (5.6% against 9.6%; 0.857 against 0.812). P4 (the MP r2r^{2}SCAN and SNUMAT HSE06 holders take the two largest shares in every set and team): fail (S1 flat: JARVIS OptB88vdW second). P5 (coordinator share negative under the sum rule and positive under the best-group rule in every coordinated set): fail (S3 best-group share −0.064, positive in 2 of 8 campaigns). P6 (no hub in the 10-agent flat team: no agent cited more than twice the median): pass (maximum 1.24 times the median).

The finer task with a rotated coordinator. Five Qwen3-4B agents ranked the 13 anion-element groups of the measured compilation (groups with at least 30 compounds), where the databases’ standalone orderings all differ (Kendall τ\tau against the measured order: SNUMAT HSE06 0.90, MP r2r^{2}SCAN 0.85, JARVIS TB-mBJ 0.82, JARVIS OptB88vdW 0.79, MP GGA 0.69, computed from each database’s full group order; the same TB-mBJ > OptB88vdW > GGA order against measured gaps that Borlido et al. [10] report). There were 20 flat campaigns and 100 coordinator-led campaigns, 20 with each database’s holder as coordinator (8,499 model calls, 26 unparseable replies replaced by the fallback rule). Team τ\tau: flat 0.773 [0.724, 0.819]; coordinated by MP GGA 0.828, MP r2r^{2}SCAN 0.808, JARVIS OptB88vdW 0.836, JARVIS TB-mBJ 0.822, SNUMAT HSE06 0.835. Registered predictions: Q1 (role premium, pooled, interval excluding zero) fail, −0.004 [−0.030, +0.021] (seed-level cluster bootstrap as for Q4); Q2 (premium positive in at least 4 of 5 rotations) fail, 3 of 5; Q3 (SNUMAT HSE06 in the top two and MP GGA last with probability at least 0.9 in both team types) fail (coordinated 0.73 and 0.995; flat 0.36 and 0.83); Q4 (coordinator raises team τ\tau, interval excluding zero) pass, +0.052 [+0.006, +0.102], positive on average but not robust across seeds: the lower bound is just above zero and the effect was uneven across seeds (better in 62 of 100 pairs and in 10 of 20 seeds). Because each seed’s flat run serves all five coordinator rotations, this interval is a seed-level cluster bootstrap (B=20,000B = 20{,}000) that resamples the 20 seeds with all five rotations of each; a tt interval on the 20 seed means gives [−0.001, +0.105]; Q5 (SNUMAT HSE06 coordinator outranks MP GGA coordinator with probability at least 0.9) fail, difference +0.007, probability 0.62; Q6 (flat team issues fewer distinct queries) pass, 0.125 against 0.146, difference −0.021 [−0.038, −0.003]. Shares by database in the flat team: MP r2r^{2}SCAN 0.253, JARVIS TB-mBJ 0.220, SNUMAT HSE06 0.210, JARVIS OptB88vdW 0.184, MP GGA 0.133 (Spearman correlation with standalone accuracy 0.7; coordinated teams 0.9).

Further detail

The meetings were read from the logs as recurring message and notebook-citation links, not scheduled meetings, and v(S)v(S) was measured by replaying the trays of the members of SS (campaign counts and tray statistics in § 13).

Flat teams loaded few distinct trays and trailed the random-tray search baseline. Only 5–10% of the trays of a flat team were distinct, and the five-agent flat team trailed the random-tray search baseline at the same budget (0.756 against 0.876, sd 0.096 over 20 random searches of 240 evaluations each, a search baseline whose seeds are not paired with those of the team runs); its members often measured the same families, although tray overlap is not shown to cause the shortfall: a coordination loss against the random-tray search baseline rather than against a nominal group of the same members, akin to the process loss by which a group falls short of what its members could reach with their efforts combined well [66]. Coordination, the management of dependencies between activities [67], is meant to reduce such loss, and misaligned or duplicated work between agents is a recurring failure of multi-agent language-model systems [68].

As a process count, coordinated minus flat, paired by seed, the coordinator raised the share of distinct trays in every set, by +1.75 ([+0.42, +3.25]), +11.67 ([+8.17, +15.33]), and +2.81 ([+1.56, +3.85]) percentage points in sets S1, S2, and S3, and coordinated teams still repeated most of their trays. In set S1 only, sequential minus coordinated (five Qwen agents choosing in turn without a coordinator), the share differed by +0.58 percentage points ([−0.67, +1.75]). Two confounds stay open: the coordinator chose in an assigned order while flat agents chose at once, and in sets S1–S3 it always held the MP GGA ordering.

Under replay valuation, which values what each member did rather than an intervention on it [69, 70], the coordinator’s effect is instead assigned to the agents it directed: its role premium, the coordinator’s share of team τ\tau minus the share of the agent holding the same database in the flat team of the same seed, was −0.004-0.004 [−0.030-0.030, +0.021+0.021] on the finer task (seed-level cluster bootstrap, since each flat run again serves all five rotations; 10 of 20 seeds positive).

In the flat teams the agent holding SNUMAT HSE06 carried the largest share (0.33 of team τ\tau for five Qwen agents), the database that, with SNUMAT GGA, also contributed most in the database campaign.

Language-model teams. Agents ran Qwen3-4B-Instruct-2507 or Nemotron 3.5 Lightning 30B-A3B (NVFP4) (no thinking mode), served with vLLM 0.29.0 [95] on an NVIDIA DGX Spark at temperature 0.7.

Twelve predictions for the extended runs were recorded on 4 October 2026, before the campaigns, in time-stamped commits of the project’s version-controlled repository (files PREDICTION.md, included in the code release and listed in § 19), and six of the twelve held, including the three on the coordinator and on duplicated trays; prompts, the retry rule, and parse failures are given with them (§ 13).

The emergent hierarchical swarm and several bodies of one kind

Setting and results

Here a swarm is a team of computational members whose coordinator re-ranks them into tiers by their record after every batch (the adaptive team of the main text), not a self-organizing swarm without central control. In the games above the structure of the team is declared before the campaign: a pool, a star, a set of meetings, or a coordinator. A materials group that runs many advisers at once can instead let the team organize itself during the campaign, as a swarm [106] of heterogeneous members that follow local rules and share part of what they learn. The question here is whether such a team outperforms a structure fixed by its designer, and how the contribution of each member changes when roles are not assigned in advance. Seven decision bodies run the Al–Mg–Si campaign of § 2.4 of the main text together (168 compounds, batches of four, PBE as the receiver): five hold a low-cost ordering (MACE-MP-0 relaxation, NEP89 MD and relaxation, MEAM MD, and the Materials Project archive, which covers 36 of the compounds), one is an active learner with no prior ordering, and one is a random scout. Every member scores the remaining compounds with its own rule, mixing its ordering with a composition surrogate fitted on the receiver results it has seen and trusting the ordering by its agreement with those results, and every member sees only the results its position in the team gives it access to.

Figure S1. On the held-out HSE06 receiver the number of tiers of the team whose tiers change varied between one and six during a campaign without lowering its recall below that of the three fixed tiers. (a) Share of the ten compounds with the lowest formation energy per atom found against the fraction of HSE06 compute; line, mean over n=100n = 100 campaigns; dotted lines, 5–95% range over campaigns for the fixed tiers and for tiers that change. (b) Number of tiers and number of members in the top tier of the team whose tiers change, mean and 5–95% band over campaigns (learned settings: mean number of tiers only); dotted line, the three fixed tiers.

Eight structures were compared over 100 campaigns each. Four are fixed: no communication, a flat team with equal batch slots and full sharing, a fixed supervisor (the active learner) that fills each batch from the members’ scores, and fixed tiers assigned by nominal fidelity (MACE-MP-0 and the archive above the potentials, the potentials above the learner and the scout, with batch slots in the ratio 4:2:1 and results passed down from higher tiers). The other four are emergent, in the sense of a non-egalitarian emergent hierarchical swarm: all members start in one tier; after every batch each member’s prequential score (the agreement between the scores it gave the batch before the results arrived and the results it can see) updates a running average and the members are re-ranked; a member with a negative score falls to a floor weight; and members whose current orderings agree merge into a temporary cell that proposes once, with the weight of its strongest member, and dissolves when the agreement drops. Every position in an emergent team is therefore temporary. The emergent structures differ in whether the shape of the hierarchy is also allowed to change. With a three-tier shape the members are re-ranked into one leader, a second tier of two and a third tier, so the number and size of the tiers are fixed. With a changing shape the tiers are re-derived after every batch from the gaps between the running scores: a new tier begins wherever the drop from one member to the next exceeds a threshold, so the team stays flat while the records are close, deepens as they separate, and flattens again where they converge, and the advantage an incumbent receives for its current tier expires after a fixed number of batches, so a position must then be held on the record alone. Each protocol is either implemented with fixed cold-start parameters or learned: its parameters were trained with the cross-entropy method [107] on the PBE receiver only and then applied unchanged to two held-out receivers, HSE06 [7, 8] and r2r^{2}SCAN [5] computed for the same compounds. All 128 sub-teams were run for each structure, so each member’s Shapley value is measured within that structure (protocol and role statistics in the subsections below).

The emergent team outperforms every fixed structure on all three receivers (Figure 9a of the main text). With the three-tier shape, on PBE its early share is 0.70, against 0.64 for the fixed tiers, 0.61 for the flat team and 0.58 for the fixed supervisor; on the held-out receivers it is 0.49 against at most 0.40 (HSE06) and 0.46 against at most 0.44 (r2r^{2}SCAN), gaps of +0.088 (90% bootstrap interval +0.071 to +0.105) and +0.021 (+0.006 to +0.034) over 100 campaigns per structure, whose seeds differ between structures, so the intervals are unpaired; and the compute needed to find eight of the ten falls from 0.23 to 0.19 of the HSE06 total. The fixed tiers, which rank the members by their nominal fidelity, improve on the flat team by at most 0.03 and fall below it on HSE06. Merging is the part of the protocol that matters most. The cell that forms most often (about half of the early rounds on every receiver) contains every member except MACE-MP-0: once the potentials and the archive have lost the trust of their own orderings, they all propose from the same surrogate fitted on the same results, and merging them returns their duplicated batch slots to the leader. Without cells the protocol falls below the flat team on HSE06 (0.34); without tiers it keeps most of the gain (0.46; both ablations with their 90% bootstrap intervals in Figure 9a of the main text). MACE-MP-0 holds the leader position in 80–88% of the early rounds and the archive in the rest. Learning the promotion protocol does not add to the implemented one: the learned parameters change the leader in most campaigns, against at most one campaign in five for the implemented parameters, but the learned and implemented teams reach the same value on the held-out receivers (0.49 and 0.49 on HSE06, 0.46 and 0.46 on r2r^{2}SCAN), so the implemented parameters are an adequate cold start for this team.

Tiers that are temporary and change shape keep the gain (Figure 9a of the main text). Under the three-tier protocol the hierarchy is nearly static: 96% of the members keep their tier from one batch to the next, and in most campaigns the leader holds its position for the whole campaign. With a changing shape the number of tiers changes about 13 times in a campaign of 42 batches and ranges from one to six. After the first batch the team has two tiers on average (it is still flat in about one campaign in eight), and it deepens to about four tiers by the end, as the records of the members separate. The top tier first holds about three members and narrows towards one, MACE-MP-0, as the campaign proceeds (fewer than 1.4 members on average after 40% of the HSE06 compute); a member’s stay in the top tier lasts a median of three to four batches, and the NEP89 potentials, MEAM MD and the active learner each lead in 6–13% of the early batches, when a few PBE results cannot yet separate their orderings from that of MACE-MP-0. The changing shape reaches the value of the three-tier shape on PBE (0.69 against 0.70, difference −0.008, 90% interval −0.023 to +0.006) and r2r^2SCAN (0.47 against 0.46) and exceeds it on HSE06 (0.51 against 0.49, difference +0.024, interval +0.002 to +0.046). The learned version sets a narrower gap, builds four to five tiers on average, and reaches the same values. The contribution follows the positions: the active learner receives +0.028 under the changing shape against +0.014 under three tiers (PBE), and on HSE06 the NEP89 relaxation receives +0.021 against 0.000, while MACE-MP-0 keeps the largest contribution (0.22 on PBE, 0.20 on HSE06). For a group running many agents, the organization chart therefore does not need to be fixed in advance, either in who holds a tier or in how many tiers there are: a hierarchy that forms, deepens and dissolves with the record performs as well as a fixed three-tier one, and members contribute most in the stages of the campaign in which they led.

The structure moves contribution to the members it promotes (Figure 9b of the main text). Relative to the flat team, the emergent team gives MACE-MP-0 +0.06 and the archive +0.04 (PBE) and +0.09 (HSE06) more contribution, and takes it from MEAM MD (−0.008 PBE, −0.045 HSE06) and the NEP89 relaxation (−0.005, −0.017); NEP89 MD, the third member whose potential wrongly predicts Mg2_2Si to be unstable, is not down-weighted (+0.001, +0.021). The archive gains most on HSE06, where it holds the leader position in a fifth of the early rounds (Table S17); a fixed tier cannot give a member a position that changes with the campaign. No team of seven reaches the best single member: MACE-MP-0 alone has an early share of 0.81 on PBE and 0.73 on HSE06. The emergent structure closes 43% of the gap between the flat team and this member on PBE and 30% on HSE06, without being told which member is the best. For a group that runs many agents, a team that promotes members by their record and merges members that have become redundant recovers part of the value lost to pooling; the attribution computed within the emergent structure separates the contribution a member receives for its knowledge from the contribution it receives for the position it has been promoted to.

Implemented protocolLearned protocol
ReceiverPBEHSE06r2SCANr^{2}\mathrm{SCAN}PBEHSE06r2SCANr^{2}\mathrm{SCAN}
Leader share, MACE-MP-00.880.800.850.820.740.78
Leader share, MP archive0.110.200.140.180.250.22
Leader changes per campaign0.080.210.120.850.960.76
Tier persistence between batches0.960.960.960.920.920.92
Cells per batch1.181.141.161.081.051.06
Share of batches with the five-member cell10.520.550.570.520.480.54

Table S17. Roles in the emergent teams with the three-tier shape over the first twelve batches (48 compounds), 100 campaigns per receiver.

Several bodies of one kind

The swarm above holds one body of each kind. A materials group often runs several of one kind: three students each choosing compositions at random as scouts, a random-search script started with three seeds, two group members who each relax the pool with MACE-MP-0, or several agents of one language model. Whether such copies form one player or several decides how a contribution statement is written, so the question was tested on the Al–Mg–Si campaign of Section S6.1 (flat team, batches of four, PBE, HSE06 and r2r^2SCAN receivers). In the first game MACE-MP-0 and the active learner are joined by one to four random scouts, each drawing its own random order of the pool, or, as a control, all repeating one draw. In the second game the active learner and one scout are joined by two copies of MACE-MP-0, which either share all results or see only the results they proposed. Every coalition of every game was run in the same 200 campaigns, so the copies of a campaign meet identical conditions, and the Shapley values are exact and computed per campaign (Figure S2).

Figure S2. Several decision bodies of one kind as separate players (computed Al–Mg–Si campaign, 200 campaigns per coalition). (a) Early share of the ten compounds with the lowest formation energy per atom found by a team of MACE-MP-0, the active learner and one to four random scouts, per receiver; solid: each scout draws its own random order; dashed: the scouts repeat one draw. (b) Shapley value of copy 1 against copy 2 in each campaign (PBE receiver): two of three random scouts with independent draws (gray), and two copies of MACE-MP-0 that see only the results they proposed (blue); diagonal: equal contribution.

Random scouts with their own draws are separate players. Two such scouts receive different contribution in 86–93% of the campaigns, by about 0.06 on average, because each draw places different compounds in the early batches (Figure S2). Their expected contributions are equal (−0.056, −0.050 and −0.053 for three scouts on PBE, with overlapping intervals), and a given scout receives positive contribution in 16–17% of the PBE campaigns and in up to 39% of the r2r^2SCAN campaigns. An audit of one campaign that merged the scouts into one player would hide which scout found the compound; merging them also changes the total, because the scouts then enter every coalition together: with four scouts their joint contribution is −0.204-0.204 as one player against −0.178-0.178 as the sum of four (PBE). Scouts that repeat one draw receive identical contribution in every campaign, as the symmetry axiom (members that add the same to every coalition receive the same value) requires. The number of copies is a design variable: each added scout takes batch slots from MACE-MP-0, and the early share falls from 0.540.54 with one scout to 0.380.38 with four (PBE; 0.400.40 to 0.330.33 for HSE06, 0.450.45 to 0.350.35 for r2^{2}SCAN), whether the scouts draw independently or not. The independent scouts lower the outcome additively: the Harsanyi dividend of a pair of scouts is between −0.003-0.003 and +0.012+0.012.

Copies of one potential are redundant even when their outputs differ. When the two MACE-MP-0 copies see only their own results, their surrogate parts are fitted on different compounds, and in a team of the two copies alone their orderings differ in 8989–94%94\% of the batches (0%0\% when the results are shared). Their contribution is nevertheless equal in every campaign (+0.289+0.289 each on PBE), because every sub-team containing copy 1 performs as the same sub-team containing copy 2: the copies are interchangeable players, and interchangeability, not identical outputs within the team, is what the symmetry axiom tests. The pair dividend is −0.68-0.68 on PBE (−0.60-0.60 HSE06, −0.34-0.34 r2^{2}SCAN), equal in size and opposite in sign to the value of one copy over an empty team, so the second copy adds nothing to the first. For a group, two members running one potential on one pool are one adviser whose contribution is split between them, whereas scouts with their own random draws are several advisers, each of which can gain or lose contribution for the compounds its own draw reached. The same distinction applies to agents of one language model: sampled at nonzero temperature, or holding different knowledge as in § 13, they are separate players; queried deterministically with one prompt, they are one.

Protocols and role statistics

This subsection specifies the structures of § 14.1 and gives the role and shape statistics of the emergent teams.

Members and local rules. A member holding an ordering scores each remaining compound by wrown+(1−w)rsurw r_{\mathrm{own}} + (1-w) r_{\mathrm{sur}}, where rownr_{\mathrm{own}} is its ordering as an evenly spaced rank score (0.5 for compounds it does not hold), rsurr_{\mathrm{sur}} is the rank score of a composition-kernel lower confidence bound (bandwidth 0.12 in the Al, Mg, Si fractions) fitted on the receiver results the member has seen, and w=max⁡(0,τ)n/(n+8)w = \max(0,\tau)n/(n+8) with τ\tau the Kendall τ\tau between its ordering and those nn results (w=0.5w = 0.5 for n<3n < 3). The active learner uses rsurr_{\mathrm{sur}} alone; the scout uses a fixed random order.

Batch slots. Each unit (a member, or a cell of merged members) carries a persistent slot contribution that grows by kwu/∑wk w_u/\sum w per batch (k=4k = 4) and falls by one for every compound the unit places; the unit with the largest contribution places its best-scored compound not yet in the batch. A cell scores compounds by the mean of its members’ scores. The fixed supervisor places all four compounds by the mean score of all members.

Sharing. In the flat team and under the fixed supervisor every result is shared. Without communication a member sees only the results of the compounds it placed. In the tiered structures a member sees the results of compounds placed at its own tier or above, the leader sees all results, and in the emergent structures the members of one cell share within the cell.

Promotion protocol. After each batch, member ii receives the prequential score qi=τini/(ni+8)q_i = \tau_i n_i/(n_i+8), where τi\tau_i is the Kendall τ\tau between the scores ii gave the evaluated compounds just before their results arrived and those results, over the nin_i results ii can see. The running average is mi←(1−α)mi+αqim_i \leftarrow(1-\alpha)m_i + \alpha q_i. Members are ranked by mim_i plus an incumbency margin δ\delta per tier above the third; the first nℓn_\ell become leaders, the next two the second tier, the rest the third. Slot weights are ρ2\rho^2, ρ\rho and 1 for the three tiers, and ff for a member with mi<0m_i < 0. Two members merge when the Kendall τ\tau between their current scores (on at most 60 remaining compounds) reaches μ\mu; cells are the connected components and are recomputed every batch. The implemented parameters are α=0.4\alpha= 0.4, ρ=2\rho= 2, δ=0.05\delta= 0.05, μ=0.85\mu= 0.85, f=0.25f = 0.25, nℓ=1n_\ell= 1; the learned parameters (cross-entropy method, population 16, six iterations, 20 training campaigns on PBE) are α=0.42\alpha= 0.42, ρ=3.8\rho= 3.8, δ=0.007\delta= 0.007, μ=0.78\mu= 0.78, f=0.29f = 0.29, nℓ=1n_\ell= 1. The ablations set μ>1\mu> 1 (no cells) or ρ=1\rho= 1, f=1f = 1 (no tiers).

Changing shape. The changing-shape protocol uses the same scores, running averages, floor weight and cells, but no fixed number of tiers. After each batch the members are sorted by mi+δ(T−ti)[si≤L]m_i + \delta(T-t_i)[s_i \le L], where TT is the current number of tiers, tit_i the current tier of member ii (1 at the top), sis_i the number of batches since its tier last changed, and LL the tenure of the incumbency margin. A new tier begins between consecutive members whenever their sorted scores differ by more than the gap gg, so TT ranges from one (a flat team) to the number of members. Slot weights are ρT−ti\rho^{T-t_i}, and sharing follows the tiered rule above. The team starts flat. The implemented parameters are those of the three-tier protocol with g=0.08g = 0.08 and L=4L = 4; the learned parameters (same training) are α=0.41\alpha= 0.41, ρ=2.8\rho= 2.8, δ=0.005\delta= 0.005, μ=0.83\mu= 0.83, f=0.11f = 0.11, g=0.034g = 0.034, L=4.9L = 4.9.

Value and attribution. The team value is the early share of the ten compounds with the lowest formation energy per atom found over the first quarter of the receiver compute. Team values use 100 campaigns per structure and receiver. The Shapley values use all 128 sub-teams per structure, each the mean of 16 campaigns, so they sum to the 100-campaign team value only approximately; the empty team selects at random.

Three tiersChanging shapeChanging shape, learned
Number of tiers, range over all batches31–61–7
Mean number of tiers after the first batch3.002.05–2.084.37–4.40
Mean number of tiers at the end3.004.04–4.225.00–5.24
Share of campaigns flat after the first batch00.12–0.130
Changes in the number of tiers per campaign012.8–13.412.8–13.7
Members in the top tier after the first batch13.14–3.241.51–1.53
Stay in the top tier, median (batches)423–46–14
Tier persistence between batches0.960.73–0.750.72–0.73
Leader changes per campaign0.08–0.213.22–3.321.43–1.88
Early share of the ten found (PBE / HSE06 / r2SCANr^{2}\mathrm{SCAN})0.70 / 0.49 / 0.460.69 / 0.51 / 0.470.70 / 0.51 / 0.47

Table S18. Shape of the emergent teams over the whole campaign (42 batches), 100 campaigns per receiver. Each entry is the range over the PBE, HSE06 and r2SCANr^{2}\mathrm{SCAN} receivers; the persistence and the leader changes refer to the first twelve batches, as in Table S17.

Further detail

The team closed 43% of the gap (computed from unrounded means) between the flat team (0.61) and MACE-MP-0 alone (0.81) without being told which member was best, a simple analogue of the credibility part of transactive memory, by which a group learns whom to trust for what [71, 72].

These organization results generate hypotheses; they do not confirm them.

A member allowed to sit in a second group, like a student shared between two laboratories and akin, for computational members such as these, to a boundary spanner [108], raised it only with the learned, near-zero agreement threshold for sharing.

Several swarms in one campaign

Setting and results

A large campaign is often run by several groups at once, each organized as its own swarm, that share one furnace, one beamline allocation, or one compute allocation. Three questions then arise that a single swarm does not pose: whether the same members do better as one swarm or as several, how the shared batch should be divided between swarms of the same or of different size, and how the contribution is divided first between the swarms and then between their members. The seven bodies of Section S6.1 were grouped into two, three, or seven swarms (4 + 3; 2 + 5 with MACE-MP-0 and the active learner as the small swarm, or with NEP89 MD and the scout; 1 + 6 with MACE-MP-0 alone; 2 + 2 + 3 by nominal fidelity; seven swarms of one), and in a second set the seven-body swarm was joined by copies of itself (7 + 7, 7 + 7 + 7, and 7 + 3 with a small copy of MACE-MP-0, the learner and a scout). Every swarm runs the emergent organization of Section S6.1 among its own members, and all swarms share one batch of four compounds, so every configuration spends the same receiver compute. The batch is divided between swarms in one of three ways: an equal share per swarm, a share proportional to swarm size, or a share by record, in which the promotion rule of the emergent team is applied to the swarms themselves (the swarm whose averaged scores have best anticipated the results it has seen receives the largest share). Results of one swarm are either published to the others after every batch or kept within the swarm. Contribution is computed at two levels: the Shapley value of each swarm, and for the members the Owen value [94], which treats the swarms as fixed unions: it first divides the outcome between the swarms by their Shapley values in the game played by whole swarms, and then divides each swarm’s share between its members. Equivalently, each Harsanyi dividend is split equally among the swarms it touches, then equally among the members of that coalition inside each swarm (not among all of the swarm’s members). The difference between a member’s Owen and Shapley values is therefore the contribution it gains or loses through the swarm it belongs to. Team values use 100 matched campaigns per configuration on the three receivers; the contribution games run every sub-team in 100 campaigns on the PBE and HSE06 receivers (configurations and values in the subsections below).

Splitting the same members into several swarms lowers the outcome unless the split gives the strongest member more of the batch (Figure S3). With an equal share per swarm, seven swarms of one lose 0.106 against one swarm on PBE, and three swarms grouped by nominal fidelity lose 0.08; members in different swarms cannot merge into one cell, so the members that have lost trust in their own orderings keep their duplicated slots. The one grouping that gains places MACE-MP-0 in a swarm of its own: with a share by record it gains +0.047 (90% interval +0.036 to +0.059) on PBE, +0.097 on HSE06 and +0.025 on r2r^2SCAN, because a separate swarm receives a fixed share of every batch from the first batch on, whereas inside one swarm MACE-MP-0 starts level with the others and acquires weight only through its record. The gain depends on knowing in advance which member to separate: with an equal share, a small swarm of the two weakest members takes half of every batch and loses 0.098 on PBE and HSE06. Dividing the batch by record protects the campaign when the strong swarm is not known: it brings the weak pair back to the value of one swarm (−0.008, interval −0.021 to +0.004, on PBE; −0.009 on HSE06; +0.013 on r2r^2SCAN) and gives the highest value of the three rules in five of the six groupings on every receiver. A share by size is never the best rule here: it gives each member the same slot weight wherever it sits, so it removes the advantage of a separate swarm without restoring the cells of a single one.

Figure S3. Several swarms sharing one batch of four compounds in the Al–Mg–Si campaign; every swarm uses the emergent organization of § 14.1 among its members. Color marks the receiver as in Figure 9a of the main text (PBE blue, HSE06 red, r²SCAN green). (a) The seven bodies grouped into several swarms, results published between swarms, PBE receiver: early share of the ten compounds with the lowest formation energy per atom found minus the value of one seven-body swarm, per rule for dividing the batch between swarms (marker shape); mean and 90% bootstrap interval over 100 matched campaigns. “Strong pair”: MACE-MP-0 and the active learner; “weak pair”: NEP89 MD and the random scout. (b) The seven-body swarm joined by copies of itself, equal share per swarm, per receiver, with results kept within each swarm (filled) or published (open). (c) Owen value (the Shapley value computed with the swarms as fixed blocs: swarms first divide the team value among themselves, then each swarm divides its share among its members) minus Shapley value summed over the members of each swarm, against the size of the swarm, for the two- and three-swarm groupings of (a), equal share, published results; mean and 90% interval over 100 campaigns.

Copies of the team help where no member’s ordering is reliable, and only when they do not share results (Figure S3). Copies of equal size receive equal contribution (7 + 7: +0.28 and +0.27 on PBE, +0.20 each on HSE06), as the symmetry axiom expects of swarms that are interchangeable up to run-to-run noise. On PBE, which MACE-MP-0 orders closely, a second or third copy lowers the outcome (three copies: −0.033 with results kept within each swarm, −0.014 published), because the leading member of each copy now receives a smaller part of the batch. On the held-out receivers three copies that keep their results to themselves gain +0.040 (HSE06) and +0.045 (r2r^{2}SCAN) over one swarm, and on r2r^{2}SCAN publishing the results removes the gain (isolated minus published, +0.064, interval +0.046 to +0.081). Copies that do not share results fit their surrogates on different compounds, so their orderings separate (Section S6.2) and their batches cover more of the pool early; copies that share results converge on one ordering. A small copy joined to the full team (7 + 3) takes half of every batch under an equal share and lowers the outcome on PBE by 0.056.

The contribution of a member depends on the size of its swarm (Figure S3). The Shapley value of a swarm follows what the swarm achieves alone, not how many members it has: MACE-MP-0 as a swarm of one receives +0.38 and the six-body swarm +0.18 (PBE, equal share, results kept within each swarm). Because the Owen value divides each swarm’s share only among its own members, a member of the smaller swarm receives more contribution than its Shapley value in all 16 two-swarm settings (four groupings, two receivers, equal and size shares), and members of the larger swarm receive less, by up to 0.19 summed over the swarm. The shift does not require the small swarm to be strong: NEP89 MD, with the scout as its only partner, gains +0.10 over its Shapley value on PBE and HSE06. For swarms of nearly equal size (2 + 2 + 3) the two values differ by at most 0.02 per swarm. For a consortium of computational members such as these, the two values answer different questions. The Owen value is the division when the swarms are organizations with their own budgets whose split between groups is set first, by the game played between the swarms as wholes, such as two laboratories sharing one facility; the Shapley value over members is the division of contribution for knowledge, independent of how the members were grouped. A contribution statement for several swarms should name which of the two it reports.

Definitions

Configurations. The seven bodies are MACE-MP-0 relaxation, NEP89 MD, NEP89 relaxation, MEAM MD, the Materials Project archive, the active learner and the random scout. The groupings are: 4 + 3, {MACE-MP-0, NEP89 MD, MEAM MD, scout} and {archive, NEP89 relaxation, learner}; 2 + 5 strong, {MACE-MP-0, learner} and the other five; 2 + 5 weak, {NEP89 MD, scout} and the other five; 1 + 6, {MACE-MP-0} and the other six; 2 + 2 + 3, {MACE-MP-0, archive}, {NEP89 MD, NEP89 relaxation} and {MEAM MD, learner, scout}; and seven swarms of one. The copies are the seven-body swarm joined by one or two copies of itself, or by a copy of {MACE-MP-0, learner, scout}. A copied ordering differs from its original only through the results it sees; a copied scout draws its own random order.

Dividing the batch. Each swarm jj carries a slot contribution that grows by kWj/∑WkW_j/\sum W per batch (k=4k = 4) and falls by one for each compound it places, as for the units inside a swarm (§ 14). The swarm with the largest contribution fills the next slot, and the slot is passed to one of its units by the swarm’s own slot contributions. The weights are Wj=1W_j = 1 (equal share), Wj=∣j∣W_j = \lvert j\rvert (share by size), or, for the share by record, the tier weights of the three-tier protocol applied to swarms: the swarm prediction for a compound is the mean score its members gave it before the result arrived, its prequential score is τjnj/(nj+8)\tau_j n_j/(n_j+8) over the njn_j results the swarm can see, a running average with rate 0.4 ranks the swarms with an incumbency margin of 0.05, and the leading swarm receives weight 4, the next two weight 2, the others weight 1, and a swarm with a negative average weight 0.25.

Sharing and value. Inside a swarm, results are shared by the rule of the emergent team (§ 14). Published results of other swarms are visible to every member after the batch. The team value is the early share of the ten compounds with the lowest formation energy per atom found over the first quarter of the receiver compute, averaged over 100 campaigns; the campaign seed does not depend on the configuration or the coalition, so differences are paired.

Owen value. For unions B1,…,BuB_1,\ldots,B_u with i∈Bki \in B_k,

Ow⁡i=∑T⊆U∖{k}∑R⊆Bk∖{i}∣T∣!(u−∣T∣−1)!u!∣R∣!(∣Bk∣−∣R∣−1)!∣Bk∣![v(QT∪R∪{i})−v(QT∪R)]\operatorname{Ow}_i=\sum_{T\subseteq U\setminus\{k\}}\sum_{R\subseteq B_k\setminus\{i\}}\frac{|T|!(u-|T|-1)!}{u!}\frac{|R|!(|B_k|-|R|-1)!}{|B_k|!}\left[v(Q_T\cup R\cup\{i\})-v(Q_T\cup R)\right]

where QTQ_T is the union of the swarms in TT. The values of each union sum to its Shapley value in the game between whole swarms, and with one union, or with every member in its own union, the Owen value equals the Shapley value; both properties are checked in the self-test of the experiment script. Each contribution game runs all 128 sub-teams of the seven bodies in 100 campaigns; a sub-team keeps its members’ swarms, and a swarm with no members drops out.

Swarms\multicolumn{4}{c}{PBE}\multicolumn{4}{c}{HSE06}\multicolumn{4}{c}{r2r^2SCAN}
eq.sizerec.isol.eq.sizerec.isol.eq.sizerec.isol.
One swarm of 70.700.700.700.700.500.500.500.500.480.480.480.48
4 + 30.670.650.660.680.490.470.460.490.460.450.450.50
2 + 5, strong pair0.650.660.670.640.480.450.510.470.470.450.470.49
2 + 5, weak pair0.600.660.690.570.410.470.490.380.450.460.490.44
1 + 6, MACE-MP-0 alone0.730.680.750.690.550.470.600.530.500.460.510.48
2 + 2 + 30.620.620.660.620.400.390.440.460.440.420.460.50
7 swarms of 10.600.600.650.540.370.370.390.400.420.420.430.44
7 + 7 (copies)0.680.680.690.680.510.510.510.510.470.470.460.51
7 + 7 + 7 (copies)0.690.690.690.670.520.520.510.540.460.460.470.53
7 + 3 (copies)0.650.670.640.650.510.490.480.480.450.470.450.47

Table S19. Early share of the ten compounds with the lowest formation energy per atom found by several swarms sharing one batch (emergent organization inside each swarm), mean over 100 campaigns. Columns: share of the batch per swarm (equal, by size, by record) with published results, and equal share with results kept within each swarm. The one-swarm row (0.702, 0.503, 0.481 before rounding) comes from runs separate from those of Section S6.1 and Figure 9 of the main text (0.696, 0.487, 0.465).

With a flat organization inside each swarm instead of the emergent one, the same groupings reach lower values throughout (one flat swarm of seven: 0.61, 0.38 and 0.44 on the three receivers), and MACE-MP-0 in its own swarm with a share by record gains +0.13 on PBE and +0.18 on HSE06 over the flat swarm of seven.

Swarms that combine, nest, and separate

Setting and results

In Section S7.1 the grouping of the members into swarms is fixed before the first batch. A consortium does not stay fixed: two groups whose recommendations agree hold joint meetings, a group whose recommendations keep failing against the measurements is placed under the group whose recommendations hold, and a group whose members disagree divides. Experiment 17 lets the structure of the Al–Mg–Si campaign change in this way while it runs. The structure has two levels above the members: independent groups, each made of a head swarm and zero or more nested swarms. Every second batch a coordinator, who sees every result, may apply five operations, each triggered by the orderings the members have just given the open pool and by the groups’ records: fuse two single swarms whose orderings agree (Kendall τ≥0.6\tau\ge0.6) and whose records differ by at most 0.1; nest the weaker of two agreeing groups under the stronger; adopt a group with a negative record into the group with the best record; split a swarm into the components of its agreement graph (τ≥0.2\tau\ge0.2); and detach a nested swarm that no longer agrees with its head. Nesting changes who sees which results: the head swarm’s results reach its nested swarms, and their results reach the head’s leader, while separate groups see only their own. Inside every swarm the emergent organization of § 14.1 runs unchanged. Four rule sets were compared with the fixed structure from five starting structures (seven single members; 2 + 2 + 3 by level of theory; 4 + 3; 2 + 5 with the weak pair; one swarm of seven): combine only (fuse, nest, adopt), separate only (split, detach), both kinds with the thresholds above as cold-start defaults, and both with thresholds learned by the cross-entropy method on training seeds disjoint from the evaluation seeds. When the structure changes, no fixed set of unions exists, so contribution uses the level structure value [104], the generalization of the Owen value to nested unions, evaluated for each structure that was in force and averaged with weights equal to the compute spent under it in the early window (definitions, learned thresholds and all values in the subsections below).

Combining never lowered the outcome; separating lowered it in most settings (Figure S4). With combination only, the changing structure matched or exceeded the fixed one from all four multi-group starts on all three receivers, with intervals above zero in 10 of the 12 settings; the largest gains start from seven single members (+0.029 on PBE, +0.027 on HSE06, +0.018 on r2r^2SCAN). The operation behind the gain is adoption: in the early window a campaign started from seven single members adopts 2.2 to 2.4 groups, nests 0.8 to 1.3 and fuses at most 0.2, and the adopted members then see the results of the group whose orderings hold. The number of independent groups falls from seven to between 1.9 and 2.6 at a quarter of the compute and to between 1.2 and 1.6 at half of it (Figure S4). Separation alone lowered the mean outcome in all 12 settings in which a swarm can divide (every start except seven single members; 90% interval below zero in at least 9), by up to 0.060 (2 + 5 with the weak pair, HSE06), and never raised it, because a separated swarm loses the results of its former partners. With results published between groups, the 4 + 3 start does not lose with both operations (+0.002, +0.010, +0.013 on the three receivers); seven single members that already share every result gain nothing from regrouping, which then changes only the division of the batch (−0.027, −0.014, −0.010). The learned thresholds keep both operations in use (from seven single members on PBE, 3.8 fusions, 4.3 nestings, 2.7 splits and 3.9 detachments per campaign in the early window), so the number of groups keeps moving, and they remove most of the loss from separation (one swarm of seven at the start: −0.013, −0.004 and −0.001 against −0.038, −0.030 and −0.014 with the default thresholds), but stay below combination only in 10 of the 12 multi-group settings.

Figure S4. Swarms that combine, nest and separate during the Al–Mg–Si campaign (results kept within each group, batch divided between groups by record). (a) Number of independent groups against the receiver compute spent, PBE receiver; mean (solid or dashed line) and 5th–95th percentile range over 100 campaigns (thin dotted outlines of the same color); the gray vertical line marks the end of the early window. (b) Early share of the ten compounds with the lowest formation energy per atom found with a changing structure minus the value of the fixed structure with the same start, PBE receiver; starts are seven single members, 2 + 2 + 3 by level of theory, 4 + 3, 2 + 5 with the weak pair, and one swarm of seven; mean and 90% bootstrap interval over 100 matched campaigns. (c) Contribution minus Shapley value per member for the 2 + 2 + 3 start, PBE receiver: Owen value of the fixed groups, and the level structure value averaged over the structures in force for the two changing rule sets; mean and 90% interval over 40 campaigns.

Small groups by level of theory that combine as records accumulate outperform one large swarm on the held-out receivers. Started as 2+2+32+2+3 (MACE-MP-0 with the archive; the two NEP89 members; MEAM, the learner and the scout) with combination only, the campaign exceeds the fixed seven-member swarm by +0.044 (90% interval +0.023 to +0.065) on HSE06 and +0.083 (+0.067 to +0.100) on r2r^2SCAN, and falls short by 0.018 on PBE, the receiver that MACE-MP-0 orders most closely. In these campaigns the two weaker groups are adopted under the MACE-MP-0 group (1.2 to 1.4 adoptions per campaign) but keep their own swarms inside it, so the cells that merge agreeing members stay within one level of theory, while the adopted swarms receive the leading group’s results. For a consortium of computational members such as these this suggests, as a conjecture not tested here, beginning with groups that share a level of theory or a technique and placing a failing group under the supervision of the group with the best record, rather than dividing a large group.

When the structure changes, each member’s contribution stays close to its Shapley value (Figure S4). With the fixed 2+2+32+2+3 groups the Owen value moves contribution away from the archive (−0.019) and toward NEP89 MD (+0.024), both intervals excluding zero on PBE. Over the twelve games with a changing structure the averaged level structure value differs from the Shapley value by at most 0.043 for any member and by 0.004 to 0.028 on average, and only one member in one game (the archive, combination only from 2+2+32+2+3 on PBE, where the structure stays close to its start) has an interval excluding zero. A grouping moves contribution between members only as long as it persists. For a consortium of computational members such as these that regroups during a campaign, the Shapley value over members is therefore an adequate contribution statement, whereas a persistent organization of groups, such as two laboratories with separate budgets, needs the union-structured value of Section S7.1.

Definitions

Structure and sharing. A structure is a list of independent groups; a group is a head swarm followed by zero or more nested swarms; a swarm is a set of members. The batch of k=4k=4 compounds is divided by stride slot contributions held per member at three levels: between groups, between the swarms of a group, and between the cells of a swarm (§ 14); because the contributions are held per member, they carry over when the structure changes. Groups and the swarms within a group receive tier weights from their records (leader r2r^2, second rr, others 1, a negative record the floor weight, with r=2r=2 and floor 0.25 as in § 14). Inside a swarm, results are shared by the rule of the emergent team. Within a group, results proposed by the head swarm reach every member of the group, and results proposed by a nested swarm reach its own members and the leader of the head swarm. Results of another group are visible only when results are published between groups.

Records. The record of a set of members BB is q(B)=τBn/(n+8)q(B)=\tau_B n/(n+8), where τB\tau_B is the Kendall τ\tau between the mean score the members of BB gave each evaluated compound before its result arrived and the receiver value, over all nn results so far. The coordinator that divides the batch between groups and changes the structure therefore uses every result, whereas each member uses only the results it can see. The fixed structures of this annex use the same record to divide the batch, so their values differ from the share by record of Section S7, which uses only the results a swarm can see.

Operations. Every pp batches, while more than kk compounds remain, the agreement a(B,C)a(B,C) between two sets of members is the Kendall τ\tau between their mean scores on up to 60 open compounds. The operations are applied in this order:

  1. split: a swarm with two or more members is divided into the connected components of the graph with an edge between two members when their agreement is at least θs\theta_s; when the swarm contains a member with a positive running score, members with a negative running score lose their edges. The component with the best running score keeps the position of the swarm; another component stays in the group as a nested swarm if it agrees with the head swarm at least θn\theta_n or its record is negative, and otherwise forms a new group;

  1. detach: a nested swarm whose agreement with its head swarm is below θs\theta_s and whose record is non-negative forms a new group;

  1. promote: a nested swarm whose record exceeds the head swarm’s by more than δ\delta becomes the head swarm;

  1. fuse and nest: pairs of groups with agreement at least θj\theta_j are taken in order of decreasing agreement, each group at most once, provided the stronger record is non-negative. Two single-swarm groups with non-negative records that differ by at most δ\delta fuse into one swarm; otherwise the swarms of the weaker group are appended to the stronger group as nested swarms;

  1. adopt: when the best group record exceeds δ\delta, every other group with a negative record is appended to the best group as nested swarms.

Combination only applies operations 3 to 5, separation only operations 1 to 3 (a component that leaves a swarm may stay nested in the same group), and both kinds apply all five. The cold-start defaults are θj=0.6\theta_j = 0.6, θs=0.2\theta_s = 0.2, θn=0.4\theta_n = 0.4, δ=0.1\delta= 0.1 and p=2p = 2. The learned thresholds were fitted by the cross-entropy method (population 16, 6 iterations, elite quarter) to the mean early value over the five starts on 12 training campaigns per start with the PBE receiver, with seeds disjoint from the evaluation seeds: θj=0.05\theta_j = 0.05, θs=0.80\theta_s = 0.80, θn=0.21\theta_n = 0.21, δ=0.33\delta= 0.33, p=3p = 3 (training value 0.672 against 0.656 with the defaults). The learned set fuses and nests almost any two groups and divides almost any swarm, so the structure changes at nearly every opportunity.

Contribution with a changing structure. For a structure with groups G\mathcal{G} and swarms S\mathcal{S}, the level structure value [104] of member ii is the mean marginal contribution v(Piπ∪{i})−v(Piπ)v(P_i^\pi\cup\{i\}) - v(P_i^\pi) over the orderings π\pi of the seven members in which the members of every swarm, and the members of every group, appear consecutively; PiπP_i^\pi is the set of members before ii in π\pi. With no unions it is the Shapley value and with one level of unions it is the Owen value; both reductions, efficiency, and the property that the values of a group sum to the group’s Shapley value in the game between whole groups are checked in the self-test of the experiment script. For each campaign seed the structure trajectory of the full team is logged, and the contribution is the level structure value of each structure in force weighted by the compute spent under it in the first quarter of the receiver compute. This compute-weighted average is descriptive, not a new value: it is efficient because each structure’s value is efficient and the weights sum to one, but the other axioms are not established for the average. The coalition values v(S)v(S) run every sub-team with every swarm of the starting structure restricted to its members in SS, with the same rule set, in 40 campaigns per sub-team.

StartRule setValueGroups at a quarter
PBEHSE06r2r^{2}SCANPBEHSE06r2r^{2}SCAN
Seven single membersfixed0.5920.4240.4597.07.07.0
combine only0.6210.4510.4771.92.32.6
both, default0.6130.4400.4782.43.23.4
both, learned0.6240.4350.4623.03.53.6
2 + 2 + 3fixed0.6610.5340.5343.03.03.0
combine only0.6810.5420.5511.11.51.5
separate only0.6580.5060.5273.83.73.7
both, default0.6670.5130.5401.51.71.9
both, learned0.6720.5380.5412.32.42.6
4 + 3fixed0.6450.4460.4642.02.02.0
combine only0.6500.4480.4721.21.41.4
separate only0.6180.3910.4523.32.73.0
both, default0.6230.3940.4541.41.82.0
both, learned0.6250.4290.4552.12.12.1
2 + 5, weak pairfixed0.6330.4750.4872.02.02.0
combine only0.6420.4930.4911.31.21.4
separate only0.6030.4150.4652.72.62.5
both, default0.6000.4260.4621.41.71.8
both, learned0.6310.4740.4912.22.32.4
One swarm of 7fixed0.6990.4980.4681.01.01.0
separate only0.6590.4700.4541.61.61.5
both, default0.6610.4680.4541.11.11.2
both, learned0.6860.4940.4671.71.61.3

Table S20. Early share of the ten compounds with the lowest formation energy per atom found with a changing structure (results kept within each group), mean over 100 matched campaigns, and the mean number of independent groups at a quarter of the receiver compute. Separation only is omitted for seven single members, which cannot divide.

Members that move between swarms or belong to two

Setting and results

In Sections S7.1 and S8.1 a member belongs to one swarm and changes swarm only when its whole swarm fuses, nests or splits. In a consortium a single member often changes group on its own, for example a researcher whose calculations keep agreeing with another laboratory’s, and some members belong to two groups at once, such as a student shared between two laboratories who attends both groups’ meetings. Experiment 18 adds two member rules to the campaign of Section S8.1, applied by the same coordinator every second batch. Under move, a member leaves its swarm for the other swarm whose ordering it agrees with most, when that agreement exceeds its agreement with the rest of its own swarm by at least 0.2 (Kendall τ\tau on the open pool); the receiving swarm accepts it when its record is non-negative, and a member with a negative record is not accepted into a swarm with a positive one. The mover enters at the lowest tier and waits two restructuring steps before it may move again. Under share, a member keeps its own swarm and also sits in the one other swarm it agrees with most, at agreement 0.6 or more and on the same acceptance rule: it sees that swarm’s results and that swarm sees its proposals, while its batch slots still come from its own swarm; the second membership is reassessed at every step. These thresholds are cold-start defaults; a learned set was fitted with the cross-entropy method on training seeds disjoint from the evaluation seeds (a τ\tau margin of 0.25 to move, agreement 0.07 to share, one step between moves). The campaigns use the seeds of Section S8.1, so every comparison with a fixed grouping, and with publishing every result to all swarms, is matched campaign by campaign. With shared members the structure is not a partition, so the level structure value of Section S8.1 is averaged over the partitions obtained by placing each shared member in one or the other of its swarms; this splits the member’s place between its two groups and keeps the contributions summing to the team value. The construction is related to the configuration value of Albizuri et al. [109], which extends the Owen value to overlapping unions, and to cooperative games with overlapping coalitions [110] (definitions and all values in the subsections below).

A member that moves on its own reassembles a fragmented team; within designed groups it rarely moves (Figure S5–Figure S5). Started from seven single members with a fixed grouping, the move rule with default thresholds raises the early value by +0.048 on PBE, +0.071 on HSE06 and +0.026 on r2r^2SCAN, with intervals above zero, against +0.029, +0.027 and +0.018 when whole swarms combine (Section S8.1). Each member goes to the member whose ordering it shares: on PBE, NEP89 MD joins NEP89 relaxation in 0.78 of the campaigns in the early window, MACE-MP-0 joins MACE MD in 0.76, and the active learner joins the archive in 0.43. From the 2 + 2 + 3 and 4 + 3 groupings a member moves 0.1 to 1.3 times per campaign and the value does not change (all intervals include zero on the three receivers), because these groupings already place members with agreeing orderings together. From 2 + 5 with the weak pair, NEP89 MD leaves the scout for the five-member swarm in 0.86 of the campaigns, with a gain of +0.011 on PBE.

Figure S5. Members that move between swarms or belong to two swarms during the Al–Mg–Si campaign (learned thresholds; results kept within each swarm except through shared members). (a–c) Early share of the ten compounds with the lowest formation energy per atom found minus that of the fixed grouping with the same start, for moving members, shared members and both (marker shape), per receiver (color, as in Figure S3); mean and 90% bootstrap interval over 100 matched campaigns. (d) Early-window value of the fixed grouping with results kept within each swarm, with every result published to all swarms, and with moving and shared members, for seven single members and the 4 + 3 start. (e) Contribution with moving and shared members minus the Shapley value for the 4 + 3 start, PBE and HSE06; mean and 90% interval over 40 campaigns.

A shared member carries most of the benefit of publishing every result, and on the hybrid and meta-GGA receivers more (Figure S5). At the default agreement of 0.6 fewer than one member is shared on average for most starts, and the gains stay below 0.008. The learned threshold (0.07) shares 1.5 to 2.8 members at a quarter of the compute in all starts except 2 + 5, and sharing then raises the value from every multi-swarm start on all three receivers, by +0.024 to +0.045 from 2 + 2 + 3 and +0.007 to +0.012 from 4 + 3. Publishing every result to every swarm raises seven single members on PBE further (0.678 against 0.659 with moving and shared members), but lowers 4 + 3 on HSE06 and r2r^2SCAN below the isolated groups (0.438 and 0.428 against 0.446 and 0.464), whereas moving and shared members raise both (0.458 and 0.469). A shared member links two swarms only when their orderings agree and the receiving swarm’s record holds, so results proposed by members whose orderings disagree with the receiver, chiefly NEP89 MD, do not reach the swarms they would mislead. With both rules and learned thresholds, the 2+2+32 + 2 + 3 grouping by level of theory exceeds one swarm of seven by +0.077+0.077 (90% interval +0.056+0.056 to +0.098+0.098) on HSE06 and +0.089+0.089 (+0.074+0.074 to +0.105+0.105) on r2SCAN\mathrm{r}^{2}\mathrm{SCAN}, matches it on PBE (+0.003+0.003, −0.009-0.009 to +0.013+0.013), and exceeds the same grouping with combination of whole swarms by +0.020+0.020 on PBE and +0.033+0.033 on HSE06. For a consortium of computational members such as these, a few members who attend a second group’s meetings, chosen by agreement and record, pass on knowledge more selectively than a common results server.

Moving and shared members leave contribution close to the Shapley value (Figure S5). Over the sixteen games (PBE and HSE06; 2+2+32 + 2 + 3 and 4+34 + 3; move, share and both from a fixed grouping, and both with combination of swarms) the averaged level structure value differs from the Shapley value by at most 0.043 for any member and by 0.008 to 0.028 on average, and the contributions sum to the team value to within 4×10−154 \times10^{-15}. Five games have one member with an interval excluding zero; in three of them it is MEAM MD in the 4+34 + 3 grouping on HSE06 (−0.032-0.032 to −0.035-0.035), which starts in the swarm of MACE-MP-0 and NEP89 MD. As for swarms that regroup, a membership that changes during the campaign, or is split between two groups, moves contribution between members only to the extent that it persists.

Definitions

Member rules. The structure and the swarm-level operations are those of § 16; the member rules are applied after them at every restructuring step, while more than kk compounds remain. The agreement a(i,C)a(i,C) between member ii and swarm CC is the Kendall τ\tau between the scores of ii and the mean scores of the members of CC on up to 60 open compounds. Move: in order of decreasing running score, a member ii in swarm BB whose last move is at least cc steps ago moves to the swarm C≠BC \ne B with the largest a(i,C)a(i,C) when a(i,C)−a(i,B∖{i})≥θma(i,C) - a(i,B \setminus\{i\}) \ge\theta_m (for a single-member swarm the second term is zero), q(C)≥0q(C) \ge0, and not both sˉi<0\bar{s}_i < 0 and q(C)>0q(C) > 0, where qq is the record of § 16 and sˉi\bar{s}_i the running score of § 14.1; a swarm left empty is removed, and a group whose head swarm empties takes its next swarm as head. Share: each member ii is attached to the swarm C≠BC \ne B with the largest a(i,C)a(i,C) when a(i,C)≥θba(i,C) \ge\theta_b, on the same acceptance rule; it then sees every result proposed by a member of CC, and the members of CC see every result it proposes. The cold-start defaults are θm=0.2\theta_m = 0.2, θb=0.6\theta_b = 0.6 and c=2c = 2. The learned thresholds, fitted for the fixed grouping with both rules on 12 PBE training seeds per start, five starts, population 16 and six iterations, are θm=0.251\theta_m = 0.251, θb=0.073\theta_b = 0.073 and c=1.13c = 1.13 (moves are counted in whole batches, so a member may move again two batches after its last move; training value 0.685 against 0.667 at the defaults).

Contribution with overlapping membership. When the members in a set HH also sit in a second swarm, each assignment of every member of HH to its own or to its second swarm defines a partition of the members into swarms and groups. The contribution of a structure in force is the mean, over the 2∣H∣2^{|H|} assignments, of the level structure value of § 16. It is efficient because every term is, and it equals the level structure value when HH is empty; both properties are checked in the self-test of the experiment script. As in § 16, the contribution of a campaign weights the structures in force by the compute spent under each in the first quarter of the receiver compute, and the coalition values run every sub-team with the same rules in 40 campaigns.

StartMember ruleValue PBEValue HSE06Value r2SCANr^{2}\mathrm{SCAN}Shared at a quarter PBEShared at a quarter HSE06Shared at a quarter r2SCANr^{2}\mathrm{SCAN}
Seven single membersnone0.5920.4240.4590.00.00.0
move, default0.6400.4940.4850.00.00.0
share, default0.5960.4310.4621.40.80.9
both, default0.6420.4940.4871.20.80.4
move, learned0.6400.4960.4870.00.00.0
share, learned0.6280.4480.4782.12.12.4
both, learned0.6590.5090.4892.72.82.5
2 + 2 + 3none0.6610.5340.5340.00.00.0
move, default0.6590.5360.5370.00.00.0
share, default0.6640.5390.5351.00.40.3
both, default0.6600.5370.5360.50.20.1
move, learned0.6600.5330.5360.00.00.0
share, learned0.7050.5730.5582.12.32.4
both, learned0.7020.5750.5571.51.92.0
4 + 3none0.6450.4460.4640.00.00.0
move, default0.6450.4470.4640.00.00.0
share, default0.6480.4490.4641.01.00.5
both, default0.6470.4490.4640.60.70.3
move, learned0.6440.4470.4640.00.00.0
share, learned0.6580.4570.4712.42.51.7
both, learned0.6570.4580.4691.71.91.6
2 + 5, weak pairnone0.6330.4750.4870.00.00.0
move, default0.6440.4820.4860.00.00.0
share, default0.6350.4760.4870.30.10.2
both, default0.6440.4820.4860.00.00.0
move, learned0.6410.4810.4840.00.00.0
share, learned0.6580.4990.4921.62.01.9
both, learned0.6480.4930.4890.10.10.4

Table S21. Early share of the ten compounds with the lowest formation energy per atom found with moving and shared members from a fixed grouping, mean over 100 matched campaigns, and the mean number of shared members at a quarter of the receiver compute.

Contribution that is not shared equally

Setting and results

Every value in the preceding sections divides a joint gain equally: the Shapley value gives each member of a coalition the same share of the gain that the coalition creates beyond its sub-coalitions (its Harsanyi dividend, § 2.1 of the main text), and § 17.1 places a member shared by two swarms in each with weight one half. A consortium often agrees on an unequal division instead: contribution in proportion to the compute each partner spent, to what each partner achieves alone, or to seniority. The weighted Shapley value of Kalai and Samet [111] covers these agreements. Each member keeps its own solo gain, and the dividend of a coalition TT is divided in proportion to weights wiw_i, so member ii receives wi/∑j∈Twjw_i/\sum_{j\in T} w_j of it; with an order of seniority instead of weights, the dividend goes to the most senior members present in TT, the ordered partition that Kalai and Samet [111] allow in their weight systems. Every such rule keeps the contributions summing to the team gain. Experiment 19 reruns all 128 sub-teams of the seven computed Al–Mg–Si advisers in one swarm (100 matched campaigns per sub-team on each of the PBE, HSE06 and r2r^{2}SCAN receivers) and divides the outcome by six rules: equal shares; shares in proportion to the compute each body spends producing its advice over the pool (NEP89 MD 1,027 s, NEP89 relaxation 282 s, MEAM MD 200 s, MACE-MP-0 relaxation 166 s, 1 s for the archive and for the two proposal rules); in proportion to the inverse of that compute; in proportion to the solo gain; by the three tiers of § 14.1 as seniority; and, outside the Shapley family, the team gain divided in proportion to the solo gains. It then repeats the overlapping-membership games of § 17.1 with the shared member placed in its own swarm with weight hh from 0 to 1 (definitions and all values in the subsections below).

The division rule moves contribution more than any structure tested (Figure S6,Figure S6). Against the equal division, the other rules change a member’s contribution by up to 0.27 of the early-window value (NEP89 MD on HSE06 under inverse-compute weights; 0.26 on PBE, where the team gain is 0.57), whereas changing or overlapping structures moved it by at most 0.043 (§ 16.1 and § 17.1). The cause is the size of the joint dividends: pairs of advisers that point to the same low-formation-energy compounds lose value together (on PBE, MACE-MP-0 relaxation with the archive −0.725 and the archive with NEP89 MD −0.644), while larger coalitions gain (the archive, NEP89 MD and MACE-MP-0 together +0.667), so a weight decides how much of each overlap a member is charged as well as how much of each joint gain it receives. Dividing by compute charges the overlaps to the potentials that spent the most: NEP89 MD falls from +0.085 to −0.005 on PBE and the archive, which spent no compute, rises from +0.176 to +0.441. Dividing by the inverse of compute reverses this (NEP89 MD +0.348, archive +0.042). Under seniority the senior tiers carry the losses of every overlap with the junior members, so the active learner is charged none of them and rises from +0.025 to +0.244 (its solo gain is +0.285). The intermediate weights of panel (a) move every contribution smoothly between these limits.

Figure S6. Dividing joint gains by compute moved the contribution of the MP archive from +0.18 to +0.44 and that of NEP89 MD from +0.09 to about zero, more than any team structure moved any contribution (at most 0.043). Computed Al–Mg–Si campaign, one swarm of seven members, PBE receiver, n=100n = 100 matched campaigns. (a) Contribution when each joint gain or loss is divided in proportion to computeα^{\alpha}, from the inverse of the compute (α=−1\alpha= -1) through equal shares (α=0\alpha= 0) to the compute (α=1\alpha= 1); line, mean; band, 90% bootstrap interval. (b) Contribution under six division rules (equal shares; in proportion to compute, to its inverse, or to the solo gain; seniority by tier; team gain in proportion to solo gains): one row per member in the order and color of the legend, marker for the rule, horizontal line for the 90% interval. The weight of a member shared by two swarms is examined below (Figure S7).

The members a group would keep are the same under every rule, up to ties for third; the member it would drop is not. MACE-MP-0 relaxation and the MP archive are among the three highest-contributing members, or tied for third, under all six rules on all three receivers, so the advisers a group would keep do not depend on the rule. MEAM MD, the member with negative contribution on HSE06 and r2r^{2}SCAN under equal shares, stays negative when contribution follows the compute or its inverse, but becomes positive on HSE06 when it follows the solo gain (+0.029) and indistinguishable from zero under seniority (+0.006, interval −0.008 to +0.021), and the proportional rule gives every member non-negative contribution on every receiver. A rule weighted by solo merit therefore protects a member whose solo gain is high but whose ordering adds nothing to the team, and the proportional rule, which ignores interactions altogether, cannot identify a member to drop. For a consortium of computational members such as these the practical sequence is to identify members to drop with the equal division, which charges overlaps without regard to cost or rank, and to apply an agreed weighting only to divide the outcome among the members that remain.

How a shared member is split between its swarms matters little (Figure S7).

Figure S7. Moving the weight of a shared member between its two swarms changed any contribution by at most 0.013. Two swarms of two and one of three, members that move and members shared by two swarms, HSE06 receiver: contribution when the shared member is placed in its own swarm with weight hh, minus the equal placement h=1/2h=1/2; line, mean over n=100n=100 campaigns; band, 90% bootstrap interval.

With 1.4 to 1.7 shared members on average in the early window, moving the weight of the own swarm from h=0h = 0 (contribution follows the information the member passes to its second swarm) to h=1h = 1 (contribution follows its batch slots) changes any contribution by at most 0.013 in the four games (PBE and HSE06; 4 + 3 and 2 + 2 + 3), below the structure effect itself (at most 0.028 from the Shapley value) and an order of magnitude below the effect of the division rule. The equal placement of Section S9.1 is therefore a default whose choice does not change the contribution statement, whereas the weights of the division rule do and have to be stated with it.

Definitions

Weighted values. For a game vv on the bodies NN, the Harsanyi dividends are d(T)=∑S⊆T(−1)∣T∣−∣S∣v(S)d(T)=\sum_{S\subseteq T}(-1)^{|T|-|S|}v(S) for non-empty TT, so that v(S)−v(∅)=∑∅≠T⊆Sd(T)v(S)-v(\varnothing)=\sum_{\varnothing\ne T\subseteq S}d(T). For positive weights ww, the weighted Shapley value of Kalai and Samet [111] is

ϕiw(v)=∑T∋iwi∑j∈Twjd(T),(6)\phi_i^w(v)=\sum_{T\ni i}\frac{w_i}{\sum_{j\in T}w_j}d(T), \tag*{(6)}

which equals the Shapley value for equal weights and gives each body its solo dividend d({i})=v({i})−v(∅)d(\{i\})=v(\{i\})-v(\varnothing) for any weights. For an ordered partition of NN into seniority classes, the dividend of TT is divided in equal parts among the members of TT in the most senior class present in TT. Both rules satisfy ∑iϕi=v(N)−v(∅)\sum_i\phi_i=v(N)-v(\varnothing). The proportional rule gives v(N)−v(∅) gi/∑jgjv(N)-v(\varnothing)\,g_i/\sum_j g_j with gi=max⁡(v({i})−v(∅),0)g_i=\max(v(\{i\})-v(\varnothing),0); it is efficient but is not additive over games. Weighted values give up symmetry by design: two bodies that change every coalition equally receive different contributions when their weights differ. The values are computed per campaign from the 128 coalition values of that campaign; the bootstrap is over the 100 matched campaigns.

Weights. Compute weights are the seconds each body spends computing its advice for the whole pool (GPU seconds for MACE-MP-0 and NEP89, core seconds for MEAM): NEP89 MD 1,027, NEP89 relaxation 282, MEAM MD 200, MACE-MP-0 relaxation 166; the archive, the active learner and the random scout compute no advice table and receive the floor of 1 s. Solo weights are max⁡(vˉ({i})−vˉ(∅),0.01)\max(\bar{v}(\{i\})-\bar{v}(\varnothing),0.01) from the mean over campaigns. Seniority uses the tiers of § 14.1: MACE-MP-0 relaxation and the archive first; NEP89 MD, NEP89 relaxation and MEAM MD second; the active learner and the random scout third. The family of panel (a) of Figure S6 uses wi=ciαw_i=c_i^\alpha for 21 values of α\alpha from −1-1 to 1.

Shared members. With a set HH of shared members, each assignment of the members of HH to their own or second swarm defines a partition; the assignment that places mm members in their own swarm receives weight hm(1−h)∣H∣−mh^{m}(1-h)^{|H|-m}, and the contribution of the structure in force is the weighted mean of the level structure values (§ 17). The thresholds are the learned ones of § 17.1; 100 campaigns per coalition.

BodyEqualCompute1/computeSolo gainSeniorityProportional
PBE receiver (v(N)=0.692v(N) = 0.692, v(∅)=0.123v(\varnothing) = 0.123)
MACE-MP-0 relaxation+0.213+0.089+0.210+0.161+0.122+0.139
NEP89 MD+0.085−0.005+0.348+0.069+0.088+0.101
NEP89 relaxation+0.071+0.015+0.041+0.104+0.069+0.074
MEAM MD+0.028−0.014−0.044+0.057−0.008+0.058
MP archive+0.176+0.441+0.042+0.138+0.098+0.131
Active learner+0.025+0.088−0.016+0.045+0.244+0.057
Random scout−0.028−0.044−0.012−0.006−0.044+0.010
HSE06 receiver (v(N)=0.507v(N) = 0.507, v(∅)=0.123v(\varnothing) = 0.123)
MACE-MP-0 relaxation+0.199+0.117+0.239+0.118+0.092+0.114
NEP89 MD+0.055−0.015+0.330−0.001−0.007+0.087
NEP89 relaxation+0.001+0.007−0.040+0.075+0.056+0.014
MEAM MD−0.033−0.073−0.238+0.029+0.006+0.036
MP archive+0.165+0.326+0.084+0.114+0.080+0.090
Active learner+0.004+0.053−0.001+0.041+0.157+0.034
Random scout−0.008−0.031+0.010+0.007−0.001+0.009
r2r^{2}SCAN receiver (v(N)=0.467v(N) = 0.467, v(∅)=0.111v(\varnothing) = 0.111)
MACE-MP-0 relaxation+0.147+0.075+0.156+0.126+0.105+0.084
NEP89 MD+0.084−0.000+0.299+0.025+0.080+0.102
NEP89 relaxation+0.034−0.005−0.024+0.052+0.039+0.046
MEAM MD−0.055−0.064−0.115−0.019−0.032+0.001
MP archive+0.125+0.312+0.028+0.103+0.082+0.088
Active learner+0.021+0.045+0.004+0.032+0.082+0.025
Random scout−0.000−0.007+0.008+0.037+0.000+0.010

Table S22. Contribution of the seven bodies in one swarm under six division rules, mean over 100 matched campaigns (90% interval widths 0.001 to 0.044).

Shared members. Over the four overlapping-membership games, the largest change of any contribution between h=0h = 0 and h=1h = 1 is 0.012 (HSE06, 4 + 3), 0.013 (HSE06, 2 + 2 + 3), 0.005 (PBE, 4 + 3) and 0.013 (PBE, 2 + 2 + 3); the largest difference between the equal placement and the Shapley value in the same games is 0.025, 0.028, 0.011 and 0.015. The body with the largest change is NEP89 MD in the 2 + 2 + 3 grouping (+0.013 from h=0h = 0 to h=1h = 1 on both receivers). The contributions sum to the team gain to within 2×10−142 \times10^{-14} for every hh.

Mg2Si\mathrm{Mg}_2\mathrm{Si} in each source. NEP89 and MEAM do not bind Mg2Si\mathrm{Mg}_2\mathrm{Si}; the other sources do (Table S23). Mg2Si\mathrm{Mg}_2\mathrm{Si} was not a direct fitting target of the MEAM potential: its Si–Mg pair was fitted to the B1 reference structure [62].

SourceFormation energy (eV/atom)Volume (A˚3\mathring{\mathrm{A}}^3/atom)
NEP89, relaxed at 0 K+0.08821.00
MEAM, published (Table IX of Jelinek et al. [62])+0.04223.05
MACE-MP-0, relaxed at 0 K−0.08121.36
Quantum ESPRESSO PBE relaxation (main text, Methods)−0.16021.43
Materials Project mp-1367 (corrected entry)−0.109–
CALPHAD assessment [112], as tabulated by Jelinek et al. [62]−0.225 (enthalpy)–
Measured (300 K, from the fitted lattice parameter) [113]–21.3

Table S23. Formation energy and volume of Mg2Si\mathrm{Mg}_2\mathrm{Si} by source.

This annex gives the extended games of the Al–Mg–Si case study of § 2.4 of the main text: the HSE06 receiver, the molecular-dynamics receivers at heat-treatment temperatures, and two receivers closer to a heat-treatment decision. The HSE06 formation energies carry a k-point uncertainty of about 25 meV/atom; bracketing it by shifting only the elemental references, or the compound energies with them, left the ten HSE06 targets unchanged and the Kendall τ\tau between the PBE and HSE06 orderings at 0.86–0.89. The Shapley contributions of the PBE and HSE06 games, with their intervals over runs, are shown in Figure 5 of the main text; Figure S8 shows those of the temperature games.

Shapley contributions for six molecular-dynamics receivers at 453 and 803 K

Figure S8. In each of the six molecular-dynamics receivers at 453 and 803 K the same potential at 300 K received the largest contribution under both trust modes (+0.13 to +0.23 under learned trust), and the other two potentials at the target temperature a negative or near-zero combined contribution. (a–f) One receiver per panel (MD potential and temperature). Shapley contribution (early share of the ten compounds with the lowest formation energy per atom found) of the six donors, grouped by role (the two other potentials at the target temperature are summed); all 64 coalitions run per receiver. Dark, fixed trust; light, learned trust (the encoding of Figure 5 of the main text). Whiskers: 90% percentile-bootstrap intervals of the mean over 20 runs, in each of which every MD value is drawn from its measured sampling error (Section S3, Table S12).

The same game was played with the HSE06 hybrid functional [7, 8, 9] as the receiver, computed at the PBE geometry; for these metallic compounds HSE06 is a different receiver, not a more accurate level [114, 115], and PBE added as a fifth player (all 32 coalitions run; Figure 5c of the main text). Under fixed trust PBE (+0.17) and MACE-MP-0 (+0.17) receive nearly equal contribution, followed by the archive (+0.12), MEAM MD (−0.08), and NEP89 MD (−0.21). The best coalition is PBE with MACE-MP-0 (v=0.77v = 0.77), and the full pool is again lower (v(N)=0.51v(N) = 0.51). The two leading players carry nearly the same ordering (Kendall τ\tau with HSE06 of +0.86 and +0.80): each alone reaches an early share of about 0.75, and together they reach 0.77, so the symmetry and efficiency axioms split their overlapping contribution between them. Under learned trust the contributions are PBE +0.14, MACE-MP-0 +0.10, archive +0.06, MEAM −0.002, and NEP89 −0.05, with v(N)=0.60v(N) = 0.60; NEP89 is the only player whose contribution remains clearly negative, because it orders the pool against both DFT levels, and the contribution of MEAM is indistinguishable from zero.

These values charge each query its HSE06 compute only, so they do not reflect that the PBE pool cost about 270 core-hours (61,285 wall-seconds on 16 cores) and the MACE-MP-0 pool under three GPU-minutes. Under the combined accounting each query is charged its HSE06 and PBE core-seconds (HSE06 on 32 cores), and a coalition that holds PBE is charged the PBE relaxation of all 168 compounds before its first query and then HSE06 per query; the compute of the potentials, under 0.01% of the total, is not charged. The acquisition does not see cost, so both accountings choose the same compounds in the same order and only the early window, the first quarter of the compute charged, differs. Table S26 gives both accountings for the Al–Mg–Si game and for the band-gap game of § 2.4 of the main text. Under the combined accounting the PBE ordering, which is the most accurate prior, receives negative contribution in both systems and under both trust modes, the contribution of the other donors changes by at most 0.04 (Al–Mg–Si) and 0.02 (band gaps), and the best coalition does not contain PBE.

SystemDonorHSE06 onlyHSE06 + PBE
Al–Mg–SiPBE+0.171 / +0.145−0.109 / −0.203
MACE-MP-0, 0 K+0.167 / +0.101+0.147 / +0.100
MP archive+0.120 / +0.064+0.094 / +0.067
MEAM MD−0.084 / −0.002−0.070 / −0.004
NEP89 MD−0.209 / −0.051−0.166 / −0.040
v(∅)v(\emptyset) ; v(N)v(N)0.345 ; 0.509 / 0.6020.359 ; 0.255 / 0.279
Band gapsPBE (computed)+0.203 / +0.128−0.155 / −0.261
MP GGA+0.040 / +0.093+0.053 / +0.093
JARVIS TB-mBJ+0.040 / +0.121+0.055 / +0.120
SNUMAT HSE06+0.018 / +0.026+0.024 / +0.030
measured gaps+0.020 / +0.088+0.034 / +0.089
v(∅)v(\emptyset) ; v(N)v(N)0.377 ; 0.697 / 0.8340.384 ; 0.395 / 0.455

Table S26. Shapley contribution (early share of the ten targets found) of the donors to the two HSE06 campaigns under the two cost accountings: HSE06 compute only, and HSE06 plus PBE compute with the PBE relaxation of the whole pool charged upfront to every coalition that uses the PBE ordering. Fixed / learned trust; all 32 coalitions run, 20 seeds and batches of one and four per coalition. Band gaps: 142 candidates without halogens, 170 of 171 HSE06 gaps computed. Best coalitions under HSE06 + PBE cost: MACE-MP-0 alone (0.758 / 0.676) for Al–Mg–Si; the measured gaps, TB-mBJ, and MP GGA (0.672, fixed) and TB-mBJ alone (0.909, learned) for band gaps.

Sharing the relaxation cost.

The combined accounting is one end of a range. When the PBE relaxations of the band-gap pool serve several campaigns, each can be charged a share ff of their cost, the division of a common cost among its users studied in airport games [59]. Because the acquisition does not see cost, every coalition chooses the same compounds at every ff, and only the early window of a coalition that uses the PBE ordering changes; the early window was recomputed from the logged campaigns at 41 values of ff from 0 to 1. The contribution of the PBE ordering falls with its own charge, as monotonicity requires [116], from +0.220 at f=0f = 0 to −0.155 at $f = 1 under fixed trust and from +0.135 to −0.261 under learned trust, while the contributions of the four archives change by at most 0.02 between f=0f = 0 and $f = 1 (Table S26; only the contribution of the PBE ordering was recomputed at intermediate ff). At f=1f = 1 the sweep reproduces the combined accounting; at f=0f = 0 it gives +0.220 and +0.135 rather than the +0.203 and +0.128 of the HSE06-only accounting, because every query is still charged its PBE core-seconds and only the upfront relaxation of the pool is shared. It is positive for f<0.567f < 0.567 (fixed) and f<0.341f < 0.341 (learned), so the relaxations pay once they serve 1/f≈1.81/f \approx1.8 or 2.9 campaigns.

Table S24 gives the break-even by window. The rule of the main text is stated for the first quarter of the receiver compute; judged over the whole budget the relaxations pay after 1.2 to 2.2 campaigns, and the Al–Mg–Si fixed-trust PBE ordering stays positive even when charged in full, so the rule is a statement about the early window. In the same campaign neither MEAM nor NEP89 binds Mg2Si\mathrm{Mg}_2\mathrm{Si}; no run separates this from the full-budget result, and neither is offered as the cause of the other. With the hypothetical ordered alloys removed from the band-gap pool (118 compounds, all ten targets archived), the fixed-trust multiple at a quarter of the compute is 2.1; with the alloys kept as candidates but barred as targets (142 compounds) it is 2.4. Result files: compute/live_semiconductor/results/campaigns/window_cost_sweep.json, archived_targets_check.json and archived_targets_only.json.

GameTrust1/81/41/2whole budget
Band gapfixed0.24 [0.21, 0.27], 4.10.57 [0.50, 0.63], 1.80.83 [0.73, 0.94], 1.20.84 [0.74, 0.95], 1.2
Band gaplearned0.21 [0.18, 0.23], 4.80.34 [0.28, 0.39], 2.90.46 [0.36, 0.56], 2.20.46 [0.36, 0.57], 2.2
Al–Mg–Sifixed0.32 [0.30, 0.33], 3.20.62 [0.58, 0.66], 1.60.99 [0.95, 1.00], 1.0never
Al–Mg–Silearned0.27 [0.25, 0.29], 3.70.43 [0.40, 0.47], 2.30.55 [0.51, 0.59], 1.80.61 [0.56, 0.65], 1.7

Table S24. Break-even charge share f∗f^* of the PBE relaxations (95% bootstrap interval, 2000 resamples) and the reuse multiple 1/f∗1/f^*, by the window in which the campaign is judged (share of the receiver compute). Never: the PBE ordering stays positive when charged in full (+0.029, [+0.025, +0.033]).

How much of the disagreement of the potentials with PBE is linear in composition. A fit of the formation energy of each potential minus that of PBE to the atomic fractions of Al, Mg and Si (Table S25) explains about half of the NEP89 difference and less than a fifth of the MEAM difference. Without its linear part, NEP89 orders the pool with PBE (τ=+0.49\tau= +0.49 in MD, +0.50+0.50 relaxed, against −0.18-0.18 and −0.22-0.22 raw); MEAM changes little. Two 0 K checks on Mg2Si\mathrm{Mg}_2\mathrm{Si} agree with the MD values: the NEP89 relaxation with an equation-of-state fit gives +0.088 eV/atom at 21.00 A˚3/atom21.00\ \AA^3/\mathrm{atom}, and the published MEAM value is +0.042 eV/atom at 23.05 A˚3/atom23.05\ \AA^3/\mathrm{atom} (Table IX of Jelinek et al. [62]). Result file: compute/live_almgsi/results/nep89_mg2si_check.json, written by nep89_mg2si_check.py.

MembernnAl, Mg, SiR2R^2residual sdτ\tau rawτ\tau without linear part
NEP89 MD188−0.079, +0.014, −0.2800.470.054−0.18+0.49
NEP89 relaxation179−0.087, +0.019, −0.2750.480.054−0.22+0.50
MEAM MD185−0.150, +0.098, +0.2530.180.143+0.46+0.36
MEAM relaxation179−0.008, +0.014, +0.1590.080.088+0.46+0.40

Table S25. Linear composition fit of the formation energy of each potential minus that of PBE (coefficients for the Al, Mg and Si fractions, eV/atom) and Kendall τ\tau against PBE before and after removing the fitted part.

Accuracy of the computed band-gap receiver. The band-gap campaign ranks candidates by their computed HSE06 gaps, so only the ordering of those gaps enters its games. For the 110 halogen-free compounds that are also in the measured compilation [50], the HSE06 gaps (PBE geometry, self-consistent kk mesh, no spin–orbit coupling) lie below the measured gaps by 0.50 eV on average (mean absolute deviation 0.82 eV, Kendall τ=0.55\tau= 0.55): by 0.79 eV for 23 oxides, 0.65 eV for 8 nitrides, 0.41 eV for 55 chalcogenides, 0.23 eV for 17 pnictides, and 0.72 eV for 7 other compounds. The 28 halides, which the campaign leaves out, lie 1.26 eV below, so the mean over all 138 compounds with a measured gap is 0.65 eV.

A further family of games tests a change of processing environment. Each potential was run at 453 K (T6 artificial ageing) and 803 K (solution treatment; the 803 K receivers may be partly molten, so these games test transfer between potentials rather than solution-treatment physics), and six donors were offered to each of the six hot MD receivers: the same potential at 300 K, the two other potentials at the target temperature, the 0 K MACE-MP-0 relaxation, and PBE and HSE06 at 0 K. All 64 coalitions were run for each receiver (Figure S8). The same potential at 300 K receives the largest contribution in every receiver (+0.13 to +0.23 under learned trust); the two other potentials at the target temperature have a negative combined contribution in all six receivers (for example −0.106 in the MEAM campaign at 803 K, of which NEP89 carries −0.093); with the MD sampling error included, this held in every run for the MEAM receivers and in about half of the runs for the MACE-MP-0 receivers (Section S3), so being at the right temperature does not compensate for a different potential; and for the MACE-MP-0 receivers the second-largest contribution goes to the 0 K relaxation with the same potential. The two DFT levels receive small contribution (−0.05 to +0.08; −0.05 to +0.05 under learned trust), and under learned trust HSE06 receives less than PBE in all six receivers, by a margin below 0.001 for MACE-MP-0 MD at 803 K: the more expensive calculation is not the more useful prior to a potential fitted to PBE-level data. The best coalition for every receiver is the same potential at 300 K alone. For a materials group, when a campaign moves to a new heat-treatment step, the prior that contributes is the group’s own model in the previous step, and the other models at the new step reduce the outcome.

Two further games use receivers closer to a heat-treatment decision. With MACE-MP-0 MD at 300 K as the receiver and six donors (all 64 coalitions run), the fixed-trust contributions are the 0 K relaxation of the same potential +0.16, PBE and the archive +0.09 each, HSE06 +0.07, MEAM MD −0.03, and NEP89 MD −0.09, and the full pool (v(N)v(N) = 0.74) is again below the best single donor (0.82). With quasi-harmonic formation free energies at 803 K as the receiver, computed from MACE-MP-0 phonons [117] for the 49 dynamically stable compounds, five donors (the 0 K relaxation, PBE, and the three potentials in MD at 803 K; 32 coalitions) receive fixed-trust contributions of +0.07, +0.06, +0.06 (MACE-MP-0 MD), +0.05 (NEP89 MD), and −0.16 (MEAM MD). Under learned trust MEAM’s contribution rises to −0.005. Once entropy enters the receiver, the static and the MD donors share the contribution nearly equally, and the potential that orders the pool differently is the one whose removal improves the campaign.

In summary, with the HSE06 hybrid functional as the receiver and PBE as a fifth player, PBE and MACE-MP-0 share the leading contribution when only HSE06 compute is charged, PBE receives negative contribution once its relaxations are charged as well, the full pool again scores below the best pair of donors, and NEP89 is the only player whose contribution remains clearly negative under learned trust, because it orders the pool against both DFT levels. When each potential is run at the ageing and solution-treatment temperatures, the same potential at 300 K receives the largest contribution in each of the six receivers, while the two other potentials at the target temperature have a negative combined contribution: when a campaign moves to a new heat-treatment step, the prior that contributes is the group’s own model in the previous step. Receivers closer to a heat-treatment decision, MD at 300 K and quasi-harmonic formation free energies at 803 K, give the same picture: the potential that orders the pool differently is the one whose removal improves the campaign.

The observed precipitation sequence as an ordinal check

Every value function of Section 2.4 of the main text is defined by a computed receiver, and against an ordering observed in the laboratory the contribution of a donor can change sign as well. The precipitation sequence of Al–Mg–Si alloys, established by electron microscopy and atom-probe studies [118, 119, 120, 121], orders 17 solute states from the dispersed solid solution to equilibrium β\beta-Mg2_2Si. MEAM MD, which receives small positive contribution in the learned-trust PBE game, orders this sequence in reverse (Kendall agreement −0.50), while PBE (+0.70) and MACE-MP-0 (+0.56 at 0 K) reproduce it. The binding energies of nine Mg/Si solute clusters separate the donors in the same way (Kendall τ\tau against PBE: MACE-MP-0 +0.78, MEAM MD −0.67).

Further detail

A subset without PBE pays the PBE relaxation of each compound it queries, which its HSE06 calculation needs as geometry, so no relaxation is charged twice; the borrowed ordering simply exists only once the whole pool has been relaxed.

The test target was the ten of 168 Al–Mg–Si compounds with the lowest formation energy per atom, 141 of them ordered fcc decorations (ordered arrangements of Al, Mg and Si on a face-centered cubic lattice) that no archive holds, with PBE as the receiver and four low-cost sources: the MACE-MP-0 foundation potential [60] relaxed at 0 K, MD at 300 K with NEP89 [61] and with the modified embedded-atom method (MEAM) potential of Jelínek et al. [62], and the Materials Project archive. Under fixed trust MACE-MP-0 contributed +0.23 and the archive +0.15, while NEP89 MD, which orders the pool against PBE (τ=−0.18\tau= -0.18; about half of its offset from PBE is linear in composition, and without that part τ=+0.49\tau= +0.49, Section S11), and MEAM MD (τ=+0.46\tau= +0.46) held the campaign back as misaligned members (−0.20, −0.11).

PBE supplies the prior closest to HSE06 (τ=+0.86\tau= +0.86, against +0.80 for MACE-MP-0), but its relaxations spent the early budget that MACE-MP-0 leaves to the receiver.

Computed campaigns. The band-gap pool held 171 compounds of at most six atoms per primitive cell, relaxed with PBE in Quantum ESPRESSO 7.4 [58] on 16 cores and with HSE06 single points at the PBE geometry; its ordered MgCaO\mathrm{MgCaO} and ZnBeO\mathrm{ZnBeO} alloys come from strongly immiscible oxides and are hypothetical ranking candidates, not synthesizable phases. Three of the ten HSE06 targets are such alloys (CaMgO2\mathrm{CaMgO}_{2} in the CuPt-111 and CuAu-001 orderings, ranks 2 and 6, and layered-wurtzite BeZnO2\mathrm{BeZnO}_{2}, rank 8), and NaNO2\mathrm{NaNO}_{2} is also a target; with the alloys removed as targets, or from the pool, the break-even multiple stays near two (Section S11). The Al–Mg–Si pool held 168 binary and ternary compounds (Materials Project structures near the hull and ordered fcc decorations).

MD formation energies (NEP89, release 2025-04-09, in GPUMD 5.8 [90]; MEAM in LAMMPS 2025.7.22 [91]; MACE-MP-0 with mace-torch 0.3.16) were time averages over 3 ps after 1 ps of equilibration, and in each of 20 runs every MD value was drawn from its measured standard error. v(S)v(S) was the mean share of the receiver’s ten best candidates found over the first quarter of its compute.

Code, data, and result files

All quantitative results are produced by the scripts and stored in the result files below (archived release v1.3.0, https://doi.org/10.5281/zenodo.23876457). Exact computations are deterministic; sampled ones are seeded. The archive manifest hash is f2158ecdc78e0229 (261,053 records, 8 sources); the campaigns use every record, and no held-out split of the archive enters any result. Campaigns on the archive draw each record at most once per campaign (campaign_protocols/code/experiments/_draw.py, called by 26_paired_campaign.py).

  • Eight-donor campaign (MP GGA and its GGA+U (TM-O/F) subset as separate donors), 256 subsets with per-seed checkpoints (experiments 08a–08c): ateki/results_8donor/.

  • Seven-source campaign, 128 subsets with per-seed checkpoints (experiment 08; meeting games and truncation checks): ateki/results/database_attribution/.

  • Harsanyi dividends (experiment 09): ateki/results/harsanyi_dividends/expt_zhuo/.

  • Measures, star and declared meetings (experiment 10): ateki/results/materials_structure_attribution/.

  • Sum and best-group rules (experiment 12): ateki/results/component_rule/.

  • Data-valuation measures (experiment 20): ateki/results/data_valuation_estimators/.

  • Language-model teams (experiments 11 and 13): ateki/results/recorded_meetings_scale/ (sets S1–S3, with the sequential arm in S1_seq/ and the per-campaign coordinator shares in coordinator_per_campaign.txt) and ateki/results/recorded_meetings_4b/.

  • Organization experiments 14–19: ateki/results/{emergent_swarm, replicated_bodies, multiple_swarms, morphing_swarms, mobile_members, unequal_credit}/.

  • Sub-team coalition values of the emergent team, including MACE-MP-0 alone (0.808 on PBE in this swarm setting, the reference for the share of the gap closed; 0.88 in the four-source game of § 2.4 of the main text, a different game with other members, so the two values are not compared): ateki/results/emergent_swarm/shapley.json; early shares in team_values.json.

  • Computed campaigns: compute/live_semiconductor/ (band gaps) and compute/live_almgsi/ (Al–Mg–Si).

  • Statistical re-analyses (held-out pool choice, final-window split): ateki/results/stats_reanalysis/.

  • Figures: writings/papers_2026/regen_figs/ (mgea_*.py for the main-text figures of this version).

References

  1. [1]Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, and Kristin A. Persson. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Materials, 1(1):011002, 2013. doi: 10.1063/1.4812323.DOI
  2. [2]Kamal Choudhary, Kevin F. Garrity, Andrew C. E. Reid, Brian DeCost, Adam J. Biacchi, Angela R. Hight Walker, Zachary Trautt, Jason Hattrick-Simpers, A. Gilad Kusne, Andrea Centrone, Albert Davydov, Jie Jiang, Ruth Pachter, Gowoon Cheon, Evan Reed, Ankit Agrawal, Xiaofeng Qian, Vinit Sharma, Houlong Zhuang, Sergei V. Kalinin, Bobby G. Sumpter, Ghanshyam Pilania, Pinar Acar, Subhasish Mandal, Kristjan Haule, David Vanderbilt, Karin Rabe, and Francesca Tavazza. The joint automated repository for various integrated simulations (JARVIS) for data-driven materials design. npj Computational Materials, 6(1):173, 2020. doi: 10.1038/s41524-020-00440-1.DOI
  3. [3]Sangtae Kim, Miso Lee, Changho Hong, Youngchae Yoon, Hyungmin An, Dongheon Lee, Wonseok Jeong, Dongsun Yoo, Youngho Kang, Yong Youn, and Seungwu Han. A band-gap database for semiconducting inorganic materials calculated with hybrid functional. Scientific Data, 7(1):387, 2020. doi: 10.1038/s41597-020-00723-8.DOI
  4. [4]John P. Perdew, Kieron Burke, and Matthias Ernzerhof. Generalized gradient approximation made simple. Physical Review Letters, 77(18):3865–3868, 1996. doi: 10.1103/PhysRevLett.77.3865.DOI
  5. [5]Ryan Kingsbury, Ayush S. Gupta, Christopher J. Bartel, Jason M. Munro, Shyam Dwaraknath, Matthew Horton, and Kristin A. Persson. Performance comparison of r²SCAN and SCAN metaGGA density functionals for solid materials via an automated, high-throughput computational workflow. Physical Review Materials, 6(1):013801, 2022. doi: 10.1103/PhysRevMaterials.6.013801.DOI
  6. [6]Fabien Tran and Peter Blaha. Accurate band gaps of semiconductors and insulators with a semilocal exchange-correlation potential. Physical Review Letters, 102(22):226401, 2009. doi: 10.1103/PhysRevLett.102.226401.DOI
  7. [7]Jochen Heyd, Gustavo E. Scuseria, and Matthias Ernzerhof. Hybrid functionals based on a screened Coulomb potential. The Journal of Chemical Physics, 118(18):8207–8215, 2003. doi: 10.1063/1.1564060.DOI
  8. [8]Jochen Heyd, Gustavo E. Scuseria, and Matthias Ernzerhof. Erratum: “Hybrid functionals based on a screened Coulomb potential” [J. Chem. Phys. 118, 8207 (2003)]. The Journal of Chemical Physics, 124(21):219906, 2006. doi: 10.1063/1.2204597.DOI
  9. [9]Aliaksandr V. Krukau, Oleg A. Vydrov, Artur F. Izmaylov, and Gustavo E. Scuseria. Influence of the exchange screening parameter on the performance of screened hybrid functionals. The Journal of Chemical Physics, 125(22):224106, 2006. doi: 10.1063/1.2404663.DOI
  10. [10]Pedro Borlido, Thorsten Aull, Ahmad W. Huran, Fabien Tran, Miguel A. L. Marques, and Silvana Botti. Large-scale benchmark of exchange-correlation functionals for the determination of electronic band gaps of solids. Journal of Chemical Theory and Computation, 15(9):5069–5079, 2019. doi: 10.1021/acs.jctc.9b00322.DOI
  11. [11]G. Pilania, J.E. Gubernatis, and T. Lookman. Multi-fidelity machine learning models for accurate bandgap predictions of solids. Computational Materials Science, 129:156–163, March 2017. ISSN 0927-0256. doi: 10.1016/j.commatsci.2016.12.004. URL http://dx.doi.org/10.1016/j.commatsci.2016.12.004.
  12. [12]Maxwell L. Hutchinson, Erin Antono, Brenna M. Gibbons, Sean Paradiso, Julia Ling, and Bryce Meredig. Overcoming data scarcity with transfer learning, 2017.arxiv.org/abs/1711.05099
  13. [13]Chi Chen, Yunxing Zuo, Weike Ye, Xiangguo Li, and Shyue Ping Ong. Learning properties of ordered and disordered materials from multi-fidelity data. Nature Computational Science, 1(1):46–53, January 2021. ISSN 2662-8457. doi: 10.1038/s43588-020-00002-x. URL http://dx.doi.org/10.1038/s43588-020-00002-x.
  14. [14]Danial Khatamsaz, Abhilash Molkeri, Richard Couperthwaite, Jaylen James, Raymundo Arróyave, Douglas Allaire, and Ankit Srivastava. Efficiently exploiting process-structure-property relationships in material design by multi-information source fusion. Acta Materialia, 206:116619, 2021. doi: 10.1016/j.actamat.2020.116619.DOI
  15. [15]Zahra Zanjani Foumani, Amin Yousefpour, Mehdi Shishehbor, and Ramin Bostanabad. Mitigating the effects of source-dependent bias and noise on multi-source Bayesian optimization: Application to materials design. In Volume 3B: 49th Design Automation Conference (DAC), page V03BT03A010. ASME, 2023. doi: 10.1115/detc2023-114414.DOI
  16. [16]Zahra Zanjani Foumani and Ramin Bostanabad. Cost-aware Bayesian optimization with automatic stop condition under multi-fidelity constraints and data. Journal of Mechanical Design, 148(2):021711, 2026. doi: 10.1115/1.4070314.DOI
  17. [17]Juan J. de Pablo, Nicholas E. Jackson, Michael A. Webb, Long-Qing Chen, Joel E. Moore, Dane Morgan, Ryan Jacobs, Tresa Pollock, Darrell G. Schlom, Eric S. Toberer, et al. New frontiers for the materials genome initiative. npj Computational Materials, 5(1):41, 2019. doi: 10.1038/s41524-019-0173-4.DOI
  18. [18]Ankit Agrawal and Alok Choudhary. Perspective: Materials informatics and big data: Realization of the “fourth paradigm” of science in materials science. APL Materials, 4(5):053208, 2016. doi: 10.1063/1.4946894.DOI
  19. [19]Rampí Ramprasad, Rohit Batra, Ghanshyam Pilania, Arun Mannodi-Kanakkithodi, and Chiho Kim. Machine learning in materials informatics: Recent applications and prospects. npj Computational Materials, 3(1):54, 2017. doi: 10.1038/s41524-017-0056-5.DOI
  20. [20]Barbara Levitt and James G. March. Organizational learning. Annual Review of Sociology, 14(1):319–338, 1988. doi: 10.1146/annurev.so.14.080188.001535.DOI
  21. [21]Linda Argote and Ella Miron-Spektor. Organizational learning: From experience to knowledge. Organization Science, 22(5):1123–1137, 2011. doi: 10.1287/orsc.1100.0621.DOI
  22. [22]Linda Argote and Paul Ingram. Knowledge transfer: A basis for competitive advantage in firms. Organizational Behavior and Human Decision Processes, 82(1):150–169, 2000. doi: 10.1006/obhd.2000.2893.DOI
  23. [23]Gabriel Szulanski. Exploring internal stickiness: Impediments to the transfer of best practice within the firm. Strategic Management Journal, 17(S2):27–43, 1996. doi: 10.1002/smj.4250171105.DOI
  24. [24]Wesley M. Cohen and Daniel A. Levinthal. Absorptive capacity: A new perspective on learning and innovation. Administrative Science Quarterly, 35(1):128–152, 1990. doi: 10.2307/2393553.DOI
  25. [25]Mark S. Granovetter. The strength of weak ties. American Journal of Sociology, 78(6):1360–1380, 1973. doi: 10.1086/225469.DOI
  26. [26]Ronald S. Burt. Structural Holes: The Social Structure of Competition. Harvard University Press, Cambridge, MA, 1992.
  27. [27]James G. March. Exploration and exploitation in organizational learning. Organization Science, 2(1):71–87, 1991. doi: 10.1287/orsc.2.1.71.DOI
  28. [28]James P. Walsh and Gerardo Rivera Ungson. Organizational memory. Academy of Management Review, 16(1):57–91, 1991. doi: 10.2307/258607.DOI
  29. [29]Milad Abholsani and Eugenia Kumacheva. The rise of self-driving labs in chemical and materials sciences. Nature Synthesis, 2(6):483–492, 2023. doi: 10.1038/s44160-022-00231-0.
  30. [30]A. Gilad Kusne and Austin McDannald. Scalable multi-agent lab framework for lab optimization. Matter, 6(6):1880–1893, 2023. doi: 10.1016/j.matt.2023.03.022.DOI
  31. [31]Stanley Lo, Sterling G. Baird, Joshua Schrier, Ben Blaiszik, Nessa Carson, Ian Foster, Andrés Aguilar-Granda, Sergei V. Kalinin, Benji Maruyama, Maria Politi, Helen Tran, Taylor D. Sparks, and Alán Aspuru-Guzik. Review of low-cost self-driving laboratories in chemistry and materials science: the “frugal twin” concept. Digital Discovery, 3:842–868, 2024. doi: 10.1039/D3DD00223C.DOI
  32. [32]Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23, pages 1–22, 2023. doi: 10.1145/3586183.3606763.DOI
  33. [33]Alireza Ghafarollahi and Markus J. Buehler. SciAgents: Automating scientific discovery through bioinspired multi-agent intelligent graph reasoning. Advanced Materials, 37(22):2413523, 2025. doi: 10.1002/adma.202413523.DOI
  34. [34]Sirui Hong, Mingchen Zhuge, Jiaqi Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. Metagpt: Meta programming for a multi-agent collaborative framework, 2023.arxiv.org/abs/2308.00352
  35. [35]Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors, 2023.arxiv.org/abs/2308.10848
  36. [36]Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI), pages 8048–8057, 2024. doi: 10.24963/ijcai.2024/890.
  37. [37]Zhihao Zhan, Ting Song, Li Dong, Shaohan Huang, Jianxun Lian, Yan Xia, and Furu Wei. Agensh: Scaling organizational intelligence to 1,024 agents. arXiv preprint arXiv:2609.26781, 2026. doi: 10.48550/arXiv.2609.26781.DOI
  38. [38]Brian Rohr, Helge S. Stein, Dan Guevarra, Yu Wang, Joel A. Haber, Muratahan Aykol, Santosh K. Suram, and John M. Gregoire. Benchmarking the acceleration of materials discovery by sequential learning. Chemical Science, 11(10):2696–2706, 2020. doi: 10.1039/C9SC05999G.DOI
  39. [39]Qiaohao Liang, Aldair E. Gongora, Zekun Ren, Armi Tiihonen, Zhe Liu, Shijing Sun, James R. Deneault, Daniil Bash, Flore Mekki-Berrada, Saif A. Khan, Kedar Hippalgaonkar, Benji Maruyama, Keith A. Brown, John Fisher, III, and Tonio Buonassisi. Benchmarking the performance of Bayesian optimization across multiple experimental materials science domains. npj Computational Materials, 7(1):188, 2021. doi: 10.1038/s41524-021-00656-9.DOI
  40. [40]Lance Kavalsky, Vinay I. Hegde, Eric Muckley, Matthew S. Johnson, Bryce Meredig, and Venkatasubramanian Viswanathan. By how much can closed-loop frameworks accelerate computational materials discovery? Digital Discovery, 2(4):1112–1125, 2023. doi: 10.1039/D2DD00133K.DOI
  41. [41]Huan Chen, Xiang Song, Jian Jin, Pan Ren, and Liang-Jie Zhang. Toward an organizational science of multi-agent LLM systems: Decoupling who, how, and which algorithm, 2026. URL https://arxiv.org/abs/2607.25446.
  42. [42]L. S. Shapley. 17. A value for n-person games. In Contributions to the Theory of Games (AM-28), Volume II, pages 307–318. Princeton University Press, 1953. doi: 10.1515/9781400881970-018.
  43. [43]John C. Harsanyi. A simplified bargaining model for the n-person cooperative game. International Economic Review, 4(2):194, 1963. doi: 10.2307/2525487.DOI
  44. [44]Roger B. Myerson. Graphs and cooperation in games. Mathematics of Operations Research, 2(3):225–229, 1977. doi: 10.1287/moor.2.3.225.DOI
  45. [45]R. B. Myerson. Conference structures and fair allocation rules. International Journal of Game Theory, 9(3):169–182, 1980. doi: 10.1007/BF01781371.DOI
  46. [46]Tianshu Song, Yongxin Tong, and Shuyue Wei. Profit allocation for federated learning. In 2019 IEEE International Conference on Big Data (Big Data), pages 2577–2586. IEEE, December 2019. doi: 10.1109/bigdata47090.2019.9006327. URL http://dx.doi.org/10.1109/bigdata47090.2019.9006327.
  47. [47]Tianhao Wang, Johannes Rausch, Ce Zhang, Ruoxi Jia, and Dawn Song. A Principled Approach to Data Valuation for Federated Learning, pages 153–167. Springer International Publishing, 2020. ISBN 9783030630768. doi: 10.1007/978-3-030-63076-8_11. URL http://dx.doi.org/10.1007/978-3-030-63076-8_11.
  48. [48]Rachael Hwee Ling Sim, Yehong Zhang, Mun Choon Chan, and Bryan Kian Hsiang Low. Collaborative machine learning with incentive-aware model rewards, 2020.arxiv.org/abs/2010.12797
  49. [49]M. G. Kendall. A new measure of rank correlation. Biometrika, 30(1-2):81–93, 1938. doi: 10.1093/biomet/30.1-2.81.DOI
  50. [50]Ya Zhuo, Aria Mansouri Tehrani, and Jakoah Brgoch. Predicting the band gaps of inorganic solids by machine learning. The Journal of Physical Chemistry Letters, 9(7):1668–1673, 2018. doi: 10.1021/acs.jpclett.8b00124.DOI
  51. [51]Alec Belsky, Mariette Hellenbrandt, Vicky Lynn Karen, and Peter Luksch. New developments in the Inorganic Crystal Structure Database (ICSD): accessibility in support of materials research and design. Acta Crystallographica Section B Structural Science, 58(3):364–369, 2002. doi: 10.1107/S0108768102006948.
  52. [52]Jiří Klimeš, David R. Bowler, and Angelos Michaelides. Chemical accuracy for the van der Waals density functional. Journal of Physics: Condensed Matter, 22(2):022201, 2010. doi: 10.1088/0953-8984/22/2/022201.DOI
  53. [53]Anubhav Jain, Geoffrey Hautier, Shyue Ping Ong, Charles J. Moore, Christopher C. Fischer, Kristin A. Persson, and Gerbrand Ceder. Formation enthalpies by mixing GGA and GGA+u calculations. Physical Review B, 84(4):045115, 2011. doi: 10.1103/PhysRevB.84.045115.DOI
  54. [54]Wen Zhang, Lingfei Deng, Lei Zhang, and Dongrui Wu. A survey on negative transfer. IEEE/CAA Journal of Automatica Sinica, 10(2):305–329, 2023. doi: 10.1109/JAS.2022.106004.DOI
  55. [55]Olga Ohrimenko, Shruti Tople, and Sebastian Tschiatschek. Collaborative machine learning markets with data-replication-robust payments, 2019.arxiv.org/abs/1911.09052
  56. [56]Dongge Han, Michael Wooldridge, Alex Rogers, Olga Ohrimenko, and Sebastian Tschiatschek. Replication robust payoff allocation in submodular cooperative games. IEEE Transactions on Artificial Intelligence, 4(5):1114–1128, 2023. doi: 10.1109/TAI.2022.3195686.DOI
  57. [57]Xinyi Xu, Zhaoxuan Wu, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Validation free and replication robust volume-based data valuation. In Advances in Neural Information Processing Systems, volume 34, pages 10837–10848, 2021.
  58. [58]P Giannozzi, O Andreussi, T Brumme, O Bunau, M Buongiorno Nardelli, M Calandra, R Car, C Cavazzoni, D Ceresoli, M Cococcioni, et al. Advanced capabilities for materials modelling with quantum espresso. Journal of Physics: Condensed Matter, 29(46):465901, 2017. doi: 10.1088/1361-648X/aa8f79.DOI
  59. [59]Stephen Littlechild and Guillermo Owen. A simple expression for the Shapley value in a special case. Management Science, 20(3):370–372, 1973. doi: 10.1287/mnsc.20.3.370.DOI
  60. [60]Ilyes Batatia, Philipp Benner, Yuan Chiang, Alin M. Elena, Dávid Péter Kovács, Janosh Riebesell, et al. A foundation model for atomistic materials chemistry. The Journal of Chemical Physics, 163(18):184110, 2025. doi: 10.1063/5.0297006. Preprint arXiv:2401.00096 (2024).DOI
  61. [61]Ting Liang, Ke Xu, Eric Lindgren, Zherui Chen, Rui Zhao, Jiahui Liu, et al. NEP89: universal neuroevolution potential for inorganic and organic materials across 89 elements. Nature Computational Science, 6(7):789–801, 2026. doi: 10.1038/s43588-026-01009-6.DOI
  62. [62]B. Jelinek, S. Groh, M. F. Horstemeyer, J. Houze, S. G. Kim, G. J. Wagner, A. Moitra, and M. I. Baskes. Modified embedded atom method potential for Al, Si, Mg, Cu, and Fe alloys. Physical Review B, 85(24):245102, 2012. doi: 10.1103/PhysRevB.85.245102.DOI
  63. [63]Morten T. Hansen. The search-transfer problem: The role of weak ties in sharing knowledge across organization subunits. Administrative Science Quarterly, 44(1):82–111, 1999. doi: 10.2307/2667032.DOI
  64. [64]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chenggen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025.arxiv.org/abs/2505.09388
  65. [65]Garold Stasser and William Titus. Pooling of unshared information in group decision making: Biased information sampling during discussion. Journal of Personality and Social Psychology, 48(6):1467–1478, 1985. doi: 10.1037/0022-3514.48.6.1467.DOI
  66. [66]Ivan D. Steiner. Group Process and Productivity. Academic Press, New York, 1972.
  67. [67]Thomas W. Malone and Kevin Crowston. The interdisciplinary study of coordination. ACM Computing Surveys, 26(1):87–119, 1994. doi: 10.1145/174666.174668.DOI
  68. [68]Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. Why do multi-agent LLM systems fail?, 2025.arxiv.org/abs/2503.13657
  69. [69]Dominik Janzing, Lenon Minorics, and Patrick Blöbaum. Feature relevance quantification in explainable AI: A causal problem. In International Conference on Artificial Intelligence and Statistics (AISTATS), volume 108 of Proceedings of Machine Learning Research, pages 2907–2916. PMLR, 2020. URL https://proceedings.mlr.press/v108/janzing20a.html.arxiv.org/abs/1910.13413
  70. [70]Tom Heskes, Evi Sijben, Ioan Gabriel Bucur, and Tom Claassen. Causal Shapley values: Exploiting causal knowledge to explain individual predictions of complex models. In Advances in Neural Information Processing Systems, volume 33, pages 4778–4789, 2020.arxiv.org/abs/2011.01625
  71. [71]Daniel M. Wegner. Transactive memory: A contemporary analysis of the group mind. In Brian Mullen and George R. Goethals, editors, Theories of Group Behavior, pages 185–208. Springer, New York, 1987. doi: 10.1007/978-1-4612-4634-3_9.DOI
  72. [72]Kyle Lewis. Measuring transactive memory systems in the field: Scale development and validation. Journal of Applied Psychology, 88(4):587–604, 2003. ISSN 0021-9010. doi: 10.1037/0021-9010.88.4.587. URL http://dx.doi.org/10.1037/0021-9010.88.4.587.
  73. [73]Gayle W. Hill. Group versus individual performance: Are N+1 heads better than one? Psychological Bulletin, 91(3):517–539, 1982. doi: 10.1037/0033-2909.91.3.517.DOI
  74. [74]James R. Larson, Jr. In Search of Synergy in Small Group Performance. Psychology Press, New York, 2010.
  75. [75]Robert M. Grant. Toward a knowledge-based theory of the firm. Strategic Management Journal, 17(S2):109–122, December 1996. ISSN 1097-0266. doi: 10.1002/smj.4250171110. URL http://dx.doi.org/10.1002/smj.4250171110.
  76. [76]Xin Luna Dong, Barna Saha, and Divesh Srivastava. Less is more: Selecting sources wisely for integration. Proceedings of the VLDB Endowment, 6(2):37–48, 2012. doi: 10.14778/2535568.2448938.DOI
  77. [77]Zhi-Hua Zhou, Jianxin Wu, and Wei Tang. Ensembling neural networks: Many could be better than all. Artificial Intelligence, 137(1–2):239–263, 2002. doi: 10.1016/S0004-3702(02)00190-X.DOI
  78. [78]Verlin B. Hinsz, R. Scott Tindale, and David A. Vollrath. The emerging conceptualization of groups as information processors. Psychological Bulletin, 121(1):43–64, 1997. ISSN 0033-2909. doi: 10.1037/0033-2909.121.1.43. URL http://dx.doi.org/10.1037/0033-2909.121.1.43.
  79. [79]Xin Luna Dong, Laure Berti-Equille, and Divesh Srivastava. Integrating conflicting data: The role of source dependence. Proceedings of the VLDB Endowment, 2(1):550–561, 2009. doi: 10.14778/1687627.1687690.DOI
  80. [80]Amirata Ghorbani and James Zou. Data Shapley: Equitable valuation of data for machine learning. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2242–2251. PMLR, 2019. URL https://proceedings.mlr.press/v97/ghorbani19c.html.arxiv.org/abs/1904.02868
  81. [81]Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas J. Spanos. Towards efficient data valuation based on the Shapley value. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), volume 89 of Proceedings of Machine Learning Research, pages 1167–1176, 2019. arXiv:1902.10275.arxiv.org/abs/1902.10275
  82. [82]Amirata Ghorbani, Michael P. Kim, and James Zou. A distributional framework for data valuation. In Proceedings of the 37th International Conference on Machine Learning (ICML), volume 119 of Proceedings of Machine Learning Research, pages 3535–3544, 2020. arXiv:2002.12334.arxiv.org/abs/2002.12334
  83. [83]Benedek Rozemberczki, Lauren Watson, Péter Bayer, Hao-Tsung Yang, Olivér Kiss, Sebastian Nilsson, and Rik Sarkar. The Shapley value in machine learning. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, pages 5572–5579, 2022. doi: 10.24963/ijcai.2022/778.DOI
  84. [84]Yongchan Kwon and James Zou. Beta Shapley: a unified and noise-reduced data valuation framework for machine learning. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 8780–8802. PMLR, 2022. URL https://proceedings.mlr.press/v151/kwon22a.html.arxiv.org/abs/2110.14049
  85. [85]Jiachen T. Wang and Ruoxi Jia. Data Banzhaf: A robust data valuation framework for machine learning. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pages 6388–6421. PMLR, 2023. URL https://proceedings.mlr.press/v206/wang23e.html.arxiv.org/abs/2205.15466
  86. [86]David Lazer and Allan Friedman. The network structure of exploration and exploitation. Administrative Science Quarterly, 52(4):667–694, 2007. doi: 10.2189/asqu.52.4.667.DOI
  87. [87]R. L. Plackett. The analysis of permutations. Applied Statistics, 24(2):193, 1975. doi: 10.2307/2346567.DOI
  88. [88]R. Duncan Luce. Individual Choice Behavior: A Theoretical Analysis. Wiley, New York, 1959. Dover reprint 2005, ISBN 0486441369.
  89. [89]A. Philip Dawid. Present position and potential developments: Some personal views: Statistical theory: The prequential approach. Journal of the Royal Statistical Society. Series A (General), 147(2):278–292, 1984. doi: 10.2307/2981683.
  90. [90]Zheyong Fan, Yanzhou Wang, Penghua Ying, Keke Song, Junjie Wang, Yong Wang, Zhezhu Zeng, Ke Xu, Eric Lindgren, J. Magnus Rahm, et al. Gpumd: A package for constructing accurate machine-learned potentials and performing highly efficient atomistic simulations. The Journal of Chemical Physics, 157(11):114801, 2022. doi: 10.1063/5.0106617.DOI
  91. [91]Aidan P. Thompson, H. Metin Aktulga, Richard Berger, Dan S. Bolintineanu, W. Michael Brown, Paul S. Crozier, Pieter J. in ’t Veld, Axel Kohlmeyer, Stan G. Moore, Trung Dac Nguyen, et al. Lammps - a flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales. Computer Physics Communications, 271:108171, 2022. doi: 10.1016/j.cpc.2021.108171.DOI
  92. [92]R. Dennis Cook. Detection of influential observation in linear regression. Technometrics, 19 (1):15–18, 1977. doi: 10.1080/00401706.1977.10489493.DOI
  93. [93]Pradeep Dubey and Lloyd S. Shapley. Mathematical properties of the Banzhaf power index. Mathematics of Operations Research, 4(2):99–131, 1979. doi: 10.1287/moor.4.2.99.DOI
  94. [94]G. Owen. Values of games with a priori unions. In Mathematical Economics and Game Theory, Lecture Notes in Economics and Mathematical Systems, pages 76–88. Springer Berlin Heidelberg, 1977. ISBN 9783642454943. doi: 10.1007/978-3-642-45494-3_7.DOI
  95. [95]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles, pages 611–626. ACM, 2023. doi: 10.1145/3600006.3613165.DOI
  96. [96]Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2):65–70, 1979.DOI
  97. [97]B. Efron. Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7(1):1–26, 1979. doi: 10.1214/aos/1176344552.DOI
  98. [98]Mark D. Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, et al. The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3(1):160018, 2016. doi: 10.1038/sdata.2016.18.DOI
  99. [99]David R. Hunter. MM algorithms for generalized Bradley-Terry models. The Annals of Statistics, 32(1):384–406, 2004. doi: 10.1214/aos/1079120141.DOI
  100. [100]Ralph Allan Bradley and Milton E. Terry. Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika, 39(3-4):324–345, 1952. doi: 10.1093/biomet/39.3-4.324.DOI
  101. [101]Gian-Carlo Rota. On the foundations of combinatorial theory I. Theory of Möbius functions. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 2(4):340–368, 1964. doi: 10.1007/BF00531932.DOI
  102. [102]Michel Grabisch and Marc Roubens. An axiomatic approach to the concept of interaction among players in cooperative games. International Journal of Game Theory, 28(4):547–565, 1999. doi: 10.1007/s001820050125.
  103. [103]A. van den Nouweland, P. Borm, and S. Tijs. Allocation rules for hypergraph communication situations. International Journal of Game Theory, 20(3):255–268, 1992. doi: 10.1007/BF01253780.DOI
  104. [104]E. Winter. A value for cooperative games with levels structure of cooperation. International Journal of Game Theory, 18(2):227–240, 1989. doi: 10.1007/bf01268161.DOI
  105. [105]Linton C. Freeman. A set of measures of centrality based on betweenness. Sociometry, 40(1):35, 1977. doi: 10.2307/3033543.DOI
  106. [106]Eric Bonabeau, Marco Dorigo, and Guy Theraulaz. Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, 1999. ISBN 9780197561485. doi: 10.1093/oso/9780195131581.001.0001.DOI
  107. [107]Reuven Rubinstein. The cross-entropy method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability, 1(2):127–190, 1999. doi: 10.1023/A:1010091220143.DOI
  108. [108]Michael L. Tushman. Special boundary roles in the innovation process. Administrative Science Quarterly, 22(4):587–605, 1977. doi: 10.2307/2392402.DOI
  109. [109]M. J. Albizuri, J. Aurrecoechea, and J. M. Zarzuelo. Configuration values: Extensions of the coalitional Owen value. Games and Economic Behavior, 57(1):1–17, 2006. doi: 10.1016/j.geb.2005.08.016.DOI
  110. [110]G. Chalkiadakis, E. Elkind, E. Markakis, M. Polukarov, and N. R. Jennings. Cooperative games with overlapping coalitions. Journal of Artificial Intelligence Research, 39:179–216, 2010. doi: 10.1613/jair.3075.DOI
  111. [111]E. Kalai and D. Samet. On weighted Shapley values. International Journal of Game Theory, 16(3):205–222, 1987. doi: 10.1007/BF01756292.DOI
  112. [112]Suzana G. Fries and Tatjana Jantzen. Compilation of CALPHAD formation enthalpy data. Thermochimica Acta, 314(1–2):23–33, 1998. doi: 10.1016/S0040-6031(97)00478-4.DOI
  113. [113]Hidetaka Kasai, Lirong Song, Henrik Lyder Andersen, Hao Yin, and Bo Brummerstedt Iversen. Multi-temperature structure of thermoelectric Mg₂Si and Mg₂Sn. Acta Crystallographica Section B, 73(6):1158–1163, 2017. doi: 10.1107/S2052520617014044.DOI
  114. [114]Joachim Paier, Martijn Marsman, and Georg Kresse. Why does the B3LYP hybrid functional fail for metals? The Journal of Chemical Physics, 127(2):024103, 2007. doi: 10.1063/1.2747249.DOI
  115. [115]Alessandro Stroppa and Georg Kresse. The shortcomings of semi-local and hybrid functionals: what we can learn from surface science studies. New Journal of Physics, 10(6):063020, 2008. doi: 10.1088/1367-2630/10/6/063020.DOI
  116. [116]H. P. Young. Monotonic solutions of cooperative games. International Journal of Game Theory, 14(2):65–72, 1985. doi: 10.1007/BF01769885.DOI
  117. [117]Atsushi Togo, Laurent Chaput, Terumasa Tadano, and Isao Tanaka. Implementation strategies in phonopy and phono3py. Journal of Physics: Condensed Matter, 35(35):353001, 2023. doi: 10.1088/1361-648X/acd831.DOI
  118. [118]G. A. Edwards, K. Stiller, G. L. Dunlop, and M. J. Couper. The precipitation sequence in Al–Mg–Si alloys. Acta Materialia, 46(11):3893–3904, 1998. doi: 10.1016/S1359-6454(98)00059-7.DOI
  119. [119]H. W. Zandbergen, S. J. Andersen, and J. Jansen. Structure determination of Mg₅Si₆ particles in Al by dynamic electron diffraction studies. Science, 277(5330):1221–1225, 1997. doi: 10.1126/science.277.5330.1221.DOI
  120. [120]R. Vissers, M. A. van Huis, J. Jansen, H. W. Zandbergen, C. D. Marioara, and S. J. Andersen. The crystal structure of the β′ phase in Al–Mg–Si alloys. Acta Materialia, 55(11):3815–3823, 2007. doi: 10.1016/j.actamat.2007.02.032.DOI
  121. [121]S. J. Andersen, C. D. Marioara, R. Vissers, A. Frøseth, and H. W. Zandbergen. The structural relation between precipitates in Al–Mg–Si alloys, the Al-matrix and diamond silicon, with emphasis on the trigonal phase U₁-MgAl₂Si₂. Materials Science and Engineering: A, 444 (1–2):157–169, 2007. doi: 10.1016/j.msea.2006.08.084.DOI

Paper details

vlnc:W_w7kaxtk4
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents