WorksComputer Science
Goal Revelation as a Bridge Between Information-Theoretic Empowerment and Capability-Based Human Power
Abstract
Power matters for AI safety. Optimal agents tend to acquire power across a wide range of objectives (instrumental convergence). As we embed AI agents into socio-technical systems, humans risk gradual disempowerment. Both concerns call for a formal, quantitative measure of power. Two such measures in the context of Markov games are information-theoretic empowerment, the capacity of the channel from an agent's actions to future states, and capability-based individual power, a measure of goal attainment built from the probabilities that a human would attain each goal in a family of candidates were they pursuing it. We show where the two meet. From individual power, Fano's inequality yields a floor . This floor is a lower bound on what we term goal revelation, the mutual information between the human's latent goal and the realised trajectory. Empowerment caps revelation from above, so that . Each quantity can be large while the one below it stays at zero, highlighting the following failure modes. An assistant maximising empowerment can prefer an interface offering broad influence over goal-irrelevant degrees of freedom to one that lets the human attain every goal they might hold. An assistant maximising revelation cannot tell a working interface from one rewired so that every goal yields the wrong outcome, since relabelling goals leaves mutual information unchanged while zeroing every attainment probability. Positive guarantees therefore come only from the attainment end of the chain. A satisfaction floor, computable from two statistics of interaction logs, offers a way to monitor the power users retain in a deployed system, and simulations in random environments show the three quantities are correlated but far from interchangeable.
Introduction
Power matters for AI safety on both sides of the human–AI relationship. Optimal agents tend to acquire power across a wide range of objectives [Tur+21], so recognising an AI that is amassing power requires the ability to measure power. And as AI agents are embedded into socio-technical systems, human control can erode through many individually unremarkable transfers [19], so avoiding gradual disempowerment requires tracking the power humans retain. Both concerns call for a formal, quantitative measure. Here we concern ourselves with two existing such measures. Empowerment [18] is the capacity of the channel from an agent’s actions to future states: a measure, in bits, of how much influence the environment affords them. Individual power in the capability sense of Heitzig and Potham [14] is a measure of goal attainment: for each goal in a family of candidates, the (discounted) probability that an agent would attain it were they pursuing it, aggregated over the family. Whilst goals are a central object in the capability-based measure, they are completely absent in information-theoretic one.
This paper connects the two through the mutual information between the goal of the agent of interest, treated as an unobserved random variable, and the state trajectory produced from the current state . We call it goal revelation, as it measures how much an observer of the trajectory could learn about which goal was being pursued. As we will show, it does not measure influence, since a agent whose goal-relevant options have all been removed cannot reveal information about the goal, however many other degrees of freedom they command. It also does not measure attainment, since a trajectory can identify the pursued goal while the goal goes unmet.
Our main result (Theorem 4.1) is a chain of bounds connecting the three quantities. From the individual power (Equation (7)) proposed in [14], Fano’s inequality yields a floor (4.2) on revelation. Revelation is capped in turn by the directed information that the realised behaviour generates, by Massey’s converse for channels with feedback [23], because the goal reaches the trajectory only through the actions. Finally, the directed information is capped by empowerment , because the realised behaviour is one feasible input among the ones the capacity maximises over. Together,
The floor is a strictly increasing function of that meets the maximum possible revelation at the maximum.
The converse implications all fail in ways that help us understand the problem. Capacity does not imply revelation: we exhibit environments in which the interface maximising offers unbounded capacity over degrees of freedom worthless to every candidate goal, while an alternative interface with maximal revelation scores arbitrarily worse (Proposition 2). Revelation does not imply attainment: relabelling the goal–outcome interface by any permutation leaves unchanged while zeroing every attainment probability (Proposition 3). Revelation is a property of the ensemble of goal-conditioned behaviours, indifferent to whether the goal each trajectory identifies is the goal it fulfils. Attainment capability restores the reference to fulfilment. Positive guarantees therefore come only from the attainment end of the chain.
Three further results sharpen the picture. We can decompose the induced behaviour class (Proposition 1), splitting the observer’s residual uncertainty into goal–policy aliasing, when distinct goals induce the same behaviour, and policy–trajectory ambiguity, when distinct behaviours produce indistinguishable outcomes. Revelation is thereby capped by the entropy of the behaviour ensemble, rather than the entropy of the goal ensemble. A satisfaction floor (Theorem 6.1) extends the lower bound to families of overlapping goals. It is the divergence between the rate at which pursuit satisfies its goal and the rate at which chance would, two statistics estimable from interaction logs. Further, simulations in random one-step environments (§ 7) measuring how the three quantities co-vary in generic settings show that the ordering holds pointwise everywhere, and even in an ensemble that excludes both failure mechanisms by construction, the scores are correlated but far from interchangeable, agreeing on the best state about as often as a coin flip.
We make no claim that any of the three quantities is, by itself, a safe objective for an AI system. Here the aim is the relationship between the quantities, stated plainly and proved. In a companion manuscript we study what each quantity incentivises when maximised, including how an attainment capability objective can game the choice of its own goal family.
Setup
For simplicity, our setup consists of a robot that wants to empower a single human . We model the interaction as a fully observable, discrete-time Markov game (a multi-agent MDP). is the finite state space, with current state . At each step the agents choose actions and simultaneously, and the state moves by the kernel . Over a horizon we write for the human’s action sequence viewed as a tuple of random variables.
The human holds a goal from a finite set of candidates, unobserved by the robot. denotes the goal viewed as a random variable. Section 3.2 provides a precise definition of a goal in terms of trajectory properties. Writing for the probability distributions over a finite set , the human acts by a goal-conditioned policy with , which may also condition on the history of past states and own past actions where noted. The robot, with no access to , acts by a state-conditioned Markov policy . We write for the joint policy when the human pursues .
A trajectory is the random state sequence generated by rolling out a joint policy from ; every law written is the law of this rollout. All per-interaction quantities are evaluated over a finite window of steps, making the trajectory space . An environment whose interaction ends early, in an absorbing state, is read as padded to the full window by repetition of that state. Interaction is segmented into such windows, and the goal is fixed within a window. The distribution the goal is drawn from, and its interpretation, are discussed in Assumption A1, § 3.3. Throughout, logarithms are base 2 and information is measured in bits. is entropy, mutual information (directed where marked with an arrow, introduced in § 3.1), and the indicator function.
Three Measures
Empowerment: Potential Influence
Empowerment [18] is the capacity of the channel from an agent’s actions to future states. Klyubin et al. defined it as the -step open-loop capacity between an action sequence and the endpoint state . Since the human of § 2 observes the state as they act, we use the closed-loop form on the full trajectory (in the spirit of Tiomkin et al. [TSP25]) defining the capacity in Massey’s directed information [23],
The maximisation ranges over all history-dependent behaviour policies for the human, defined as causally conditioned input distributions , in which each action may depend on everything realised so far but not subsequent states. This maximiser is a device for scoring the state , not a model of what the human will do. In particular, it is not the goal-conditioned policy (and no goal variable appears in Equation (1)). The channel being maximised against is the environment with the robot’s action marginalised out,
extended over the window as a controlled Markov chain. The feasible set of input distributions is a compact convex polytope and directed information is continuous on it, so the maximum is attained. We adopt the following two conventions, which will matter for comparisons later on. is an unnormalised -use capacity, in bits per window rather than per step, in order to make it comparable with the goal entropy . The closed-loop form dominates the open-loop and endpoint-state forms: open-loop action sequences are among the policies maximised over, for them directed information equals mutual information [23], and the endpoint state is part of the trajectory. It is therefore the most generous choice of “influence afforded” quantity1. As with any capacity, the operational interpretation is a coding statement over repeated uses (in the limit). In the deterministic environments of our worked instances it reduces to a zero-error count, with equal to of the number of distinguishable trajectories2.
Capability: Attainment
Heitzig and Potham [14] measure power by attainment instead. For a state and a candidate goal , the question is counterfactual: how capable of attaining is the human, were they pursuing ? In their setting a goal is a proposition about the world state, represented by the set of states in which it holds. The game form is finite and acyclic with a terminal set . The human’s behaviour is a given goal-conditioned policy that is the robot’s model of the human, rather than an optimality assumption. The capability is determined by six axioms on how attainment may depend on the trajectory law, up to a monotone transform, yielding the truncated Bellman recursion
with discount , as the simplest admissible choice. This is the discounted probability that pursuit of reaches it. We adopt the recursion as it stands. It is evaluated under the goal-conditioned policy of § 2, the robot’s model of the human3. We depart from their setting in two places. They impose coverage, meaning every state lies in some goal, so the current state always carries one attained goal and individual power below is non-negative. We adopt the opposite, i.e. no goal is certified at time zero (Assumption A2), since a window that decodes before anyone acts is trivial. The goal family need not cover every trajectory, since a window may end in , or run out, with no goal satisfied. Every capability can then vanish and diverge to , which the floor of Theorem 4.1 tolerates because it clamps at zero. We also admit the endpoint , since the finite window needs no discount for the recursion to be well posed. The capability is then the plain probability of reaching before , the setting of our worked instances.
“Eat an apple after finishing the coffee” and “never enter the room” constrain the path rather than a single state, so neither is a set of states in the state space as modelled. Both become sets of states once a state records the history that reached it, and the recursion above then applies unchanged4. We work with the trajectory property directly. It gives the certificate time an explicit form, and it keeps goals in the same space as the other two measures.
Definition 1 (Trajectory goals and certificate times). A goal is identified with a nonempty satisfaction set , such that means “ satisfies ”. The certificate time of is the time index
where collects the trajectories sharing the prefix : the first step at which the prefix alone guarantees satisfaction.
A prefix with is a good prefix in the sense of Kupferman and Vardi [20]. The definition also sorts goals along the safety–co-safety axis [24]: reaching goals are certified at first entry, while “never enter the room” is certified only at the window’s end, . Capability generalises as the expected discounted certificate,
which is the satisfaction probability when , and which additionally accounts for how soon satisfaction is guaranteed when . For the state-set goal “reach before ”, the certificate time is the first entry into and Equation (5) reduces to the recursion (3).
A single trajectory can visit several target sets in sequence. From pairwise-disjoint target sets , the first-outcome goal of has satisfaction set
whose members are the trajectories in which is the first outcome realised. These satisfaction sets are pairwise disjoint by construction. First-outcome goals are an analysis device, not a claim that humans care about outcome order. Disjointness makes goal identity readable off a trajectory (§ 4). Under this framing, attainment via a detour through another target counts as failure, which loosens the bounds. § 6 provides an alternative framing for instances where goal regions interleave. Equation (5) evaluates first-outcome goals directly, since its certificate time is the first entry into , before any other target and any state of . For such a goal, is the discounted probability that pursuing makes it the window’s first outcome.
Individual power aggregates the capabilities over the candidate goal family with an exponent , without asking which goal the human holds. Heitzig and Potham derive its form from axioms on the aggregate, among them neutrality to goal labels, invariance to adding unreachable goals, a preference for reliable over uncertain attainment, and additivity over independent games:
Their reliability axiom forces . We allow any , which Theorem 4.1 covers. Small favours breadth (many moderate capabilities), large favours peak capability. Each summand conditions the human’s policy on a different hypothetical goal while the robot’s policy stays fixed and goal-agnostic. The aggregate has a Bayesian reading that will pair with the observer of the next subsection: taking as the likelihood of an abstract success event under a uniform prior over goals, is the log-evidence of the model in which the human is observed to succeed, and the associated posterior over goals is Individual power thus scores by how credible success is from there, one hypothetical goal at a time.
Goal Revelation: Identifiability
The third quantity needs the goal as a random variable with the following assumption.
A1. Goal prior: The goal is fixed for the window and unobserved. We model it as drawn before the window begins from a reference prior over , uniform unless stated otherwise, . The prior belongs to whoever would infer the goal from the trajectory, not to whoever scores the state. It captures a designer’s or observer’s ignorance of which goal this human holds, or an empirical frequency over a population of interaction windows where one exists (as in the deployment logs of § 6). It is fixed before interaction and does not depend on the robot’s policy. Individual power (7), the score the robot itself computes, carries no prior: the robot knows the family but holds no belief over which member is live. Every information quantity involving is relative to the prior.
Definition 2 (Goal revelation). Given policies and the prior of Assumption A1, goal revelation at is the mutual information between goal and trajectory,
We write , and likewise and , when the policies are fixed and understood. Revelation is what an observer of the interaction could learn: watching the trajectory and updating the prior to the posterior , they gain bits about the pursued goal on average. Equivalently, by (8), revelation is the prior entropy minus the equivocation , the uncertainty that remains after the trajectory has been seen. It is capped by the ceiling , however rich the environment.
Revelation, as defined, has three important properties.
It is zero when behaviour is not goal-conditioned. Under a policy that ignores , the joint law factorises and . Acting differently for different goals is the only channel through which the goal reaches the world.
It is a property of the ensemble of goal-conditioned behaviours (rather than of any single window). If each goal induces a deterministic policy with a distinct outcome, then every window is deterministic given the goal and is maximal purely because the trajectory identifies which policy the ensemble selected.
It is indifferent to attainment. The definition does not consider whether satisfies the goal that produced it, only whether trajectories discern goals, which is why we call it revelation and not control. § 5 provides a theorem concerning this issue.
Prior work has optimised the same functional with the latent variable owned by the agent itself, as skill discovery. In § 8 we discuss how, in the skill discovery framing, relabelling the latent is a symmetry rather than a failure.
The Bridge
The first-outcome construction of § 3.2 helps us construct a further structural assumption needed for decoding.
A2. Disjoint goals: The satisfaction sets are pairwise disjoint (no trajectory satisfies two goals). The family need not be exhaustive (a trajectory may not satisfy any goal). None of the goals are certified at time zero from the initial state ( for every ), which excludes windows that decode trivially before anyone acts.
Theorem 4.1 (Capacity, revelation, attainment). Fix , policies with Markov, the prior of Assumption A1, and a horizon . Then
where the middle quantity is the directed information generated by the human’s realised, goal-marginalised behaviour, and the floor, which additionally requires the uniform prior (A1), disjoint goals (A2), and capabilities per Equation (5) with any , , is
with and the binary entropy function.
The full proof is in Appendix § 13. The floor uses a decoding argument: an observer who watches the trajectory guesses the goal by satisfaction, if and only if , single-valued by disjointness and defaulting to a fixed on trajectories that do not satisfy a goal. Since and on , the average correct-guess probability is at least and we obtain the resulting error bound (10) from Fano’s inequality. The clamp at the random-guessing rate prevents the bound from asserting a positive floor even when every capability is zero (, ). The middle inequality is Massey’s converse for channels with feedback [23]: the kernel (2) does not consider the goal, which influences the trajectory only through the actions. By the chain rule,
whose second term is nonnegative. The undirected chain is not Markov under feedback, because actions are common effects of the goal and past states, so the plain data-processing inequality is unavailable and is replaced by the directed form (Appendix § 13 gives a two-line counterexample). The upper inequality holds because the human’s realised behaviour, with the goal marginalised out, is itself a causally conditioned input distribution, so it competes in the maximisation defining .
Lemma 4.2 (Monotone floor). is a strictly increasing function of on , and as .
Maximising individual power therefore maximises a guaranteed floor on revelation. In the attainable-capability limit, the floor meets the ceiling, i.e., revelation is forced to its maximum wherever the environment permits perfect capability. This is the sense in which attainment implies revelation. The bound applies only in this direction (§ 5 shows how both converse directions fail). Capabilities are evaluated under whatever goal-conditioned policy is in play, optimal or not, so suboptimal pursuit and environmental noise lower rather than break the bound.
Each rung of (9) answers a different question:
| quantity | question it answers | goal variable |
| , | how attainable the candidate goals are | counterfactual |
| how much outcomes identify the pursued goal | latent | |
| how much influence-information pursuit does generate | marginalised | |
| how much influence could be exercised | none |
Table 1.
None of the above is “the amount of control the human exerted”. High asserts that outcomes distinguish goals, not that they are achieved. And the rung that looks most like control, the directed information, has its test in the deterministic pursuer of Remark 1. Given their goal, nothing in their actions is left to chance, yet they generate no directed information at all. The quantity counts variation in the actions that reaches the trajectory, not the actions themselves.
Two gaps. By (11), the distance from capacity to revelation splits as
The first term is the capacity unachieved by the realised ensemble. The capacity-achieving input is a randomisation device, rather than a rational goal pursuit policy, and a goal-pursuer needs at most bits of the channel however large its capacity, meaning that the capacity margin is not a deficiency of the human’s behaviour. The margin measures how far the environment’s influence budget exceeds what goal-conditioned behaviour uses. Section 5 shows a maximiser of chases this margin. The second term is directed information the actions generate that the goal does not explain, i.e. randomness in pursuit itself such as due to exploration, noise, mixed strategies. It vanishes for deterministic pursuit.
Remark 1 (Deterministic pursuit). If each is deterministic, then given the actions are a function of the realised states, so and . In this case, every bit of influence-information the interaction generates is a revealed-goal bit, and the whole distance to capacity is margin. Rational pursuit tends to deterministic policies, so this is the typical regime, and it is why maximising directed information and pursuing a goal are opposite activities: the first requires randomisation which the second avoids in the limit.
Aliasing and ambiguity. There are two reasons that can make revelation fall short of the ceiling: distinct goals may induce the same behaviour, and distinct behaviours may produce indistinguishable trajectories. Both are undistinguished by , but they separate cleanly through the behaviour the goal induces.
Definition 3 (Behaviour class). Fix and , and let be the set of histories with positive probability from under some . Say when for every , and let be the behaviour class: the goal equivalence class that removes those differences in induced policy that are not shown by reachable histories.
A goal reaches the trajectory only through the policy it induces, so the trajectory reveals no more and no less of the goal than of its class, and equals . The class provides a split of the equivocation into its two causes.
Proposition 1 (Aliasing–ambiguity decomposition). Under Assumption A1, is a Markov chain, and
In particular : revelation is capped by the entropy of the behaviour ensemble, not of the goal ensemble.
If a hundred goals induce two observable behaviours, interaction can reveal at most one bit, however deterministic and however well executed. In Equation (12), aliasing is a refinement of the equivocation. Under deterministic pursuit, where the goal-blind term vanishes, its effect appears entirely as capacity margin, since fewer distinct behaviours generate less influence-information. Both terms of (4.5) are computed exactly in the experiment of § 7.
Both Converses Fail
In the chain (9), each quantity can be large while the one below it stays at zero. This section makes the two failure modes behind that concrete: capacity without revelation, and revelation without attainment.
Capacity Without Revelation
Human goals are few and coarse relative to the environments that realise them. For instance, we care whether the room is clean, not about the position of each dust particle. This means that in any rich environment, while the entropy of the action-sequence variable, and with it the bound (1) on , grows linearly in the horizon. A capacity maximiser is therefore rewarded for influence that is not used by any candidate goal. The separation below realises this as an argmax reversal between two robot policies.
Proposition 2 (Capacity without revelation). For every , , and horizon there is a fully observable, deterministic environment with initial state , a goal family of first-outcome goals satisfying
Assumptions A1–A2 with , and two robot policies5 (the prior uniform), (a board of free binary toggles responds to the human; goal transitions are blocked) and (goal transitions respond; toggles are frozen), such that when the human pursues their goal:
while and every capability is 0;
while , its ceiling, and every capability is 1.
Whenever , , so the capacity score prefers , by a margin unbounded in and , while its preferred policy leaves revelation, and every capability, at zero.
| robot policy at | ||
| (toggle board) | bits | 0 |
| (assistance) | bits | bits |
Table 1 (PDF p. 8). Proposition 2 at goals, toggles, steps: capacity prefers the board by bits; under that policy the trajectory carries nothing about the goal.
Table 1 shows an instance of this proposition. The board capacity, deterministic and reversible, is worthless to every goal in , but the capacity score does not account for it. This does not depend on modelling the human as a pure goal-pursuer, since a human who repurposed the board to signal their goal could push Rev no higher than its ceiling bits, which the board capacity dwarfs. On this instance a deterministic pursuer generates no goal-blind influence, and under generates no influence-information at all, so the gap decomposition (12) is all margin.
Revelation Without Attainment
Under the uniform prior, revelation is a functional of the family of goal-conditioned trajectory laws , invariant under relabelling which law goes with which goal. Attainment, in contrast, is a property of the goal–law pairing, asking whether the law paired with reaches ’s satisfaction set.
Proposition 3 (Revelation without attainment). Fix , policies inducing laws and capabilities . For a permutation of , call a configuration -scrambled if it induces the laws for every : the human pursues through the same policy, and the interface delivers what pursuing would have delivered. Under Assumptions A1–A2 with the uniform prior:
Rev is invariant: ;
if for every (i.e., under each original law, pursuit never attains a goal other than the one pursued), then the capability of under the scrambled law, , equals when and is 0 otherwise. A fixed-point-free zeros every capability, sending and .
Consequently, the floor of Theorem 4.1 has no converse: no function of , or of any statistic invariant under goal relabelling, bounds capability from below. Configurations exist with identical revelation and capability profiles versus .
A minimal case in Table 2 illustrates how measures identifiability rather than control. Additionally, the satisfaction decoder behind Theorem 4.1 checks for the goal actually pursued, so a scramble sends its correct-guess probability (and with it the floor) to zero, while Rev stays unchanged. The guarantee runs from attainment upward.
| interface at | capability profile | floor | |
| aligned | 0.9 bits | (0.9, 0.9) | 0.531 bits |
| -scrambled (swap) | 0.9 bits | (0, 0) | 0 |
Table 2. Proposition 3 at with the transposition; the aligned interface realises each intended goal with probability and otherwise ends in a goal-independent failure trajectory. Whilst revelation is the same in both cases, the human attains their goal 90% of the time in one and never in the other.
A Floor for Overlapping Goal Families
The Fano floor (10) requires disjoint satisfaction sets (Assumption A2) and parameters that are counterfactual pursuit probabilities. We now derive a second floor without these requirements. It reads the trajectory through one bit (whether it satisfies the goal pursued), and bounds Rev by what that bit alone can certify. The restriction discards timing and route information by design, which makes the bound depend on the interaction only through two scalar statistics: the efficacy , the probability that the pursued goal is satisfied within the window, and the specificity , the expected number of goals a trajectory satisfies, intended or not. The ratio is the probability that a trajectory satisfies a goal by chance, rather than under pursuit.
Theorem 6.1 (Satisfaction floor). Under the uniform prior assumption (A1), for any finite family of trajectory goals, possibly overlapping,
the binary relative entropy between the hit rate under pursuit and the hit rate under chance, with the convention .
Expanding and collecting the terms,
with the binary entropy. Read left to right: the ceiling, less the entropy of the success bit, less the expected log-count of goals consistent with that bit, of them on success and on failure.
We now consider the limit cases for a sanity check. Satisfying only the pursued goal (, ) gives , the ceiling. Behaviour independent of the goal, satisfying goals by accident (), gives . Avoiding the pursued goal while satisfying others () gives , which again shows that revelation is not a desired target in itself. The sign of records which way pursuit departs from chance.
Remark 2 (The same floor as a variational bound). For any decoder , the variational lower bound of Barber and Agakov [1], whose use for intrinsic control is due to Mohamed and Rezende [25], gives
A softmax over the satisfaction indicators with one log-odds weight, optimised over that weight, returns Equation (14) (Appendix § 13). The hard decoder of Theorem 4.1 is its infinite-weight limit for a disjoint goal family. Theorem 6.1 is therefore also the best bound that decoder class can give, where the sign of its optimal weight is the sign of .
The two floors connect through the capabilities. The average capability lower-bounds the efficacy , since discounting each first-outcome charge only subtracts, and substituting for in (14) is conservative when , the region where increases in its first argument. Unlike the counterfactual capabilities, are population statistics of observable interactions: given per-goal satisfaction labels (which for goals expressed as regular properties a monitor automaton computes directly from logged trajectories [Tor+22]) and the pursued goal per window for , both are means of bounded per-window statistics, so finite-log estimates concentrate at standard rates. Equation (14) then gives a computable lower bound on the goal revelation a deployed system’s users retain, providing a quantitative handle on the gradual disempowerment concern [19]. However, the estimate is only as good as the labels, is relative to the family and the window segmentation, and is a measurement, not a training signal.
Numerical Illustration
Section 5 included simple designed environments to illustrate failure modes. This section asks what the chain looks like in generic environments: how much of capacity becomes revelation, how much revelation the floor certifies, and how far the three scores agree about which state is best (with no adversarial aspects).
We focus on a multi-armed bandit as the simplest setting exhibiting the properties above. The goals are the outcomes themselves, a disjoint family by construction. At horizon one a trajectory is a single transition, so everything the three measures see of a state is its induced channel (2). That channel is an stochastic matrix from the human’s actions to outcome states. We therefore identify a state with that matrix, writing for the probability that action leads to outcome , and sample states by drawing each row from a Dirichlet() distribution. Pursuit is rational and deterministic, (ties to the first index), so the capability is . The individual power is taken at , . Capacity is computed exactly as the channel capacity by Blahut–Arimoto with a duality-gap certificate of nats, cross-checked against an independent solver to bits on the first 150 channels. The behaviour-class terms from the joint laws, , is the chosen arm . The canonical configuration is , , , 1,000 independent environments of 50 states each, from a fixed seed6. The simulations highlight three regularities and one caution:
The ordering holds on every one of the 50,000 states. With deterministic pursuit, every bit of realised influence-information is a revealed-goal bit (to , illustrating Remark 1). On average the states carry bits of capacity, of which bits (85.0%) is revelation, of which bits (54.6%) is certified by the floor from .
Illustrating the equivocation split of Proposition 1, out of bits of prior entropy, ambiguity accounts for 1.687 bits on average and aliasing for 0.138. Aliasing is positive in 33.1% of states (two goals sharing an argmax arm) and quantised, equal to 2/5 of a bit whenever one pair collides at . In this ensemble the ceiling binds a third of the time, with most of what trajectories fail to reveal being channel noise rather than behavioural coincidence.
Rank agreement (Table 3): the three scores (, Rev, ) co-vary, with mean within-environment Kendall between 0.58 and 0.75, yet exact argmax agreement is close to a coin flip, 57.8% between and Rev and 41.8% between Rev and . Near-argmax behaviour is tighter, as the state maximising sits on average at the 96.4th percentile of Rev (top decile in 88.2% of environments), and the state maximising Rev at the 98.3rd percentile of . Maximising the floor is a good, not perfect, proxy for maximising revelation in generic environments. The sensitivity sweep shows every agreement falling as the outcome and action spaces grow.
| Mean Kendall (within an env’t) | 0.751 | 0.619 | 0.581 | |
| Exact argmax agreement | 57.8% | 41.8% | ||
| 3 | 4 | 1.0 | 0.863 | 0.693 |
| 5 | 8 | 1.0 | 0.754 | 0.617 |
| 8 | 12 | 1.0 | 0.700 | 0.556 |
| 5 | 8 | 0.2 | 0.809 | 0.715 |
| 5 | 8 | 5.0 | 0.746 | 0.569 |
Table 3. Left: rank agreement across the 50 states of an environment, averaged over 1,000 environments (canonical configuration). Right: sensitivity of the mean to the configuration, 300 environments each. Agreement falls as the spaces grow and rises for spikier channels ().
The caution is that this ensemble is favourable to agreement by construction. Every outcome is a candidate goal, so there are no degrees of freedom worthless to the family. The goal–outcome correspondence is identity, so nothing is scrambled. These are the two mechanisms of § 5, and introducing either one can drive the corresponding agreement to zero. The simulation adds the complement of the separations: even with both mechanisms absent, the quantities are correlated but far from interchangeable, and the scores do not agree on which state is best.
Related Work
Mutual-information objectives for agents sort by three choices: which pair of variables the information couples, who holds the latent, and whether the quantity is maximised or minimised. The chain of § 4 falls under the “goal to trajectory”, “exogenous human goal” latent, and an assistant (rather than the goal’s owner) maximising the channel. The lines below differ from our bounds in at least one of the three choices.
Empowerment. Klyubin et al. [18] introduced Empowerment as -step open-loop channel capacity from actions to a future state, as task-agnostic intrinsic motivation; Salge et al. [SGP14] survey the formal properties. Our closed-loop form follows the directed-information treatment of causal information flow in sequential settings [23 TT17], and the feedback-aware process Empowerment of Tiomkin et al. [TSP25]. Proposition 2 turns the informal worry that capacity is goal-blind into an argmax reversal against a revelation-maximal alternative.
Empowerment as assistance. Pointing the quantity at the human predates the algorithms: Salge and Polani [SP17] propose a language-free stand-in for Asimov’s laws that combines three Empowerment heuristics, one of which keeps the human’s Empowerment high. Du et al. [6] propose human Empowerment as an auxiliary reward term for an assistant, and implement not the channel capacity but a proxy based on the diversity of reachable final states. Franzmeyer et al. [10] cast altruism as preserving another agent’s choice, with reachable-state diversity again standing in. Myers et al. [26] scale the commitment with contrastive successor representations, maximising the influence of the human’s actions on future states without inferring rewards. This line of work keeps the human side goal free, so Proposition 2 applies where controllable degrees of freedom outnumber candidate goals, and Proposition 3 applies to any surrogate invariant under relabelling of the goal–outcome correspondence.
Skill discovery. In this family of prior art, the latent is owned by the agent itself. Variational Intrinsic Control [12] maximises over options with the option prior optimised, read by its authors as Empowerment over a learned option channel. Skew-Fit skews a goal prior toward rarely visited states [27]. DADS [Sha+20] maximises a one-step with the prior fixed, DIAYN [8] a per-state discriminator variant, and Choi et al. [Cho+21] subsume goal-conditioned reinforcement learning under variational Empowerment. Eysenbach et al. [7] prove that mutual-information skill discovery does not learn skills optimal for every possible reward function, which is congruent with § 5. They also show that, under an assumed adaptation procedure, the learned skill distribution is the initialisation that minimises regret against adversarially chosen rewards. METRA replaces the objective outright after diagnosing that it misaligns with useful behavioural spread [28]. For a self-assigned latent, relabelling is a symmetry and is not an issue. When applied to another agent’s goal variable, whose labels carry commitments the assistant is meant to serve, the same invariance is Proposition 3. The variational decoders of that literature are trained to tighten the bound, whereas the decoder of § 6 is restricted to satisfaction indicators so that what it certifies is estimable.
Intent legibility. On the actor’s side, legible motion [4] maximises an observer’s posterior on the actor’s actual goal from a trajectory prefix. Strouse et al. [Str+18] add or subtract the goal information in actions, , or in states, , to share or hide intent. The same information is treated as a cost to be minimised in van Dijk and Polani’s relevant goal information (the minimal goal information a policy must consult, whose spikes mark natural subgoals) [5], InfoBot’s bottleneck for transferable defaults [11], and opacity enforcement (minimising what observations leak about a secret) [SBF24]. These works either anchor to the true goal, which immunises them against relabelling, or treat the actor’s own latent, for which relabelling is again a symmetry. The satisfaction decoder behind Theorem 4.1 is anchored in Dragan’s sense, checking the trajectory against the goal actually pursued. A relabelling therefore leaves the aggregate mutual information unchanged while sending the decoder’s success rate, and with it the floor, to zero.
Interface optimisation. Reddy et al. [29] point out that randomly permuting the keybindings of an assistive interface leaves the one-step information from the user’s commands to the next state unchanged. This is a one-step instance of the invariance in Proposition 3. They offer a behavioural solution, adding to the objective on the assumption that co-adapting users issue noisier commands through unintuitive interfaces. Capability anchoring is a structural alternative that collapses under permutations with no assumption about user noise.
Multi-agent information objectives. Multi-agent reinforcement learning has a dense family of information objectives, with latents held by the team. MAVEN couples one shared latent to the joint trajectory for committed exploration [22], and role and diversity methods couple agent-specific latents to that agent’s own trajectory or observations [Wan+20; Li+21; JL21], extended hierarchically [Yan+23]. From offline data, Chen et al. [3] learn hierarchical skill codes with an auto-encoder rather than an information objective. Social influence puts the information between actions instead, rewarding [15], routed through a shared coordination latent by Kim et al. [17]. On the perception side, IMAS² couples a selected subset of agents’ observations to the joint trajectory [Shi+26]. Multi-human assistance exists in goal-free or information-free forms: coupled Empowerment maximisation combines the companion’s and the player’s capacities with a transfer capacity between them [13], multi-principal assistance games aggregate utilities by social choice with no information quantity [9], and a recent preprint reports that an assistant maximising one human’s Empowerment measurably reduces a bystander’s in 27–56% of procedurally generated environments [Yan+26]. All of these are per-agent or goal-free constructions, so neither mechanism of Section 5 is expressible in them, and a tuple of distinct private goals read through one shared trajectory has no occupant. The single-human chain proved here is the base case for that extension.
Capability-based power. Heitzig and Potham [14] construct individual power axiomatically from goal-attainment capabilities and argue for soft maximisation of its long-term aggregate as the robot’s objective, goal-agnostic by design. We adopt their capability as the attainment rung of the chain and prove, in information terms, that it provides a floor under revelation. Turner et al. [Tur+21] prove optimal policies tend to seek power for the agent. In contrast, the chain (9) is a formal account of the complementary quantity, the power the human retains, and Kulveit et al. [19] motivate measuring its erosion, for which the estimable floor of Section 6 is one candidate instrument.
Discussion
Theorem 4.1 orders individual power, goal revelation and empowerment in a chain in which attainment implies revelation, and Section 5 shows that both converses fail. The bridge licenses a floor statement, meaning that maximising individual power maximises a guaranteed lower bound on goal revelation (Lemma 4.2), a bound that meets the ceiling where the environment permits perfect capability. It does not license “maximising maximises ”, which is false in general. The two floors admit a division of labour. The Fano floor is a design-time statement in the objective’s own variable, with counterfactual parameters that a log cannot estimate, whilst the satisfaction floor is a deployment-time measurement in statistics that logs do estimate. Rewarding a system on its own measurement corrupts the measurement, so the satisfaction floor is a monitor rather than a training signal.
Limitations. All quantities live on a finite window: goals are trajectory properties certified within steps, which covers reach, pattern, and within-window maintenance goals but not unbounded safety properties, whose attainment lies outside the finitely-certifiable class [YLC22]. Full observability is assumed throughout: human, robot, and the decoding observer all see the state, and partial observability changes both the capability recursion and the decoding argument. The Fano route needs disjoint satisfaction sets, manufactured by the first-outcome construction, with the satisfaction floor as the instrument for genuinely overlapping families. Every information quantity is relative to the reference prior and the candidate family: a different gives different numbers, and we did not consider goals outside the family. Capability is counterfactual, conditioning the human’s policy on goals they may not hold, so a robot that computes must model that policy. Theorem 4.1 then bounds the revelation of the modelled human, and carries over to the actual human only as far as the model is accurate. The human holds one goal per window, drawn afresh across windows. Graded mixtures of goals within a window are outside the present decoding argument, and the soft decoder of Section 6 is the natural starting point for that extension.
Conclusion
We set out to relate three measures of a human’s power at a state, i.e. empowerment, goal revelation, and individual power. Theorem 4.1 orders them in a chain. Fano’s inequality turns individual power into a floor under revelation, so a robot that maximises individual power maximises a guaranteed lower bound on how much the interaction reveals about the goal pursued. Neither of the other two quantities provides such a guarantee. The capacity score includes influence over degrees of freedom that are not used by any candidate goal, and can prefer an interface on which every candidate goal is unattainable (Proposition 2). Revelation measures whether trajectories discern goals rather than whether goals are attained, so a relabelling that zeroes every capability leaves it unchanged (Proposition 3). This means that positive guarantees about the power a human retains come only from the attainment end of the chain. We make no claim that any of the three measures is a safe objective on its own.
The results provide four things beyond the chain itself.
The assumption that maximising empowerment preserves the human’s ability to attain their goals does not hold in general. Proposition 2 gives environments in which the empowerment-maximising interface leaves every candidate goal unattainable, by a margin that grows without bound in the number of goal-irrelevant degrees of freedom.
Proposition 1 separates the two reasons a trajectory fails to reveal intent, i.e. goal–policy aliasing, when distinct goals induce the same behaviour, and policy–trajectory ambiguity, when distinct behaviours produce indistinguishable trajectories. This shows that revelation is capped by the entropy of the behaviour ensemble rather than that of the goal ensemble.
The satisfaction floor (Theorem 6.1) requires two bounded population statistics, the efficacy and the specificity , both of which are means of per-window quantities that interaction logs record. This provides a computable lower bound on the goal revelation that the users of a deployed system retain, i.e. one quantitative handle on gradual disempowerment, which does not require the counterfactual capabilities that a log cannot estimate.
Skill-discovery objectives optimise the same mutual information with the latent held by the agent itself, for which relabelling is a symmetry. Pointed at a human’s goals, the same objective inherits the invariance of Proposition 3, so an assistant optimising it cannot distinguish a working interface from a rewired one.
Three directions follow from the limitations in § 9. The capability objective inherits the choice of its goal family, and a robot maximising it can game that choice. We study this failure and an architectural repair in a companion manuscript. The results assume full observability and a finite interaction window, and relaxing either changes both the capability recursion and the decoding argument. Finally, the floors are stated for the robot’s model of the human’s policy, so a model refined from interaction, with a bound on its error, would carry the guarantees over to the actual human.
Code availability
The code for § 7 and a script that recomputes the numbers quoted in this paper are available at https://github.com/RuizSerra/revealing-goals.
Use of AI tools
We used large language models throughout this work: Claude (Fable 5, Fable 5.1, Opus 5 and Opus 5.5; Anthropic) and Gemini Pro 3.8 (Google). The work began with an extended, interactive exchange between one of the authors and Claude, in which the research questions, framing and candidate results were developed jointly over many iterations. In this phase, before any drafting, the author set the direction of the work, deciding which lines of inquiry to pursue and which results to retain, refine or set aside. Claude subsequently drafted the text, carried out the mathematical derivations, and wrote the code and tables, while the author continued to steer the content through successive revisions. The same author then revised the prose of the whole main text for readability. Fresh Claude instances and Gemini were used to red-team the mathematics, prose and claims, and to check that the cited works support the statements attributed to them. The authors take full responsibility for the paper.
Acknowledgements
This work was carried out as part of the PIBBSS Fellowship, with support from Principles of Intelligence, a not-for-profit organisation.
Author contributions
Contributions follow the CRediT taxonomy.
JRS: Conceptualisation (lead); formal analysis (lead); funding acquisition; investigation (lead); project administration; software (lead); validation; visualisation; writing – original draft (lead); writing – review and editing.
TG: Conceptualisation; formal analysis; supervision; validation; writing – review and editing.
JH: Conceptualisation; formal analysis; investigation; software; supervision; validation; writing – review and editing.
Proofs
Proof of Theorem 4.1
Upper inequality (feasibility). The human’s realised behaviour with the goal marginalised out is the input distribution
The mixture has memory in the actions even when each per-goal policy is Markov, because past actions carry evidence about which is in play; the conditional is bookkeeping from the marginalisation, not any agent’s belief. Equation (A.1) is causally conditioned (each factor reads only the past), so it lies in the feasible set of the maximisation defining against the same channel, and .
Middle inequality (Massey’s converse). Identify as the message and as the causal channel inputs. The induced kernel (2) is the conditional law of given any history containing and does not read the goal, so
adjoining the message to the input changes nothing. Summing the chain rule over gives . Expanding the left side by the chain rule in the other order, and using (the chain rule for mutual information, having no time index), yields the identity (11); nonnegativity of its second term is the inequality [23].
The detour through directed information is forced. is not a Markov chain under feedback, so the undirected data-processing inequality is unavailable: the action is a common effect of the goal and the realised states, and conditioning on it couples them. A two-line counterexample: let the dynamics be pure noise (states independent of actions) and let the human’s single action report whether the realised first state matches their goal. Then , yet given the action the two are dependent, , so fails.
Lower inequality (Fano floor). Define the satisfaction decoder : output if , and a fixed default if satisfies no goal. By disjointness (Assumption A2) at most one goal is satisfied, so is single-valued and everywhere defined; the default only raises the correct-guess probability for , and no goal is certified at time zero, so no trivial immediate decode occurs. Given , a correct guess is at least as probable as satisfaction, which the capability lower-bounds pointwise, since gives :
Averaging over the uniform prior and using on for ,
so the decoding error obeys . Because is a function of , the data-processing inequality (valid here: is a Markov chain) gives , and Fano’s inequality bounds the latter by with . The function increases up to its peak at the random-guessing rate , where , so
holds in both regimes: for below the peak by monotonicity, since , and above it because the right side equals , which bounds the equivocation always. Substituting into gives (10). Every step uses only Assumptions A1–A2, disjointness, and the pointwise bound, so the floor holds for any disjoint family of trajectory goals under the realised (not optimal) policy.
Proof of Lemma 4.2, and a closed form
is strictly decreasing in , and is strictly increasing on , since if and only if . For the clamp is inactive, so is strictly increasing in . As we get , hence and .
Substituting into and simplifying with gives the closed form, for :
Proof of Remark 1
By induction on : is a deterministic function of , and if is a deterministic function of , then is a deterministic function of . Hence each term of conditions on variables that determine , and vanishes. The claim then follows from the identity (11).
Proof of Proposition 1
First, the goal-conditioned trajectory law depends on only through its class. Let . The law of the history under is a product of kernel factors and policy factors evaluated at realised histories ; by induction on , every history with positive probability under lies in , where the two policies agree, so the laws of all histories, and of , coincide under and . Hence is well defined, , and since is a function of , is a Markov chain.
For the decomposition: gives , and reduces the last term to . Likewise . Subtracting, .
Proof of Proposition 2
Construction. States are pairs with location (0 a hub; each an absorbing room whose target set induces the first-outcome goal ) and board configuration . The initial state is , in no target set. The human’s actions are . Under , toggles and every is inert; under , moves , absorbing thereafter (windows padded per § 2), and every is inert. Pursuit is .
(1) Under the induced channel is deterministic. The distinguishable action classes per step are the classes (every coincides with wait), and distinct class sequences yield distinct trajectories, the class at step being recoverable from . Hence , attained by the uniform open-loop input over class sequences, so . Under pursuit every chosen action is inert, so is one constant trajectory for every : , and no target is reachable, so every capability is 0.
(2) Under the achievable trajectories are “wait steps, then enter room ” for , plus the all-wait trajectory: in total, through a deterministic channel injective on distinguishable inputs, so . Pursuit enters room at with certainty, so every capability is 1 at , and determines : .
Threshold. For , Bernoulli’s inequality gives .
Proof of Proposition 3
(1) Under the uniform prior, scrambling replaces the channel by . The trajectory marginal is unchanged, , and the conditional entropy is a uniform average over the same set of distributions, by the substitution ; hence both terms of are unchanged.
(2) With goal , the scrambled trajectory is distributed as an original trajectory with goal . If , the hypothesis applied to the original law for , with , gives under the scrambled law, and the certificate-time expectation (5) vanishes with it; if the law, and the capability, are unchanged. For fixed-point-free every capability is 0, the sum inside is 0, , and by the clamp.
The final claim takes the configuration of Table 2 and its swap: identical , capability profiles and .
Proof of Theorem 6.1
Revelation is the relative entropy between the joint law of and the product of its marginals, . Relative entropy cannot increase when the same measurable map is applied to both arguments. Apply the map . Under the joint law its image is Bernoulli with parameter . Under the product law, with uniform and independent of , it is Bernoulli with parameter . The relative entropy between two Bernoulli laws is , which is (14). If then every reachable trajectory satisfies no goal, or every goal, so takes the same value and ; otherwise both arguments lie in . Expanding and collecting the terms gives (15).
The decoder route (Remark 2). With the uniform prior, choose the decoder that reads through its satisfaction indicators, a softmax with one log-odds weight :
Since each indicator is 0 or 1, the partition sum is , where counts the goals satisfies. Substituting into (16),
using . Jensen’s inequality on the concave log bounds the last term by , the argument being positive for every finite . Optimising the resulting bound over gives
and substituting back yields (15), the expanded form of (14). The zero-revelation case makes , and exactly when .
References
- [1]David Barber and Felix V. Agakov. “Information Maximization in Noisy Channels: A Variational Approach”. In: Advances in Neural Information Processing Systems. Vol. 16. 2003.
- [3]Jiayu Chen, Tian Lan, and Vaneet Aggarwal. “Variational Offline Multi-Agent Skill Discovery”. In: Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence (IJCAI). 2025, pp. 4833–4841. DOI: 10.24963/ijcai.2025/538.DOI
- [4]Anca D. Dragan, Kenton C. T. Lee, and Siddhartha S. Srinivasa. “Legibility and Predictability of Robot Motion”. In: ACM/IEEE International Conference on Human-Robot Interaction. 2013.
- [5]Sander G. van Dijk and Daniel Polani. “Grounding Subgoals in Information Transitions”. In: IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL). 2011. DOI: 10.1109/ADPRL.2011.5967384.DOI
- [6]Yuqing Du, Stas Tiomkin, Emre Kiciman, Daniel Polani, Pieter Abbeel, and Anca Dragan. “AvE: Assistance via Empowerment”. In: Advances in Neural Information Processing Systems. Vol. 33. 2020, pp. 4560–4571.arxiv.org/abs/2006.14796
- [7]Benjamin Eysenbach, Ruslan Salakhutdinov, and Sergey Levine. “The Information Geometry of Unsupervised Reinforcement Learning”. In: International Conference on Learning Representations. 2022.arxiv.org/abs/2110.02719
- [8]Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. “Diversity is All You Need: Learning Skills without a Reward Function”. In: International Conference on Learning Representations. 2019.arxiv.org/abs/1802.06070
- [9]Arnaud Fickinger, Simon Zhuang, Andrew Critch, Dylan Hadfield-Menell, and Stuart Russell. Multi-Principal Assistance Games: Definition and Collegial Mechanisms. NeurIPS 2020 Cooperative AI Workshop; non-archival. 2020. DOI: 10.48550/arXiv.2012.14536.DOI
- [10]Tim Franzmeyer, Mateusz Malinowski, and João F. Henriques. “Learning Altruistic Behaviours in Reinforcement Learning without External Rewards”. In: International Conference on Learning Representations. 2022.arxiv.org/abs/2107.09598
- [11]Anirudh Goyal, Riashat Islam, Daniel J. Strouse, Zafarali Ahmed, Hugo Larochelle, Matthew M. Botvinick, Yoshua Bengio, and Sergey Levine. “InfoBot: Transfer and Exploration via the Information Bottleneck”. In: International Conference on Learning Representations. 2019.
- [12]Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. Variational Intrinsic Control. Presented in the ICLR 2017 Workshop Track. 2016. DOI: 10.48550/arXiv.1611.07507.DOI
- [13]Christian Guckelsberger, Christoph Salge, and Simon Colton. “Intrinsically Motivated General Companion NPCs via Coupled Empowerment Maximisation”. In: IEEE Conference on Computational Intelligence and Games (CIG). 2016. DOI: 10.1109/CIG.2016.7860406.
- [14]Jobst Heitzig and Ram Potham. A Fair Objective for Human-Empowerment-Preserving AI: Desiderata, Design, and Likely Behavioral Consequences. Version 1. Aug. 8, 2026. DOI: 10.48550/arXiv.2608.08240. URL: https://arxiv.org/abs/2608.08240v1. Pre-published.
- [17]Woojun Kim, Whiyoung Jung, Myungsik Cho, and Youngchul Sung. “A Variational Approach to Mutual Information-Based Coordination for Multi-Agent Reinforcement Learning”. In: International Conference on Autonomous Agents and Multiagent Systems (AAMAS). 2023, pp. 40–48.arxiv.org/abs/2303.00451
- [18]A.S. Klyubin, D. Polani, and C.L. Nehaniv. “Empowerment: A Universal Agent-Centric Measure of Control”. In: 2005 IEEE Congress on Evolutionary Computation. 2005 IEEE Congress on Evolutionary Computation. Vol. 1. Sept. 2005, 128–135 Vol.1. DOI: 10.1109/CEC.2005.1554676.DOI
- [19]Jan Kulveit, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger, and David Duvenaud. “Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development”. In: arXiv preprint arXiv:2501.16946 (2025).arxiv.org/abs/2501.16946
- [20]Orna Kupferman and Moshe Y. Vardi. “Model Checking of Safety Properties”. In: Formal Methods in System Design 19.3 (2001), pp. 291–314. DOI: 10.1023/A:1011254632723.DOI
- [22]Anuj Mahajan, Tabish Rashid, Mikayel Samvelyan, and Shimon Whiteson. “MAVEN: Multi-Agent Variational Exploration”. In: Advances in Neural Information Processing Systems. Vol. 32. 2019.arxiv.org/abs/1910.07483
- [23]James L. Massey. “Causality, feedback and directed information”. In: Proc. Int. Symp. Inf. Theory Applic. (ISITA-90). 1990, pp. 303–305.
- [24]Zohar Manna and Amir Pnueli. “A Hierarchy of Temporal Properties”. In: Proceedings of the Ninth Annual ACM Symposium on Principles of Distributed Computing (PODC). 1990. DOI: 10.1145/93385.93442.
- [25]Shakir Mohamed and Danilo Jimenez Rezende. “Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning”. In: Advances in Neural Information Processing Systems. Vol. 28. 2015.arxiv.org/abs/1509.08731
- [26]Vivek Myers, Evan Ellis, Sergey Levine, Benjamin Eysenbach, and Anca D. Dragan. “Learning to Assist Humans without Inferring Rewards”. In: Advances in Neural Information Processing Systems. Vol. 37. 2024, pp. 71540–71567. DOI: 10.52202/079017-2285.DOI
- [27]Vitchyr H. Pong, Murtaza Dalal, Steven Lin, Ashvin Nair, Shikhar Bahl, and Sergey Levine. “Skew-Fit: State-Covering Self-Supervised Reinforcement Learning”. In: International Conference on Machine Learning. Vol. 119. Proceedings of Machine Learning Research. 2020, pp. 7783–7792.arxiv.org/abs/1903.03698
- [28]Seohong Park, Oleh Rybkin, and Sergey Levine. “METRA: Scalable Unsupervised RL with Metric-Aware Abstraction”. In: International Conference on Learning Representations. 2024.arxiv.org/abs/2310.08887
- [29]Siddharth Reddy, Sergey Levine, and Anca Dragan. “First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximization”. In: Advances in Neural Information Processing Systems. Vol. 35. 2022.arxiv.org/abs/2205.12381