WorksPhysics and Astronomy
feedback-grape: A JAX-based Python Package for GPU-Accelerated GRAPE with Feedback
Abstract
Modern quantum technology applications require accurate control of quantum systems, including a feedback loop in the control process. While open-loop control can be performed using well established gradient methods like GRadient Ascent Pulse Engineering (GRAPE), it was until recently unclear how to combine these with feedback from measurements, which are inherently non-differentiable. Recent work has successfully demonstrated how feedback can be incorporated into GRAPE using machine learning, resolving this issue. Following this progress, we have developed an efficient Python package implementing the method of ``feedback GRAPE''. In this hands-on tutorial, we discuss the package in detail based on concrete examples. Built on the GPU-accelerated JAX platform, our package offers a user-friendly interface inspired by the widely-used QuTiP library, including simplified treatment of multi-component systems.
Copyright attribution to authors.
This work is a submission to SciPost Physics Codebases.
License information to appear upon publication.
Publication information to appear upon publication.
Received Date
Accepted Date
Published Date
Introduction
The extensively developing field of quantum technologies poses ever higher demands on quantum control techniques [1, 2]. In absence of a feedback loop and if a high-quality numerical model of the system dynamics is available, well established gradient methods offer a reliable solution. A paradigmatic example of such open-loop quantum control is the method of GRadient Ascent Pulse Engineering (GRAPE) introduced in 2005 in the context of NMR spectroscopy [3, 4]. In GRAPE, the gradients of a quality measure (usually based on fidelity) with respect to the controlling fields are evaluated. The latter are then tuned towards maximization of the quality measure. These steps are repeated until convergence is achieved.
If feedback is present, however, the overall control process is not differentiable anymore and GRAPE in its standard form is not applicable. One way to circumvent this problem is offered by model-free reinforcement learning [5, 6] as demonstrated in Ref. [7, 8, 9] and experimentally in Ref. [10, 11]. In this approach, no numerical model of the system dynamics is required and the control tuning is performed based directly on the outcomes of the measurements during and/or at the end of the control process. However, this is disadvantageous in case a good quality model is present, since the discarded knowledge could otherwise be used to improve the search for the optimal control parameters. In particular, the system dynamics between the measurements remains deterministic and differentiable, and should be efficiently incorporated in the control process.
Such a combination of GRAPE and feedback has been achieved recently in Ref. [12], where some concepts from model-based reinforcement learning [5, 6] were applied to include feedback, whilst performing gradient ascent through the quantum dynamics. Note that in Ref. [12], the measurements are parametrized and optimized alongside the control fields. The efficacy of the hereby developed method, “feedback GRAPE”, was demonstrated on a number of concrete control scenarios [12, 13] covering state preparation with feedback, state purification and state stabilization with extension to quantum error correction. Given the high potential value to the community, we saw the need to provide an efficient implementation of the methods developed in Ref. [12]. Two implementations of feedback-GRAPE are already available online: the code accompanying Ref. [12] [14], and the code used for Gottesman-Kitaev-Preskill (GKP) quantum error correction in Ref. [13] [15]. However, both were written for the specific problems studied in those works. Our goal here is to provide a general implementation that can be applied to a wide range of feedback-control tasks.
In this work, we demonstrate an efficient GPU-accelerated Python package primarily designed to perform feedback GRAPE. As the backbone of our codes, we employ the JAX library [16] which allows to combine physics modeling with machine learning techniques in highly efficient vectorized GPU-accelerated functions. At the same time, we provide a user-friendly interface by following and extending the patterns of the well known QuTiP library [17, 18]. To perform feedback GRAPE, the user has to encode the system’s Hamiltonian, dissipation channels (if present), a sequence of parametrized control and measurements gates, and the control quality measure, i.e., the reward. Then, by calling a corresponding function from the package, an automated optimization of the control and measurement parameters is triggered. In this hands-on tutorial, we describe these functionalities in detail and demonstrate them using ample code examples. We assume that the user is familiar with general scientific programming in Python using NumPy, and explain the functionalities from other third-party libraries as they are used in the considered examples. The package documentation can be found on the website [19]. The source code is openly available on GitHub [20] under the MIT license, and contributions are welcome.
The article is structured as follows. In § 2, we describe the package dependencies and installation process. In § 3, we introduce the feedback GRAPE algorithm. Following that, we show in § 4 how basic quantum-mechanical components (e.g. quantum states and operators) are encoded using our package. In § 5, we discuss how the gates parametrization is represented and turned into quantum-mechanical operators in the code. In § 6, we describe two application examples of feedback GRAPE using our package. In § 7, we present further useful features of the package. Finally, we conclude with an outlook and conclusions in § 8 and § 9, respectively.
Dependencies and installation
Table 1 summarizes the minimal required versions of Python and the third-party Python libraries which are used by our package. Our code employs NumPy [21], SciPy [22], and Mat-
| Python | NumPy | SciPy | JAX | Optax | FLAX | Dynamiqs |
Table 1. Software dependencies and their version requirements. We note that specific versions of orbax-checkpoint<=0.11.32 and chex>=0.1.89 are also included in the requirements.txt for compatibility.
plotlib [23], which are well-established libraries in the scientific community. The backbone of the package is the JAX library [16], recently developed at Google. JAX’s main functionality consists of highly efficient, GPU-accelerated manipulation of data arrays, represented similarly to NumPy arrays. Additionally, we use the JAX-based packages Optax [24] for optimization and Flax [25] for defining neural networks. For modeling quantum dynamics, we use our own implementations, except for solving the Lindblad master equation, for which we employ the recently developed Dynamiqs library [26], which offers GPU-accelerated and differentiable JAX-based solvers.
The feedback GRAPE Python package can be installed using the PyPI package manager. The version without GPU support is installed as
pip install feedback-grape
If an NVIDIA GPU is present on the machine, it can be leveraged by the package to significantly accelerate the computations. The GPU-capable version is installed as
pip install feedback-grape[cuda12]
which assumes compatibility with CUDA 12. Running one of these commands without further parameters will automatically identify and install/update the dependencies. Note that the two variations differ by triggering the installation of different JAX versions: without and with GPU support, respectively. As an alternative to installing via the PyPI manager, the code can be cloned directly from the GitHub repository [20].
Feedback-GRAPE algorithm
In this section, we present a high-level overview of the feedback GRAPE algorithm sketched in Figure 1 and introduced in Ref. [12]. A typical control scenario is represented by a circuit consisting of multiple repeated “elementary” control blocks. In the following, these repetitions are referred to as “timesteps”. Each timestep contains unitary control and measurement gates. Both gate types are parametrized and optimized in feedback GRAPE.
Figure 1. A high-level overview of the feedback-GRAPE algorithm. A physical system is controlled using a feedback loop mechanism: Measurement outcomes from the quantum system are fed into a recurrent neural network or a lookup table which generates the control parameters for the next time step.
The measurement gates extend the ordinary GRAPE algorithm and play a two-fold role. Firstly, they constitute non-unitary control elements removing entropy from the system and purifying its state. Secondly, they yield information on which the parameters of the subsequent gates are conditioned. In the present incarnation of our package, described in the following, we consider generalized binary measurements and denote their outcomes . Each measurement is therefore represented by an operator pair satisfying the completeness condition for positive operator-valued measures (POVMs) . If a system in a state described by the density matrix is measured, the probability of a particular outcome and the corresponding post-measurement state can be obtained as
The stochastic measurement outcomes are used to choose the control and measurement parameters in the future steps using a feedback policy. The latter is either a lookup table (LUT) containing directly the optimized parameters or a recurrent neural network (RNN). We note that both approaches maintain memory over the past measurement outcomes in the current trajectory, thus rendering the feedback non-Markovian. The described approach closely follows reinforcement learning [5]. However, exploration here isn’t a design choice, but is forced by the stochasticity of the measurements.
Training is performed by collecting statistics over multiple trajectories and evaluating the average reward gradients over the trainable parameters using the equation in Ref. [12]:
Here, the trainable parameters are denoted as . The averaging is performed over a batch of trajectories each identified by its sequence of measurement outcomes denoted as . For each such sequence , we denote the trajectory probability and the reward gained in it. The reward choice depends on the control task and will be discussed in the sections devoted to corresponding examples. The obtained gradients are then used to perform gradient ascent towards the optimal control parameters. Further details will be discussed in the context of the concrete examples introduced in the following sections. For more details on the feedback-GRAPE algorithm and equation 2, the reader is referred to Ref. [12].
Encoding a quantum-mechanical system
We begin now with demonstration of our feedback GRAPE Python package. Since the user has to encode the physical system by hand, we start here from the basic quantum mechanical entities (quantum states, operators etc.) intended for this purpose. These are implemented in the feedback_grape.utils subpackage demonstrated in the following.
As the first example, we create a qubit state and a cavity coherent state with :
from feedback_grape.utils.states import basis, coherent
N_cav = 20
basis_state = basis(n=2)
coherent_state = coherent(n=N_cav, alpha=3)
Here, we first imported the corresponding functions basis and coherent from the submodule feedback_grape.utils.states, and called them with the corresponding arguments. The parameter denotes the dimension of the truncated Hilbert space of the cavity. Note that the shown code closely follows the QuTiP syntax. The important difference to QuTiP is that the invoked functions return not objects of custom classes, but usual JAX arrays. These can be manipulated using the standard JAX techniques, which are similar to those for NumPy arrays, see e.g. the JAX documentation [16]. For instance, a density matrix can be constructed from a pure state using directly the JAX means. An alternative way is to use the function feedback_grape.utils.fidelity.ket2dm in the same way as in QuTiP.
Next, we demonstrate the most elementary quantum operators by constructing the number operator from the bosonic creation and annihilation operators:
from feedback_grape.utils.operators import create, destroy
number_operator = create(N_cav) @ destroy(N_cav)
As in the case of states, operators are represented by JAX arrays. Consequently, and in contrast to QuTiP, the operator multiplication is performed via the @ operation, whereas the * operation means element-wise multiplication of two matrices. Other operators and operator-related functions defined in feedback_grape.utils.operators can be found in Table 2 and will be used throughout this article.
| Function | Description | Function | Description |
sigmax | Pauli operator | create | Bosonic creation operator |
sigmay | Pauli operator | destroy | Bosonic annihilation operator |
sigmaz | Pauli operator | identity | Identity operator |
sigmap | Pauli operator | cosm | Cosine of an operator |
sigmam | Pauli operator | sinm | Sine of an operator |
Table 2. Operators and functions defined in utils.operators.
As the next step, we show an important function tensor, used for creating states and operators for compound systems. Here, we construct a system consisting of a cavity coupled to a qubit. In this example, the ground state of this system is created with the following code:
from feedback_grape.utils.tensor import tensor
ground_qubit = basis(n=2)
ground_cavity = basis(n=N_cav)
ground_state = tensor(ground_cavity, ground_qubit)
The tensor function can also be applied to quantum operators represented by 2D JAX arrays. Note that whereas a system with few components can be encoded using such tensor-ing, this approach becomes cumbersome for more components, as e.g. qubit chains. For this case, we have implemented an alternative convenient interface described in § 7.2.
Two other important functions of our package are fidelity and purity in the submodule feedback_grape.utils.fidelity. For two quantum states characterized by density matrices and , the fidelity function computes the Uhlmann-Jozsa fidelity [27] given by the formula . The purity function evaluates the quantum state purity . Depending on the control task, fidelity and purity can be used to construct the trajectory reward.
This finishes the description of the basic components for encoding quantum systems using our package. As an alternative to the described functionalities, the users can create basic quantum-mechanical entities directly in QuTiP and then apply the jaxify function from the feedback_grape.utils.operators submodule in order to obtain a corresponding JAX representation, see § 7.3.
Parametrized control and measurement gates
The feedback-GRAPE method aims to achieve the control goal by optimizing the parameters of the control and measurement gates. In this section, we discuss how these can be encoded in our package.
Description of a control example
We now turn to a concrete example of a cavity coupled to two qubits, following Example C in Ref. [12]: a control qubit, which is driven and interacts with the cavity, and an ancilla qubit, which is used only to measure the cavity. This system is subject to a control sequence illustrated in Figure 2. Each timestep is repeated times and consists of a parametrized measurement followed by two parametrized unitary gates and described below. The measurement yields one of two possible outcomes, which is fed to a LUT or RNN. The latter predicts the gate parameters for the following operations, including the measurement in the next timestep (dashed).

Figure 2. Control loop of a qubit-cavity system consisting of timesteps. Measurement outcomes are fed to a LUT or RNN, which predicts the parameters for the subsequent operations, including the next measurement (dashed). In this work, only binary measurements are considered.
In this example, the measurement is represented by the following operator pair
where is the photon annihilation operator, and and are the (real) parameters to be optimized. Physically, this measurement is performed by (i) putting the ancilla qubit in a state lying on the equator of the Bloch sphere; (ii) letting it freely evolve under the coupling to the cavity; (iii) projectively measuring the ancilla qubit in a basis in the equatorial plane, which yields a binary outcome. The parameter includes the ancilla qubit-cavity interaction strength and the evolution time in (ii), whereas represents the angle between the axes in (i) and (iii).
The parametrized unitary gates are given by the following equations:
The gate represents an external drive on the control qubit with containing the drive strength and time. The gate describes the interaction between the control qubit and the cavity with the coupling strength and the interaction time contained in the parameter . Note that and in Eqs. (5), (6) are complex, and the actual optimized parameters are their real and imaginary parts.
Unitary operations and measurements as Python functions
In order to represent the behavior of a parametrized control (measurement) gate in the code, the user creates a function receiving the parameters (and the measurement outcome) as arguarnissaat ments and returning the corresponding concrete matrix. Below, we will employ the functionalities described in § 4, which are imported as:
import jax
import jax.numpy as jnp
from jax.scipy.linalg import expm
from feedback_grape.utils.operators import (
sigmap, sigmam, create, destroy, identity, cosm, sinm
)
We now encode the measurement given by Eqs. (3), (4):
def meas_operator(measurement_outcome, params):
gamma, delta = params
cav_operator = (
gamma * create(N_cav) @ destroy(N_cav)
+ delta / 2 * identity(N_cav)
)
angle = tensor(cav_operator, identity(2))
meas_op = jnp.where(
measurement_outcome == 1,
cosm(angle),
sinm(angle),
)
return meas_op
The optimization routine calling this function will assume that the measurement outcome is passed as the first argument, whereas the optimized parameters are provided in an array as the second argument. Note that it is important to use here the matrix cosm / sinm functions, and not jnp.cos / jnp.sin acting on each matrix element independently. Additionally, note that the conditional statement in the code is represented using the jnp.where function rather than the standard Python if-else clause. This is necessary for compatibility of the created function with JAX’s just-in-time (JIT) compilation [16], which significantly improves the computation speed and is crucial for many functionalities of the feedback GRAPE package.
The parametrized unitary control gates are represented in the code as follows:
def qubit_unitary(alphas):
alpha_re, alpha_im = alphas
alpha = alpha_re + 1j * alpha_im
h_dt = (alpha*sigmap() + alpha.conjugate()*sigmam()) / 2
return tensor(identity(N_cav), expm(-1j * h_dt))
def qubit_cavity_unitary(betas):
beta_re, beta_im = betas
beta = beta_re + 1j * beta_im
a_sigp = tensor(destroy(N_cav), sigmap())
adag_sigm = tensor(create(N_cav), sigmam())
h_dt = (beta * a_sigp + beta.conjugate() * adag_sigm) / 2
return expm(-1j * h_dt)
Here, the optimization parameters are passed to the functions as a single array. We stress again that both and in Eqs. (5), (6) are generally speaking complex. Therefore, each function above receives an array of two real arguments, which are the real and the imaginary part of the corresponding complex parameter.
Python class for managing optimized parameters
As exemplified above, the users encode the behavioral pattern of parametrized control and measurement gates as Python functions receiving the optimization parameters as arguments. In this section we present a Python class Gate which stores the actual parameter values and manages the concrete action of the operation (unitary control or measurement) on the physical system. Objects of the Gate class are created from the following three main components: (i) a function gate encoding the operation as described in § 5.2; (ii) an array initial_params with the initial parameter values (usually random); (iii) a boolean flag measurement_flag indicating if the operation is a measurement. There is also another optional argument param_constraints (not used here) which serves as a constraint for the optimized parameters, see [19] for details. The parameters can be initialized using the JAX randomization mechanism, which we briefly describe in the paragraph below. The objects of the Gate class for the measurement and the unitary gates defined in § 5.2 are instantiated as follows:
from feedback_grape.fgrape import Gate
key = jax.random.key(seed=52)
key, subkey = jax.random.split(key)
measure = Gate(
gate=meas_operator,
initial_params=jax.random.uniform(
subkey,
shape=(2,), # gamma, delta
minval=-jnp.pi,
maxval= jnp.pi,
),
measurement_flag=True,
)
key, subkey = jax.random.split(key)
qub_unitary = Gate(
gate=qubit_unitary,
initial_params=jax.random.uniform(
subkey,
shape=(2,), # alpha_re, alpha_im
minval=-jnp.pi,
maxval= jnp.pi,
),
measurement_flag=False,
)
key, subkey = jax.random.split(key)
qub_cav = Gate(
gate=qubit_cavity_unitary,
initial_params=jax.random.uniform(
subkey,
shape=(2,), # beta_re, beta_im
minval=-jnp.pi,
maxval= jnp.pi,
),
measurement_flag=False,
)
Since the seed determines the initial parameters, and optimization outcomes can be sensitive to this initialization, we advise users to similarly explore multiple seeds.
Randomization mechanism in JAX. The JAX randomization strategy [16] significantly differs from the more common approach based on a hidden randomizer state (like e.g. in NumPy). The latter would be incompatible with JIT compilation which assumes that the compiled function be “pure”, i.e. with an outcome depending only on the function arguments and without any hidden state influencing it. Therefore, the randomizer state in JAX is encapsulated in a so called “random key” which is passed explicitly to a function performing randomization, e.g. jax.random.uniform or jax.random.normal. The first key is created from a seed, whereas new random keys are created by “splitting” the old ones. Using the same key twice in a particular randomizing function leads to the same output1. Moreover, reusing a key in different functions can still lead to correlated results and is therefore considered a bad practice. It is common to split a key into two keys just before invoking a function with randomization. One of the keys is used as the called function argument, whereas the other one is kept for later randomization using the described approach. Note that by convention these two keys are often assigned to the variable names subkey and key, respectively. We refer the user to the JAX documentation [16] for more details and examples on the JAX randomization mechanism.
Illustration examples
At the moment, our package addresses state preparation, purification and stabilization tasks, with fidelity and purity as the available reward functions. In this section, we showcase our code on two concrete examples of closed-loop quantum control: state preparation from a thermal initial state and state stabilization. As the feedback policy, we use LUTs and RNNs, respectively.
Example task: Cavity state preparation with feedback
Following the description of the control example in § 5.1 and Example C in Ref. [12], we illustrate preparation of the following superposition of the Fock state in the cavity:
We assume here that the system is initially in a thermal state, which necessitates measurements in the preparation procedure. The initial cavity state reads
where for this example the average number of photons is assumed.
Using the utilities introduced in § 4, the density matrices representing the initial and the target states are encoded as:
n_average = 1
beta = jnp.log((1 / n_average) + 1)
diags = jnp.exp(-beta * jnp.arange(N_cav))
normalized_diags = diags / jnp.sum(diags, axis=0)
rho_cav0 = jnp.diag(normalized_diags)
rho_qub0 = basis(2, 0) @ basis(2, 0).conj().T
rho0 = tensor(rho_cav0, rho_qub0)
For the target state, we use the fock function (an alias for basis) to construct Fock states of the cavity:
from feedback_grape.utils.states import fock
psi_cav_target = (
fock(N_cav, 1) + fock(N_cav, 2) + fock(N_cav, 3)
) / jnp.sqrt(3)
psi_qub_target = basis(2, 0)
psi_target = tensor(psi_cav_target, psi_qub_target)
rho_target = psi_target @ psi_target.conj().T
As the next step, we arrange the Gate objects created here and in § 5 in a Python list. The order in the list corresponds to the application order of the corresponding operations in a timestep (which in this example will be repeated 5 times):
system_params = [measure, qub_unitary, qub_cav]
The control procedure is now defined and we can switch to the search of the optimal gate parameters. This is achieved by calling the function optimize_pulse, which is the central tool in our package. We show below how it can be done for our particular example, and summarize more generally the optimize_pulse arguments in Table 3. We also show the type of each variable following the standard type annotation rules in Python.
| Parameter | Description | ||
init_state | (jnp.ndarray) Initial state or density matrix. | ||
target_state | (jnp.ndarray) Target state or density matrix. | ||
system_params | (list[Gate]) List of Gate objects in the order of the corresponding operations in a timestep. | ||
num_time_steps | (int) Number of repeated timesteps in a trajectory. | ||
max_iter | (int) Maximum number of optimization iterations. | ||
convergence_threshold | (float | None) The smallest improvement in mean reward, measured over convergence_window iterations. Provide None to disable convergence checking and always run for max_iter iterations. | ||
convergence_window | (int) Number of iterations averaged into each of the two disjoint windows compared when checking convergence. Only used when convergence_threshold is not None. Default: 50 | ||
| Parameter | Description | ||
convergence_patience | (int) Number of consecutive window comparisons that must fall below convergence_threshold before stopping. Only used when convergence_threshold is not None. Default: 3 | ||
learning_rate | (float) Learning rate used by the Adam optimizer [29], employed for gate parameter search. | ||
evo_type | (str) Type of state representation used during evolution, e.g., "state" or "density". | ||
reward_weights | (`list[float | int] | None`) Weights for the reward at each time step. If None, only the final step is weighted. |
goal | (str) Optimization objective: "purity", "fidelity", or "both". Default: "fidelity" | ||
batch_size | (int) Number of trajectories evolved in parallel during training. Default: 1 | ||
eval_batch_size | (int) Number of trajectories used during evaluation. Default: 10 | ||
eval_time_steps | (`int | None) Number of time steps used during evaluation. If None, defaults to num_time_steps`. | |
mode | (str) Feedback policy: "nn" (neural network) or "lookup" (lookup table). Default: "lookup" | ||
rnn | (callable) Recurrent model used when mode = "nn". Ignored otherwise. | ||
rnn_hidden_size | (int) Hidden dimension of the RNN model. Only used with mode = "nn". Default: 30 | ||
progress | (bool) If True, prints progress (cost every 10 iterations). Useful for debugging but may slow optimization. Default: False | ||
| Parameter | Description | ||
initial_trainable_parameters | (jnp.ndarray) Parameters to warm start from, continuing training from a previous FgResult’s optimized_trainable_parameters instead of a fresh initialization. Default: None |
Table 3. Summary of arguments of the optimize_pulse function with their types shown following standard Python type annotation rules.
from feedback_grape.fgrape import optimize_pulse
result = optimize_pulse(
init_state=rho0,
target_state=rho_target,
system_params=system_params,
num_time_steps=5,
mode="lookup",
goal="fidelity",
max_iter=3000,
learning_rate=0.02,
evo_type="density",
progress=True,
batch_size=10,
eval_time_steps=5
)
After the optimization process is complete, an object of a custom class FgResult is returned. It has a number of attributes intended for representing the computation results and their further use. Those attributes are discussed in Table 4.
| Field | Description |
optimized_trainable_parameters | (jnp.ndarray) Optimized control amplitudes. |
iterations | (int) Number of iterations taken for optimization. |
final_state | (jnp.ndarray) Final operator after applying the optimized control amplitudes. |
returned_params | (list[jnp.ndarray]) Array of final parameters for each time step. |
final_purity | (jnp.ndarray | None) Final purity of the optimized control. (None) when goal does not include purity. |
final_fidelity | (jnp.ndarray | None) Final fidelity of the optimized control. (None) when goal does not include fidelity. |
fidelity_each_timestep | (list[jnp.ndarray] | None) Fidelity values of the optimized control at each time step and batch. (None) when goal does not include fidelity. |
purity_each_timestep | (list[jnp.ndarray] | None) Purity values of the optimized control at each time step and batch. (None) when goal does not include purity. |
reward_history | (jnp.ndarray) Reward at each training iteration. |
Table 4. Summary of fields in FgResult.
Numerical results
The training in this example is performed for 3000 iterations. After the optimization process is complete, we extract the evaluation fidelity in each time step via result.fidelity_each_timestep and examine how the state evolves with time. As shown in Figure 3, the parameters converged to a state which optimizes the fidelity in the final time step (in this example, ). This behavior is natural for the state preparation task. Note that other tasks, e.g. state stabilization, may require keeping the fidelity high over many time steps. This can be controlled by the argument reward_weights of the optimize_pulse function which we discuss and demonstrate in the next example.
Figure 3. Fidelity evolution during evaluation of the trained feedback-GRAPE procedure, where only the fidelity at the final time step was optimized. Faint blue curves show the individual stochastic measurement trajectories under the learned look-up policy. The shaded band shows their min–max range, and the solid black curve their mean. Note that some trajectories coincide and are indistinguishable in the plot.
The progress in the training process can be investigated using the result.reward_history attribute. The optimization procedure may be sensitive to the parameter initialization and the hyperparameter values, which should be explored by the users for the case at hand. In Figure 4 we show the result of such exploration for the present example. In each case, the training was performed for 3000 iterations. It can be seen that the convergence pattern and the result indeed shows dependence on the initial random seed and the learning rate (indicated as “lr”). We attribute this behavior to complex reward landscape with many local minima.
Figure 4. Progress with training iterations for different random seeds and learning rates (indicated as “lr”). The training was performed in all cases for 3000 iterations.
We note that we ran the training for a fixed number of iterations and do not employ the early stopping mechanism. The latter is available in our package and can be leveraged when calling the optimize_pulse function by setting the following arguments (see also Table 3):
convergence_threshold– The smallest improvement in mean reward, measured overconvergence_windowiterations.convergence_window– Number of iterations averaged into each of the two disjoint windows compared when checking convergence.convergence_patience– Number of consecutive window comparisons that must fall belowconvergence_thresholdbefore stopping.
Example task: State stabilization
In this section we consider the task of stabilization of a quantum state in presence of decoherence. We depart from the system considered in the previous section and showcase stabilization with the feedback-GRAPE method for a system of five qubits subject to amplitude damping and dephasing described below. The code for this example is available in the GitHub repository [30]; since it closely follows the structure of the state-preparation task, we highlight here only the aspects specific to this example. Specifically, we aim stabilize the following superposition state:
In general, the evolution of the density matrix in presence of noise can be modeled using the Lindblad equation:
where is the Hamiltonian governing the unitary (noiseless) evolution and are the jump operators of the noise channels. We assume that the system idles for time with and consider each qubit to undergo amplitude damping and dephasing modeled by the jump operators and with the rates and . Under these assumptions, the decoherence result is fully determined by the products . In this example we assume .
We encode this by passing the jump operators of the system into a new class called Decay as follows:
from feedback_grape.utils.operators import (
sigmaz, sigmam, identity
)
gamma_z = 0.1 # dephasing rate
gamma_m = 0.1 # decay rate
I = identity(2)
``:VEVENT
dephasing = [
gamma_z**0.5 * tensor(sigmaz(), I, I, I, I),
gamma_z**0.5 * tensor(I, sigmaz(), I, I, I),
gamma_z**0.5 * tensor(I, I, sigmaz(), I, I),
gamma_z**0.5 * tensor(I, I, I, sigmaz(), I),
gamma_z**0.5 * tensor(I, I, I, I, sigmaz()),
]
decay = [
gamma_m**0.5 * tensor(sigmam(), I, I, I, I),
gamma_m**0.5 * tensor(I, sigmam(), I, I, I),
gamma_m**0.5 * tensor(I, I, sigmam(), I, I),
gamma_m**0.5 * tensor(I, I, I, sigmam(), I),
gamma_m**0.5 * tensor(I, I, I, I, sigmam()),
]
decay_gate = Decay(
c_ops=dephasing + decay,
)
The Decay object takes c_ops as input, which is a list of JAX arrays corresponding to the jump operators2. This decay_gate object is then inserted in the system_params list where the measurement and control gates are accumulated to form the structure of a time step. When the decay_gate is applied, the Lindblad equation (10) with is integrated numerically over a unit time interval using the master-equation solver of the dynamiqs library [26].
In our time-step ansatz, the decay gate is followed by a sequence of four fully parametrized generalized measurements interleaved with fully parametrized unitaries. We note that such encoding of decay as a discrete gate can be used to approximately model decoherence during the control. This is achieved by repeatedly interrupting the control process and inserting partial decay gates; see Ref. [12] for details.
For the state preparation task considered in § 6, the reward was naturally determined by the fidelity in the last time step. For stabilization, it is important to incorporate the fidelity also in the intermediate time steps. In our package, this can be achieved using the reward_weights argument of the optimize_pulse function. For time steps, it receives an array with weights . Then the reward is expressed as , where is the fidelity in the -th time step. The default value is , which was implicitly used in the state preparation example.
In Figure 5 we show the fidelity achieved in each time step of the stabilization process, using a neural network for the optimization (mode="nn" in optimize_pulse), compared against an uncontrolled system. The RNN was trained on 3 time steps and applied to stabilize the state over 20 time steps. For each set of training parameters, we trained three randomly initialized RNNs and selected the best performing model based on its average fidelity over 20 time steps. We demonstrate the results for three distinct values passed to the reward_weights argument: , and . The time-steps whose fidelity enters the reward are highlighted by the bold black contour. We see that the stabilization is efficient only if the fidelities in all three time-steps contribute to the reward .
Figure 5. Comparison of ensemble fidelity of RNN models trained for state stabilization of a five-qubit superposition state against amplitude damping and dephasing. The training was performed for three distinct values passed to the reward_weights argument: (from top to bottom) [0, 0, 1], [0, 1, 1] and [1, 1, 1]. The time-steps whose fidelity enters the reward are highlighted by the bold black contour and the fidelities at the 0th time step represent the initial state before any gates are applied. The results are compared with an uncontrolled system (top row).
Using the exemplary control tasks for state preparation from a thermal state and state stabilization of a state subject to decoherence, we have covered the most relevant functionalities of our package. Below, we show other helpful features and refer the user to the package documentation [19] for further details.
Discussion of other features
In this section we introduce further features helpful for solving quantum control tasks.
GRAPE without feedback
Our package supports the original GRAPE algorithm [3] for optimizing control pulses in absence of feedback. Note that we do not use the analytical approach presented in Ref. [3], but rather employ automatic differentiation offered by the JAX library.
There are two ways to perform GRAPE optimization within the package. The first approach uses the same gated pipeline described in the preceding sections, but without supplying a measurement operator. In this case, the control sequence consists entirely of parametrized unitary gates, and the optimization proceeds exactly as in the feedback-GRAPE setting, except that no measurement-conditional branching occurs. There, the user passes mode="no-measurement" in the fgrape.optimize_pulse function and proceeds as described in the preceding sections.
The second approach follows the traditional GRAPE formulation more closely. Here, the user directly supplies the system Hamiltonian, the initial and target states, and the number of time intervals into which the control pulse is discretized. This is accessed through the interface demonstrated in the following code snippet:
from feedback_grape.grape import optimize_pulse
res = optimize_pulse(
H_drift=H0,
H_control=H_ctrl,
U_0=psi0,
C_target=psi_target,
num_t_slots=num_of_intervals,
total_evo_time=T,
evo_type=evolution_type,
optimizer=optimizer_type,
)
Note that we imported here from feedback_grape.grape rather than feedback_grape.fgrape.
A comprehensive description of the traditional GRAPE in our package with example notebooks for both methods are available in the package documentation [19].
Composite system interface
Alongside the normal QuTiP-like API based on explicitly constructing tensor products for subsystem operators, we provide also an alternative API which allows users to avoid manually tensoring subsystem operators with identities and therefore can be convenient for working with large compound systems. This API is modular and can be expanded to a general composite system structure. However it is currently implemented only for systems consisting of qubits and cavities, and we refer to it as to QubitCavity API.
The central object of this API is the QubitCavity class defined as follows:
class QubitCavity:
def __init__(self, num_qubits: int, *cavity_dims: int):
"""
Qubit comes first in the Hilbert space, followed by
cavities.
Args:
num_qubits (int): Number of qubits.
*cavity_dims (int): Dimensions of each cavity.
"""
Here, the first argument specifies the number of qubits, and the remaining arguments specify the cavity dimensions. For example, a system of two qubits and two cavities can be encoded as follows:
from feedback_grape.utils.modeling import QubitCavity
qc = QubitCavity(2, 10, 20)
where we model the first 10 Fock states of the first cavity and the first 20 Fock states of the second cavity. The individual subsystems can then be addressed explicitly. For example, the first qubit and the first cavity are accessed as:
q0 = qc.qubits[0]
c0 = qc.cavities[0]
q0_sigmaz = q0.sigmaz
c0_annihilation = c0.a
Here, q0.sigmaz returns the Pauli operator acting on qubit 0 and the identity elsewhere, and c0.a returns the annihilation operator acting on cavity 0 and the identity elsewhere. This is done automatically without requiring the user to manually tensor with identity operators for the spectator subsystems. More examples of the QubitCavity API are given in the documentation [19].
Conversion of operators from QuTiP
QuTiP is one of the most used platforms for simulating quantum dynamics and it offers an extensive library of predefined operators. However, the corresponding Python objects are incompatible with JAX. Therefore, we implemented a function jaxify converting QuTiP’s object to JAX arrays which can be used in our package. For example the sigmax operator (which is also directly available in our package) can be converted from QuTiP as follows:
import qutip as qt
from feedback_grape.utils.operators import jaxify
sigx = jaxify(qt.sigmax())
Outlook
The current implementation of feedback-grape provides a flexible and efficient framework for feedback-based quantum optimal control. In the following we point out several possible extensions which can be implemented in the future.
Attention-based neural networks such as transformers have revolutionized many fields, for instance, natural language processing, genomics, and computer vision. Recently, the use of transformers for quantum feedback control was demonstrated [31]. Such architecture could be added to the package as an extra option for finding the optimized parameters which may help capture long-range temporal correlations more efficiently than the present models.
The interface for compound systems demonstrated in § 7.2 currently supports only systems consisting of qubits and cavities. Since this API is especially advantageous for systems with many components, its generalization would be helpful for usage of the package in such cases.
In the current version of the package, the optimal control strategy for producing, purifying or stabilizing a specific quantum state is sought. An extension to ensembles of initial states would enable e.g. discovery of protocols for quantum error-correction, which can be treated as stabilization of a subspace of states rather than a single state.
In addition, the library of numerical solvers can be further expanded. Different physical regimes and control scenarios may benefit from specialized integration schemes or propagation methods with improved efficiency, stability, or memory scaling. Providing a wider set of solvers would allow users to better tailor simulations to their specific applications.
Finally, a further useful extension would be the option to supply a custom reward function. Currently, the optimization target is fixed to fidelity, purity, or a combination thereof; allowing users to define arbitrary reward functions would enable optimization for other quantities relevant to their specific application.
Conclusion
We have developed and demonstrated a general and efficient GPU-accelerated Python package implementing the method of feedback GRAPE. We illustrated its use on two concrete closed-loop control tasks — cavity state preparation from a thermal state and multi-qubit state stabilization — and described its further features, including standard GRAPE without feedback, a composite-system interface, and conversion of QuTiP operators. Built on the JAX library, the package provides a user-friendly interface that follows and extends the patterns of the well-known QuTiP library. We warmly invite contributions from the community and hope that this package helps further advancements in the field of feedback-based quantum optimal control.
References
- [1]S. J. Glaser, U. Boscain, T. Calarco, C. P. Koch, W. Köckenberger, R. Kosloff, I. Kuprov, B. Luy, S. Schirmer, T. Schulte-Herbrüggen et al., Training schrödinger’s cat: Quantum optimal control: Strategic report on current status, visions and goals for research in europe, The European Physical Journal D 69(12), 279 (2015).
- [2]C. P. Koch, Controlling open quantum systems: tools, achievements, and limitations, Journal of Physics: Condensed Matter 28(21), 213001 (2016).DOI
- [3]N. Khaneja, T. Reiss, C. Kehlet, T. Schulte-Herbrüggen and S. J. Glaser, Optimal control of coupled spin dynamics: design of nmr pulse sequences by gradient ascent algorithms, Journal of Magnetic Resonance 172(2), 296 (2005), doi:https://doi.org/10.1016/j.jmr.2004.11.004.
- [4]S. Machnes, U. Sander, S. J. Glaser, P. de Fouquieres, A. Gruslys, S. Schirmer and T. Schulte-Herbrüggen, Comparing, optimizing, and benchmarking quantum-control algorithms in a unifying programming framework, Physical Review A—Atomic, Molecular, and Optical Physics 84(2), 022305 (2011).DOI
- [5]R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, The MIT Press, second edn. (2018).
- [6]M. Bukov and F. Marquardt, Reinforcement learning for quantum technology (2026), arXiv: 2601.18953.arxiv.org/abs/2601.18953
- [7]T. Fösel, P. Tighineanu, T. Weiss and F. Marquardt, Reinforcement learning with neural networks for quantum feedback, Physical Review X 8(3), 031084 (2018).DOI
- [8]R. Porotti, A. Essig, B. Huard and F. Marquardt, Deep reinforcement learning for quantum state preparation with weak nonlinear measurements, Quantum 6, 747 (2022).DOI
- [9]V. V. Sivak, A. Eickbusch, H. Liu, B. Royer, I. Tsioutsios and M. H. Devoret, Model-free quantum control with reinforcement learning, Phys. Rev. X 12, 011059 (2022), doi:10.1103/PhysRevX.12.011059.DOI
- [10]K. Reuer, J. Landgraf, T. Fösel, J. O’Sullivan, L. Beltrán, A. Akin, G. J. Norris, A. Remm, M. Kerschbaum, J.-C. Besse et al., Realizing a deep reinforcement learning agent for real-time quantum feedback, Nature Communications 14(1), 7138 (2023).DOI
- [11]Y. Baum, M. Amico, S. Howell, M. Hush, M. Liuzzi, P. Mundada, T. Merkh, A. R. Carvalho and M. J. Bierićuk, Experimental deep reinforcement learning for error-robust gate-set design on a superconducting quantum computer, PRX Quantum 2(4), 040324 (2021).DOI
- [12]R. Porotti, V. Peano and F. Marquardt, Gradient-ascent pulse engineering with feedback, PRX Quantum 4(3), 030305 (2023).DOI
- [13]M. Puviani, S. Borah, R. Zen, J. Olle and F. Marquardt, Non-markovian feedback for optimized quantum error correction, Physical Review Letters 134(2), 020601 (2025).DOI
- [14]R. Porotti, feedback_grape, https://github.com/rporotti/feedback_grape, GitHub repository.
- [15]M. Puviani, GQF, https://github.com/Matteo-Puviani/GQF, GitHub repository.
- [16]J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne and Q. Zhang, JAX: composable transformations of Python +NumPy programs, URL http://github.com/google/jax (2018).
- [17]J. Johansson, P. Nation and F. Nori, Qutip 2: A python framework for the dynamics of open quantum systems, Computer Physics Communications 184(4), 1234 (2013), doi:10.1016/j.cpc.2012.11.019.DOI
- [18]P. D. Nation and J. R. Johansson, QuTiP: Quantum Toolbox in Python (2014).
- [19]Y. Elbrolosy, A. Bories, F. Marquardt and P. Bilous, Python package for grape with feedback, URL https://feedbackgrape.readthedocs.io/en/latest/ (2025).
- [20]Y. Elbrolosy, A. Bories, F. Marquardt and P. Bilous, Python package for grape with feedback, URL https://github.com/YousefElbrolosy/feedbackGRAPE (2025).
- [21]C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus et al., Array programming with NumPy, Nature 585(7825), 357 (2020), doi:10.1038/s41586-020-2649-2.DOI
- [22]P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett et al., SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python, Nature Methods 17, 261 (2020), doi:10.1038/s41592-019-0686-2.DOI
- [23]J. D. Hunter, Matplotlib: A 2d graphics environment, Computing in Science & Engineering 9(3), 90 (2007), doi:10.1109/MCSE.2007.55.DOI
- [24]DeepMind, I. Babuschkin, K. Baumli, A. Bell, S. Bhupatiraju, J. Bruce, P. Buchlovsky, D. Budden, T. Cai, A. Clark, I. Danihelka, A. Dedieu et al., The DeepMind JAX Ecosystem, URL http://github.com/deepmind (2020).
- [25]J. Heek, A. Levskaya, A. Oliver, M. Ritter, B. Rondepierre, A. Steiner and M. van Zee, Flax: A neural network library and ecosystem for JAX, URL http://github.com/google/flax (2024).
- [26]P. Guilmin, A. Bocquet, É. Genois, D. Weiss and R. Gautier, DynamIQs: an open-source python library for gpu-accelerated and differentiable simulation of quantum systems, URL https://github.com/dynamiqs/dynamiqs (2025).
- [27]R. Jozsa, Fidelity for mixed quantum states, Journal of Modern Optics 41(12), 2315 (1994), doi:10.1080/09500349414552171.DOI
- [28]“write your model in jax they said. it’ll be deterministic they said.”, https://github.com/google/jax/discussions/10674.
- [29]D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014).arxiv.org/abs/1412.6980
- [30]A. Bories, Y. Elbrolosy, F. Marquardt and P. Bilous, Stabilization of a 5-qubit state using feedback-grape, URL https://github.com/AdrianBories/fgrape_qubit_stabilization (2026).
- [31]P. Vaidhyanathan, F. Marquardt, M. T. Mitchison and N. Ares, Quantum feedback control with a transformer neural network architecture, Physical Review Research 8(1), L012043 (2026).DOI



