WorksEngineering
A Personalized and Adaptive Intelligent Tutoring System for Psychomotor Learning
Abstract
Intelligent tutoring systems (ITSs) have advanced learner motivation and personalization in cognitive domains, but extending these gains to psychomotor tasks remains challenging. Existing psychomotor ITSs rely on learner models built before instruction or updated once per module, leaving open how to use real-time assessments of learner cognitive state for finer-grained personalization. Psychomotor learning involves multiple tutoring objectives, including calibrating self-confidence to performance, calibrating workload per the Yerkes-Dodson Law, and driving skill progression toward higher learning stages. How to balance these objectives for learners, and adjust that balance over time, remains open. This study proposes and evaluates an online personalization algorithm that continuously reprioritizes tutoring objectives to match learners' evolving needs. Rather than weighting objectives equally, the actor-critic-inspired algorithm dynamically updates reward weights using recent measures of self-confidence, workload, and performance. These weights inform a multi-input, multi-output cognitive control framework that synthesizes learner-specific tutoring policies via model-based planning. The approach was evaluated in a human-subject study (N = 40) with a 2D quadrotor landing task. Clustering of the learned reward weights revealed distinct prioritization patterns across learners, indicating the algorithm captures heterogeneity in learner needs. Learners whose weights shifted more substantially tended to reach higher final performance, and those whose weights consistently emphasized workload calibration achieved the strongest outcomes. These findings advance online learner modeling and control-based personalization in psychomotor ITSs, showing how adaptive tutoring can better support learning by aligning instructional decisions with learners' evolving cognitive and performance states.
Introduction
Intelligent tutoring systems (ITS) aim to personalize education for learners by emulating human tutors through algorithmic frameworks that draw on models of human learning and data collected from the learner [1, 2]. ITSs have been successful in closing the feedback loop by considering learner behavior in cognitive learning domains such as language acquisition, computer programming, and mathematics [3]. In particular, many of these consider increasing learner motivation by supporting self-efficacy, a learner’s self-perception of their competence and achievements [4], as increased learner self-efficacy has been consistently associated with improved learning performance and heightened learner motivation [4].
While ITSs have predominantly been used for cognitive learning domains, there have been several recent developments to advance ITSs designed for psychomotor tasks. A psychomotor task is defined as any motor action that is dependent on conscious mental processes [5, 6, 7]. Recent psychomotor ITS development has largely focused on structured knowledge representation and the design of adaptive, self-improving tutoring systems that enhance tutor–learner interaction. Selfit [8] enables users to self-report fatigue, motivation to train, sleep quality, and stress levels before each training session. During training, users may manually record exercise failures and report whether additional repetitions could have been performed, including the number of additional repetitions. After completing a session, users can evaluate the perceived difficulty of the training session. This information is then used by Selfit to update the training curricula. Another example is GIFT, the generalized intelligent framework for tutoring [6], which updates a model of the student using historical learner information, self-reports, performance data, and sensor-derived behavioral and physiological measures. In addition to this, the framework leverages a motivation assessment tool (MAT) over longitudinal training. Together, these measurements are utilized to inform GIFT on how to adapt via tutoring strategies according to the INSPIRE (Intelligent, Nurturant, Socratic, Progressive, Indirect, Reflective, Encouraging) model of tutoring success [9]. In prior work [10], an alternative approach is proposed, in which learner engagement and motivation are decomposed into the underlying cognitive states of self-confidence and workload. Seminal learning literature indicates that self-confidence and workload directly influence self-efficacy.
Additionally, these cognitive factors are widely recognized as key determinants of human–automation interaction quality [11, 12, 13]. Therefore, to model evolution during learning, a Markov Decision Process (MDP) framework is used to probabilistically capture the dynamics of self-confidence, workload, and learning stages characterized in by the authors in prior work [14]. The action space includes assistance provided during task execution and the tone of the formative feedback delivered after each task attempt. The MDP framework enables the representation of qualitative tutoring objectives through reward function design, providing a quantitative basis for synthesizing instructional policies that address multiple learning objectives.
Despite these advancements, existing psychomotor ITS are not fully realizing the goal of personalization, particularly when it comes to adaptation to achieve cognitive objectives. Supporting learner motivation in this setting requires online assessment of learner self-efficacy along with targeted interventions that can effectively influence the learner. Although researchers have recognized the need for intelligent tutoring systems (ITSs) to adapt to human cognitive behavior in cognitive learning domains [1, 3, 15, 16], an open question remains: how should an ITS use these ongoing assessments to improve personalization strategies in psychomotor domains? Currently, personalization requires an initial model of the individual student, like that provided by the aforementioned motivational assessment tool in GIFT [9] and the self-reports collected by Selfit [17]. Both are implemented before instruction, or can also be implemented once per distinct module or task. However, the personalization of psychomotor ITSs could benefit from more frequent online updates to the model of student behavior, especially as psychomotor tasks increase in complexity. Furthermore, the tutoring policies employed by these psychomotor ITS should be updated according to the evolving needs of a given student by leveraging adaptive control methods. This allows online updates to both the model and the policies tutoring agents use when helping the learner and would further enhance the tutoring strategies chosen by current ITSs. One example is adapting how and when information is presented by the ITS according to an individual’s learning style [18].
The contribution of this paper is an adaptive algorithm that updates a psychomotor ITS tutoring policy via reprioritization of tutoring objectives during the learning process for each learner. The actor-critic algorithm from reinforcement learning inspires the algorithm and considers both performance and cognitive states (self-confidence and mental workload) to determine how well the student and ITS interaction is meeting tutoring objectives. This is then used to solve for reward weights, in which the weights represent the prioritizations of each tutoring objective. The proposed algorithm leverages a learner model and multi-objective tutoring policy designed for psychomotor ITS described in the authors’ prior work [10, 19]. The main research questions we answer are: (RQ1) Does the algorithm give rise to distinct patterns in the tutoring objective weights across all learners? (RQ2) Do the reward weights dynamically change to adapt to the learner’s needs? (RQ3) Is the personalization algorithm successful in prioritizing the tutoring objectives learners need? Our findings show that the personalization algorithm produces distinct patterns in the tutoring objective prioritization across learners. Through statistical analysis, the algorithm is shown to appropriately prioritize tutoring objectives according to the learners’ needs.
The paper is organized in the following order. The necessary preliminaries, including the chosen psychomotor task, learning metrics, and model training, are described in § 2. We then describe policy synthesis, followed by further detail on personalization in Section III. In Section IV, we describe the experiment implemented to evaluate the proposed personalization. Data processing and results are described in § 5, followed by a discussion of the results in § 6. Finally, conclusions and comments for future work are in § 9.
Preliminaries
In this section, we provide the necessary background for the reader. First, we describe the psychomotor task considered in this work in § 2-A, followed by the definition of two key learning metrics for the task in § 2-B. We then describe the model training approach employed to obtain a learner model capturing the dynamics of self-confidence, workload, and learning stage progression. These preliminaries are based upon existing work by the authors [10, 14, 19].
Psychomotor Task: 2D quadrotor landing simulator
The psychomotor task considered in this work is a 2D quadrotor landing simulator. The learner’s objective is to safely guide the quadrotor onto a landing pad while satisfying constraints on its speed () and roll angle (), using the desktop simulator shown in Figure 1a. The quadrotor’s state consists of its position (, ), velocity (, ), roll angle , and roll angular velocity . Participants control the quadrotor through a T.Flight Hotas 4 controller, using the throttle to command thrust force, , and the joystick to command roll angular acceleration, . A landing is classified as safe if the quadrotor reaches the pad within the speed and roll angle limits and as unsafe if it reaches the pad but violates one or both of these conditions. If the quadrotor crashes before reaching the pad, the attempt is considered unsuccessful. During each trial, the participant either lands the quadrotor entirely on their own (manual control mode, ) or receives automation assistance (shared control mode, ). In manual mode, the participant has full control over the quadrotor’s thrust force and roll angular acceleration inputs. Conversely, in shared control mode, the participant’s inputs are supplemented by inputs from automation consisting of a linear-quadratic regulator that tracks a reference trajectory derived from expert demonstrations, using a linear time-invariant model of the quadrotor dynamics together with a quadratic cost function that drives all quadrotor states toward zero [20].


Figure 1. The designed psychomotor task.
After each landing attempt, learners receive negative, neutral, or positive formative feedback generated using GPT-4o [21], following the approach of Jensen et al. [22], who showed that AI-generated formative feedback can meaningfully improve learning outcomes. The automation assistance and feedback tone are controllable inputs that the ITS can adjust before each trial of the task.
Online Quantification of Learning
To quantify learning in the psychomotor task online (i.e., after each trial), we employ two complementary metrics. The first is a discrete-valued metric defined in terms of psychomotor learning stages that is well-suited for a Markov decision process model. The second is a continuous-valued measure. based on a scoring function that enables gradient descent-based policy updates. We describe each metric and its utility in the overall ITS in the subsections that follow.
1) Discrete-valued Learning Stage Classification: The Dreyfus model of skill acquisition learning stages was first proposed by [23] as a means of characterizing learning progression in psychomotor tasks by outlining five sequential stages of learning a skill and has evolved in the decades since. Prior work [14] developed a classifier for determining a participant’s learning stage after completing a trial of the quadrotor landing task based on trajectory data associated with task execution. Specifically, the first four stages of the five-stage learning theory model proposed by Dreyfus [23] are applied to the quadrotor landing task: novice (learning stage 1), advanced beginner (learning stage 2), intermediate (learning stage 3), and competent (learning stage 4). Features of the quadrotor dynamics as well as feasible landing types for each learning stage are summarized as follows for the reader’s benefit.
Learning stage 1 corresponds to the novice learner, who is still decomposing the task and developing an understanding of the overall goal. At this stage, the learner struggles to stabilize the quadrotor, and feasible landings are limited to unsuccessful or unsafe outcomes.
Learning stage 2 corresponds to the advanced beginner, who begins developing strategies, identifying sub-tasks, and recognizing cues that indicate instability. The learner can now stabilize the quadrotor and complete landings, though the resulting trajectories remain inefficient and inconsistent. Feasible landings at this stage include unsuccessful, unsafe, and safe landings.
Learning stage 3 corresponds to the competent learner, who refines their strategy, prioritizes among sub-tasks, and shows increasing consistency, though still under a high attention load. The learner can control the quadrotor and produces less variable trajectories, with feasible landings limited to unsafe and safe outcomes.
Learning stage 4 corresponds to the proficient learner, whose strategy has become fixed and continues to improve with practice, accompanied by a decreasing attention load. The learner maintains control and lands the quadrotor consistently, producing efficient trajectories, with safe landings as the only feasible outcome.
Each learning stage definition is quantified using a representative distribution of trajectories associated with that stage; example trajectories for each learning stage are shown in Figure 1b. These trajectories capture the full quadrotor states of position , velocity , roll angle , and roll angular velocity together with the thrust force input and roll angular acceleration input . Each of these distributions is represented as a kernel embedding within a reproducing kernel Hilbert space, from which a learning stage classifier is built. For a detailed description of the development of the classifier, see [14].
2) Continuous-valued Performance Score: A gamification approach is utilized to define a continuous-valued scoring function that is better suited for online policy adaptation than the discrete characterization of learning progression provided by the learning stage definitions. The total score is given by
where the position score, velocity score, roll angle score, and proficiency score for each trial are given by , , , and , respectively. The position, velocity, and roll angle scores quantify how close the landing conditions of a learner’s performance are to the safe landing conditions. The proficiency score quantifies how similar one’s landing attempt is to those of proficient learners (learning stage 4) based on trajectory data representative of proficient learners. The individual score components are defined as follows:
Eqns. (2)–(4) are defined using Sigmoid functions that are parameterized such that the total score of 750 out of 1000 points is indicative of someone who has successfully landed the quadrotor, albeit not in a manner that is representative of a canonically proficient learner. In Eqn. (5), is the maximum mean discrepancy [24] between the landing attempt trajectory and canonical distribution representing learning stage 4 .
Human Learner Model
To design an optimal tutoring policy and adapt it online, we require a model of the human learner. Given that the tutoring objectives of interest rely on knowledge of the learner’s self-confidence, workload, and learning progression, we utilize a Markov decision process (MDP) model that captures stochastic transitions between these learner states. The model is based on a previously developed and validated framework [10] and is summarized here for completeness. An MDP is defined by the 5-tuple , where denotes a finite set of states and a finite set of actions. The transition function represents the probability of transitioning to the next state given the current state and action . The MDP framework is particularly useful for designing a tutoring policy because tutoring objectives can be captured using the reward function , which is explicitly a function of the learner’s states.
The model captures the dynamics of cognitive states of self-confidence and workload, as well as a learning progression state defined by learning stage. Each state is treated as independent, allowing separate models that map the actions and onto each state to be trained independently. Moreover, the transition probabilities for all state combinations are given by
This independence assumption has been supported previously by [10]. The self-confidence and workload states, and respectively, are each discretized into three levels of low, medium and high.
The discretization thresholds are set at the 33% and 66% quantiles of the self-reported data, which is collected on a 0–100 numerical scale; each self-report is then sorted into its corresponding discrete state based on these thresholds. The learning stage MDP instead defines three states : learning stage 1 (), learning stage 2 (), and a combined category for learning stages 3 and 4 (). Stages 3 and 4 are merged into a single category by design, due to the similarity of quadrotor trajectory behavior in both stages.
Across all models, the action set represents the decisions available to the tutoring agent. The automation action indicates whether within-trial automation assistance is off () or on (), while the feedback tone action indicates whether the feedback delivered is negative (), neutral (), or positive ().
For the reader’s reference and convenience, we describe the model training method in Appendix A.
Tutoring Policy Design and Adaptive Personalization
In order to personalize the psychomotor tutoring policy to individual learners, we need a tutoring policy structured in a way that enables adaptation, as well as the adaptation algorithm itself. In this section, we first introduce the design of a weighted reward function that captures the three tutoring objectives of interest. We then propose an algorithm for adapting the weights in the reward function based on individual performance over consecutive trials.
Reward Function Design and Policy Training
We aim to achieve the following three tutoring objectives with a psychomotor ITS: (1) calibrating self-confidence to learning stage; (2) calibrating workload in accordance with the Yerkes-Dodson law [25]; and (3) rewarding advancement in learning stages from novice toward proficient. The first two objectives are motivated by the self-efficacy literature [26, 27, 28, 29], which holds that greater self-efficacy makes learners more realistically self-aware about their progress and accomplishments, in turn boosting performance and motivation [4, 29]. Maintaining appropriately calibrated self-confidence and workload levels are thus both components of self-efficacy. The third objective explicitly encourages progression through the learning stages. Therefore utilize a multi-objective and weighted reward function defined as
where , , and are weights reflecting the relative priority assigned to each tutoring objective, and , , and are the components of the total reward corresponding to three tutoring objectives. For brevity, detailed descriptions of , , and are provided in Appendix A. The optimal policy is computed using the Bellman optimality equations [30, 31]. The action-value function is given by
where and correspond to Eqn. (6) and Eqn. (7), respectively. The optimal policy is then obtained by selecting, for each state, the action that maximizes . As discussed in the Introduction, different learners may require more emphasis on one tutoring objective based on their individual characteristics. Moreover, even for the same learner, the appropriate prioritization will change over time. This motivates the design of a reward weight adaptation scheme, discussed next.
Reward Weight Adaptation Algorithm
To update the reward weights online, one may initially turn to online reinforcement learning (RL) algorithms such as actor-critic [32]. However, in the case of the psychomotor ITS, the available dataset is not well-suited for online reinforcement learning. First, the data is sparse and collected in short, fixed-length episodes (e.g., the most recent trials), which limits the ability to perform long-horizon credit assignment or reliably estimate value functions from interaction data alone. Second, the data are generated under a restricted set of previously deployed policies, resulting in limited state–action coverage and insufficient exploration for stable policy improvement. Third, learner behavior is inherently variable and non-stationary, as performance, self-confidence, and workload evolve, violating the stationarity assumptions commonly required for RL. Finally, the human-in-the-loop nature of the system makes aggressive exploration undesirable, as frequent or suboptimal policy updates may negatively impact learning outcomes and user experience.
To address these challenges, we propose a data-driven reward re-weighting approach that leverages recent observations to adapt the relative importance of tutoring objectives, followed by model-based -iteration to compute the corresponding policy. This framework can be interpreted as an actor–critic-inspired architecture, in which the reward weights serve as high-level actor parameters that modulate the reward structure underlying policy optimization, while the objective functions (described in Sec. III-B1) computed from recent learner data provide a critic-like signal that evaluates the extent to which the tutoring objectives are satisfied. Unlike classical actor–critic methods, however, policy updates are not performed through online interaction; instead, the policy is recomputed via model-based planning under the learned reward function, resulting in a more stable and data-efficient approach.
Objective Function Definition
To adapt the relative prioritization of tutoring objectives, we define three objective loss functions that correspond directly to the reward components . These objective functions encode the same three tutoring objectives defined in the reward function and provide a differentiable signal for adjusting the reward weights. We utilize self-reported self-confidence and self-reported workload and scaled score . Scaled score is the total score (Eqn. (1)) divided by 10, such that self-reported self-confidence, self-reported workload, and scaled score are all in the range . The objective functions are computed from recent trial history, specifically the last ten trials, using the empirical mean of self-reported self-confidence , empirical mean of self-reported workload , and empirical mean of scaled score , together with the scaled score standard deviation .
The first objective function aligns learner self-confidence with observed performance and is defined as the squared difference of and ,
such that both over-confidence and under-confidence are penalized. The second objective function promotes an inverse relationship between workload and performance. For example, as learners gain proficiency in executing the skill of landing the quadrotor, the cognitive workload should decrease. We take a similar approach to Eqn. (9) by taking a squared difference. Therefore, we define as
The third objective function aims to drive learners’ performance to exceed a desired proficiency threshold while maintaining consistent task execution across trials. It is defined using a rectified linear unit (ReLU) function-based penalty [33] for insufficient performance, combined with a variance penalty, given by
where denotes the desired performance threshold and controls the strength of the variance penalty. This formulation encourages learners to maintain both high and consistent performance.
Reward Weight Adaptation via Gradient Descent
Rather than minimizing a weighted sum of the objective functions directly, we first convert the functions into a target priority distribution over reward components. This formulation ensures that if a given objective is not being met, its reward weight is increased, thereby increasing the influence of under-satisfied tutoring objectives in subsequent policy optimization. Specifically, let denote the target reward-weight distribution induced by , , and , where
and is a parameter that controls how strongly the objective least satisfied dominates the target distribution. Under this construction, larger values of yield larger target weights , where denotes the tutoring objective. The learned reward weights are constrained to form a convex combination such that , , and are parameterized using a softmax transformation of unconstrained parameters given by .
To obtain interpretable reward-weight updates, we optimize so that it tracks the target distribution while remaining close to a prior reward-weight distribution . Therefore, the loss function used for optimizing the tutoring objective weights is
where denotes the Kullback–Leibler divergence [34] and controls the strength of regularization toward the prior distribution. This formulation serves two purposes. First, the term drives the learned reward weights toward the priority distribution, ensuring that tutoring objectives that have not been met receive greater emphasis. Second, the prior regularization term discourages abrupt shifts in reward priorities to promote continuity across learner interactions. Reward-weight adaptation is completed via gradient descent applied to the unconstrained parameter vector such that
where is the learning rate.
This procedure separates tutoring objective prioritization from policy optimization. Reward weights are first adapted using recent learner data, after which the resulting reward function is used within a model-based -iteration procedure to compute the optimal action-value function and the corresponding tutoring policy . Algorithm 1 depicts the pseudo-code of the entire process used to optimize tutoring objective weights.
Algorithm 1 Reward weight learning followed by -iteration
Require: Recent trial data ; reward components ; discount factor ; transition model ; prior weight vector ; learning rate , gradient steps ; performance threshold ; variance penalty ; temperature parameter ; KL regularization coefficient
Ensure: Learned weights , weighted reward , optimal action-value function , and policy
Compute empirical summaries:
Compute objective functions:
Construct target weight distribution:
Initialize parameters:
Initialize logits
for to do
end for
Construct weighted reward:
Run -iteration:
Initialize
for to do
for all do
end for
end for
return
Human User Study
With the proposed reward-weight adaptation algorithm and associated tutoring policy, we seek to answer the following research questions: (RQ1) Does the algorithm give rise to distinct patterns in the tutoring objective weights across all learners? (RQ2) Do the reward weights dynamically change to adapt to the learner’s needs? (RQ3) Is the personalization algorithm successful in prioritizing the tutoring objectives learners need? To do this, we design and conduct a user study in which participants learn to land the quadrotor in the 2D simulator while receiving tutoring assistance guided by Algorithm 1.
Experiment Procedure
Participants complete the psychomotor task described in § 2-A. A flowchart depicting the sequence of events in the experiment is shown in Figure 2. In the first two trials, participants complete the task manually and without formative feedback, providing a measurement of baseline manual performance. The next 30 trials are divided equally into three phases, where the initial position of the quadrotor changes in each phase; the sequence of the changes in position is randomized to remove ordering effects.

Figure 2. Experiment Flowchart
At the beginning of Phase 1, the reward weights are initialized uniformly for every participant, and the associated tutoring policy is synthesized. The participant then completes one practice trial in manual mode before the tutoring policy is turned on for the next 7 trials. During these 7 trials, the policy determines whether the automation assistance should be on or off () at each trial as well as what tone of feedback () should be used when providing formative feedback to the participant. Recall that after each trial , participants receive feedback on their previous landing attempt. This includes the time it took to land the quadrotor, the final speed and roll angle, and the achieved landing type. Additionally, the participants receive their total score out of 1000 points. Participants are then asked to self-report their confidence in achieving a safe landing with assistance together, as well as their perceived mental workload during the landing attempt. For the 9th and 10th trials of Phase 1, the participant again completes two trials in manual mode. At this stage, Algorithm 1 is executed to update the reward weights, after which a new tutoring policy is synthesized for each participant using the Phase 1 data. For this experiment, in Eqn. (11). This value is chosen based on the experimental design in which a safe landing, with a trajectory that differs substantially from expert behavior, yields at most a score of 750/1000. Therefore, a threshold of 850/1000 indicates that the learner’s trajectory is beginning to resemble that of a proficient operator. The scaling parameter of Eqn. (12) is set to 0.25 so that the least satisfied objective receives sufficient emphasis in the reward-weight vector. Parameter in Eqn. (13) is set to 0.01 to allow the reward weights to adapt more rapidly to each learner’s needs.
Phases 2 and 3 are conducted similarly to Phase 1, with the only change being the initial position of the quadrotor on the screen and the implemented policy being based on an updated reward-weight vector. After Phase 3 ends, the participant completes 5 trials in manual mode without formative feedback to provide a measure of their performance manually landing the quadrotor.
Participants
In total, 40 participants (10 male, 29 female, 1 non-binary) completed the user study with ages ranging from 18 to 70 years (, ). All participants were compensated $20/hr. The Institutional Review Board at Purdue University, USA, approved the study.
Apparatus
The quadrotor landing training simulator was developed on Python 3.12.0 using Pygame 2.6.1. The simulator was adapted for this study based on the platform described in [20] and uses a Thrustmaster T. Hotas 4 with joystick and throttle to control the quadrotor.
Data Collection
For each participant, the collected trial-by-trial data includes self-reported self-confidence, self-reported workload, and achieved total score. From this, we use data from the evaluation trials (the last five trials, all completed manually) to calculate the following metrics. Final performance is the mean score of the last five trials completed manually. Change in performance is the difference between the mean score of the last five trials and the mean score of the first two trials, which are all completed manually. The final self-reported self-confidence and workload are computed as the mean of the self-reports collected during these final five manual trials.
In addition to the trial-by-trial data, for each participant, the learned reward-weight vectors obtained after each phase are recorded. The reward-weight vector learned after Phase 1 and implemented during Phase 2 is denoted . Likewise, the reward-weight vectors learned after Phases 2 and 3 are denoted and , respectively.
Data Processing and Statistical Analysis
To identify patterns in the learned reward-weight vectors determined by Algorithm 1 across all participants, we apply -means clustering to the learned reward-weight vectors for obtained after phases 1-3. Recall that each vector represents the relative prioritization of the reward components, and each participant contributes three reward-weight vectors, one learned after each phase of the experiment. Changes in cluster assignment between consecutive phases are interpreted as shifts in the underlying tutoring strategy. The number of clusters is determined using silhouette analysis [35], with the elbow method [36] used for secondary verification. The silhouette method evaluates clustering quality by comparing, for each data point, its average distance to points within its assigned cluster against its distance to points in the nearest neighboring cluster. The resulting silhouette score lies in the range , where higher values indicate well-separated and cohesive clusters.
The optimal is selected based on the average silhouette score, with preference given to the smallest that achieves near-maximum performance (within the 95% threshold of the maximum performance) to balance clustering quality and model simplicity. The elbow method examines the within-cluster sum of squares (WCSS), defined as
as a function of , and identifies the point at which additional clusters yield diminishing reductions in variance. This “elbow” provides a structural indication of an appropriate number of clusters and is used to verify the consistency of the silhouette-based selection. The results are shown in Figure 3. Although the elbow method suggests clusters and the silhouette method suggests clusters, the discrepancy is limited to a single cluster, indicating that both methods capture a similar underlying structure. The resulting clusters are depicted in Figure 4.

Figure 3. (a) According to the silhouette tradeoff method, has the highest silhouette value while minimizing the number of clusters. (b) According to the elbow method, .

Figure 4. -means clustering results for the learned reward-weight vectors. Each reward-weight vector is defined by , where corresponds to the tutoring objective of self-confidence calibration, corresponds to workload calibration, and corresponds to progression through the learning stages. (a) Three-cluster solution selected using silhouette analysis. (b) Four-cluster solution obtained using the elbow method. Cluster 1 prioritizes self-confidence calibration, Cluster 2 prioritizes workload calibration, Cluster 3 prioritizes learning-stage progression, and the additional Cluster 4 in the four-cluster solution represents a balanced emphasis on self-confidence and workload calibration. The three-cluster solution was selected for subsequent analysis because it provides the best trade-off between cluster quality and model complexity.
We prioritize the silhouette-based selection, using clusters, as it directly quantifies clustering quality through intra-cluster cohesion and inter-cluster separation (as opposed to the elbow method, which provides a heuristic assessment based on variance reduction and may yield less distinct transition points). Cluster 1 represents prioritizing tutoring objective 1 of self-confidence calibration, cluster 2 represents prioritizing tutoring objective 2 of workload calibration, and cluster 3 represents prioritizing tutoring objective 3 of progressing in learning stage.
We leverage the resulting cluster centroids to determine the patterns of reward-weight vectors and categorize participants into one of four groups. From the 40 participants, nine participants have consistent reward-weight vectors that prioritize tutoring objective 1 (calibrating self-confidence). Eight participants have consistent reward-weight vectors that prioritize tutoring objective 2 (calibrating workload), and seven participants have consistent reward-weight vectors that prioritize tutoring objective 3 (rewarding forward progression in learning). Sixteen participants have reward-weight vectors that change such that the cluster to which those weights belong is different, indicating a fundamental shift in tutoring objective priority. For these participants, Table X in Appendix A lists the clusters that the reward-weight vectors and were classified into, as well as their mean score, self-reported self-confidence, and self-reported workload in the five final evaluation trials completed manually. For brevity, we will refer to these different groups as the SC group, W group, LS group, and change group, respectively.
| Part. | Mean Score | Mean | Mean | ||
| 1 | 1 | 2 | 937.6 | 72 | 80 |
| 5 | 1 | 2 | 786.4 | 67 | 43 |
| 7 | 2 | 1 | 872.0 | 89 | 6 |
| 10 | 3 | 2 | 888.6 | 57 | 54 |
| 12 | 3 | 2 | 574.0 | 71 | 83 |
| 16 | 1 | 2 | 920.2 | 85 | 54 |
| 18 | 1 | 2 | 941.8 | 81 | 34 |
| 19 | 1 | 2 | 933.8 | 89 | 44 |
| 23 | 3 | 2 | 921.0 | 80 | 80 |
| 24 | 3 | 1 | 733.6 | 49 | 20 |
| 25 | 1 | 2 | 925.8 | 86 | 30 |
| 28 | 1 | 2 | 801.4 | 73 | 80 |
| 30 | 3 | 2 | 924.6 | 75 | 64 |
| 32 | 3 | 2 | 780.0 | 42 | 79 |
| 33 | 3 | 1 | 294.8 | 10 | 90 |
| 39 | 1 | 2 | 794.6 | 87 | 35 |
Table X. Change group participants whose cluster assignment changed between phases 1 and 2. The provided information includes the assigned cluster for phase 1 and assigned cluster for phase 2 as well as the mean score, mean self-reported self-confidence, and mean self-reported workload over trials .
Using these cluster-based group definitions, we perform one-way ANOVA to determine whether final performance, change in performance, final self-reported self-confidence, and final self-reported workload differ across groups. The one-way ANOVA shown in Table I allows us to investigate whether there are statistically significant differences in metrics between groups of participants. This is followed by post hoc multiple-comparison tests to determine which groups differ from one another. As shown in Table I, there are significant differences
| Variable | -value | -value | ||
| Final Performance | 3 | 36 | 4.1286 | 0.0129 |
| Change in Performance | 3 | 36 | 0.9578 | 0.4231 |
| Final Self-Reported Self-Confidence | 3 | 36 | 6.0819 | 0.0019 |
| Final Self-Reported Workload | 3 | 36 | 0.2877 | 0.8339 |
Table I. SUMMARY OF ONE-WAY ANOVA RESULTS ACROSS OUTCOME VARIABLES.
in groups for the metrics of final performance and self-confidence. Therefore, post hoc multiple-comparison tests are conducted for both metrics. Tables II and Table III show the results of the multiple comparison test for final performance and final self-confidence, respectively. Each table lists the groups, their means , standard deviations , the mean difference between groups, the lower and upper confidence intervals (CI) of the differences between groups, and the significance of the comparison via -value.
| Group 1 | Group 2 | Mean diff. | Lower CI | Upper CI | -value | ||||
| Change | SC | 69.56 | 21.22 | 46.67 | 26.94 | 22.896 | -1.207 | 46.998 | 0.0677 |
| Change | W | 69.56 | 21.22 | 88.50 | 13.42 | -18.938 | -43.985 | 6.110 | 0.1939 |
| Change | LS | 69.56 | 21.22 | 55.29 | 21.55 | 14.277 | -11.937 | 40.490 | 0.4675 |
| SC | W | 46.67 | 26.94 | 88.50 | 13.42 | -41.833 | -69.941 | -13.725 | 0.0016 |
| SC | LS | 46.67 | 26.94 | 55.29 | 21.55 | -8.619 | -37.771 | 20.533 | 0.8556 |
| W | LS | 88.50 | 13.42 | 55.29 | 21.55 | 33.214 | 3.276 | 63.152 | 0.0248 |
Table III. Pairwise comparisons for self-confidence. Self-confidence is reported on a scale from 0 to 100.
For the reader’s convenience, we summarize the means and standard deviations of final performance, final self-reported self-confidence, and final self-reported workload in Table IV.
| Group | Mean Final Performance | Final Performance Standard Deviation | Mean of Final | Final Standard Deviation | Mean of Final | Final Standard Deviation |
| Change | 814.39 | 170.96 | 69.56 | 21.22 | 54.75 | 25.69 |
| SC | 793.27 | 107.95 | 46.67 | 26.94 | 52.00 | 24.92 |
| W | 933.95 | 35.24 | 88.50 | 13.42 | 45.75 | 20.70 |
| LS | 651.66 | 235.35 | 55.29 | 21.55 | 53.14 | 9.91 |
Table IV. Mean and standard deviations of final performance, self-reported self-confidence, and self-reported workload.
Results and Discussion
Here we answer and discuss the three research questions posed in this work.
RQ1: Does the algorithm give rise to distinct patterns in the tutoring objective weights across all learners?
Yes, distinct patterns in the tutoring objective weights across all learners are evident based on the clustering analysis presented in § 5. We first consider the case of three distinct clusters (Figure 4a). Cluster 1 represents prioritizing tutoring objective 1 of self-confidence calibration, cluster 2 represents prioritizing tutoring objective 2 of workload calibration, and cluster 3 corresponds to prioritizing tutoring objective 3, namely progression through the learning stages. In the case of four distinct clusters (Figure 4b), we observe the same reward-weight vector patterns as those resulting from , with the addition of a fourth cluster which seems to more equally weight tutoring objectives 1 and 2 of self-confidence and workload calibration, respectively. In other words, this fourth cluster prioritizes cognitive state calibration more generally, relative to helping the learner progress in the learning stage.
RQ2: Do the reward weights dynamically change to adapt to the learner’s needs?
Yes. For some participants, the reward-weight vector changes after Phase 1, resulting in a dynamically updated tutoring policy that adapts to the learner’s needs. A participant is considered to exhibit a change in reward weighting if the reward-weight vectors learned for two consecutive phases are assigned to different clusters, as shown in Table X in Appendix A. Most participants in the change group achieve a mean final evaluation score above 750/1000 among the last five trials. This is indicative of safe landings according to the score design in Eq. (1). The mean of the change group’s mean final evaluation scores among the last five trials is 814.39 with a standard deviation of 170.96 (Table II). Note that given the between-subjects design of the user study, we cannot explicitly determine whether a participant in the change group benefited from the updated tutoring policy relative to experiencing a static one. However, the fact that participants in the change group did not have statistically different learning outcomes compared to the other groups (see Tables II and Table III) suggests that these participants benefited from the personalization.
| Group 1 | Group 2 | Mean diff. | Lower CI | Upper CI | -value | ||||
| Change | SC | 814.39 | 170.96 | 793.27 | 107.95 | 21.121 | -153.605 | 195.847 | 0.9879 |
| Change | W | 814.39 | 170.96 | 933.95 | 35.24 | -119.562 | -301.143 | 62.018 | 0.3025 |
| Change | LS | 814.39 | 170.96 | 651.66 | 235.37 | 162.730 | -27.301 | 352.761 | 0.1155 |
| SC | W | 793.27 | 107.95 | 933.95 | 35.24 | -140.683 | -344.447 | 63.081 | 0.2633 |
| SC | LS | 793.27 | 107.95 | 651.66 | 235.37 | 141.610 | -69.719 | 352.939 | 0.2878 |
| W | LS | 933.95 | 35.24 | 651.66 | 235.37 | 282.293 | 65.262 | 499.323 | 0.0065 |
Table 2. Pairwise comparisons for final performance score. Score is defined in the range of 0 to 1000, and a score of 750 is required for a safe landing. A score of 850 is indicative that the learner’s trajectory is beginning to resemble that of a proficient operator.
RQ3: Is the proposed algorithm successful in prioritizing the tutoring objectives learners need?
The results suggest that the personalization algorithm successfully prioritizes tutoring objectives according to learner needs. This conclusion is supported by the statistical analysis described in § 5. First, we found no statistically significant difference among the groups for change in performance or final self-reported workload (Table I). Second, as shown in Tables II and Table III, pairwise comparisons between groups showed that with few exceptions, there is no statistically significant differences in final performance score or final self-reported self-confidence. This suggests that each learner benefited from the personalized tutoring policy they experienced.
To further examine whether the proposed algorithm adapted the tutoring objectives correctly to each learner, we consider the analysis summarized in Table IV. Notably, the learning stage (LS) group exhibited the lowest mean final performance and the highest standard deviation of final performance among all groups, as shown in Table IV. The mean final score of 651.66 is near the midpoint of the scoring scale (500/1000), and so the LS group’s mean final self-confidence (55.29) and mean workload (52.00) are calibrated to performance according to the designed objective functions described in Eqns. (9) and (10). This is consistent with the interpretation that the LS group received reward weights prioritizing forward progression in learning, because the learning stage tutoring objective was the least met of the three objectives.
In contrast, the learners whose reward-weight vectors were biased toward workload calibration achieved the highest mean final scores and the lowest standard deviation across all groups. The W group achieved a mean final score of 933.95/1000 with a standard deviation of 35.24, indicating that learners in the W group were most consistently able to achieve safe quadrotor landing trajectories consistent with learning stages 3 and 4, according to the score design in Eqn. (1). In fact, as shown in Table II, the significant pairwise comparison is between the W and LS groups, with the W group achieving a significantly higher mean final score than the LS group (the lowest and least consistent performing group). Consistent with achieving the highest performance and performance consistency in the evaluation trials, the W group also reported the highest mean and lowest standard deviation of final self-reported self-confidence across all groups, as shown in Table IV. According to Table III, the W group achieved significantly higher final self-reported self-confidence than both the self-confidence (SC) and LS groups. However, the objective defined in Eqn. (10) targets low workload as performance increases; the W group reported a mean final workload of 45.75/100 with a standard deviation of 20.70, which is elevated relative to their high performance. This miscalibration between workload and performance is consistent with the interpretation that the W group received a reward-weight vector prioritizing workload calibration throughout the experiment. Finally, despite achieving the second-highest mean final performance of 793.27/1000 with a standard deviation of 107.95, indicative of unsafe and safe landings categorized in learning stages 2 and 3, the SC group reported substantially lower self-confidence than expected under calibrated conditions. According to Eqn. (9), a calibrated self-confidence for this level of performance would be approximately 80.00/100; however, the SC group self-reported a mean self-confidence of only 46.67/100 with a standard deviation of 26.94, the highest standard deviation of self-reported self-confidence across all groups. For comparison, the W group, which achieved the highest mean final performance, reported a mean self-confidence of 88.50/100, closer to the value expected for the SC group given its performance level. As shown in Table III, the SC group self-reported significantly lower mean final self-confidence than the W group, further confirming this miscalibration. Because self-confidence was the least calibrated tutoring objective for this group, the SC group received a reward-weight vector prioritizing self-confidence calibration throughout the experiment.
Conclusion
In this paper, we presented and evaluated an algorithm for personalizing tutoring assistance to individual learners by dynamically updating the relative prioritization of tutoring objectives within a multi-input, multi-output cognitive control framework for psychomotor intelligent tutoring systems (ITSs). The three tutoring objectives considered are aligning self-confidence with performance, calibrating workload to performance, and promoting progression through psychomotor learning stages. We enable the tutoring agent to adapt to changes in a learner’s cognitive states and learning stage via an actor–critic-inspired architecture in which the reward weights serve as high-level actor parameters that shape policy optimization, while tutoring objective-specific functions computed from recent learner data provide a critic-like signal. Unlike classical actor–critic methods, the policy is not updated through online interaction but instead recomputed via model-based planning under the learned reward function. Moreover, the algorithm overcomes limitations posed by the data requirements of traditional actor–critic reinforcement learning by using gradient descent to optimize the reward weight parameters.
To evaluate the proposed adaptive tutoring algorithm, we designed and conducted a human user study. We showed, through -means clustering, that the personalization algorithm did result in distinct patterns in the tutoring objective weights across all learners. Using these clusters, we identified which learners’ reward-weight vectors changed sufficiently to move them to a different cluster after each phase of the study. Participants whose learned reward-weight vectors changed sufficiently to transition between clusters generally achieved higher final performance than participants whose reward-weight vectors remained within the same cluster, with the exception of the workload-calibration group.
This was followed by a statistical analysis, which showed that the personalization algorithm successfully prioritized tutoring objectives according to the observed needs of individual learners. A limitation of the results is the number of participants; despite collecting data from a total of 40 participants, we could not control how many people were categorized into each cluster (and hence the groups defined based on these clusters). As a result, some groups contained relatively few participants, with the smallest groups containing only seven participants.
Nevertheless, this work provides a computational framework and associated algorithms for achieving personalization during psychomotor learning and demonstrates, quantitatively, the benefits of this personalization. Possible directions for future work include a more extensive user study to further validate the efficacy of the proposed ITS, consideration of interdependencies between cognitive states and their influence on tutoring objectives, and personalization of the timescale on which reward weights are updated for individual learners.
Acknowledgments
This material is based upon work supported by the National Science Foundation under Award No. 1836952. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Appendix
Parameters for these self-confidence, workload, and learning stage MDP models were estimated by iteratively optimizing the initial and transition probabilities that maximize the likelihood of the observed state and action sequences. Optimization was performed with the genetic algorithm available in MATLAB’s Optimization Toolbox [37].
Both states and actions are recorded at each trial, and the likelihood was computed from the joint probability of the full state-action history through trial , . Applying the chain rule together with the Markov property [30, 31] reduces this joint probability to
This expression gives the likelihood of the observed trajectory up through the final trial , using the transition probabilities defined in Eqn. (6).
A 5-fold cross-validation is implemented to train the model. The dataset was split into five randomly assigned folds, with four folds used for training and the remaining fold for testing. The procedure is repeated so that each fold serves as the test set once. The selected model achieved the highest combined log-likelihood across both the training and test partitions.
For each model, the transition probabilities form a array. With three states and six actions, each MDP model yields a array encoding all 54 possible state transitions. As an illustration, Table V lists the transition probabilities .
| Actions States | ||||||
| 0.705 | 0.243 | 0.0528 | 0.781 | 0.153 | 0.0660 | |
| 0.207 | 0.632 | 0.161 | 0.176 | 0.725 | 0.0989 | |
| 0.121 | 0.110 | 0.769 | 0.115 | 0.244 | 0.642 |
Table V. Transitions probability where feedback tone is positive and assistance is both off (left) and on (right).
To confirm that the trained models were suitable for policy synthesis, their predictive accuracy was evaluated using multi-class receiver operating characteristic (ROC) analysis, comparing the predicted self-confidence, workload, and learning-stage classifications against participants’ self-reported cognitive states and observed learning stages. Additional details of the validation procedure are available in [10].
Each tutoring objective is associated with a reward function grounded in established learning-science literature. The reward functions are adapted from the formulation presented in [10] and are reproduced here for completeness.
Objective 1: The reward combines two equally weighted components: a calibration term based on the automation-assistance action, , and a calibration term based on the feedback-tone action, (Table VI). Both terms push the learner’s self-reported confidence state toward agreement with their actual learning stage. Consider as an example: a learner who reports high confidence while still at learning stage 1, a case of severe overconfidence, receives a reward of 0.60 for negatively toned feedback versus only 0.20 for positively toned feedback, reflecting that corrective tone is more appropriate in this case. Similarly, leaving automation off yields a higher reward than turning it on, such that the learner is shown that they perform worse without assistance. Table VII gives the resulting combined values for .
| 0.75 | 1.0 | 0.10 | 0.50 | 0.40 | ||
| 0.75 | 0.50 | 0.50 | 0.30 | 0.20 | ||
| 0.50 | 0.25 | 0.60 | 0.30 | 0.20 | ||
| 0.50 | 0.75 | 0.20 | 0.30 | 0.50 | ||
| 0.75 | 0.75 | 0.10 | 0.50 | 0.40 | ||
| 0.75 | 0.50 | 0.50 | 0.30 | 0.20 | ||
| 0.25 | 0.50 | 0.10 | 0.30 | 0.60 | ||
| 0.50 | 0.75 | 0.10 | 0.40 | 0.50 | ||
| 1.0 | 0.75 | 0.10 | 0.50 | 0.40 |
Table VI. Component reward functions and used to calibrate self-confidence to learning stage. Green highlighting indicates calibrated cases, while yellow and red highlighting indicate light and severe mis-calibration, respectively.
| 0.43 | 0.65 | 0.78 | 1.00 | 0.70 | 0.91 | |
| 0.78 | 0.57 | 0.61 | 0.39 | 0.52 | 0.30 | |
| 0.65 | 0.43 | 0.39 | 0.17 | 0.22 | 0.010 | |
| 0.30 | 0.52 | 0.39 | 0.61 | 0.57 | 0.78 | |
| 0.44 | 0.43 | 0.79 | 0.77 | 0.70 | 0.69 | |
| 0.78 | 0.57 | 0.61 | 0.39 | 0.52 | 0.30 | |
| 0.010 | 0.22 | 0.17 | 0.39 | 0.43 | 0.65 | |
| 0.22 | 0.43 | 0.48 | 0.70 | 0.57 | 0.78 | |
| 0.65 | 0.43 | 1.00 | 0.78 | 0.91 | 0.70 |
Table VII. Reward for calibrating self-confidence to learning stage. Green highlights represent cases of calibration, and yellow and red highlights represent cases of light and severe mis-calibration, respectively.
Objective 2: The reward encourages workload levels appropriate to the learner’s current stage. This is consistent with learning stage theory from [23], which states that task-related cognitive demand declines as expertise grows. This trend is consistent with the learning-stage characterization reported in [14]. Table VIII implements this logic. At learning stages 3 and 4, low workload paired with no assistance earns the maximum reward of 1.0, reflecting that proficient learners should be able to land the drone manually with minimal cognitive demand. Conversely, at learning stage 1, low workload also earns 1.0, since a novice attempting a manual landing should experience an elevated workload, and the reward is highest when the workload is appropriately challenging for their current level. When the learner’s workload is high, assistance is rewarded more than its absence, since the learner likely lacks the spare mental capacity to manage the task unaided. Only the assistance action enters , since assistance is provided during the trial, whereas feedback is delivered only after the attempt.
| ASSIST | |||
| 1.0 | 0.010 | ||
| 0.60 | 0.40 | ||
| 0.30 | 0.70 | ||
| 0.60 | 0.40 | ||
| 0.45 | 0.55 | ||
| 0.40 | 0.60 | ||
| 1.0 | 0.010 | ||
| 0.70 | 0.30 | ||
| 0.40 | 0.60 |
Table VIII. Reward for calibrating workload. Green highlighting indicates calibrated cases, while yellow and red highlighting indicate light and severe mis-calibration, respectively.
Objective 3: The reward incentivizes progression through the learning stages, as shown in Table IX. With automation off, reaching learning stage 3 or 4 earns the maximum reward of 1.0, with progressively lower rewards assigned for stages 1 and 2, encouraging the learner to advance toward expertise in manual mode. With automation on, the pattern inverts: learning stage 1 earns the maximum reward of 1.0, reflecting that a learner at this stage is most likely to benefit from assistance during the trial, while rewards decline across stages 2 through 4 as the need for assistance diminishes. As with Objective 2, only the assistance action enters , since it directly shapes the learner’s performance within each trial.
| ASSIST | |||
| 0.10 | 1.0 | ||
| 0.55 | 0.45 | ||
| 1.0 | 0.25 |
Table IX. Reward for advancing learning stage.
All reward function components are designed to fall within the range [0, 1].
Table X summarizes the mean self-reported self-confidence, and mean self-reported workload over the final evaluation trials for change group participants in Table X.
References
- [1]B. P. Woolf, Building intelligent interactive tutors: Student-centered strategies for revolutionizing e-Learning. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2008.
- [2]S. Doroudi, “Mastery Learning Heuristics and Their Hidden Models,” in Artificial Intelligence in Education, ser. Lecture Notes in Computer Science, I. I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, and E. Millán, Eds. Cham: Springer International Publishing, 2020, pp. 86–91.DOI
- [3]E. Mousavinasab, N. Zarifsanayey, S. R. Niakan Kalhori, M. Rakhshan, L. Keikha, and M. Ghazi Saeedi, “Intelligent tutoring systems: a systematic review of characteristics, applications, and evaluation methods,” Interactive Learning Environments, vol. 29, no. 1, pp. 142–163, jan 2021.
- [4]A. Bandura, Self-Efficacy: The Exercise of Control. New York, NY: H. Freeman, 1997.
- [5]J. Laine, T. Lindqvist, T. Korhonen, and K. Hakkarainen, “Systematic Review of Intelligent Tutoring Systems for Hard Skills Training in Virtual Reality Environments,” International Journal of Technology in Education and Science, vol. 6, no. 2, pp. 178–203, may 2022.DOI
- [6]R. A. Sottilare and J. J. LaViola, “Extending Intelligent Tutoring beyond the desktop to the psychomotor domain,” in Proceedings of the 2015 Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC). Arlington, VA: National Training and Simulation Association, 2015.
- [7]O. C. Santos, “Training the Body: The Potential of AIED to Support Personalized Motor Skills Learning,” International Journal of Artificial Intelligence in Education, vol. 26, no. 2, pp. 730–755, jun 2016.DOI
- [8]L.-M. Neagu, E. Rigaud, M. Dascalu, and S. Travadel, “Selfit v2 – Challenges Encountered in Building a Psychomotor Intelligent Tutoring System,” in Intelligent Tutoring Systems, S. Crossley and E. Popescu, Eds. Cham: Springer International Publishing, 2022, pp. 350–361.DOI
- [9]E. Lameier, L. Reinerman-Jones, G. Matthews, E. Biddle, and M. Boyce, “The Motivational Assessment Tool (MAT) Development and Validation Study,” in Augmented Cognition: Intelligent Technologies, D. D. Schmorrow and C. M. Fidopiastis, Eds. Cham: Springer International Publishing, 2018, pp. 46–58.DOI
- [10]M. S. Yuh and N. Jain, “Designing Cognitively-Aware Psychomotor Intelligent Tutoring Systems via Multi-Objective and Human-Centric Optimization,” in 2025 34th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). Eindhoven, Netherlands: IEEE, aug 2025, pp. 35–42.DOI
- [11]V. Riley, “Operator reliance on automation: Theory and data,” in Automation and human performance: Theory and applications, 1st ed., R. Parasuraman and M. Mouloua, Eds. Mahwah, NJ: CRC Press, jun 1996, pp. 19–35.
- [12]R. Parasuraman and M. Mouloua, Eds., Automation and human performance: Theory and applications, ser. Automation and human performance: Theory and applications. Hillsdale, NJ, US: Lawrence Erlbaum Associates, Inc, 1996.
- [13]R. Parasuraman and V. Riley, “Humans and Automation: Use, Misuse, Disuse, Abuse,” Human Factors: The Journal of the Human Factors and Ergonomics Society, vol. 39, no. 2, pp. 230–253, jun 1997.
- [14]M. S. Yuh, K. R. Ortiz, K. S. Sommer-Kohort, M. Oishi, and N. Jain, “Classification of Human Learning Stages via Kernel Distribution Embeddings,” IEEE Open Journal of Control Systems, vol. 3, pp. 102–117, 2024.DOI
- [15]A. K. Erümit and İ. Çetin, “Design framework of adaptive intelligent tutoring systems,” Education and Information Technologies, vol. 25, no. 5, pp. 4477–4500, sep 2020, accessed: 2026-06-09.DOI
- [16]V. J. Shute and J. P. Psotka, “Intelligent Tutoring Systems: Past, present, and future,” Defense Technical Information Center, Fort Belvoir, VA, Tech. Rep., may 1994.DOI
- [17]L.-M. Neagu, E. Rigaud, V. Guarnieri, S. Travadel, and M. Dascalu, “Selfit – An Intelligent Tutoring System for Psychomotor Development,” in Intelligent Tutoring Systems, ser. Lecture Notes in Computer Science, A. I. Cristea and C. Troussas, Eds. Cham: Springer International Publishing, 2021, pp. 291–295.DOI
- [18]P. Phobun and J. Vichianpanya, “Adaptive intelligent tutoring systems for e-learning systems,” Procedia - Social and Behavioral Sciences, vol. 2, no. 2, pp. 4064–4069, jan 2010.DOI
- [19]M. Yuh, “Designing Cognitively Aware Intelligent Tutoring Systems via Model-Based Cognitive Control,” s 2026.DOI
- [20]S. Byeon, D. Sun, and I. Hwang, “Skill-level-based Hybrid Shared Control for Human-Automation Systems,” in 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Melbourne, Australia: IEEE, oct 2021, pp. 1507–1512.DOI
- [21]OpenAI, “ChatGPT,” mar 2025. [Online]. Available: https://chatgpt.com
- [22]E. Jensen, S. Sankaranarayanan, and B. Hayes, “Automated Assessment and Adaptive Multimodal Formative Feedback Improves Psychomotor Skills Training Outcomes in Quadrotor Teleoperation,” in Proceedings of the 12th International Conference on Human-Agent Interaction, ser. HAI ’24. New York, NY, USA: Association for Computing Machinery, nov 2024, pp. 185–194.arxiv.org/abs/2405.15982
- [23]S. E. Dreyfus, “The Five-Stage Model of Adult Skill Acquisition,” Bulletin of Science, Technology & Society, vol. 24, no. 3, pp. 177–181, jun 2004.DOI
- [24]A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola, “A kernel two-sample test,” J. Mach. Learn. Res., vol. 13, no. 1, p. 723–773, Mar. 2012.DOI
- [25]R. Yerkes and J. Dodson, “The Relation of Strength of Stimulus to Rapidity of Habit Formation,” Journal of Comparative Neurology & Psychology, vol. 18, pp. 459–482, 1908.
- [26]R. Bénabou and J. Tirole, “Self-Confidence and Personal Motivation,” in Psychology, Rationality and Economic Behaviour: Challenging Standard Assumptions, ser. International Economic Association Series, B. Agarwal and A. Vercelli, Eds. London: Palgrave Macmillan UK, 2005, pp. 19–57.
- [27]I.-C. Hung, K.-J. Chao, L. Lee, and N.-S. Chen, “Designing a robot teaching assistant for enhancing and sustaining learning motivation,” Interactive Learning Environments, vol. 21, no. 2, pp. 156–171, apr 2013.DOI
- [28]J. M. Keller, “Development and use of the ARCS model of instructional design,” Journal of instructional development, vol. 10, no. 3, pp. 2–10, sep 1987.DOI
- [29]A. Bandura, “Self-efficacy: Toward a unifying theory of behavioral change,” Psychological Review, vol. 84, no. 2, pp. 191–215, 1977.
- [30]R. S. Sutton and A. G. Barto, Reinforcement learning: an introduction, ser. Adaptive computation and machine learning. Cambridge, Mass: MIT Press, 1998.DOI
- [31]A. Adhikari and J. Pitman, Data 140: Probability for Data Science. Berkeley, CA: Data Science Education Program, University of California, Berkeley, 2022.
- [32]V. Konda and J. Tsitsiklis, “Actor-critic algorithms,” in Advances in Neural Information Processing Systems, S. Solla, T. Leen, and K. Müller, Eds., vol. 12. MIT Press, 1999.
- [33]J. Brownlee, “A Gentle Introduction to the Rectified Linear Unit (ReLU),” jan 2019.
- [34]S. Kullback and R. A. Leibler, “On Information and Sufficiency,” The Annals of Mathematical Statistics, vol. 22, no. 1, pp. 79–86, mar 1951.DOI
- [35]P. J. Rousseeuw, “Silhouettes: A graphical aid to the interpretation and validation of cluster analysis,” Journal of Computational and Applied Mathematics, vol. 20, pp. 53–65, nov 1987.DOI
- [36]GeeksforGeeks, “Elbow Method for optimal value of K in KMeans,” 2026. [Online]. Available: https://www.geeksforgeeks.org/machine-learning/elbow-method-for-optimal-value-of-k-in-kmeans/
- [37]MathWorks, “MATLAB optimization toolbox,” Natick, MA, 2026. [Online]. Available: https://www.mathworks.com/products/optimization.html