betaFeedback

WorksComputer Science

Dynamic Learning and Productivity for Data Analysts: A Bayesian Hidden Markov Model Perspective

Abstract

Data analysts are essential in organizations, transforming raw data into insights that drive decision-making and strategy. This study explores how analysts' productivity evolves on a collaborative platform, focusing on two key learning activities: writing queries and viewing peer queries. While traditional research often assumes static models, where performance improves steadily with cumulative learning, such models fail to capture the dynamic nature of real-world learning. To address this, we propose a Hidden Markov Model (HMM) that tracks how analysts transition between distinct learning states based on their participation in these activities. Using an industry dataset with 2,001 analysts and 79,797 queries, this study identifies three learning states: novice, intermediate, and advanced. Productivity increases as analysts advance to higher states, reflecting the cumulative benefits of learning. Writing queries benefits analysts across all states, with the largest gains observed for novices. Viewing peer queries supports novices but may hinder analysts in higher states due to cognitive overload or inefficiencies. Transitions between states are also uneven, with progression from intermediate to advanced being particularly challenging. This study advances understanding of into dynamic learning behavior of knowledge worker and offers practical implications for designing systems, optimizing training, enabling personalized learning, and fostering effective knowledge sharing.

Introduction

Context

Data analysts are vital to modern organizations, transforming raw data into actionable insights that inform decision-making. Their contributions enable businesses to identify trends, understand customer preferences, and improve operational efficiency. For example, retail companies rely on analysts to determine top-selling products during holiday seasons or forecast future consumer behavior. Similarly, social media platforms depend on analysts to identify the types of content that generate the highest user engagement. These insights are critical for shaping strategies that help organizations remain competitive in fast-paced markets.

A key aspect of a data analyst’s work is writing and executing SQL (Structured Query Language) queries. These queries allow analysts to extract and organize data from large databases, perform calculations, and generate reports. For instance, an analyst might write a query to identify the ten most searched items on an e-commerce website during a given month or calculate the average time users spend browsing certain product categories. The ability to create efficient queries is essential for addressing business questions quickly and accurately. The productivity of data analysts directly influences organizational performance. Faster and more accurate query writing leads to quicker insights, which are particularly valuable for time-sensitive decisions such as marketing adjustments during a product launch or supply chain realignments.

Developing proficiency in query writing is not a straightforward process. It is a learned skill that requires mastering technical tools, understanding database structures, and refining problem-solving strategies. Organizational learning theory defines learning as a change in knowledge that occurs through experience at the individual, group, or organizational level [1, 2]. Much of the early research on learning focused on manufacturing and service industries [3, 4], but more recent studies emphasize knowledge-intensive fields such as IT consulting and software development [5, 6, 7].

Our empirical setting is a collaborative platform where data analysts access large-scale data, write and execute queries, and view queries created by their peers. This platform facilitates both independent problem-solving and collaborative knowledge sharing. For example, an analyst tasked with identifying e-commerce trends might review peer queries to understand alternative approaches, adapt existing solutions, or gain inspiration for tackling new challenges. These collaborative features foster a rich environment for both individual and collective learning, enhancing query-writing efficiency and accuracy.

This study focuses on two key modes of learning for data analysts: writing queries (direct experience) and viewing peer queries (indirect experience). Writing queries exemplifies ”learning-by-doing,” where individuals gain knowledge through hands-on experience. The concept of learning-by-doing has been extensively studied in individual learning curves. [8] demonstrated how repetition improves memory retention, while Wright [9] observed cost reductions in airframe production through cumulative experience, formalizing the idea of the learning curve [10]. More recent studies confirm the value of direct experience across industries. For example, software developers improve productivity through specialized tasks [5], and IT consultants enhance problem resolution times through experience [6]. Evidence from healthcare and open-source software projects further underscores the benefits of cumulative experience [11, 12, 13].

Indirect experience, represented by viewing peer queries, offers complementary benefits. Analysts gain insights by observing how colleagues approach problems, adapt solutions, or solve complex tasks. Research on organizational learning highlights the value of shared knowledge and collaborative interactions [14, 15]. The situated perspective emphasizes that learning is inherently social and interactive [16]. Empirical evidence supports the importance of peer-based learning across contexts. For instance, repair technicians share practical knowledge through storytelling [17], and programmers leverage online platforms like Stack Overflow to acquire new skills and address technical challenges efficiently [18]. However, the benefits of indirect experience are not universal. For example, [19] found no evidence of peer effects on researcher productivity in certain scientific fields.

Combining direct and indirect experience has been shown to enhance learning outcomes in diverse settings, from manufacturing to collaborative software development [20, 21]. For data analysts, understanding the interplay between these two modes of learning is essential for designing effective training programs and fostering productive collaboration. This study contributes to this understanding by examining how these learning modes shape productivity within a collaborative platform.

Research Gap and Motivation

Most studies on learning assume static models where learning happens at a constant rate over time. These models also assume that an individual’s ability, which affects performance, remains stable. However, these assumptions do not reflect the reality of learning processes. Individuals learn at different rates, and even the same person’s learning rate can change over time. As experience builds up, it contributes to ”human capital,” which can enhance an individual’s ability to learn and apply new knowledge in the future [22].

The concept of absorptive ability, introduced by [22], emphasizes the importance of prior knowledge in acquiring and using new information. Studies in cognitive and behavioral science have shown that past experiences influence how individuals approach new learning tasks [23, 24, 25]. Static models fail to capture these dynamics, leading to biased results when evaluating learning activities. For example, [26] pointed out the reflection problem, where static models mistakenly attribute individual performance to aggregated group effects. This issue can misrepresent the actual drivers of learning and productivity.

Dynamic models provide a better way to study learning behaviors over time. For instance, [27] examined how peer-based learning accumulates and changes the productivity of salespeople. Similarly, [13] developed a stochastic model to study how developers in open-source projects improve their skills. These studies highlight the value of dynamic models, but there is limited research on how such models can be applied to individual learning in knowledge-intensive fields like data analytics.

This study addresses this gap by proposing a Hidden Markov Model (HMM) to capture the dynamic learning processes of data analysts. The HMM framework models how analysts’ productivity and learning ability change over time as they engage in different learning activities. These activities include writing queries (learning from direct experience) and viewing peer queries (learning from indirect experience), which together build their knowledge and expertise. The model represents each data analyst as transitioning between a finite set of states, where each state reflects a specific level of productivity and learning ability. Transitions between states depend on participation in learning activities. Additionally, the model accounts for individual differences in learning through random effects, capturing unobserved factors like personal preferences or past experiences. To estimate the model, we use a hierarchical Bayesian approach with Markov Chain Monte Carlo (MCMC) techniques. This approach allows us to analyze variations across individuals and provides reliable estimates by combining prior knowledge with observed data. By applying this model to data on query-writing activities, we aim to reveal patterns in learning and productivity, offering new insights into dynamic learning processes.

Previous research [28] investigated the effects of learning-by-doing and learning-by-viewing on data analyst productivity, with a focus on how peer characteristics, such as productivity and popularity, influenced the effectiveness of indirect peer learning. While insightful, this earlier work relied on a static framework that assumed constant learning effects over time. Building on the same empirical setting, this study adopts a dynamic perspective to examine how analysts transition between distinct learning states. By employing a HMM framework, it captures the evolution of learning behaviors and provides a deeper understanding of the relationship between productivity, learning activities, and state transitions. This approach addresses the limitations of static models and offers novel insights into the dynamics of learning processes in knowledge-intensive environments.

Key Results and Contribution

The key findings from our analysis are summarized below:

  1. Dynamic Learning States and Productivity Patterns The HMM identified three distinct learning states: novice, intermediate, and advanced. Analysts in higher states demonstrated greater intrinsic productivity, as reflected by reduced query completion times. The results confirm that learning is not static but evolves dynamically as analysts participate in different activities.

  1. Differential Effects of Learning Activities The effects of learning activities—writing queries (learning by doing) and viewing peer queries (learning by observing)—varied significantly across learning states. Writing queries positively influenced transitions to higher states for both novice and intermediate analysts, while viewing peer queries helped novices but hindered analysts in intermediate and advanced states. This highlights the importance of tailoring learning activities to the current state of the analyst.

  1. Learning State Transition Dynamics The model revealed that it is significantly harder for analysts to move from the intermediate to the advanced state than from novice to intermediate. This finding underscores the diminishing returns of learning activities at higher levels of expertise and suggests a need for tailored strategies to sustain learning momentum in advanced analysts.

The contribution of this study is three-fold:

  1. This research advances the understanding of learning dynamics by introducing a dynamic framework that incorporates state transitions and heterogeneous effects of learning activities. Unlike static models, the proposed HMM captures the evolving nature of productivity and learning, shedding light on how prior knowledge influences future learning behaviors. This contributes to the broader literature on organizational learning and individual skill development.

  1. The study employs an innovative Hidden Markov Model with hierarchical Bayesian estimation, enabling the analysis of individual-level learning trajectories. By integrating random effects, the model accounts for unobserved heterogeneity, providing robust and personalized insights. The methodological approach offers a new way to study learning dynamics in knowledge-intensive tasks, which can be adapted to other domains.

  1. The findings provide actionable insights for designing adaptive systems and interfaces in collaborative platforms. By identifying learning states and tailoring activities accordingly, platforms can optimize both individual productivity and learning outcomes. For example, the results inform strategies for designing onboarding programs, personalized learning paths, and feedback mechanisms that support continuous development. These insights are particularly

valuable for managers and system designers aiming to enhance productivity in knowledge-driven organizations.

The remainder of this paper is structured as follows. Section 2 presents the materials and methods, detailing the empirical setting, data sources, and measurement approaches, including the dependent and related variables. This section also provides a comprehensive introduction to the HMM framework, covering its theoretical foundation, model development, and estimation strategy. Section 3 reports the key results, highlighting the findings on learning dynamics and productivity patterns. Finally, Section 4 discusses the implications of these findings, outlines limitations, and suggests directions for future research. The disclosure and data availability statements are shared in section 5 and section 6.

Material and Methods

Empirical Setting

This study investigates the behavior of data analysts engaging with a collaborative data platform widely adopted in industry for managing large-scale analytics workflows. The platform serves as an integrated environment for data analysts, providing functionalities for query composition, execution, and knowledge sharing. First, the platform acts as a centralized repository for metadata in analytics data warehouses, enabling seamless access to data from various services, such as relational databases (Teradata, MySQL, SQL Server, etc) and visualization tools like Tableau. Analysts with appropriate permissions can access datasets and integrate these tools into their workflows efficiently to produce comprehensible results. Second, the platform supports a vibrant collaborative ecosystem beyond data access and analysis. A data analyst can share her knowledge with the community by publishing her queries, writing articles about her good practices or participating in conversations on technical issues. A data analyst can also seek knowledge from the community by viewing queries authored and published by other peers, searching for relevant articles or asking for help in the conversation board. By fostering collaboration and knowledge sharing, the platform facilitates organizational learning and innovation. It supports diverse activities, including expertise location, resource reuse, and social networking among analysts [29, 30, 31, 32]. These features make it a representative example of modern collaborative technologies in large-scale industrial settings, enabling both novice and expert analysts to improve their productivity and skills.

Data and Measures

Dataset

The raw dataset for this study comprises two primary sources:

  1. Usage Data: Backend logs automatically collected from a collaborative data platform, capturing analysts’ interactions over a four-year period from January 2014 to March 2018. All user identifiers were hashed to ensure privacy. The dataset includes:

    • Query Records: Metadata for all queries, such as query ID, title, author (hashed user ID), time of creation, brief descriptions, and publication sta-

tus.

  • Execution Records: Logs of query executions, detailing query ID, hashed user ID, execution timestamps, and query length.

  • Viewing Records: Comprehensive records of user interactions, documenting views of query pages on the digital platform.

(2) Employee Information: Organizational data available through the public company hub, including hashed user ID and the corresponding tenure, title, and hierarchical position. No additional personal identifiers were accessed or used in the study.

The final dataset was curated with the following criteria:

  • Inclusion Criteria: Analysts who authored at least one query on the platform during the study period were included.

  • Exclusion Criteria: We then excluded 1) platform administrators and admin employees of the platform vendor to avoid skewing the analysis with non-representative user behaviors; 2) Analysts with missing organizational data (e.g., due to employment termination during the study period) were also removed.

After applying the inclusion and exclusion criteria, the dataset initially comprised 2,059 analysts. These analysts were categorized by organizational levels based on their reporting structure, ranging from entry-level employees to senior staff. To ensure representativeness, extremely senior or junior analysts were excluded, resulting in a final sample of 2,027 active analysts who collectively authored 101,327 queries during the study period. Queries that were never executed or contained incomplete metadata were subsequently removed, leaving a final dataset of 79,797 queries written by 2,001 data analysts. Notably, only about one in eight queries was ever viewed by an analyst other than the original author, with 10,049 (approximately 1 in 8) queries authored by 1,097 analysts being reused by peers.

Variables

2.2.2.1. Dependent variable. To develop a productivity measure for data analysts in our study, we borrow the concept of Pre-alpha phase from the software development life cycle [33, 34, 35].The development process consists of several phases, as summarized in prior research [33, 36]. During the Pre-alpha phase, developers focus on writing preliminary source code, a critical step that takes place before Alpha testing. We define FirstCompletionTimei,kFirstCompletionTime_{i,k} as the time interval between the point when the data analyst ii clicks the button to create an empty query kk and the point when she executes this query for the first time:

FirstCompletionTimeanalyst i, query k=Timestampi first executesk−Timestampi createsk(1)FirstCompletionTime_{\mathrm{analyst}\ i,\ \mathrm{query}\ k} = Timestamp^{k}_{i\ \mathrm{first\ executes}} - Timestamp^{k}_{i\ \mathrm{creates}} \tag*{(1)}

This time interval, comparable to the Pre-alpha phase in software development, characterizes the time that a data analyst spends to shape her idea into a testable, prototype query. The longer such time interval is, the later this analyst can proceed the following steps. We, therefore, use FirstCompletionTimei,kFirstCompletionTime_{i,k} as a proxy for data analyst productivity. These query writing start time-stamps and the first execution time-stamps are automatically logged at the back end of the collaborative data platform. We believe that the data analysts in our study cannot manipulate this data directly nor have no incentive to act strategically, as only the administrators of the collaborative data platform are informed and have access to these data.

We also include other variables related with analyst’s participation in learning activities (i.e. Learning activitiesit_{it} and variables related with analyst’s productivity in writing query (i.e. ZitkZ_{itk} in Equation (11)).

Variables Related with Learning Activities.
  • Number of Queries Writtenit_{it}: this is defined as the number of queries that a data analyst ii has written by herself on the collaborative data platform during period tt. The mean of Number of Queries Writtenit_{it} is 0.782 and the standard deviation is 3.82.

  • Number of Queries Viewedit_{it}: this is defined as the number of queries that a data analyst ii has viewed from her peers on the collaborative data platform during period tt. The mean of Number of Queries Viewedit_{it} is 0.306 and the standard deviation is 2.03.

Variables Related with Productivity in Writing Queries.

Prior work suggests that individual adeptness, multi-tasking and task complexity are likely to affect productivity. A more adept data analyst may write a query faster; a data analyst who has piles of work may become less productive because of stress and pressure; a data analyst may spend more time on writing a complex query that consists of many statements. We incorporate the variables to calculate the probabilities of state-dependent productivity in writing queries. We explain the definitions of these control variables for the scenario in which the analyst ii creates and first executes query kk.

  • Workloadi,k_{i,k}: This is the average number of queries that analyst ii composes simultaneously together with the focal query kk during the FirstCompletionTimei,k_{i,k}. This definition is inspired by the workload developed in [37] for restaurant workers. For example, suppose FirstCompletionTimei,k_{i,k} lasts 40 minutes. During this period, the author data analyst only creates another query that overlaps with the focal query kk for 20 minutes. The Workloadi,k_{i,k}, therefore, is (40 min + 20 min)/(40 min) = 1.5 queries.

  • Query Sizek_k: This is the number of statements that were executed in the first execution of query kk.

  • Saved Queryk_k: This is a binary variable that indicates whether query kk has been saved.

  • Migrated Queryk_k: This is a binary variable that indicates whether part of query kk was migrated from a different platform. We acquire such information by parsing the title or description of query kk.

  • Tenure on collaborative data platformi,k_{i,k}: This is the number of months between the date when the author analyst ii joined the collaborative data platform and the time she creates query kk.

The descriptive statistics for these variables can be found in Table 1. In addition to these variables, we include a state-specific constant in ZitkZ_{itk}. This constant term captures a state-specific fixed effect in query-writing productivity, which can be interpreted as the intrinsic productivity in writing queries for an average analyst in the corresponding state.

StatisticMeanSt. Dev.MinPctl(25)Pctl(75)Max
FirstCompletionTime90,7111,200,93512026283,798,568
Aggregate Direct Experience116.2200.20161261,775
Aggregate Indirect Experience35.970.801351,510
Direct Experience with Different Databases52.4124.802491,389
Direct Experience with the focal Database63.911306681,092
Workload1.10.711128
Tenure on collaborative data platform14.913.1042450
Saved Query0.30.50011
Migrated Query0.010.10001
Query Size4.6117.611118,095

Table 1. Descriptive statistics of the raw data capturing N=79797N = 79797 queries written and executed by 2001 data analysts at the organization during 2014-2018.

Hidden Markov Model for Data Analysts Learning

Theory

The latent nature and structure of such dynamic learnings make them very difficult to capture. To analyze the learning dynamics of data analysts in our setting, we propose a Hidden Markov Model (HMM) similar to that in [13]. The HMM is a model of bivariate Markov process {St,Ot}t=1∞\{S_t,O_t\}_{t=1}^{\infty} where {St}t=1∞\{S_t\}_{t=1}^{\infty} is a Markov process that consists of a finite state space and {Ot}t=1∞\{O_t\}_{t=1}^{\infty} is a sequence of observations that is determined by a state-specific stochastic process. Generally {St}t=1∞\{S_t\}_{t=1}^{\infty} is assumed not directly observable (hidden). The possibility of modeling complex structures of {Ot}t=1∞\{O_t\}_{t=1}^{\infty} through a simple formulation makes HMMs attractive and widely applied in areas ranging from bioinformatics [38], ecology [39] and criminology, to speech recognition [40] and transportation [41, 42]. They are also found to be very important in econometrics and macro-economics. For example, [43] proposed an HMM as a very tractable approach to modeling unobservable regime shifts and to estimating the impact of such shifts on the level of U.S. real gross national product. Another well-known tool in the quantitative finance literature with essentially the same structure of HMM is the Kalman filter [44, 45].

The HMM model described in this section is an individual-level model of query writing and learning behaviors. In our HMM, we characterize these hidden states with respect to the latent productivity and learning ability of data analysts, or absorptive ability in [22]’s term. Such latent characteristics can include programming skills, knowledge on the databases and other factors that may influence analyst productivity and subsequent learning. At any point of time, a data analyst resides in only one state. The transitions between states over a period are affected by her participation in learning activities. Such activities can include the data analyst’s writing her own queries (i.e. learning by doing) and her viewing on queries written by other analysts (i.e. learning by viewing). The relationship of a state with a unique productivity behavior in writing query, along with the Markovian transitions between states allow us to capture data analyst’s productivity and learning dynamics.

To do so, we use a nonhomogeneous HMM [46] in which the Markovian transitions are a function of time-varying variables. This is important for us to understand the drivers of the dynamics rather than merely building a model that fits the dynamics in the data. Specifically, we first assume that a data analyst’s probability for moving from one state to another is determined by her observed participation in learning activities as well as the unobserved individual random effects. Second, the observation in our HMM is the productivity of individual analyst in writing a query in different periods of time. The query-specific productivity is a function of analyst’s intrinsic productivity, observed characteristics of the query, and unobserved individual random effects. Adding random effects to both parts of the model grants us the possibility to capture the heterogeneity of analysts’ performance in both learning and writing queries.

Figure 1 shows how our HMM relates the transitions between latent states to observed dynamics in productivity. Specifically, in period tt, an analyst probabilistically resides in one of total NsN_s states based on her accumulated learning stock. Each realized state corresponds to a unique stochastic process that characterizes the analyst’s productivity in writing queries in period tt. Her participation in learning activities during period tt determines whether she will move up to a higher state, stay where she is, or move down to a lower state in the next period.

Image not extracted

Figure 1. A Hidden Markov Model of Data Analyst Learning and Writing Query

This model has several advantages compared with the standard learning curve model. First, it allows us to capture the learning dynamics of a data analyst by investigating the differential impact of learning activities on her transitions between different hidden states, conditional on the state she currently resides. Second, this model also provides us a dynamic segmentation of data analysts based on their time-varying states, whereas an ad-hoc specification of ss is usually added to an otherwise static model. Third, the use of state and the inherent dependency between serial states within the model enable us to fully account the effect of autocorrelation. Autocorrelation in our setting may occur when an unobservable shock (e.g. unexpected illness of the individual analyst) commonly affects serial or multiple query-writing observations. This unobservable shock, if only exists in one period, can be captured by the state-specific query-writing process that is common to all observations in that period. If the shock spans across two serial periods, the state that corresponds to the previous period can carry on the effect of the unobservable shock to the current state thanks to the Markovian dependency between serial states. Fourth, in this model the current query-writing behavior only depends on the lag participation in learning activities. By doing so it diminishes reverse causality between learning and change in productivity. An analyst may view many queries from peers because she needs to write an extremely complex query. Such reverse causality may lead to erroneous estimate of learning effects in the classical static model.

Model Development

To analyze the learning dynamics and productivity of data analysts, we developed a Hidden Markov Model (HMM). This framework is designed to capture the evolution of an analyst’s latent learning states over time and link these hidden states to their observable query-writing productivity. In essence, the model enables us to infer the unseen “learning journey” of data analysts and connect it to measurable outcomes.

In our HMM, the latent states represent distinct levels of productivity and learning ability, ranging from novices (state 1) to highly proficient analysts (state NsN_s). These states are ordered to reflect progression as analysts accumulate skills and knowledge over time. The entire time horizon is divided into a finite number of periods, t∈{1,2,...,T}t \in\{1, 2, ..., T\}. At the beginning of any given period, a data analyst resides in a latent state. This state determines her productivity level for that period, and her participation in learning activities influences her transition to another state (or staying in the same state) in the next period. For each data analyst ii we observe a sequence of query-writing productivity outcomes over time, denoted as Oi={O1i,O2i,...,OTi}O^i = \{O^i_1,O^i_2,...,O^i_T\}, where OtiO^i_t represents the observed productivity in period tt. These observed outcomes are assumed to be influenced by the hidden states and transitions are estimated. Below, we provide an overview of the key components of modeling framework, with detailed technical specifications available in the appendix.

The State Transition Probabilities

The transitions between latent learning states are modeled as a Markov process where only transitions to adjacent states are feasible. Specifically, the conditional probability that analyst ii moves from state ss in period tt to state s′s' in period t+1t+1 is denoted as qs,s′i=P(St+1i=s′∣Sti=s)q^i_{s,s'} = P(S^i_{t+1}=s'|S^i_t=s). This structure reflects the gradual nature of learning progression and regression, ensuring that analysts cannot “skip” states in their learning journey.

Transitions between states are determined by the stock of learning accumulated by the analyst during the period. The stock of learning, denoted as LSsitLS_{s}^{it}, represents the cumulative effect of participation in learning activities, which include both learning by doing (e.g., writing queries) and learning by observing (e.g., reviewing queries authored by others). The following equation defines how the stock of learning determines the likelihood of state transitions, emphasizing the role of learning activities and unobserved heterogeneity:

LSsit=βsLearning activitiesit+ζi+eist,LS_{s}^{it} = \beta_s \text{Learning activities}_{it} + \zeta_i + e_{ist},

where Learning activitiesit\text{Learning activities}_{it} is a vector capturing analyst ii’s participation in various learning activities during period tt. The vector βs\beta_s represents state-specific coefficients that quantify the effect of these activities on building the learning stock. This state-specific design reflects the intuition that analysts in higher states may derive diminishing returns from the same learning activities compared to those in lower states, as suggested by research on program cognition and learning effects [47].

The term ζi\zeta_i represents analyst-specific random effects, capturing unobserved heterogeneity such as participation in offline learning activities or differences in baseline abilities. The residual term eiste_{ist} accounts for random fluctuations in learning stock and is assumed to follow an extreme value distribution.

The transition probabilities are modeled using an ordered logit framework [48], where the likelihood of moving to a higher or lower state depends on whether the learning stock exceeds a state-specific threshold. Analysts move up to a higher state if their learning stock during the period exceeds a threshold μ(s+1,s)\mu(s+1,s), move down to a lower state if their learning stock falls below μ(s−1,s)\mu(s-1,s), or remain in their current state otherwise. These thresholds are ordered to ensure consistency:

μ(s+1,s)>μ(s−1,s),∀s∈{1,2,...,Ns}.\mu(s+1,s) > \mu(s-1,s), \quad\forall s \in\{1, 2, ..., N_s\}.

Additionally, boundary constraints are imposed to ensure that analysts in the highest or lowest states transition appropriately (e.g., μ(Ns+1,Ns)=∞\mu(N_s+1,N_s)=\infty and μ(0,1)=−∞\mu(0,1)=-\infty). This formulation provides a behaviorally interpretable framework for understanding the drivers of learning dynamics. By linking participation in learning activities to transitions between states, the model highlights how targeted efforts (e.g., engaging in peer query reviews or creating more queries) influence analysts’ progression or regression in latent learning states. Furthermore, the ordered logit structure ensures that transitions remain probabilistically coherent while capturing the nuanced effects of learning activities and individual differences.

State-Dependent Productivity in Writing Queries

We use FirstCompletionTimekitFirstCompletionTime_k^{it} as a proxy for data analyst productivity. Specifically, it is the time interval between the point when the data analyst ii clicks the button to create an empty query kk and the point when she executes this query for the first time, in period tt::

FirstCompletionTimekit=Timestampi first executes k−Timestampi creates the empty k.FirstCompletionTime_k^{it} = Timestamp_{i\ \text{first executes }k} - Timestamp_{i\ \text{creates the empty }k}.

Given that FirstCompletionTimekitFirstCompletionTime_k^{it} is a count variable and suffers from over-dispersion, i.e. the variance is significantly larger than the mean, we assume it follows a statedependent negative binomial distribution:

P(FirstCompletionTimekit=τ∣Sti=s)=e−λitkλitkτΓ(τ+1),P(\mathrm{FirstCompletionTime}_{k}^{it} = \tau\mid S_{t}^{i} = s) = \frac{e^{-\lambda_{itk}}\lambda_{itk}^{\tau}}{\Gamma(\tau+ 1)},

where τ\tau represents the observed completion time, and λitk\lambda_{itk} is the conditional mean, defined as:

log⁡(λitk)=ρsZitk+ηi+ϵitk.\log(\lambda_{itk}) = \rho_{s}Z_{itk} + \eta_{i} + \epsilon_{itk}.

Here, ZitkZ_{itk} is a vector of observable, time-varying factors that influence analyst ii’s productivity in writing query kk during period tt. The coefficients ρs\rho_{s} are state-specific, capturing how these external factors impact productivity differently depending on the analyst’s current latent state ss. For instance, analysts in higher latent states may exhibit greater resilience to external challenges due to accumulated learning and experience.

The term ηi\eta_{i} represents analyst-specific random effects, encapsulating unobserved heterogeneity such as individual programming style or domain expertise [49]. Meanwhile, ϵitk\epsilon_{itk} is a log-gamma distributed error term, capturing residual variation not explained by ZitkZ_{itk} or ηi\eta_{i}.

To accommodate the over-dispersed nature of the data, we reformulate the probability distribution using a state-specific dispersion parameter δs>0\delta_{s} > 0, the log-gamma distribution of the error term ϵitk\epsilon_{itk} [50]

P(FirstCompletionTimekit=τ∣Sti=s)=Γ(δs+τ)Γ(τ+1)Γ(δs)hitkδs(1−hitk)τ,P(\mathrm{FirstCompletionTime}_{k}^{it} = \tau\mid S_{t}^{i} = s) = \frac{\Gamma(\delta_{s} + \tau)}{\Gamma(\tau+ 1)\Gamma(\delta_{s})}h_{itk}^{\delta_{s}}(1-h_{itk})^{\tau},

where

hitk=δseρsZitk+ηi+δs.h_{itk} = \frac{\delta_{s}}{e^{\rho_{s}Z_{itk}+\eta_{i}}+\delta_{s}}.
The Likelihood of an Observed Sequence of Productivity Outcome

2.3.2.3. The Likelihood of an Observed Sequence of Productivity Outcome. Note that in our empirical setting, a data analyst can write multiple queries in one period. That is, given a data analyst ii has written KK queries in period tt, the productivity outcome we can observe in the period is a sequence of the FirstCompletionTime\mathrm{FirstCompletionTime} as the follows:

Oti={FirstCompletionTime1it,FirstCompletionTime2it,…,FirstCompletionTimeKit}(2)O_{t}^{i} = \{\mathrm{FirstCompletionTime}_{1}^{it},\mathrm{FirstCompletionTime}_{2}^{it},\ldots,\mathrm{FirstCompletionTime}_{K}^{it}\} \tag*{(2)}

Therefore, the probability of observing OtiO_{t}^{i} is:

P(Oti∣Sti=s)=∏k=1,2,…,KP(FirstCompletionTimekit∣Sti=s)(3)P(O_{t}^{i} \mid S_{t}^{i} = s) = \prod_{k=1,2,\ldots,K} P(\mathrm{FirstCompletionTime}_{k}^{it} \mid S_{t}^{i} = s) \tag*{(3)}

Because of the Markovian structure of this model, the probabilities of an individual’s productivity outcome are correlated through the underlying sequence of the hidden states. Specifically, consider the realized state path for an analyst ii is Si={S1i,S2i,...,STi}S^{i} = \{S^{i}_{1}, S^{i}_{2}, ..., S^{i}_{T}\} and the observed sequence of outcome Oi={O1i,O2i,...,OTi}O^{i} = \{O^{i}_{1}, O^{i}_{2}, ..., O^{i}_{T}\}. Then we have the likelihood of observing the sequence of OiO^{i} is

L(Oi)=∑S1i=1n∑S2i=1n...∑STi=1nP(Si,Oi)(4)L(O^{i}) = \sum_{S^{i}_{1}=1}^{n} \sum_{S^{i}_{2}=1}^{n} ... \sum_{S^{i}_{T}=1}^{n} P(S^{i}, O^{i}) \tag*{(4)}

The full likelihood across analysts is simply the production of individual likelihood over i∈{1,2,...,Nind}i \in\{1, 2, ..., N_{\mathrm{ind}}\}.

Estimation Strategy

Two main approaches have been proposed for estimating the parameters of Hidden Markov Models (HMMs): (1) maximum likelihood estimation, typically achieved through direct optimization or the Expectation-Maximization (EM) algorithm [51, 52], and (2) Bayesian estimation. For this study, we employ a hierarchical Bayesian estimation procedure, as described in [53], due to its flexibility in capturing cross-sectional heterogeneity among data analysts.

Bayesian estimation offers several key advantages for modeling learning dynamics and productivity. By integrating prior information with observed data, it provides robust inference and generates posterior distributions of model parameters, enabling a richer characterization of variability. This is particularly valuable for analyzing individual-level differences in state transitions and productivity patterns, as Bayesian methods account for heterogeneity more effectively than traditional approaches. Prior research [54] emphasizes the risks of neglecting such heterogeneity, which can lead to spurious or overstated effects in dynamic models.

To implement Bayesian estimation, we use Markov Chain Monte Carlo (MCMC) techniques, including Gibbs sampling and the Metropolis-Hastings algorithm [55, 56, 57]. These methods efficiently explore the posterior distribution, even for models with complex dependencies and random effects. Detailed descriptions of the MCMC algorithms and their implementation are provided in the appendix, including the rationale for their selection in this study. Briefly, Gibbs sampling is employed for parameters with tractable conditional distributions, while Metropolis-Hastings complements this by handling parameters without closed-form solutions. Together, these techniques ensure accurate and computationally efficient estimation of both state-specific and individual-level effects.

Results

We estimate two models of a single state and of three states respectively. The one state model assumes contemporaneous effect of query writing and viewing on analyst productivity. It is equivalent to the classic learning curve model. We use the maximum likelihood procedure to estimate this model. In contrast, the three-state model accounts for learning dynamics over time and was estimated using a hierarchical Bayesian approach, implemented via Markov Chain Monte Carlo (MCMC) methods. Specifically, the Gibbs sampler and random-walk Metropolis-Hastings algorithm were employed, leveraging the Julia programming language for computation [58]. The MCMC procedure was initialized with random values, running for 100,000 iterations across two chains until convergence was achieved. Convergence was assessed using the ratio of within-chain variance to between-chain variance for each parameter, following established guidelines [59]. The first 85,000 iterations were discarded as burn-in, and the remaining 15,000 iterations were retained for inference.

Table 2 compares the fit measures for both models, demonstrating that the three-state model consistently outperforms the single-state model across all criteria. This underscores the importance of incorporating the latent learning states of analysts to better capture the complexity of their productivity and learning behaviors. The results suggest that modeling learning as a dynamic process provides a more nuanced and accurate understanding of the interaction between query writing, query viewing, and productivity improvements.

-2LogLikelihoodAICBIC
One-state Model1 232 380.81 232 400.81 232 493.6
Three-state Model433 477.1441 551.1479 043.7

Table 2. Fit Measures for Comparing Models. Note: The more negative the number, the better the fit. AIC, Akaike information criterion; BIC, Bayesian information criterion.

HMM Estimates

Table 3 presents the summary statistics, including means, standard deviation (Std) and 95% highest posterior density (HPD) intervals, calculated from the posterior distribution for all parameters. Estimates with HPDs that do not overlap zero are considered statistically significant.

ParameterPosterior meanPosterior Std95% HPD intervals
β1writing\beta_{1}^{\mathrm{writing}}5.8641.416(4.450, 8.351)
β2writing\beta_{2}^{\mathrm{writing}}4.3791.763(2.542, 7.537)
β3writing\beta_{3}^{\mathrm{writing}}0.3380.377(-0.127, 0.988)
β1viewing\beta_{1}^{\mathrm{viewing}}5.2671.163(4.021, 7.001)
β2viewing\beta_{2}^{\mathrm{viewing}}-8.3680.814(-9.964, -7.480)
β3viewing\beta_{3}^{\mathrm{viewing}}-8.3061.440(-10.411, -6.482)
μ(2,1)\mu(2, 1)20.5471.585(17.167, 22.590)
μ(2,3)\mu(2, 3)-3.4582.906(-6.975, 0.383)
μ(1,2)\mu(1, 2)-13.2622.006(-16.626, -11.211)
log⁡(μ(3,2)−μ(1,2))\log(\mu(3, 2) - \mu(1, 2))12.9550.784(11.431, 13.996)
log⁡δ1\log\delta_{1}0.0000.000(-0.001, 0.000)
log⁡δ2\log\delta_{2}10.7590.806(9.759, 12.003)
log⁡δ3\log\delta_{3}-2.2870.186(-2.782, -2.081)
ρ1constant\rho_{1}^{\mathrm{constant}}5.8472.351(3.667, 10.039)
ρ2constant\rho_{2}^{\mathrm{constant}}3.2180.577(2.462, 4.019)
ρ3constant\rho_{3}^{\mathrm{constant}}-6.8420.636(-7.844, -6.183)
ρ1workload\rho_{1}^{\mathrm{workload}}9.8840.530(8.925, 10.788)
ρ2workload\rho_{2}^{\mathrm{workload}}2.7942.304(-0.072, 6.611)
ρ3workload\rho_{3}^{\mathrm{workload}}7.6551.212(5.347, 8.734)
ρ1tenure of author\rho_{1}^{\mathrm{tenure\ of\ author}}0.0060.057(-0.087, 0.117)
ρ2tenure of author\rho_{2}^{\mathrm{tenure\ of\ author}}-8.1471.756(-11.554, -6.130)
ρ3tenure of author\rho_{3}^{\mathrm{tenure\ of\ author}}-8.9751.780(-11.075, -6.648)
ρ1migrated query\rho_{1}^{\mathrm{migrated\ query}}2.3183.203(-1.430, 6.844)
ρ2migrated query\rho_{2}^{\mathrm{migrated\ query}}-4.6421.830(-6.962, -2.277)
ρ3migrated query\rho_{3}^{\mathrm{migrated\ query}}9.1850.434(8.305, 9.676)
ρ1saved query\rho_{1}^{\mathrm{saved\ query}}0.2830.094(0.177, 0.544)
ρ2saved query\rho_{2}^{\mathrm{saved\ query}}8.4042.942(4.814, 12.674)
ρ3saved query\rho_{3}^{\mathrm{saved\ query}}-0.0110.644(-1.031, 1.186)
ρ1query size\rho_{1}^{\mathrm{query\ size}}-3.5182.373(-8.646, 0.002)
ρ2query size\rho_{2}^{\mathrm{query\ size}}-4.8152.519(-9.414, 0.629)
ρ3query size\rho_{3}^{\mathrm{query\ size}}0.2170.329(-0.092, 0.960)

Table 3. Estimation Results for the Three-state Hidden Markov Models Parameter Note: Bolded coefficients’ HPDs do not cross zero.

State Dependent Productivity

Our analysis reveals clear variations in state-specific intrinsic productivity among analysts, as indicated by the constant terms ρ1constant\rho_{1}^{\mathrm{constant}}, ρ2constant\rho_{2}^{\mathrm{constant}}, and ρ3constant\rho_{3}^{\mathrm{constant}}. The posterior means for these constants are 5.8, 3.2, and −6.8-6.8 for states 1, 2, and 3, respectively, all of which are statistically significant. These results suggest that intrinsic productivity improves as analysts transition to higher states (a more negative constant implies shorter FirstCompletionTime\mathit{FirstCompletionTime}). Based on this pattern, we interpret the states as follows: analysts in state 1 exhibit characteristics of inexperience and slower query-writing productivity, which we label as novice. Similarly, analysts in state 2, with moderate productivity, are categorized as intermediate, while state 3, reflecting the highest productivity, is labeled as advanced.

Workload Effects

The posterior means for workload effects are 9.9, 2.8, and 7.7 for states 1, 2, and 3, respectively. These positive values indicate that higher workloads decrease analysts’ productivity across all states. However, the effect is smallest in magnitude and statistically insignificant for analysts in the intermediate state. This suggests that analysts at the intermediate level may better manage workloads compared to their novice or advanced counterparts.

Tenure Effects

The posterior means for the effect of analysts’ tenure on the platform are 0.006, −8.1, and 9.0 for states 1, 2, and 3, respectively. These results indicate that tenure positively impacts productivity for analysts in the intermediate and advanced states. However, the effect is insignificant for analysts in the novice state, suggesting that tenure benefits may not manifest until analysts gain sufficient experience or skills.

Query-Specific Characteristics

Our results highlight the relationship between certain query-specific characteristics and FirstCompletionTime. One notable factor is whether a query is migrated (originating on another platform and later moved to the current system). Our results show that analysts in novice state are insensitive to whether the current query is migrated or not. In contrast, intermediate-state analysts exhibit significant productivity gains on migrated queries, with a significant posterior mean of −4.6. Conversely, analysts in the advanced state show a significant slowdown when working on migrated queries. One potential explanation for this pattern is analysts in advanced state are usually asked to migrate large and complex queries. Ensuring those queries work appropriately on a new platform requires more time and caution as more calibration and comments may be necessary.

Similarly, our results show that queries saved by analysts (potentially signaling complexity) increase the time required to complete the query for novice and intermediate analysts. However, this effect is insignificant for advanced analysts, suggesting that they are less hindered by query complexity. Interestingly, we observe no significant effect of query size on FirstCompletionTime across all states. This lack of effect may stem from the fact that most queries (more than 75%) in our study consist of only one statement, which likely minimizes variability in query size effects.

Learning and State Transitions

The estimated thresholds for state transitions provide valuable insights into the learning dynamics of data analysts. Transitioning from the novice to the intermediate state requires reaching a threshold of 20.5, while moving down from the advanced to the intermediate state occurs at a threshold of −3.5. For analysts in the intermediate state, the thresholds are −13.3 to regress to the novice state and an exceptionally high 422, 932.8 to advance to the advanced state. These thresholds represent the accumulation of learning activities, such as writing or viewing queries, needed for an analyst to transition between states. The results show that progressing to higher states becomes increasingly difficult as analysts advance in their learning. Specifically, moving from the intermediate to the advanced state requires far greater effort compared to progressing from the novice to the intermediate state.

Effects of Learning Activities

The impact of learning activities on state transitions reveals nuanced, state-dependent effects. For novice analysts, both writing their own queries and viewing queries authored by peers significantly increase the likelihood of transitioning to the intermediate state. However, for intermediate analysts, while the effect of writing queries remains positive, its magnitude diminishes. This observation aligns with prior research, such as [60], which highlights that individuals with higher expertise often experience slower rates of subsequent learning within their area of specialization.

In contrast, analysts in the intermediate state or higher exhibit an increased likelikelihood of transitioning to a lower state when they engage in viewing peers’ queries. This counterintuitive pattern may stem from the variability in the quality of published queries. Here we only consider the aggregate views for an analyst without differentiating the source of knowledge. Analysts may encounter poorly designed or irrelevant queries that detract from their productivity. Moreover, viewing peer queries, though seemingly beneficial, often competes with the time available for actual programming or query-writing tasks, creating delays and inefficiencies in productive work.

Cognitive and Behavioral Implications

This competition for time and attention is particularly pronounced for analysts in higher states, who may derive diminishing returns from additional peer-based learning. These findings resonate with the behavioral observation that ”a wealth of information creates a poverty of attention” [61]. Analysts in advanced states may find the abundance of available materials less helpful because of the cognitive burden associated with evaluating and integrating new information. For these individuals, the effort required to sift through peers’ queries may outweigh the benefits, especially when the materials are not immediately relevant or aligned with their expertise.

Another plausible explanation for this pattern is domain switching. When intermediate or advanced analysts engage more frequently in viewing peers’ queries, it may indicate a shift in their responsibilities or focus, such as transitioning to a new business domain or database. This change often necessitates a temporary return to novice-like performance as analysts acclimate to unfamiliar contexts. In such cases, increased viewing activity reflects the challenges of adapting to new information and workflows rather than a decline in skill. Consequently, these observations emphasize the importance of considering the contextual and temporal aspects of learning behaviors when interpreting productivity dynamics.

Discussion

Classical literature on individual learning curves often assumes a static model, which risks producing biased estimates of learning effects when individual behavior evolves dynamically. This paper presents a Hidden Markov Model (HMM) to capture these dynamics in the productivity and learning behaviors of data analysts. By comparing our model to the standard learning curve framework, we demonstrate that the HMM is a more appropriate approach for our empirical setting. The HMM effectively captures three ordered learning states through which analysts progress, revealing increasing intrinsic productivity as they transition to higher states.

Our findings underscore the importance of participation in two distinct learning activities—writing queries (learning from direct experience) and viewing peers’ queries (learning from indirect experience)—in shaping analysts’ transitions between these states. The results indicate that the effects of these activities differ depending on the current learning state of the analyst, providing valuable insights into the nuanced nature of learning dynamics.

Design Implications for Systems and Interfaces

This study makes several significant contributions to the design and improvement of data platforms and collaborative systems, particularly in optimizing user productivity and learning trajectories. By leveraging insights from the proposed Hidden Markov Model (HMM), we underscore the importance of mechanisms for identifying user states, guiding learning activities, and delivering personalized experiences.

  • Mechanisms to Identify User Learning States: Our framework demonstrates the feasibility of detecting an analyst’s latent learning state based on their productivity and activity patterns. Integrating real-time state detection into platforms enables targeted support tailored to each user’s needs. For instance, novice analysts can receive prompts for viewing peer queries and guided tutorials, while advanced analysts might benefit from features that streamline the creation of original, complex queries. These mechanisms lay the foundation for adaptive interfaces that evolve with the user’s learning progress.

  • Guided Learning Activities for Optimized Productivity The differential effects of learning activities across states underscore the need for systems to prioritize state-appropriate activities. Platforms should dynamically recommend activities aligned with the user’s current state. For novice analysts, a mix of writing and viewing queries could enhance foundational skills, while intermediate users may benefit from features that encourage independent problem-solving. For advanced analysts, tools that minimize cognitive distractions—such as personalized filters or curated peer query recommendations—can help sustain high productivity.

  • Personalized Mechanisms for Learning Progression: Platforms can leverage predictive capabilities of the model to anticipate when analysts are likely to transition between states. For example, analysts approaching a state transition could receive motivational feedback, skill-building tasks, or curated learning resources aligned with their progression trajectory. Additionally, platforms can identify behaviors, such as excessive peer query viewing, that may hinder growth for advanced users. Visualizing progress through dashboards that display state-specific milestones and learning goals can further engage users, providing both a sense of accomplishment and actionable guidance.

  • Cost-Effective User Development and Onboarding: The insights from this study can inform the design of onboarding programs that accelerate analysts’ transitions from novice to advanced states. Tailored learning paths, curated based on an analyst’s initial state and progression, can reduce the time and resources required to achieve proficiency. Automating aspects of learning support also minimizes the need for costly external training, allowing organizations to invest in scalable, efficient development solutions.

  • Enhancing Collaboration without Overload: The observed risks of cognitive overload for analysts in higher states highlight the importance of managing information flow effectively. Features such as intelligent query recommendations, relevance-based filtering, and quality indicators for peer contributions can mitigate the negative effects of excessive peer query viewing. By prioritizing high-value collaborative activities and reducing irrelevant distractions, platforms can ensure that users derive maximum benefit from peer-based learning while maintaining their productivity.

By implementing these mechanisms, platforms can foster an environment that not only optimizes individual productivity but also enhances the overall learning experience. These contributions underscore the role of adaptive and personalized design in maximizing the potential of technology-mediated work and collaboration.

Limitations and Future Directions

This study provides significant insights into the dynamic learning and productivity of data analysts through the application of a Hidden Markov Model (HMM). However, there are several limitations that warrant further exploration. First, we restricted our analysis to one-state and three-state HMMs. While our results indicate that a dynamic model outperforms static models, the three-state configuration may not fully capture the complexity of analyst behavior. Future work should explore HMMs with varying numbers of states to determine the optimal model configuration that best represents the data. Second, our assumption of a uniform initial state distribution may be overly simplistic. Incorporating a more flexible approach by estimating the initial state probabilities at the population level could enhance the model’s accuracy and robustness, as suggested in prior work [62]. Additionally, our hierarchical modeling framework could be extended to incorporate individual random effects as functions of observable characteristics, such as demographics, organizational roles, or geographic locations [63, 64]. This extension would allow us to explore how specific individual or contextual factors influence learning dynamics and state transitions, offering a more nuanced understanding of variability across users. Furthermore, while this study focuses on aggregate learning activities, future research could refine the analysis by examining the quality and relevance of viewed queries, particularly in advanced states, where cognitive overload risks are prominent.

Lastly, the generalizability of our findings should be validated across other collaborative systems, organizational settings, and broader categories of knowledge workers. While this study focuses on data analysts, many other knowledge workers—such as software engineers, research scientists, or technical writers—may share similar learning behaviors, including the interplay of independent work and collaborative learning. Extending the framework to examine these groups would provide valuable insights into how learning dynamics and productivity manifest in diverse professional contexts. Moreover, testing the model across different platforms and industries could uncover platform-specific or organizational influences on learning and productivity. For instance, variations in collaborative tools, team structures, or organizational cultures might alter the effectiveness of learning activities and the observed state transitions. Testing the framework across such settings would not only validate its robustness but also refine its applicability.

By addressing these limitations and pursuing these directions, future work can deepen our understanding of knowledge workers’ learning dynamics while providing actionable insights for the design of more effective collaborative and adaptive systems.

Disclosure

This research did not receive any specific grant and no potential competing interest was reported by the author(s).

Data Availability Statement

Data supporting the findings of this study are not publicly available due to confidentiality agreements and non-disclosure obligations with the data provider. As such, the dataset cannot be shared.

HMM Model Development

Consider a finite set of states s∈{1,2,…,Ns}s \in\{1,2,\ldots,N_s\} in our HMM, with 1 being the lowest state and NsN_s being the highest. The entire time horizon is divided in to a finite number of periods t∈{1,2,…,T}t \in\{1,2,\ldots,T\}. At the beginning of any given period, a data analyst resides in an unknown state. For each period and each data analyst, we observe the productivity of the data analyst in writing queries, other factors that can affect her productivity and her participation in various learning activities during the period. The observed sequence of a data analyst ii’s productivity over periods is denoted as her outcome sequence Oi={O1i,O2i,…,OTi}O^i = \{O^i_1,O^i_2,\ldots,O^i_T\}, where OtiO^i_t characterizes her temporal productivity in writing queries in period tt. Given these primitives, our HMM has three main components:

  1. The initial state distribution, π\pi;

  2. The sequence of the state-transition probability matrix, QQ, with Qt−1→tQ_{t-1\to t} denoting the Markovian transition matrix from t−1t-1 to tt;

  3. The observed productivity with their state-dependent probability PP. The probability that data analyst ii is observed with certain productivity at time tt conditional on her state is P(Oti∣sti)P(O^i_t|s^i_t), where StiS^i_t is the state that ii belongs to in period tt and OtiO^i_t is the observed productivity of ii in the same period.

The Initial State Distribution

The initial state distribution characterizes the probability that an average data analyst starts in a particular state at the beginning of time horizon (period t=1t = 1). Let πs\pi_s be the probability that an average analyst belongs to the state ss in period t=1t = 1, where s∈{1,2,…,n}s \in\{1,2,\ldots,n\}. We have ∑s=1,2,…,Nsπs=1\sum_{s=1,2,\ldots,N_s}\pi_s = 1. In our HMM, we assume initially the probabilistic membership of all data analysts to different states follows the same distribution:

π=(π1,π2,…πNs)(5)\pi= (\pi_1,\pi_2,\ldots\pi_{N_s}) \tag*{(5)}

Many studies assuming time-homogeneous HMM define the initial state distribution as the stationary distribution of the transition matrix [41, 64, 65]. However, in our HMM the transition matrix is time-varying, thereby violating the assumption of homogeneity. Some other studies estimate their HMMs with fixed initial state distribution. Such assumption can come from prior information [66] or is simply specified for the computational convenience [13]. An alternative shown in a recent study by [62] is to directly estimate the initial state probability distribution at the population level. In our HMM we fix the initial state distribution to be uniform across states, i.e. πs=1Ns,∀s∈{1,2,…,Ns}\pi_s = \frac{1}{N_s}, \forall s \in\{1,2,\ldots,N_s\}.

The State Transition Matrix

We model the transitions between latent learning states as a Markov process where only transitions to adjacent states are feasible. The transition matrix is defined as the follows:

Qt→t+1it=(q1,1itq1,2it0⋯00q2,1itq2,2itq2,3it⋯00⋮⋮⋮⋱⋮⋮000⋯qNs,Ns−1itqNs,Nsit)(6)Q^{it}_{t\to t+1} = \begin{pmatrix} q^{it}_{1,1} & q^{it}_{1,2} & 0 & \cdots& 0 & 0 \\ q^{it}_{2,1} & q^{it}_{2,2} & q^{it}_{2,3} & \cdots& 0 & 0 \\ \vdots& \vdots& \vdots& \ddots& \vdots& \vdots\\ 0 & 0 & 0 & \cdots& q^{it}_{N_s,N_s-1} & q^{it}_{N_s,N_s} \end{pmatrix} \tag*{(6)}

where qs,s′it=P(St+1it=s′∣Stit=s)q^{it}_{s,s'} = P(S^{it}_{t+1}=s' \mid S^{it}_{t}=s) is the conditional probability that analyst ii moves from state ss in period tt to state s′s' in period tt, abd 0≤qs,s′it≤10 \le q^{it}_{s,s'} \le1 ∀s,s′\forall s,s', ∑s′=1,2,…,Nsqs,s′it=1\sum_{s'=1,2,\ldots,N_s} q^{it}_{s,s'} = 1 ∀s\forall s. We model the transitions between the states as a threshold model; what transition occurs depends on how the stock of learning compares to the threshold. The stock of learning, termed as LSsiLS^{i}_{s} is assumed to be a function of analyst ii’s participation in learning activities during the period as the follows:

LSsit=βsLearning activitiesit+ζi+eist(7)LS^{it}_{s} = \beta_s\mathrm{Learning\ activities}_{it} + \zeta_i + e_{ist} \tag*{(7)}

where Learning activitiesit\mathrm{Learning\ activities}_{it} is vector of ii’s participation in learning activities in this period and βs\beta_s is a vector of coefficients capturing the effect of participation in these activities on building up the analyst’s learning stock. Note that the effects of participation in learning activities are state-specific. Researchers in program cognition models have shown that past experiences and knowledge base of a programmer greatly affect her program understanding, thereby affecting the return of learning [47]. ζi\zeta_i is the random effect that captures the unobserved heterogeneity in state transitions across analysts. Such unobserved heterogeneity may include participation in offline learning activities that we do not observe. Assuming the unobserved eiste_{ist} is independently and identically distributed (IID) of the extreme value type, the non-homogeneous transition probabilities are modeled as an ordered logit [48]:

qs,s+1it=1−exp⁡(μ(s+1,s)−βsLearning activitiesit−ζi)1+exp⁡(μ(s+1,s)−βsLearning activitiesit−ζi)qs,s−1it=exp⁡(μ(s−1,s)−βsLearning activitiesit−ζi)1+exp⁡(μ(s−1,s)−βsLearning activitiesit−ζi)qs,sit=exp⁡(μ(s+1,s)−βsLearning activitiesit−ζi)1+exp⁡(μ(s+1,s)−βsLearning activitiesit−ζi)−exp⁡(μ(s−1,s)−βsLearning activitiesit−ζi)1+exp⁡(μ(s−1,s)−βsLearning activitiesit−ζi)(8)\begin{aligned} q^{it}_{s,s+1} &= 1 - \frac{\exp\left(\mu(s+1,s)-\beta_s\mathrm{Learning\ activities}_{it}-\zeta_i\right)}{1+\exp\left(\mu(s+1,s)-\beta_s\mathrm{Learning\ activities}_{it}-\zeta_i\right)} \\ q^{it}_{s,s-1} &= \frac{\exp\left(\mu(s-1,s)-\beta_s\mathrm{Learning\ activities}_{it}-\zeta_i\right)}{1+\exp\left(\mu(s-1,s)-\beta_s\mathrm{Learning\ activities}_{it}-\zeta_i\right)} \\ q^{it}_{s,s} &= \frac{\exp\left(\mu(s+1,s)-\beta_s\mathrm{Learning\ activities}_{it}-\zeta_i\right)}{1+\exp\left(\mu(s+1,s)-\beta_s\mathrm{Learning\ activities}_{it}-\zeta_i\right)} -\frac{\exp\left(\mu(s-1,s)-\beta_s\mathrm{Learning\ activities}_{it}-\zeta_i\right)}{1+\exp\left(\mu(s-1,s)-\beta_s\mathrm{Learning\ activities}_{it}-\zeta_i\right)} \tag*{(8)} \end{aligned}

where μ(s+1,s)\mu(s+1,s) is the logit threshold for analysts in state ss to move up to state s+1s+1 and μ(s−1,s)\mu(s-1,s) the logit threshold for analysts in state ss to move down to state s−1s-1. Put it into words, a data analyst will move up to a higher state if her stock of learning during the period is greater than μ(s+1,s)\mu(s+1,s), or move down to a lower state if her temporal learning is smaller than μ(s−1,s)\mu(s-1,s). We constrain μ(s+1,s)>μ(s−1,s),∀s∈{1,2,…,n}\mu(s+1,s)>\mu(s-1,s), \forall s \in\{1,2,\ldots,n\} to ensure the order of these thresholds. We also fix μ(Ns+1,n)=∞\mu(N_s+1,n)=\infty and μ(0,1)=−∞\mu(0,1)=-\infty to ensure that analysts already in the lowest or highest state transit correctly. Analysts of states s∈{2,3,…,Ns−1}s \in\{2,3,\ldots,N_s-1\} can move up to a higher state, move down to a lower state or stay in the current state.

State-Dependent Productivity in Writing Queries

We use FirstCompletionTimekitFirstCompletionTime_k^{it} as a proxy for data analyst productivity. For the detailed definition of FirstCompletionTimekitFirstCompletionTime_k^{it}, see 1.3.3.1. Specifically, it is the time interval between the point when the data analyst ii clicks the button to create an empty query kk and the point when she executes this query for the first time, in period tt:

FirstCompletionTimekit=Timestampi first executes k−Timestampi creates the empty k(9)FirstCompletionTime_k^{it} = Timestamp_{i\ \text{first executes }k} - Timestamp_{i\ \text{creates the empty }k} \tag*{(9)}

Because FirstCompletionTimekitFirstCompletionTime_k^{it} is a count variable and suffers from over-dispersion, i.e. the variance is significantly larger than the mean, we assume it follows a state-dependent negative binomial distribution:

P(FirstCompletionTimekit=τ∣Sti=s)=e−λitkλitkτΓ(τ+1)(10)P(FirstCompletionTime_k^{it}=\tau\mid S_t^i=s)=\frac{e^{-\lambda_{itk}}\lambda_{itk}^{\tau}}{\Gamma(\tau+1)} \tag*{(10)}

where the conditional mean λitk\lambda_{itk} is defined as the following:

log⁡(λitk)=ρsZitk+ηi+ϵitk(11)\log(\lambda_{itk})=\rho_s Z_{itk}+\eta_i+\epsilon_{itk} \tag*{(11)}

where ZitkZ_{itk} is a vector of time-varying factors that can affect the productivity of analyst ii in writing query kk in period tt. ρs\rho_s is a vector of state-specific coefficients that capture the effect of ZitkZ_{itk}, where ss is the state that ii resides in period tt. The random effect ηi\eta_i captures the analyst-specific unobserved heterogeneity, which may include individual programming style [49]. With the log-gamma distribution of the error term ϵitk\epsilon_{itk}, we can reformulate equation A6 as the following [50]:

P(FirstCompletionTimekit=τ∣Sti=s)=Γ(δs+τ)Γ(τ+1)Γ(δs)hitkδs(1−hitk)τ(12)P(FirstCompletionTime_k^{it}=\tau\mid S_t^i=s)=\frac{\Gamma(\delta_s+\tau)}{\Gamma(\tau+1)\Gamma(\delta_s)}h_{itk}^{\delta_s}(1-h_{itk})^\tau \tag*{(12)}

where

hitk=δseρsZitk+ηi+δs(13)h_{itk}=\frac{\delta_s}{e^{\rho_s Z_{itk}+\eta_i}+\delta_s} \tag*{(13)}

with δs>0\delta_s>0 as a state-specific dispersion parameter. Note that in our empirical setting, a data analyst can write multiple queries in one period. That is, given a data analyst ii has written KK queries in period tt, the productivity outcome we can observe in the period is a sequence of the $FirstCompletionTime as the follows:

Oti={FirstCompletionTime1it,FirstCompletionTime2it,…,FirstCompletionTimeKit}(14)O_t^i = \{\mathrm{FirstCompletionTime}_1^{it},\mathrm{FirstCompletionTime}_2^{it},\ldots,\mathrm{FirstCompletionTime}_K^{it}\} \tag*{(14)}

Therefore, the probability of observing OtiO_t^i is:

P(Oti∣Sti=s)=∏k=1,2,…,KP(FirstCompletionTimekit∣Sti=s)(15)P(O_t^i \mid S_t^i=s)=\prod_{k=1,2,\ldots,K}P(\mathrm{FirstCompletionTime}_k^{it}\mid S_t^i=s) \tag*{(15)}

The Likelihood of an Observed Sequence of Productivity Outcome

Because of the Markovian structure of this model, the probabilities of an individual’s productivity outcome are correlated through the underlying sequence of the hidden states. Specifically, consider the realized state path for an analyst ii is Si={S1i,S2i,…,STi}S^i=\{S_1^i,S_2^i,\ldots,S_T^i\} and the observed sequence of outcome Oi={O1i,O2i,…,OTi}O^i=\{O_1^i,O_2^i,\ldots,O_T^i\}. Then we have

P(Oi∣Si)=∏t=1TP(Oti∣Sti)(16)P(O^i\mid S^i)=\prod_{t=1}^{T}P(O_t^i\mid S_t^i) \tag*{(16)}

And the probability of seeing such state path is:

P(Si)=P(S1i)∏t=1T−1qSti,St+1iit(17)P(S^i)=P(S_1^i)\prod_{t=1}^{T-1}q_{S_t^i,S_{t+1}^i}^{it} \tag*{(17)}

where P(S1i)P(S_1^i) is the probability that the initial state of ii is S1iS_1^i; here P(S1i=s)=πsP(S_1^i=s)=\pi_s. So the joint probability of SiS^i and OiO^i is

P(Si,Oi)=P(S1i)∏t=1T−1qSti,St+1iit∏t=1TP(Oti∣Sti)(18)P(S^i,O^i)=P(S_1^i)\prod_{t=1}^{T-1}q_{S_t^i,S_{t+1}^i}^{it}\prod_{t=1}^{T}P(O_t^i\mid S_t^i) \tag*{(18)}

Accordingly, the likelihood of observing the sequence of OiO^i is given by summing equation A14 over all possible state paths that ii could take overtime:

L(Oi)=∑S1i=1n∑S2i=1n⋯∑STi=1nP(Si,Oi)(19)L(O^i)=\sum_{S_1^i=1}^{n}\sum_{S_2^i=1}^{n}\cdots\sum_{S_T^i=1}^{n}P(S^i,O^i) \tag*{(19)}

Following [41], we can simplify equation A15 into the following matrix notation:

L(Oi)=πP1iQ1→2iP2iQ2→3i⋯PTi1′(20)L(O^i)=\pi P_1^iQ_{1\to2}^iP_2^iQ_{2\to3}^i\cdots P_T^i\mathbf{1}' \tag*{(20)}

where

Pti=(P(Oti∣Sti=1)00⋯000P(Oti∣Sti=2)0⋯00⋮⋮⋮⋮⋮⋮000⋯0P(Oti∣Sti=Ns))(21)\mathbf{P}^{i}_{t} = \begin{pmatrix} P(O^{i}_{t} \mid S^{i}_{t}=1) & 0 & 0 & \cdots& 0 & 0 \\ 0 & P(O^{i}_{t} \mid S^{i}_{t}=2) & 0 & \cdots& 0 & 0 \\ \vdots& \vdots& \vdots& \vdots& \vdots& \vdots\\ 0 & 0 & 0 & \cdots& 0 & P(O^{i}_{t} \mid S^{i}_{t}=N_s) \end{pmatrix} \tag*{(21)}

and 1′\mathbf{1}^{\prime} is an Ns×1N_s \times1 vector of ones. The full likelihood across analysts is simply the production of individual likelihood over i∈{1,2,...,Nind}i \in\{1, 2, ..., N_{\mathrm{ind}}\}.

HMM Estimation Strategy

We estimate our HMM using a standard hierarchical Bayes estimation procedure described in [53], for its great flexibility and efficiency in estimating models that account for cross-sectional heterogeneity. Instead of focusing on the point estimate of true parameter value as the frequentist approach, the main interest of the Bayes approach is in generating the entire distribution of the parameters given the data and a prior. Bayes theorem provides the mechanism for how data and prior beliefs are translated in to posterior beliefs:

p(θ∣y)=p(y∣θ)p(θ)p(y)∝p(y∣θ)p(θ)(22)p(\theta\mid y) = \frac{p(y \mid\theta)p(\theta)}{p(y)} \propto p(y \mid\theta)p(\theta) \tag*{(22)}

where θ\theta is the parameters we want to estimate and p(θ∣y)p(\theta\mid y) is the posterior distribution of θ\theta, given observed data yy and prior p(θ)p(\theta). Inference here refers to summarizing the posterior distribution with typical statistics, such as the posterior mean E[θ]=∫θp(θ∣y)dθE[\theta] = \int\theta p(\theta\mid y)d\theta and posterior standard deviation. Note that bayesian inference here is conducted only using formal probability theory and does not require asymptotic approximations, which is particularly useful in modeling data with limited size.

To apply a standard hierarchical Bayes estimation procedure, we distinguish two sets of parameters: random-effect parameters {Θi}\{\Theta_i\} and parameters that are common across individuals Ψ\Psi. Specifically,

Θi={ζi,ηi}Ψ={μ(2,1),μ(1,2),μ(3,2),...,μ(s−1,s),μ(s+1,s),...,μ(Ns−1,Ns),δ1,δ2,...,δs,...,δNs,β1,β2,...,βs,...,βNs,ρ1,ρ2,...,ρs,...,ρNs}(23)\begin{aligned} \Theta_i &= \{\zeta_i, \eta_i\} \\ \Psi&= \{\mu(2,1), \mu(1,2), \mu(3,2), ..., \mu(s-1,s), \mu(s+1,s), ..., \mu(N_s-1,N_s), \\ &\qquad\delta_1, \delta_2, ..., \delta_s, ..., \delta_{N_s}, \\ &\qquad\beta_1, \beta_2, ..., \beta_s, ..., \beta_{N_s}, \\ &\qquad\rho_1, \rho_2, ..., \rho_s, ..., \rho_{N_s}\} \tag*{(23)} \end{aligned}

and we assume a prior distribution for the random parameters as the following:

Prior⁡(Θi)=Normal⁡(0,ΣΘ)(24)\operatorname{Prior}(\Theta_i) = \operatorname{Normal}(0, \Sigma_{\Theta}) \tag*{(24)}

Conforming to convention, we complete this specification by assuming the typical diffuse priors for Ψ\Psi and the hyperparameter Σθ\Sigma_{\theta}, following [64]

Prior⁡(ΣΘ)=Inverse Wishart⁡(NΘ+5+Nind,INΘ)Prior⁡(Ψ)=Normal⁡(0,30INΨ)(25)\begin{aligned} \operatorname{Prior}(\Sigma_{\Theta}) &= \operatorname{Inverse\ Wishart}(N_{\Theta}+5+N_{\mathrm{ind}}, I_{N_{\Theta}})\\ \operatorname{Prior}(\Psi) &= \operatorname{Normal}(0, 30I_{N_{\Psi}}) \tag*{(25)} \end{aligned}

Conducting bayes estimation requires sequential drawing from the conditional posterior p({Θi},Ψ∣y)p(\{\Theta_i\},\Psi\mid y) using the Markov Chain Monte Carlo (MCMC) algorithm. Note that Equation (22) along with Equation (21) suggests that the conditional posterior does not have a closed form. Thus, we apply Metropolis-Hasting algorithm [55, 56] to sample from the posterior distribution, using a Gibbs sampler [57].

The basis of the Metropolis algorithm consists of: (1) in the mthm^{\mathrm{th}} iteration, simulate a candidate sample of parameters, say Ωcand\Omega^{\mathrm{cand}}, from the proposal distribution Φm−1(⋅)\Phi^{m-1}(\cdot) based on the last accepted sample Ωm−1\Omega^{m-1}; (2) accept the candidate sample with the probability calculated via the acceptance function α(Ωcand∣Ωm−1)\alpha(\Omega^{\mathrm{cand}}|\Omega^{m-1}). We choose the Gaussian distributions as the proposal distributions, taking advantage of its symmetry. That is Φm−1(⋅)=Normal⁡(Ωm−1,Σ)\Phi^{m-1}(\cdot)=\operatorname{Normal}(\Omega^{m-1},\Sigma). The typical acceptance function is the following:

α(Ωcand∣Ωm−1)=min⁡{1,P(Ωcand∣y)P(Ωm−1∣y)}(26)\alpha(\Omega^{\mathrm{cand}|\Omega^{m-1}})=\min\left\{1,\frac{P(\Omega^{\mathrm{cand}}|y)}{P(\Omega^{m-1}|y)}\right\} \tag*{(26)}

where P(⋅∣y)P(\cdot|y) is the posterior joint density given the data. This acceptance function is designed to send the sampler to move to higher probability areas under the posterior joint density. In practice, we draw a random number uniformly between 0 and 1, and compare it with P(Ωcand∣y)P(Ωm−1∣y)\frac{P(\Omega^{\mathrm{cand}}|y)}{P(\Omega^{m-1}|y)}. If P(Ωcand∣y)P(Ωm−1∣y)\frac{P(\Omega^{\mathrm{cand}}|y)}{P(\Omega^{m-1}|y)} is larger than the uniform random number, we accept the candidate sample; otherwise we reject it. After running the algorithm for a long time, we end up with a sequential sample from the target posterior distribution of the parameters.

Gibbs sampler is an MCMC sampler developed by [57], which allows us to divide variables of interest into blocks consisting of two or more variables and sample the joint distribution of each block conditional on all other variables. Gibbs sampler is particularly suitable for hierarchical models. With Gibbs sampler, we can recursively update the individual random effects {Θi}\{\Theta_i\}, the covariance Σθ\Sigma_{\theta}, and the common effects Ψ\Psi following Algorithm 1:

Algorithm 1: Hierarchical Bayes Estimation Algorithm

Initialize: Generate a random sample of

ΣΘ0∼Inverse Wishart⁡(NΘ+5+Nind,INΘ)\boldsymbol{\Sigma}_{\Theta}^{0} \sim\operatorname{Inverse\ Wishart}(N_{\Theta}+5+N_{\mathrm{ind}}, I_{N_{\Theta}})

{Θi0}∼Normal⁡(0,ΣΘ0)\{\boldsymbol{\Theta}_{i}^{0}\} \sim\operatorname{Normal}(\mathbf{0},\boldsymbol{\Sigma}_{\Theta}^{0})

Ψ0∼Normal⁡(0,30INΨ)\boldsymbol{\Psi}^{0} \sim\operatorname{Normal}(\mathbf{0},30I_{N_{\Psi}})

for iteration m=1,2,⋯m=1,2,\cdots do

Step 1 for i←1i \leftarrow1 to NindN_{\mathrm{ind}} do

  1. Simulate a candidate Θicand∼Normal⁡(Θim−1,λθΣ~θ)\boldsymbol{\Theta}_{i}^{\mathrm{cand}} \sim\operatorname{Normal}(\boldsymbol{\Theta}_{i}^{m-1},\lambda_{\theta}\widetilde{\boldsymbol{\Sigma}}_{\theta}), where Θim−1\boldsymbol{\Theta}_{i}^{m-1} is the accepted draw in iteration m−1m-1, and λθ\lambda_{\theta} and Σ~θ\widetilde{\boldsymbol{\Sigma}}_{\theta} are chosen adaptively following the Algorithm 4 in [67] and [68]. We adapt λθ\lambda_{\theta} and Σ~θ\widetilde{\boldsymbol{\Sigma}}_{\theta} to reduce the autocorrelation between draws and to maintain the optimal acceptance rate.

  1. Accept Θicand\boldsymbol{\Theta}_{i}^{\mathrm{cand}} to be Θim\boldsymbol{\Theta}_{i}^{m} with the probability of

min⁡{exp⁡(−12Θicand(ΣΘm−1)−1Θi′cand)Pi(yi∣Θicand,Ψm−1)exp⁡(−12Θim−1(ΣΘm−1)−1Θi′m−1)Pi(yi∣Θim−1,Ψm−1),1}\min\left\{\frac{\exp\left(-\frac{1}{2}\boldsymbol{\Theta}_{i}^{\mathrm{cand}}(\boldsymbol{\Sigma}_{\Theta}^{m-1})^{-1}\boldsymbol{\Theta}_{i}^{\prime\mathrm{cand}}\right)P_i(y_i\mid\boldsymbol{\Theta}_{i}^{\mathrm{cand}},\boldsymbol{\Psi}^{m-1})}{\exp\left(-\frac{1}{2}\boldsymbol{\Theta}_{i}^{m-1}(\boldsymbol{\Sigma}_{\Theta}^{m-1})^{-1}\boldsymbol{\Theta}_{i}^{\prime m-1}\right)P_i(y_i\mid\boldsymbol{\Theta}_{i}^{m-1},\boldsymbol{\Psi}^{m-1})},1\right\}

if reject, Θim=Θim−1\boldsymbol{\Theta}_{i}^{m}=\boldsymbol{\Theta}_{i}^{m-1}.

Step 2 Sample ΣΘm∼Inverse Wishart⁡(NΘ+5+Nind,INΘ+∑i=1NindΘimΘi′m)\boldsymbol{\Sigma}_{\Theta}^{m}\sim\operatorname{Inverse\ Wishart}(N_{\Theta}+5+N_{\mathrm{ind}},I_{N_{\Theta}}+\sum_{i=1}^{N_{\mathrm{ind}}}\boldsymbol{\Theta}_{i}^{m}\boldsymbol{\Theta}_{i}^{\prime m})

Step 3 Given the updated {Θim}\{\boldsymbol{\Theta}_{i}^{m}\}

  1. Simulate a candidate Ψcand∼Normal⁡(Ψm−1,λΨΣ~Ψ)\boldsymbol{\Psi}^{\mathrm{cand}}\sim\operatorname{Normal}(\boldsymbol{\Psi}^{m-1},\lambda_{\Psi}\widetilde{\boldsymbol{\Sigma}}_{\Psi}), where Ψm−1\boldsymbol{\Psi}^{m-1} is the accepted draw in iteration m−1m-1. λΨ\lambda_{\Psi} and Σ~Ψ\widetilde{\boldsymbol{\Sigma}}_{\Psi} are also adaptively chosen using the same algorithm as in Step 1.

  1. Accept Ψcand\boldsymbol{\Psi}^{\mathrm{cand}} as Ψm\boldsymbol{\Psi}^{m} with the probability of

min⁡{exp⁡(−12Ψcand(VΨ0)−1Ψ′cand)Pi(yi∣Θim,Ψcand)exp⁡(−12Ψm−1(VΨ0)−1Ψ′m−1)Pi(yi∣Θim,Ψm−1),1}\min\left\{\frac{\exp\left(-\frac{1}{2}\boldsymbol{\Psi}^{\mathrm{cand}}(V_{\Psi}^{0})^{-1}\boldsymbol{\Psi}^{\prime\mathrm{cand}}\right)P_i(y_i\mid\boldsymbol{\Theta}_{i}^{m},\boldsymbol{\Psi}^{\mathrm{cand}})}{\exp\left(-\frac{1}{2}\boldsymbol{\Psi}^{m-1}(V_{\Psi}^{0})^{-1}\boldsymbol{\Psi}^{\prime m-1}\right)P_i(y_i\mid\boldsymbol{\Theta}_{i}^{m},\boldsymbol{\Psi}^{m-1})},1\right\}

where VΨ0=30INΨV_{\Psi}^{0}=30I_{N_{\Psi}}. If reject, Ψm=Ψm−1\boldsymbol{\Psi}^{m}=\boldsymbol{\Psi}^{m-1}.

References

  1. [1]C.M. Fiol and M.A. Lyles, Organizational learning, Academy of management review 10 (1985), pp. 803–813.DOI
  2. [2]M.M. Crossan, H.W. Lane, and R.E. White, An organizational learning framework: From intuition to institution, Academy of management review 24 (1999), pp. 522–537.DOI
  3. [3]C.L. Benkard, Learning and forgetting: The dynamics of aircraft production, Tech. Rep., National bureau of economic research, 1999.
  4. [4]E.D. Darr, L. Argote, and D. Epple, The acquisition, transfer, and depreciation of knowledge in service organizations: Productivity in franchises, Management Science 41 (1995), pp. 1750–1762.DOI
  5. [5]W. Fong Boh, S.A. Slaughter, and J.A. Espinosa, Learning from experience in software development: A multilevel analysis, Management Science 53 (2007), pp. 1315–1331.DOI
  6. [6]Y. Kim, R. Krishnan, and L. Argote, The learning curve of knowledge workers in a computing call center, Information Systems Research 23 (2012), pp. 887–902.DOI
  7. [7]K. Kang and J. Hahn, Learning and forgetting curves in software development: Does type of knowledge matter?, ICIS 2009 Proceedings (2009), p. 194.
  8. [8]H. Ebbinghaus, Memory: A contribution to experimental psychology, Annals of neurosciences 20 (1885), p. 155.
  9. [9]T.P. Wright, Factors affecting the cost of airplanes, Journal of the aeronautical sciences 3 (1936), pp. 122–128.DOI
  10. [10]L. Argote, Organizational learning: Creating, retaining and transferring knowledge, Springer Science & Business Media, 2012.DOI
  11. [11]D.S. Kc and B.R. Staats, Accumulating a portfolio of experience: The effect of focal and related experience on surgeon performance, Manufacturing & Service Operations Management 14 (2012), pp. 618–633.
  12. [12]J.R. Clark, R.S. Huckman, and B.R. Staats, Learning from customers: Individual and organizational effects in outsourced radiological services, Organization Science 24 (2013), pp. 1539–1557.
  13. [13]P.V. Singh, Y. Tan, and N. Youn, A hidden markov model of developer learning dynamics in open source software projects, Information Systems Research 22 (2011), pp. 790–807.DOI
  14. [14]B. Levitt and J.G. March, Organizational learning, Annual review of sociology 14 (1988), pp. 319–338.
  15. [15]G.P. Huber, Organizational learning: The contributing processes and the literatures, Organization Science 2 (1991), pp. 88–115.DOI
  16. [16]M.J. Tyre and E. Von Hippel, The situated nature of adaptive learning in organizations, Organization science 8 (1997), pp. 71–83.DOI
  17. [17]J. Brown and P. Duguid, Organizational learning and communities of practice: Toward a unified view of working, learning, and innovation, in Knowledge and communities, Elsevier, 2000, pp. 99–121.
  18. [18]B. Vasilescu, V. Filkov, and A. Serebrenik, Stackoverflow and github: Associations between software development and crowdsourced knowledge, in Social computing (SocialCom), 2013 international conference on. IEEE, 2013, pp. 188–195.DOI
  19. [19]F. Waldinger, Peer effects in science: Evidence from the dismissal of scientists in nazi germany, The Review of Economic Studies 79 (2011), pp. 838–861.
  20. [20]R. Reagans, L. Argote, and D. Brooks, Individual experience and experience working together: Predicting learning rates from knowing who knows what and knowing how to work together, Management Science 51 (2005), pp. 869–881.
  21. [21]D. Kc, B.R. Staats, and F. Gino, Learning from my success and from others’ failure: Evidence from minimally invasive cardiac surgery, Management Science 59 (2013), pp. 2435–2449.
  22. [22]W.M. Cohen and D.A. Levinthal, Absorptive capacity: A new perspective on learning and innovation, Administrative science quarterly 35 (1990), pp. 128–152.
  23. [23]H.C. Ellis, The transfer of learning. (1965).
  24. [24]P.H. Lindsay and D.A. Norman, Human information processing: An introduction to psychology, Academic press, 2013.
  25. [25]G.H. Bower, E.R. Hilgard, et al., Theories of learning, Prentice-Hall, 1981.
  26. [26]C.F. Manski, Identification of endogenous social effects: The reflection problem, The review of economic studies 60 (1993), pp. 531–542.
  27. [27]T.Y. Chan, J. Li, and L. Pierce, Learning from peers: Knowledge transfer and sales force productivity growth, Marketing Science 33 (2014), pp. 463–484.
  28. [28]Y. Yin, I. Gurvich, S. McReynolds, D. Seys, and J.A. Van Mieghem, Learning by doing versus learning by viewing: An empirical study of data analyst productivity on a collaborative platform at ebay, Proceedings of the ACM on Human-Computer Interaction 2 (2018), p. 193.DOI
  29. [29]T. Matthews, J. Chen, S. Whittaker, A. Pal, H. Zhu, H. Badenes, and B. Smith, Goals and perceived success of online enterprise communities: what is important to leaders & members?, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2014, pp. 291–300.DOI
  30. [30]T. Matthews, S. Whittaker, H. Badenes, B.A. Smith, M. Muller, K. Ehrlich, M.X. Zhou, and T. Lau, Community insights: helping community leaders enhance the value of enterprise online communities, in Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 2013, pp. 513–522.DOI
  31. [31]M. Muller, K. Ehrlich, T. Matthews, A. Perer, I. Ronen, and I. Guy, Diversity among enterprise online communities: collaborating, teaming, and innovating through social media, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2012, pp. 2815–2824.DOI
  32. [32]M. Rowe, M. Fernandez, H. Alani, I. Ronen, C. Hayes, and M. Karnstedt, Behaviour analysis across different types of Enterprise Online Communities, in Proceedings of the 4th Annual ACM Web Science Conference. ACM, 2012, pp. 255–264.
  33. [33]V. Tiwari, Some observations on open source software development on software engineering perspectives, International Journal of Computer Science & Information Technology (IJCSIT) 2 (2010), pp. 113–125.DOI
  34. [34]J. Piggot and C. Amrit, How healthy is my project? open source project attributes as indicators of success, in IFIP International Conference on Open Source Systems. Springer, 2013, pp. 30–44.
  35. [35]R.P. Buse and W.R. Weimer, A metric for software readability, in Proceedings of the 2008 international symposium on Software testing and analysis. ACM, 2008, pp. 121–130.DOI
  36. [36]G.J. Rothfuss and K. Bauknecht, A framework for open source projects, Department of Information Technology. Zurich, Switzerland, University of Zurich 157 (2002).
  37. [37]T.F. Tan and S. Netessine, When does the devil make work? an empirical study of the impact of workload on worker productivity, Management Science 60 (2014), pp. 1574–1593.DOI
  38. [38]B.G. Leroux and M.L. Puterman, Maximum-penalized-likelihood estimation for independent and markov-dependent mixture models, Biometrics (1992), pp. 545–558.pubmed.ncbi.nlm.nih.gov/1637977
  39. [39]H. Morf, The stochastic two-state solar irradiance model (stsim), Solar Energy 62 (1998), pp. 101–112.DOI
  40. [40]L.R. Rabiner and B.H. Juang, An introduction to hidden markov models, ieee assp magazine 3 (1986), pp. 4–16.DOI
  41. [41]I.L. MacDonald and W. Zucchini, Hidden Markov and other models for discrete-valued time series, Vol. 110, CRC Press, 1997.
  42. [42]S.L. Scott, Bayesian methods for hidden markov models: Recursive computing in the 21st cen- tury, Journal of the American Statistical Association 97 (2002), pp. 337–351.
  43. [43]J.D. Hamilton, A new approach to the economic analysis of nonstationary time series and the business cycle, Econometrica: Journal of the Econometric Society (1989), pp. 357–384.DOI
  44. [44]S.J. Koopman, Exact initial kalman filtering and smoothing for nonstationary time series models, Journal of the American Statistical Association 92 (1997), pp. 1630–1638.
  45. [45]J.C. Duan and J.G. Simonato, Estimating and testing exponential-affine term structure models by kalman filter, Review of Quantitative Finance and Accounting 13 (1999), pp. 111–135.
  46. [46]J.P. Hughes, P. Guttorp, and S.P. Charles, A non-homogeneous hidden markov model for precipitation occurrence, Journal of the Royal Statistical Society: Series C (Applied Statistics) 48 (1999), pp. 15–30.DOI
  47. [47]A. Von Mayrhauser and A.M. Vans, Program comprehension during software maintenance and evolution, Computer (1995), pp. 44–55.DOI
  48. [48]W.H. Greene and D.A. Hensher, Modeling ordered choices: A primer, Cambridge University Press, 2010.DOI
  49. [49]P. Blikstein, M. Worsley, C. Piech, M. Sahami, S. Cooper, and D. Koller, Programming pluralism: Using learning analytics to detect patterns in the learning of computer programming, Journal of the Learning Sciences 23 (2014), pp. 561–599.DOI
  50. [50]W. Greene, Econometric Analysis, Pearson Education, 2011, Available at https://books.google.com/books?id=LWQuAAAAQBAJ.
  51. [51]A.P. Dempster, N.M. Laird, and D.B. Rubin, Maximum likelihood from incomplete data via the em algorithm, Journal of the Royal Statistical Society: Series B (Methodological) 39 (1977), pp. 1–22.DOI
  52. [52]L. Baum, An inequality and associated maximization technique in statistical estimation of probabilistic functions of a markov process, Inequalities 3 (1972), pp. 1–8.
  53. [53]P.E. Rossi, G.M. Allenby, and R. McCulloch, Bayesian statistics and marketing, John Wiley & Sons, 2012.
  54. [54]J.J. Heckman, Heterogeneity and state dependence, in Studies in labor markets, University of Chicago Press, 1981, pp. 91–140.
  55. [55]N. Metropolis, A.W. Rosenbluth, M.N. Rosenbluth, A.H. Teller, and E. Teller, Equation of state calculations by fast computing machines, The journal of chemical physics 21 (1953), pp. 1087–1092.
  56. [56]W.K. Hastings, Monte carlo sampling methods using markov chains and their applications (1970).
  57. [57]S. Geman and D. Geman, Stochastic relaxation, gibbs distributions, and the bayesian restoration of images, IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-6 (1984), pp. 721–741.
  58. [58]J. Bezanson, A. Edelman, S. Karpinski, and V.B. Shah, Julia: A fresh approach to numerical computing, SIAM Review 59 (2017), pp. 65–98, Available at https://epubs.siam.org/doi/10.1137/141000671.DOI
  59. [59]S.P. Brooks and A. Gelman, General methods for monitoring convergence of iterative simulations, Journal of Computational and Graphical Statistics 7 (1998), pp. 434–455.
  60. [60]S.M. Shafer, D.A. Nembhard, and M.V. Uzumeri, The effects of worker learning, forgetting, and heterogeneity on assembly line productivity, Management Science 47 (2001), pp. 1639–1653.DOI
  61. [61]J.R. Anderson, L.M. Reder, and H.A. Simon, Situative versus cognitive perspectives: Form versus substance, Educational researcher 26 (1997), pp. 18–21.
  62. [62]C. Lee, E. Ofek, and T.J. Steenburgh, Personal and social usage: The origins of active customers and ways to keep them engaged, Management Science 64 (2017), pp. 2473–2495.DOI
  63. [63]G.M. Allenby and J.L. Ginter, Using extremes to design products and segment markets, Journal of Marketing Research 32 (1995), pp. 392–403.DOI
  64. [64]O. Netzer, J.M. Lattin, and V. Srinivasan, A hidden markov model of customer relationship dynamics, Marketing science 27 (2008), pp. 185–204.
  65. [65]J.M. Montoya and D. Raffaelli, Climate change, biotic interactions and ecosystem services (2010).DOI
  66. [66]D.A. Schweidel, E.T. Bradlow, and P.S. Fader, Portfolio dynamics for customers of a multi-service provider, Management Science 57 (2011), pp. 471–486.
  67. [67]C. Andrieu and J. Thoms, A tutorial on adaptive mcmc, Statistics and computing 18 (2008), pp. 343–373.DOI
  68. [68]B. Shaby and M.T. Wells, Exploring an adaptive metropolis algorithm, Currently under review 1 (2010), pp. 1–17.

Paper details

vlnc:W_c3f6jk6u
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 12

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents