WorksComputer Science
Contextuality and Contradictions: Exploring Real-World Identifications of AI-Generated Writing
Abstract
As AI-generated content spreads into nearly every domain of human interaction, practices and discourses of AI detection, both formal and informal, have become an everyday part of how people interact with text and with each other. In this paper, we explore the real-world practices and discourses of AI identification that have emerged in response to a shifting discursive landscape where the human provenance of a text can no longer be taken for granted. We pair qualitative coding with computational analysis of lexical change, grounded in linguistic anthropological perspectives on language and communication, to investigate AI identifications among English Wikipedia editors and students. We find that these identifications draw on a diversity of evidentiary types, from discrete textual features and judgments about content to circular sense-making claims and presumptions about the author's moral character, and that they are contradictory, dynamic, and contextually dependent. In practice, what often motivates an AI identification is not the presence of any specific textual element, but pragmatic infelicity: a perceived mismatch between the elements present and those deemed appropriate within the text's particular context. Consequently, real-world identifications of AI-generated writing do not merely neutrally point out the perceived presence of "AI-like" features, but can be leveraged as a discursive resource to maintain and define sociolinguistic norms: what can be said in a given context, what is appropriate, and, ultimately, what is human. Further, the design and implementation of systems and tools that treat AI-likeness as a context-free property of text may therefore inherit, and enforce at scale, the norms of whichever contexts they were trained on.
Introduction
As AI-generated content has become increasingly widespread within contemporary information ecologies, there has been a simultaneous proliferation of AI detection practices, both formal and informal. Discourses and practices of AI identification have become an everyday part of how people interact with texts: from tools to help teachers determine the authorship of an essay to Redditors helping someone figure out if their aunt’s new online boyfriend is true love or a romance scam. Despite the fact that detection studies in controlled environments show that untrained human raters typically perform near chance in distinguishing AI-generated from human-generated text [10, 15, 18, 25, 28, 42, 43], folk theories, practices, and discourses of AI identification flourish, both online and off, and there seems to be a widespread sense that there exists a recognizable, describable AI “voice” or “accent”: a way of writing that is distinctly “AI-esque” or “AI-y”, on account of being full of “AI-isms” and “AI stuff”1.
Associations, accurate or not, between form and context are not unique to AI. Research in linguistic anthropology and sociolinguistics has long demonstrated how people make inferences about speakers based on the formal features of their speech and these indexical inferences, regardless of accuracy, can in turn spur evaluations of the trustworthiness and value of both the speaker and the utterance itself [1, 7, 16, 26, 30, 37]. Thus, to understand communication–when, and why, it succeeds or breaks down–in this contemporary moment, it is imperative to take these folk discourses seriously and further our understanding of how, when, and why people flag writing as AI-generated in real-world encounters.
In our study, we analyze survey responses () from high school and undergraduate students in Vermont and 11,882 instances of English Wikipedia editors flagging AI-generated text between October 2024 and September 2026. In this study, we ask what evidence people cite when they identify writing as AI-generated (RQ1), whether and how that evidence changes within a community over time (RQ2), and what an open-ended exploratory analysis of the survey data (not tied to a fixed research question), might reveal about the phenomenon more broadly (EA1).
We find that the cited motivations for AI identifications are diverse, but also often contradictory. Types of evidence range from discrete textual features, judgments about content, inferences about the author, and circular appeals to an “AI-like” quality; and the two communities contradict each other in ways that reflect their own sociolinguistic norms (e.g., students read neutrality and lack of emotion as AI, while Wikipedia editors read enthusiasm and promotional tone as AI) (RQ1). On Wikipedia, the vocabulary of stated reasons is growing, yet also compressing. The justifications editors use to flag something as AI are shortening over time, and a growing share of editors in later months only cite an external community noticeboard rather than naming any feature of the text at all (RQ2). Finally, some students interpreted the statement “this feels very chatgpt” as a dismissal or chastisement of an interlocutor’s stance or behavior, rather than an observation about the text itself (EA1).
From nebulous vibes to contradictory, contextually variable descriptions, it might be easy to dismiss these inconsistent explanations and circular inferencing as just random guesswork. Yet we suggest that these decisions to flag something as AI-generated are often rooted in something real: judgments of pragmatic infelicity, where an otherwise well-formed utterance fails to align with contextual expectations. What often motivates an AI identification, in practice, is not the presence of any stable, universal set of features, but a perceived mismatch between the elements present and those deemed appropriate for the context in which the text occurs. These judgments are not necessarily accurate about a text’s provenance, and that distinction is key.
This paper offers a mixed-methods account of AI identification both as it happens in the wild, in Wikipedia editors’ stated reasons for flagging text, and as people reflect on it, in students’ survey responses about what makes writing seem AI-generated. We argue that “AI-likeness” is a contextual judgment about appropriateness, and that tools and people alike enforce that judgment when they treat detection solely as a property of text. In doing so, we highlight how AI detection practices are not merely a straightforward, neutral practice of identification, but also a social practice that entails defining and policing the boundaries of acceptable human discourse, behavior, and thought.
Background
(Meta)pragmatics. Scholars of pragmatics have “challenged propositional and lexical-grammatical idealizations of language” [46], arguing that language exists only as an abstraction outside its social context: neither the meaning nor effect of an utterance can be comprehensively understood without reference to the pragmatic norms—the culturally-informed rules and expectations that govern the use of language in real-world interactions—of the situation in which it occurs. Formal linguistic competence (i.e., mastery of the code, its grammatical, syntactic, and semantic structure) proves largely inadequate for real-world interactions unless accompanied by “communicative competence” (i.e., how to appropriately use language in a given context) [24]. Discussing how language functions not only to convey propositional information but also to perform social actions, J.L. Austin describes how otherwise well-formed (i.e. grammatical) utterances can fail in their intended effects if pragmatically infelicitous [3]—if the utterance “conflict[s] with contextual expectations or assumptions” [19]. Because pragmatic norms partially determine both meaning and effect, any analysis of metapragmatic discourse must take context into account—and identifications of text presumed to be AI-generated constitute a type of metapragmatic discourse.
Linguistic anthropologists have widely documented how metalinguistic and metapragmatic discourses (i.e. talk about talk; commentary, often evaluative, about language itself and language use, respectively) not only comment on sociolinguistic realities, but bring them into being [27, 44]. Through metapragmatic discourse, pragmatic norms are established and enforced [8, 9, 38, 41] and particular ways of communicating come to be seen as appropriate for particular contexts or associated with particular sorts of persons and practices [14, 29, 47]. It is through metapragmatic discourses that conceptualizations of AI writing solidify and circulate, undergoing enregisterment [49]—the process by which constellations of formal features of language (e.g. accent, style, word choice) come to be recognized as belonging to distinct personae or social contexts, producing a recognizable “register” [2]. Importantly, a register does not necessarily accurately or wholly describe the ways in which language is truly used in the associated context or by its associated speakers [32]. Yet the accuracy of metapragmatic discourse has little bearing on its social consequences. This phenomenon helps explain why the unreliability of human and automated AI detection2 discussed next does not prevent judgments of AI authorship from becoming socially meaningful and consequential: for example, AI accusations can enforce divisions of power by policing participation in knowledge stewardship [35, 36, 48]. Although people regularly refer to a single “AI voice,” we show that what counts as AI-sounding differs between communities, and that when people justify an identification, they often reach past the register’s catalog of features to the surrounding context.
AI detection. People vary widely in their ability to accurately detect AI-generated text (AI detection) depending on the context of the task [15]. Detection studies show that human evaluators often cannot reliably differentiate between LLM-generated and human-generated text [10, 18, 28, 31], although under certain conditions people can improve their detection accuracy [15, 18, 43]. However, people do appear to rely partially on perceived textual features during AI detection, as well as socio-emotional cues [15, 28, 31, 35].
Like human detection, automated AI detection is inconsistent, depending on the specific conditions under which it takes place, including the detection method, the genre and domain of the examined text, the distribution of text the method expects, and the system through which the text was produced [11, 13, 20, 34, 48]. Automated AI detection tools can disproportionately flag text unlike their training data, such as text written in non-native or translated English [36, 48]. For example, Weber-Wulff et al. [48] found that the overall accuracy of 14 automated AI detectors for their corpus3 of human-authored text was 96%, but that the accuracy dropped to 76% when the tools were applied to a corpus of originally human-authored text which had been machine-translated into English. In addition, while 6/14 automated AI detectors yielded false positives, that risk dramatically increased for machine-translated text [48]. Even relatively robust methods are not general detectors of AI provenance, but operate under particular assumptions about AI architecture, how AI is being used, and the domain from which the text originates [34, 39].
Recent work advises that automated AI detection should move away from relying on features known to be context-dependent, instead aiming to detect inconsistencies.4 However, inconsistency must still be contextually defined. The resultant technical, practical, and logistical choices as to training, design, intended use, affordances, and deployment are therefore not neutral, but situated. In these ways, automated AI detection can also inherit and enforce the norms and beliefs of the communities in which it is embedded. Overall, detecting AI-generated text remains a difficult problem [11, 20, 34, 48, 55]. More fundamentally, whether general AI detection is tractable at all is theoretically contested [12, 21, 34, 48, 57, 58].
Methods
Data
We draw on two datasets from two different contexts: English Wikipedia editors flagging text as AI and surveys of students from Vermont. What makes these comparative datasets insightful is that both give us people explaining, in their own words, what made something look like AI to them, but in contexts with quite different expectations about how one ought to communicate. The Wikipedia data is particularly illuminating in how it highlights the process of unprompted AI identifications as part of an ongoing community project.
Our first dataset is our survey. We conducted 74 total surveys, yielding 220 relevant free-text responses, between June and October 2026, which asked high school students and undergraduates in Vermont to respond to open-ended questions about AI writing. Across survey sites, the survey content was identical, but required separate IRB processes and different informed consent processes (due to the age ranges and settings involved). The surveys were administered and collected in person during educational outreach workshops. The relevant surveys were administered in person to ensure that participants were engaged, motivated, and not rushed while they completed them. Workshops involved pre- and post-survey administration where possible, although they may have happened on different days. The survey took people around 5 minutes to complete. See Table 4 in the Appendix for a breakdown of how the survey data was gathered and for survey questions. The adult survey was IRB-approved on June 5, 2026, and the child survey was IRB-approved on June 9, 2026.
| Group | Population | Timing | Total N | Min No. | Inc N | Min Inc No. |
| Adult (18+) | ||||||
| City HS students (2 classes) | Pre Module 2, October 26 | 4 | 4 | 4 | 4 | |
| Undergraduates | Post Day 3, July 10 | 12 | - | 12 | - | |
| Librarians | Pre, June 19–22 | 5 | 5 | 0 | 0 | |
| Librarians | Post, July 7 | 3 | - | 0 | - | |
| Undergraduates | Pre Day 1, June 11 | 15 | 15 | 15 | 15 | |
| Child (<18) | ||||||
| City HS students (2 classes) | Pre Module 2, October 26 | 6 | 6 | 6 | 6 | |
| Statewide HS students | Post Workshop, June 26 | 18 | - | 18 | - | |
| Statewide HS students | Pre Workshop, June 26 | 19 | 19 | 19 | 19 |
Table 4. Survey data collection sessions. Survey administered in-person except for the pre-, Librarian group. 39 total responses for the Adult survey and 43 total responses for the Child survey. The undergraduate group of responses may include up to two somewhat older respondents who were supervising the group of students. The N column provides the number of responses collected during that event. The Min No. column provides the minimum number of unique people taking the survey at that event. Because we did not link Pre- and Post- responses by individual, depending on how people chose to participate, more individuals could have taken the survey than the minimum estimate. However, in general, we were observing the participants as they took the survey, and approximately the same group of people chose to participate in both pre- and post- surveys. The Inc (Included) N column provides the number of responses used as data for this paper. The Min Inc No. column provides the minimum number of unique people who took the survey whose responses we relied on in this work. The Librarian group was excluded from this work to simplify interpretation, as being substantially demographically different from the other participant groups.
The second dataset in our study comes from English Wikipedia, where editors flag pages they believe contain AI-generated text. Editors do this by placing the {{AI-generated}} cleanup template on the page. This renders a maintenance banner and adds the page to a tracking category. The template takes an optional free-text reason= parameter in which the tagging editor can explain what prompted the flag. We collected every placement of this template from October 2024 to September 2026: 11,882 tags across 11,780 unique Wikipedia pages, of which 6,046 (50.8%) carried a stated reason (of the subset with stated reasons, there were 6,004 unique Wikipedia pages). Stated reasons were rather rare before December 2025 (N = 2,717 tags, 52 with a stated reason). Because tags with reasons are rare before December 2025, our time series will typically begin in December 2025 (see Figure 3 for distribution). We pool earlier reasons into a single pre-December bin or omit them if the analysis is just about tags. Analysis that looks at cumulative measures will, however, use all reason-carrying tags. Interestingly, fluctuations in whether editors use reason tags may suggest how much evidence an editor feels they need to bring to the table when claiming something was written by AI. Tags were placed by 797 distinct editors, and the top 3 account for 70% of tags in the pre-December 2025 to September 2026 bins (10,978 of 11,882 tags for which the placing editor could be resolved), so counts are of tags, not of independent decisions.
Figure 3. Compression of the reason= parameter, (a) All tags in the month by what the reason holds. Numbers under the axis are tags in the bin. (b) Coded labels per reason among reasons with at least one prose word (median and interquartile band).
The Wikipedia data observes judgments being made as part of an unprompted collaborative community practice. The Wikipedia reasons were written as part of the editor’s work, to justify to other editors the legitimacy of a tag on a specific page. The template, its reason= field, and the WP:AISIGNS essay page [53] show that editors act as though there is a register. They pull features from the text, build a vocabulary to name them, and then formalize that vocabulary in the essay page, which in turn supplies the next tagger’s wording. The reasons are therefore not just a neutral record of what was in the text, but they are a record of what editors have learned to look for and what registers as suspicious amidst all the text. For that reason, we ask not whether the editor was correct, but what goes into the identification.
Survey Design and Context
The survey was designed jointly for two related projects: this work on indexicality and judgments of AI origin, and the VALI project [59], an ongoing project developing open-source AI Literacy materials. The VALI materials are being co-designed and piloted with a wide range of people in various contexts, and we administered the survey in a small subset of those piloting workshops (5 of them).5 The workshops included willing participants, were IRB approved, and were long enough and were conducted with an appropriate format for the survey to be a relatively minor part of the event from the participants’ perspective.
The full survey covers 18 questions, including 8 regarding the perception, reception, pragmatics, and indexicality of putatively AI-generated text (see Appendix § 10 for the survey questions). For this paper, we relied only on responses to 3 open-ended questions: Q8 (what signs/cues make something seem AI-generated to you?, displayed conditionally to respondents who felt it was at least sometimes obvious when something was written by AI), Q11 (how would you describe what AI writing is like?), and Q14 (why do you think Person A said “this feels very chatgpt”?, following a short example conversation). These yielded 34, 29, and 37 non-blank responses from the adult survey and 37, 40, and 43 from the child survey, 220 in total. The survey was administered through an institutional Qualtrics platform.
Analytical methods
For the survey data, we conducted iterative, thematic analysis of the free-response survey questions 8, 11, and 14. To begin, we generated a list of ’evidence of AI-generated text (for Vermont students)’: each feature survey participants expressly associated with AI-generated text. When extracting features, we retained as much elaboration and variation as possible without overlapping the next index in the respondent’s statement (i.e., “very formal style of writing”, “seems overly formal”, and “tends to be more on the formal side of writing” instead of simply “formal”.) To directly address RQ1, we manually clustered the stated evidence into semantically similar groups, across which we then identified patterns regarding the type of evidence. (e.g., relating to discrete textual features or to the content of a text). We then conducted another thematic analysis of the original list to inductively identify additional patterns and themes in the evidence given, but not constrained to evidentiary type (e.g., this surfaced the themes regarding contradictions in stated evidence and use of context in AI identifications). The final thematic analysis returned to the full corpus of free-response text from survey questions 8, 11, and 14 (particularly 14) to conduct an exploratory, entirely inductive and open-ended analysis, not guided by RQ1, to surface salient patterns and themes that do not directly speak to our stated research question but could inform, complicate, and deepen our understanding of the phenomenon more broadly. The inductive analysis was conducted on the June and July responses. The October 26 responses were then coded against the resulting categories and checked for evidence that would require revising or extending them; none did, so the scheme was retained.
For Wikipedia in RQ1, we characterize the vocabulary editors use when they give a reason for tagging text as AI-generated. The unit is the free-text reason= string in the order editors wrote them. Three coders labeled all unique reasons with one or more labels naming the feature or features the editor cited. Coder 1 trained Coders 2 and 3 in several online sessions on how to recognize a distinct reason and apply labels. Coders 1-3 labeled independently, and ambiguous labels were settled by discussion between all coders. After labeling, Coders 1 and 4 grouped the labels into a two-level schema derived from the labels themselves (Level-1 band (Textual Features, Content, Author, Vibes, External Resources) and a Level-2 sub-band within it (for Textual Features, say, lexicon, structure, quality, error. See Table 5 for complete list)). Coders 1 and 4 also marked each label as either making a pragmatic infelicity claim or not. Note that External Resources labels are excluded from the pragmatic infelicity denominators because here the editors are pointing toward bundled collections of AI markers, some which make infelicity claims and some which do not, and it is not possible to disambiguate which the editors were thinking about.
| Category | Sub-type | Description | Example |
| Text. Features | Lexicon | Word choice | “used highlight as a verb”; “flowery” |
| Text. Features | Orthography | Writing convention | “excessive dashes” |
| Text. Features | Structure | Grammar, syntax, or text formatting | “markdown dividers” |
| Text. Features | Quality | Descriptive or evaluative statement about the textual features as a whole | “high density of AI vocabulary words” |
| Text. Features | Error | Mistakes in grammar or usage | “grammatical errors” |
| Text. Features | Circular | Refers to AI and/or presumes the existence of a knowable register | “prose reminiscent of an essay/thesis written by a language model” |
| Content | Quality | Descriptive or evaluative statement about aspect of the content seen to be inherently “AI-like” | “puffery” |
| Content | Inappropriate | Descriptive or evaluative statement about aspect of the content seen as unfit for Wikipedia article | “overattribution” |
| Content | Remnant | Text leftover from or referring to the interaction with the chatbot that produced it | “infobox placeholders” |
| Content | Inaccuracy | Factually incorrect or fabricated material | “hallucinated citations” |
| Author | Trait | Characteristic attributed to the individual who input the text | “User Newcomer” |
| Author | Behavior | Action or editing pattern attributed to the individual who input the text | “Pattern of AI use from a particular user” |
| Vibes | Circular | Refers to AI and/or presumes the existence of a knowable register | “AI-esque”; “etc etc” |
| Vibes | Sensory | Appeal to intuition or impression | “feels” like AI |
| Vibes | Inappropriate | Vague or undescribed irregularity | “odd” |
| External Rsrc. | Authority | Pointing to external links with authority | “item from noticeboard” |
| External Rsrc. | Tools | Pointing to external detection tools | “Rates high on LLM detection tool” |
Table 5. Coding scheme for evidence cited when flagging text as AI-generated.
For RQ2, we ask whether the vocabulary of reasons is changing over time. We apply Heaps’ law (also sometimes called Herdan’s law) [17, 22, 23] to the coded labels (seen in Figure 2). Heaps’ law describes how the number of distinct types in a text grows with the number of tokens read. An open vocabulary therefore keeps producing new types as the text lengthens. If the vocabulary, on the other hand, is closed, it will saturate. It can be formalized as , , where is the number of tokens, the number of distinct types among them, and and are fitted. is the quantity of interest. Near 1, almost every new token is a new type, and near 0, the vocabulary has closed. For English prose, typically sits around 0.4 to 0.6 [22]. In our analysis, a token is one use of a label on one reason, a type is a distinct label, and we take the reasons in the order editors wrote them, so the curve is the vocabulary as editors produce it. If the features editors cite had settled into a fixed catalog, the curve would flatten, and if editors are still naming new features, it keeps rising.
Figure 2. Vocabulary growth in the reason = parameter. Solid lines are observed distinct types against tokens in the order editors wrote the reasons; dashed lines are power-law fits over . (a) Raw wording (word tokens after stripping links and URLs) and the coded labels applied by the team. (b) Raw wording of reasons that cite the WP:AISIGNS essay against those that do not.
For a corpus of this size, describes the shape of the curve over the observed range rather than estimating an asymptotic exponent. We use as opposed to type-token ratio (TTR) because TTR falls with sample size by construction. So comparing it across groups of different size is invalid [4]. is size-corrected in principle, but a fit over a short range of is still sensitive to that range. So we also rarely, meaning, we subsample each group to a matched many times and compare the number of distinct labels at that , reporting the interquartile band across draws. As a second read on the same question, we report the hapax legomena share, the proportion of label types used exactly once, which stays high while a vocabulary is open and falls as it closes [4]. We compute it on both the raw wording and the coded labels (Figure 11), which also checks that coding did not collapse the tail of rare forms.
Figure 11. Hapax rate per month at matched sample size, raw wording versus coded labels.
We also compare editors who cite the WP:AISIGNS essay page [53] with those who do not (see Figure 2). A reason counts as citing the AI signs essay page when it links to or names the essay or one of its shortcuts (e.g., WP:AISIGNS). We asked whether a shared catalog like the essay page indeed narrows the vocabulary editors draw on or not. We fit Heaps’ law separately for each of the two groups (those who cite the essay and those who do not) and then compare . Because the two groups are different sizes, we made sure to rarefy both to a matched and compare the number of distinct labels at that with interquartile bands across draws.
We conducted two sensitivity checks for features of the data that could have produced these patterns on their own. The first is editor concentration: tagging often takes the editor rather than the page as its unit, so a few high-volume taggers can dominate any monthly count (§ 3.1; Appendix Figs. 12 and 13). We therefore reran the time-series measures on two reduced corpora: one reason per editor per bin, drawn at random and repeated, and the full corpus with the top three editors removed (Figure 14). The second concerns the period before the reason= parameter was regularly used. Since only 52 of the 2,717 tags placed before December 2025 carried a reason, we leave those months off the monthly axes or pool them into a single pre-December bin where a figure shows it, but keep the reasons in every cumulative measure.

Figure 14. Four trend measures under three samplings of the reason corpus. Each panel redraws one measure from the main text on the full corpus (black), on the corpus with the three most prolific editors removed (teal), and on one randomly drawn reason per editor per month (red, dashed; seed fixed). (a) Share of reasons that are link-only. (b) Share of reasons that cite a noticeboard. (c) Mean position of policy shortcuts on the linked–bare–lowercase cline. (d) Share of head-noun mentions with no AI-pointing modifier, on the prose subset. Numbers under the axis are reasons per month with the top three editors removed; a variant’s point is drawn only where that month has at least ten reasons, which is why the reduced series begin late in (d).
Results
RQ1: Evidence Cited in AI Identifications
What evidence do people in these communities cite when identifying text as AI-generated? We show that, in practice, the AI identification practices in which members of these communities engage are contextually-dependent and grounded in a diversity of evidentiary types, from discrete textual features to circular sense-making claims and presumptions regarding the author’s character. Furthermore, people often draw on evidence that is inconsistent and inconstant–contradictory both within and between communities and individuals, and prone to change over time in response to both a shifting sociotechnical landscape and processes of linguistic enregisterment. Importantly, we do not intend to make any claims about the accuracy of these identifications, just about what motivates them. Folk discourses and practices of AI detection are consequential regardless of their accuracy as they have concrete social impacts and reveal powerful social norms.
4.1.1 Types of Evidential Categories. Both student survey respondents and Wikipedia editors cited a range of evidence for their AI identifications. This evidence was not limited to discrete textual features–such as the much-maligned em-dash–but also included more abstract qualities and information external to the text itself. From the survey data, we categorized this evidence into four general kinds:
Textual Features. This included lexical choices such as “fancy words and metaphors”, syntactical patterns such as the construction “Not only X, but Y”, and qualities such as “overly formal” or “overly sweet and solicitous”.
Content. This included assessments of the content, rather than the form, such as having a neutral perspective and “refusal to commit to an opinion” or even expressing an opinion that the survey respondents deemed not one that an actual human being would ever hold.
Author. This involved presumptions of the author’s character or behavior that might explain their use of AI, such as being someone who does not care enough to “really put any effort into the conversation”.
Vibes. This final category included more nebulous, and often circular, evidence such as the text is “AI-y”, “Ai sounding” “lacking genuine human vibes”, “uncanny”, “weird”, “strange”, “soulless”, or “just doesn’t seem right” and “sounds fake”.
The Wikipedia editors’ labels mostly fell into these four broad categories of evidence, but this data also presented an additional category, (5) External Resources where editors referred to external documents compiling “known” markers of AI writing in the WP:AISIGNS essay page or AI detection tools such as Pangram or GPTZero, and allowed us to bring a finer resolution to the analysis and further develop sub-types within these broader categories. The full coding schema with examples and descriptions of evidence cited when flagging text as AI-generated can be seen in Appendix Table 5.
4.1.2 Confident, But Contradictory Evidence. While over 90% of survey respondents stated they felt confident that they could “at least sometimes” tell when a text was written by AI, there was far less uniformity in their descriptions of what signs or cues made something seem AI-generated to them (see Table 1). Many of the properties they cited as evidence of AI writing were expressly contradictory. Some of the students characterized AI-generated texts as unusual and overwrought, with “flowery”, “advanced”, or “uncommon” words and “long and descriptive”, “too detailed” sentences. In contrast, others said they could identify AI-generated texts because it was so bland and simplistic, with “plain”, “low sophistication” vocabulary and “a lot of short sentences”. Similarly, the students often made reference to errors when explaining why they might identify a text as being AI-generated, but did so in conflicting ways. Sometimes, the presence of errors indexed AI, the sign of an incompetent technology that lacks the capacity to avoid mistakes. Other times, it was an absence of errors that indexed AI, the sign of a precise machine that lacks inherently human imperfections.
| Unusual / Overwrought | Bland / Simplistic |
| “too many unnecessary fancy words and metaphors”; “too detailed”; “flowery language”; “uncommon words and grammar”; “Advanced language”; “Very long and descriptive”; “long sentences” | “the writing style is very basic”; “bland”; “plain”; “very plain”; “low sophistication in vocabulary”; “Short, punchy sentences”; “a lot of short sentences” |
| Errors Present | Errors Absent |
| “often has foolish and simple mistakes”; “errors in grammar or lots of weird grammar”; “weird grammar. Strange wordings”; “inaccurate”; “It can have major errors [. . . ]” | “it doesnt have any spelling/grammar mistakes”; “lack of grammatical and syntactical errors”; “smart”; “[if] their writing is too good people will get skeptical and say it’s ai” |
| Friendly / Polite | Formal / Academic |
| “polite”; “overly friendly”; “overly sweet and solicitous”; “overly nice”; “It feels like it is being weirdly friendly”; “a high amount of politeness” | “very acedemic language style”; “very formal style of writing”; “seem overly formal”; “uses grammar that is often believed to be extremely formal” |
Table 1. Examples of contradictory AI indices cited by participants. Quotes are verbatim.
These contradictions could even be seen within individuals. One student stated in one place that AI writing could be identified by the “lack of opinions” expressed and its tendency toward hedging, and, in another place, stated that it was “very direct”. Another student described AI writing as both having “extremely formal” grammar and making many “errors in grammar” and yet another characterized AI writing as being full of “Short punchy sentences” and also as something that “communicates deep ideas but in ways that dont focus on concision”. These contradictions persisted, even when moving beyond discrete formal features to more abstract qualities and characteristics. For example, survey participants overwhelmingly described AI-writing as “overly nice”, “weirdly friendly”, and “polite”, while also describing it as “overly formal” and “academic”.
There were a handful of features identified by our survey participants as signs of AI writing that had very few contradictions. This included some textual features that, likely not incidentally, have become the object of widespread metalinguistic discourses in the media, such as the em-dash [33, 56], as well as a widely shared sense that AI writing is neutral to a fault, lacking emotion and opinion. While multiple respondents discussed how the information presented in an AI-generated text could be biased, the text itself was largely characterized as having a “neutral tone”, “no emotion”, and a distinct “lack of opinions”. This seemed to be viewed negatively, not as a marker of impartiality and open-mindedness, but of inauthenticity and a lack of commitment or care. As one student stated, AI-generated text “seems like a boiler-plate answer that says all the ‘right’ things but doesn’t seem like a person who cares about the issue is sharing it.” There were a handful of respondents who associated AI writing with expressions of sentiment, but even these individuals associated it specifically with failed expressions of sentiment: “the text is trying to be sentimental”, “Tries and Fails to write with emotion” [emphasis ours].
If the most consistent quality seen to index AI for the students was neutrality and a lack of emotion or opinion, the most consistent quality seen to index AI for the Wikipedia editors was the exact opposite: “non-neutral language”, writing that was “too enthusiastic” and suggested the author has a personal investment in expressing a particular perspective, stance, or opinion (seen in Table 2).
| Students’ AI Indices | Wikipedia Editors’ AI Indices |
| “neutral tone, playing both sides”; “super neutral”; “lack of opinions”; “refusal to commit to an opinion”; “no emotion”; “doesn’t lean into emotions”; “Unemotional”; “it takes the emotions out”; “not really having an opinion” | “non-neutral language”; “random adjectives that feel opinionated”; “editorializing tone”; “undue emphasis on significance”; “undue emphasis on importance”; “promotional tone”; “extreme promotional tone”; “Tone seems very promotional and unencyclopedic”; “significant puffery”; “peacock phrasing”; “fake profundity”; “too enthusiastic” |
| “lack of personal connection between text and author”; “doesn’t seem like a person who cares about the issue”; “soulless”; “sounds like the smile given by a depressed chick fil a employee as they tell you my pleasure, except the soul of that employee has been replaced by a rock.” |
Table 2. Examples of neutral versus editorial AI indices cited by participants across Wiki and survey data. Quotes are verbatim. Survey respondents described writing that seems neutral, that lacks opinion, or lacks emotion as indexing AI. Wikipedia editors described writing that seems non-neutral, that has an editorializing or promotional tone, as indexing AI.
The Salience of Context
These results demonstrate that real-world identifications of AI-generated content significantly rely on contextual information. Wikipedia editors occupy a distinctly different context from our survey respondents: one concerned with the production of encyclopedic knowledge, rather than casual, interpersonal interactions. To participate appropriately in casual, personal interactions with fellow students is to be authentic and invested in the interaction, not someone who “says like all the ‘right’ things” but is emotionally absent. On the other hand, to participate appropriately in the production of Wikipedia articles is to be neutral and not have a conflict of interest, writing in an “encyclopedic tone” that reports what sources say without “undue emphasis on significance” or a “promotional tone,” not someone who sounds personally invested in the subject. The two contexts have different sociolinguistic norms and expectations, and these contextual differences directly influence when and how people identify text as AI-generated. In practice, therefore, what often motivates AI identifications is not the presence of a stable, universal set of features or qualities, but the presence of features or qualities deemed pragmatically inappropriate for that specific context.
This is further attested by the abundant use of intensifiers such as “too” and “overly” used in descriptions of AI writing. Often it was not the fact that the text was formal, detailed, friendly, enthusiastic, or had a particular feature that prompted AI identifications; it was that the text was perceived as “overly formal”, “too detailed”, “overly friendly”, “too enthusiastic”, or had “too much” or “too many” instances of a feature, which implies that the sign of AI writing is not the quality itself, but specifically an excessive amount of that quality for the context. Some survey respondents expressed this directly, with one stating that the “dashes” that prompted their AI identification seem “kinda out of place for a conversation that seems more informal” and others that “it has all [the] grammar stuff you do not need for texting someone” or used “formal structure in an informal conversation”.
To determine the significance of contextual expectations in real-world AI identifications, we manually coded the editors’ reasons into labels, then assigned each unique label a meta-label indicating whether it makes an infidelity claim (seen in Figure 1). Of the 11,882 {AI-generated} reason tags collected, 6,046 carried a non-empty reason= parameter and form the basis of this analysis. Labels assigned the pragmatic infelicity meta-label included both those that directly commented on the contextual mismatch (e.g., “not using encyclopedic tone” when the context is explicitly an encyclopedia) as well as more implicit references, such as a markup style that is not commonly used on Wikipedia.
Figure 1. Share of reasons that make a pragmatic-infelicity claim. Red line: percentage of reason-carrying tags in the month whose reason received at least one label coded as infelicity. Gray line: percentage of all label uses in the month that are infelicity labels. External/Authority labels are not coded for infelicity and are excluded from both lines, so link-only reasons are omitted; numbers under the axis are the reasons that remain.
RQ2: Features changing over time
The Wikipedia data, because it was temporal, let us watch these editors’ judgments accumulate over time. This allowed us to ask what features editors cite when they flag something as AI-written, and how these features change as the vocabulary grows and the editors begin to form into a community of practice around the act of flagging text. Our results show that the vocabulary of reasons is forming into a register over time and that, in recent months, it is compressing. The referents to the community of practice are acquiring more weight and attention in the reasons. For example, between December 2025 and September 2026, the share of reasons citing a textual feature falls from roughly 80% to under 20%, while the share citing external authority rises to above 80% (Appendix Fig. 10).
We also see that the vocabulary is still under construction. Editors are continuously creating new variants of features (hapax legomena). Of the 467 label types applied to at least one reason (the team’s inventory holds 472; five were never applied), 212 (45.4%) are used exactly once, yet these account for only 1.17% of the 18,106 label tokens (Baayen’s ), and the list of distinct features is still growing at the end of our collection window. Across the 6,046 reasons, the Heaps exponent is for the coded labels and for the raw wording, both in the range of an open natural-language lexicon, and neither curve appears to be flattening (Figure 2a). Interestingly, the vocabulary is also positioned inside a feedback loop with the Wikipedia:Signs of AI writing essay page (short label WP:AISIGNS) [53], a catalog which both inherits features from the flagging editors and, in turn, informs editors on what features to flag. Features that Wikipedia editors cite in their reasons are formalized into the catalog, and the essay in turn becomes a list the next tagger can draw on to justify a flag. We see that the catalog measurably narrows what gets said: reasons that cite WP:AISIGNS grow their vocabulary at , against for reasons that do not, a gap that survives rarefaction to matched (Figure 2b). These are not wholly distinct communities. Editors of the essay page are also flaggers: 66 editors (out of the 797 who placed tags in our dataset) both edited the essay and placed tags. In our explorations of this sub-community, we see them adding words and examples drawn from their own experience flagging text, discussing that experience on the essay’s talk page, and clarifying entries because other editors are applying them incorrectly in their reasons.
Over the same period, the reasons themselves are starting to concentrate on a core set of features that editors regularly use. The reasons are also getting shorter over time. Reasons of four or more prose words fall from 63–78% of tags in January–February to 20–26% by August–September, while link-only reasons (which were almost absent before June) reach 30–45% of all tags (Figure 3). Editors cite fewer features over time, with the median number of coded labels per reason dropping from five in January to two in August, and to one once link-only reasons are included (Figure 3). Within the external resource warrants themselves, the two main sources trade places. Through April, more than 94% of External reasons point to the WP:AISIGNS essay page, which is a catalog of textual features. From July the majority instead point to the AI noticeboard (60–81%), while the essay falls to 32% in July and to 14–20% in August–September (Appendix Fig. 4). The warrant shifts from a description of what AI text looks like to a record of what the community has already noticed. A growing share of flags defer entirely to the noticeboard: it is cited in under a quarter of reasons in any month through June (and under 5% in most of them), and in 53–70% from July onward (Appendix Fig. 6). A link to the noticeboard is the sole content of 1,475 of its 1,801 uses (Appendix Fig. 5). The AI noticeboard, importantly, is the coordination page of WikiProject AI Cleanup, the community that coalesced around identifying and cleaning up AI-generated text. A reason that reads only “Wikipedia:AI_noticeboard” usually names no textual feature at all. It warrants the tag by the fact that the community has already noticed the page.
A similar trend appears as well at the word level. We see that head nouns such as “structure”, “vocabulary”, “prose”, and “formatting” increasingly appear on their own, whereas they used to be surrounded by additional explanatory text such as “AI looking vocabulary” or “Outline Structure looks AI” or points to a specific part of the text where the AI-looking writing occurs, such as saying the AI feature is the “outline structure about challenges.” We see the bare head nouns’ share rising about 3.8 percentage points per month (Appendix Fig. 7).
These shifts are not an artifact of the few very prolific taggers in the set. The most active editor’s monthly share of tags falls from 53% in June to 25% in September while the number of distinct taggers rises from 128 to 163, so the rise in noticeboard deferral coincides with the practice spreading to more editors, not with one editor’s output growing. Under both reduced samplings (top three editors removed; one reason per editor per month) the rise in link-only reasons and in noticeboard deferral is preserved but arrives later and at a lower volume: link-only reasons reach 47% of the
remaining editors’ reasons in September rather than 53% in July (Appendix Fig. 14). Both practices appear first in the reasons of the three most prolific taggers and are taken up by the wider community roughly two months later, a diffusion pattern in which high-volume taggers lead and the wider community follows.
The growth curve in Figure 2 and the compression measures (Appendix Fig. 4 and Figure 7) show us both type and token behavior. The growth curve gives us the complete list of all of the unique features editors have cited in their reasons and we see this list still expanding. The compression measures show which of those features actually get used, and that set is shrinking toward a handful of common forms. The 212 features used only once are the clearest instance, where nearly half of the inventory is only one percent of the use. This is what enregisterment looks like in practice, where a core set of features is surrounded by an open periphery that is still growing with new variants [2].
Figure 7. Head nouns used without an AI-pointing modifier, by month. Among reasons with at least four prose words, each mention of a feature noun (vocabulary, formatting, prose, structure, language) is counted as modified if a term such as AI, LLM, ChatGPT, generated, typical, or AI-like occurs within three tokens of it, and bare otherwise. Thin lines show individual nouns (months with at least five mentions; total mentions in parentheses); the heavy line pools all nouns, and the dashed line is a weighted least-squares trend over months with at least ten pooled mentions (weights ).
EA1: Exploratory Results
Informed by the epistemological sensibilities of the anthropologist on our team, our methodology intentionally left space for the generation of results that exceeded the boundaries of our initial research questions, for answers we did not even think to seek. The final, entirely inductive, thematic analysis we conducted on the complete corpus of open-ended survey responses revealed that, in practice, real-world identifications of AI-generated content may also have a social function: they define and enforce pragmatic infelicity, not just neutrally recognize it. Question 14 asked the students to read a brief exchange between Person A and Person B on the ethics of ghosting and then comment on why Person A might have said “this feels very chatgpt” in reply to Person B’s comment that ghosting was not ideal, but some situations were more complicated than acknowledged by Person A’s initial statement which dismissed the practice as “never ok”. Many responses pointed out various aspects of the text that the student believed might have prompted Person A’s identification, such as “those dash things” and the lack of bias when “most people are more biased than that”.
Some respondents, however, interpreted the question “why do you think Person A said ‘this feels very chatgpt’” to be not about what features prompted Person A to believe Person B’s response was written by ChatGPT, but what Person A hoped to accomplish by saying it seemed like Person B response was written by ChatGPT. These respondents inferred that the purpose of Person A’s AI identification was to police Person B’s behavior, undermine their perspective, or end the discussion. These students framed AI identifications as a rhetorical strategy “that shuts down the ability to bring genuine concerns up since you will be deemed “fake/inhuman”” and, therefore, making the identification had more to do with the response being “not what [Person A] wants to hear” rather than any of its textual features (see Table 3).
| Evidentiary motivations | Social motivations |
| “not really having an opinion” | “to make [Person A’s] opinion known” |
| “because of the m dashes and the knowledge that the average person wouldn’t have” | “accuses Person B of communicating without emotion” |
| “those dash things” | “shuts down the ability to bring genuine concerns up since you will be deemed ‘fake/inhuman’” |
| “the em dashes and the sentence structure” | “minimizing”, “minimizes the comment” |
| “Because of the opening line ‘I get that it can feel. . .’ lots of LLMs kinda repeat things back when they start a response” | “shut [Person B] down using logical fallacy/attacking the speaker not their argument” |
| “unnecessary use of adjectives” | “It dismisses the valid claims made by person b. Any criticism can be dismissed making learning/improving as a person harder” |
| “A but B” |
Table 3. Why Person A said “this feels very ChatGPT” (according to high school and college students)
Discussion
This paper highlights the plurality of an emergent AI register and the salience of context for real-world identifications of AI-generated content. We show how the constellation of textual features that have come to be indexically linked to AI can differ across social contexts and that, despite the consolidation of the register(s) over time, real-world identifications of AI-generated content do not always evoke those features in their evidentiary claims, but draw on a range of other contextual factors. Taken together, our results encourage scholars, practitioners, and others to consider AI detection not merely as an act of discovery, but a social action: a prescription, a rhetorical strategy, a world-making practice.
Exploring this phenomenon across two communities with very different expectations about how writing ought to look further reveals that what often prompts someone to identify a piece of writing as AI-generated is pragmatic infelicity, a perceived mismatch between the text and its context, not a fixed set of features. This shows up most clearly where the two groups contradict each other, yet do so in a manner that reflects the differing pragmatic norms of their respective social contexts. In a text thread between friends, emotional investment is the expectation and neutrality is infelicitous. In an encyclopedia, neutrality is the expectation and enthusiasm is infelicitous. Accordingly, our student respondents described AI writing as “super neutral,” “unemotional,” and “soulless” and Wikipedia editors flagged it as the opposite: “non-neutral language,” “promotional tone,” “too enthusiastic.”
Exploring this phenomenon over time within the Wikipedia community additionally reveals qualitative and quantitative insights into the process of enregisterment. In the first months of the flagging practice, new metapragmatic vocabulary forms then get codified in the Wikipedia:Signs of AI writing essay. Then a core set of features gets adopted by flaggers while the periphery keeps growing with new variants. The reasons compress to shortened forms and bare head nouns (“vocabulary,” “structure” written on their own as justifications without context) and, finally, link to an external authority, the WikiProject AI Cleanup noticeboard, with no textual feature named at all. This evolution, along with the results from our exploratory thematic analysis that showed survey respondents interpreting AI identification as a speech act intended to police an interlocutor’s behavior, points to how real-world identifications of AI-generated content might not just comment on AI writing, but define it—and, in turn, define human writing.
Our work does not offer a definitive approach to correctly detect AI writing, and the list of features we collected should emphatically not be used this way. We demonstrate how flagging writing as AI is often a contextual act, and when we package that act as a tool, we can lose sight of the fact that part of that tool came from human judgment. For example, through ordinary community work on Wikipedia, the shift from textual warrant to external warrant unintentionally renders situated judgments invisible, giving them the appearance of objective fact. While the Wikipedia essay itself is careful to say that its features are not definitive signs of AI writing, in practice, they become them–particularly as the catalog circulates well beyond Wikipedia. Humanizer, an agent skill with over 63,400 GitHub stars at the time of writing (November 3, 2026), states it is built on the Wikipedia:Signs of AI writing essay [53] and reproduces its features as a numbered list of patterns to strip from text [6]. This tool, and other AI detection tools, can, thus, encode a specific community’s ideas about appropriate text for their local context as a universal metric for sounding human, which risks making AI writing whatever a particular detection procedure can operationalize.6 This is a familiar pattern in automation where a designer’s judgment is built into the system: the tool shows its score but not the context in which that judgment was made, and the people left to monitor it lose the skill and vigilance to catch its errors [5]–and because they tend to trust an automated score over their own judgment [40, 45], the packaged tool is read as a verdict.
Beyond what this means for designing and investigating AI detection tools, these everyday practices and discourses of AI identification generate intense social pressures to avoid getting flagged as AI. While writing this paper, we found ourselves hyper-aware of every em dash, every time we used highlight as a verb, and every list of three. It is an impossibly constrained Goldilocks situation: to be neither too enthusiastic nor too apathetic, too opinionated nor too neutral, too bland nor too flowery–and, most importantly, a perfect fit for someone else’s perception of appropriateness. The lexical space that human writers feel comfortable drawing from thus narrows and the result is writing produced under suspicion. Ultimately, this may lead to the constriction of how–and what–we can express.
Limitations & Future Research
While by intention and reflective of the core theoretical commitments of this work, the survey sample is small, particular, and not generalizable. Additionally, because students were asked to reflect on given stimuli, the survey does not observe unprompted AI identifications. Thus we have insight into types of evidence, but not what might stand out in the moment as warranting an identification. Importantly, while treated as largely distinct in this study neither the student community nor Wikipedia are closed systems–the boundaries between contexts are inevitably amorphous and permeable and future work should consider how they influence each other. (e.g., how viral videos mimicking an “AI accent” might feed into student and Wikipedia editors indexical frameworks.) In addition, The Wikipedia data only covers roughly ten months of usable time series, therefore despite showing change it is still a snapshot in time.
More broadly, the Author and External bands suggest that as LLMs make text a weaker signal of its own production conditions, people may be widening the evidentiary base they use to make language meaningful, drawing on editing behavior, timing, reputation, and community authority alongside the text itself; whether evidence of provenance is migrating outward from the linguistic artifact in this way is a question we take up in future work.
Prompted by the results of our exploratory analysis, we plan to further investigate the social and interactional components of AI identifications and their motivations. We will conduct discourse analysis of social media interactions on Reddit and TikTok where one interlocutor identifies another’s writing as AI-generated to further explore the social consequences and motivations of real-world AI identifications. When are these identifications made? How do they shape the interaction going forward? Similarly, we plan to examine the exchanges between editors who flag AI text and those they accuse, on the noticeboard and in the spaces where accused editors seek help and form community to gain insight the role AI identifications might play in community and identity formation.
Acknowledgments
The authors thank June Clark for her work on the VALI project and deployment of the survey. We would also like to thank the Fletcher Free Library, the Governor’s Institute of Vermont, Jessamyn West, Alexa Woodward, Renee Simkins, Maggie Jones, the Osher Lifelong Learning Institute, Katie Ekstrom Grenon, and Hannah Reing for helping us realize opportunities to administer the survey.
Our team acknowledges support from the Alfred P. Sloan Foundation (Grant #G-2024-22498) and the National Science Foundation (Grant #2242829).
Author Contributions
Conceptualization: M.M., J.L., J.Z.; Methodology: M.M., J.L., J.Z.; Data Analysis: J.L., J.Z., M.M.; Wikipedia Data curation: J.L.; Data Annotation: J.L., J.Z., T.T.P., M.M.; Survey development: J.Z., B.L., C.B., M.M.; Survey Administration: J.Z., K.H., B.L., C.B.; Data visualization: J.L., J.Z.; Writing original draft: M.M., J.L., J.Z.; All authors approved the final submitted draft.
Generative AI Usage Statement
The authors wrote this manuscript. One author (M.M.) used no generative AI tools at any stage of the project. The remaining authors used LLM-based tools in supporting roles such as grammar and spell-checking and copyediting of their own text (Writefull latest version as of November 2026), help with LaTeX, citation, and figure formatting, locating gaps in sources, and coding assistance for exploratory visualizations (Claude Fable 5.1, Copilot for Microsoft 365 latest version as of November 2026). No analysis, finding, or claim in this paper originated with an AI tool, and the authors designed, reviewed, and ran all code and wrote all text.
Ethics and Privacy Statement
The survey part of this study was approved by the University of Vermont Institutional Review Board (IRB protocols (behavioral) STUDY00004178 and STUDY00004269). All participants provided informed consent prior to completing the survey. The Wikipedia data that we used for this project is all public revision histories of pages which Wikipedia publishes under an open license GNU Free Documentation License and the Creative Commons Attribution-ShareAlike 4.0 International License, and through public APIs. This work is purely observational, we did not make any edits, place any tags, or send any messages to editors. This complies with Wikipedia’s policy that research projects should not treat Wikipedia as a laboratory (WP:NOTLAB) [54]. We also comply with Wikipedia’s research guidance, under which offline analysis of public data does not require subject recruitment (Wikipedia:Research, Wikipedia:Ethically researching Wikipedia) [51, 52]. Individual editors on Wikipedia act under public usernames, however, we chose to analyze flagging at the community level. We also replace usernames with salted hashes during our analysis. We do, however, quote editors’ AI-flagging reasons verbatim rather than paraphrasing them, because their wording is the object of our study. Quotes are never paired with usernames. And the strings themselves are already public under Wikipedia’s open license. co-designed and piloted Spring 2026 - Summer 2027. Paper in progress.. https://vermontcomplexsystems.org/projects/vali/
Survey Questions
The following reproduces the wording and response options presented to participants. Formatting has been adapted for presentation in the manuscript. The survey was administered via an institutional Qualtrics platform.
Included two initial pages without survey questions covering: Research Information Sheet, Voluntary Participation/Withdrawal, Questions, followed by:
Thank you for reading the information sheet. As a recap: you are being invited to take part in this research study because you are a Vermonter involved in some aspect of K-12 education.
This study is being conducted by the Vermont Complex Systems Institute at the University of Vermont
The purpose of this study is to understand how people think about LLMs and to understand and improve the effectiveness of our VALI project curriculum at:
(1) increasing knowledge on how large language models (LLMs) work
(2) increasing confidence and intuition when working with generative tools like LLMs
(3) understanding potential sources of LLM bias.
You will be asked a series of questions about your understanding and opinions of AI, as well as questions regarding the VALI curriculum (if applicable).
In the following questions, you may be asked to relay your understanding and opinions about LLMs, and generative AI. For clarity, we use the term “Chatbot” to capture any interactions with any generative AI tools (e.g., ChatGPT, Claude, Gemini, etc.)
Q1. In which Vermont county do you teach or work? [Text box]7
Q2. What is your role? (Pick the choice that best describes what you do most of the time). List of options: Educator (K-12), Educator (University), Administrator (K-12), Administrator (University), Other Educational Job (e.g. IT support, educational technologist, etc.), Student, Parent or Guardian, Librarian, Other: [Text box]
Q3. How important do you think it is to learn about AI and Chatbots? List of options: Not at all important, Slightly important, Moderately important, Very important, Extremely important
Q4. How comfortable are you using a Chatbot? List of options: Not at all confident, Slightly confident, Moderately confident, Very confident, Extremely confident
Q5. How comfortable do you feel saying “I’m not sure that’s right” when something a Chatbot says does not make sense to you? List of options: Not at all confident, Slightly confident, Moderately confident, Very confident, Extremely confident
Q6. Did you know that a Chatbot can sometimes get things wrong or be unfair, even when they seem very smart? List of options: Not at all, A little, Somewhat, Quite a bit, Extremely
Q7. Do you feel it is obvious when something was written by AI? List of options: Yes; often, Sometimes, No; not typically, I don’t know
Q8. If yes or sometimes, what signs/cues make something seem AI-generated to you? [Text box] [Conditionally displayed]
Q9. When you think something is AI-generated, how does it come across? For each pair, select the word that best describes your views. Pairs: Genuine, sincere/ Insincere, fake; Trustworthy, reliable/ Untrustworthy, unreliable; Powerful, effective/ Weak, useless
Q10. In general, which (if any) of the following words do you think describe what AI writing is like? Select all that apply. List of options: Professional, Casual, Formal, Submissive, Sycophantic, Friendly, Rude, Direct, Emotional, Logical, Human-like, Mechanical, Foolish, Unintelligent, Wise, Smart
Q11. In general, how would you describe what AI writing is like? [Text box]
Q12. How do you feel when someone sends you content you believe is AI-generated? Please mark on a scale from 1-7, where 1 represents “negatively” and 7 represents “positively”. List of options: Negatively (1), 2, 3, Neutral (4), 5, 6, Positively (7)
Q13. In general, do you usually feel... List of options: Annoyed, Skeptical, Indifferent; neutral, Excited, Impressed, Other: [Text box]
Please read the following example conversation and then answer the question that follows:
Person A: I totally disagree. I just don’t think it is ever ok to ghost someone. Cutting contact or not showing up without any explanation or excuse is so disrespectful. It hurts.
Person B: I get that it can feel hurtful, but saying it’s never okay ignores context and power differences. In some situations—especially where there are safety concerns or uneven dynamics—ghosting can be a way for someone to set boundaries without risking escalation. It’s not always ideal, but it’s not universally wrong either.
Person A: this feels very chatgpt
Q14. Why do you think Person A said “this feels very chatgpt”? What effect do you think it had on the interaction?
[Text box]
Q15. Did you participate in a full-length focus group or piloting session with the curriculum materials? List of options: Yes, No
[Conditionally displayed if Yes] Q16. Would these materials improve your students’ understanding of AI? List of options: Not at all, A little, Somewhat, Quite a bit, Extremely
[Conditionally displayed if Yes] Q17. How confident are you that these materials could be used by another teacher (without any further support)? List of options: Not at all, A little, Somewhat, Quite a bit, Extremely
[Conditionally displayed if Yes] Q18. Would these materials engage your students at an appropriate level? List of options: Not at all, A little, Somewhat, Quite a bit, Extremely
Survey Supplementary Figures and Tables
Table 4 summarizes the survey data collection sessions, including group, timing, response counts, and which responses were included in the analysis.
Wikipedia Coding Schema
Table 5 presents the full coding scheme used to classify the evidence cited in the {{AI-generated}} tag reasons, organized by Level-1 category and Level-2 sub-type with a description and example for each.
Inventory of coded labels on Wikipedia
Table 6–Table 9 list all 472 unique labels in the team’s label inventory, grouped by Level-1 band and Level-2 sub-band. Labels are reproduced as recorded in the team’s decisions file, including editors’ spelling, capitalization, and Wikipedia shortcuts (WP:. . .); near-duplicates (e.g. AIBOLD and WP:AIBOLD) are kept as separate labels because they correspond to distinct surface forms in the reason= parameter. A dagger (†) marks the 158 labels the team coded as expressing pragmatic infelicity. Labels in the External/Authority sub-band were not coded for infelicity. Counts in parentheses give the number of labels in each sub-band.
| Sub-band () | Labels |
| Lexicon (61) | “accentuates”; AI phrasing; AIVOCAB; “align with”; avoidance of basic copulatives; “central”; “conceptual turning point”; “culminated”; “depicting”; “emerged from”; emojis{sup}†; “enduring status”; “establishing a framework”; “exceptionally”{sup}†; “focal point”; “fostering”; “here is”; “is regarded as a prominent...” ; jargon; “Key X” header; LLM-typical words like “underscores”; “maintains a presence”; nonsensical language; overly consistent vocabulary{sup}†; overuse of “referred to”{sup}†; “perfect”; “phenomenal”{sup}†; “pioneering”{sup}†; “pristine”{sup}†; random adjectives; “Remains a central theme”; “revealed”; “rich tapestry”; “showcasing”; “signalling an emphasis”; “stands as a testament”; strange wod choices{sup}†; Subsequently; suspicious AI lexica; “the novel marked”; “they state”; too many adjectives{sup}†; “traced to”; typical llm verbiage/vocab; “underscored”; use of word “key”; use of word “pivotal”; use of word “signify”; used “highlight” as a verb; Used word “emphasizing”; Uses AI phrase “reflects”; Uses AI phrase “significant”; Uses AI phrase “transition”; uses wording such as “highlighting”; vocab distro typical of LLMS; vocabulary co-occurrence; “whilst”{sup}†; WP:AIVOCAB; WP:AIWORDS; WP:SAID{sup}†` |
| Orthography (13) | AIBOLD; curly quote marks; em dash; excessive dashes; LLM Looking Bolding; punctuation; reference markers in plaintext{sup}†`; spaces; spaces after sentences and citations; title case; treating article title as proper noun; WP:AIBOLD; WP:AIDASH |
| Structure (66) | AILIST; AILIST/AIBOLD combo; AITABLE; avoidance of is/are phrases; Bolding of random terms; bulleted list; canned list formatting; categories; de-tag{sup}†; deprecated lang templates{sup}†; excessive sectioning{sup}†; formatting; formatting irregularities{sup}†; grammar and use of commas; header irregularities; headings; Infobox is a liable instead of an infobox template{sup}†; inline external links{sup}†; lead structure as standalone entity{sup}†; length{sup}†; List styling looks AI; listification; Lists of threes; LLM Looking Word Chaining; malformed AfC template{sup}†; markdown{sup}†; MARKDOWN{sup}†; markdown code blocks; markdown dividers{sup}†; mechanically consistent sentence structure; negative parallelism; non-Wiki markup{sup}†; nonexisting templates,{sup}†; nonsensical use of wiki template{sup}†; nonstandard formatting{sup}†; nonstandard markdown{sup}†; outline structure about challenges{sup}†; outline structure about future prospects{sup}†; Outline Structure looks AI; overuse of lists{sup}†; parallelisms; passive voice{sup}†; reference URLs were formatted as Markdown{sup}†; repeated words; rule of three; sentence structure; Signs of AI writing; strange stub tag{sup}†; strangely formatted citations{sup}†; structural signs; suspicious link style; syntax; template structure; the text is wrapped in a markdown code block{sup}†; thematic breaks; undue list styling{sup}†; unnecessary bolding{sup}†; unusual use of tables{sup}†; weird formatting{sup}†; WP:AILIST; WP:AIMARKDOWN; WP:AIPARALLEL; WP:AITABLE; WP:AITITLECASE{sup}†; WP:MARKDOWN; WP:RO3 |
| Quality (39) | a lot of words about not much; AIDISCLAIMER{sup}†; Busy assortment of subheadings{sup}†; canned emphasis on association{sup}†; canned emphasis on identification; canned emphasis on legacy{sup}†; canned emphasis on reportage; canned emphasis on significance{sup}†; didactic disclaimer{sup}†; editorializing tone{sup}†; elaborate headings{sup}†; Florid prose{sup}†; flowery{sup}†; formulaic prose; general tone and theme{sup}†; hedging{sup}†; may be LLM revising specifically; Meandering{sup}†; non neutral tone{sup}†; not using encyclopedic tone{sup}†; Opinionated adjectives{sup}†; overly complex sentences{sup}†; overstated language{sup}†; Overuse of descriptive language{sup}†; passive language; peacock phrasing{sup}†; promotional tone{sup}†; strange styled writing{sup}†; suspicious AI style; technical evidence but unclear tone{sup}†; too enthusiastic{sup}†; tortured AIATTR; vague language{sup}†; verbose prose{sup}†; weird prose{sup}†; wordy{sup}†; WP:AIDISCLAIMER{sup}†; writing that is suspiciously sophisticated{sup}† |
| Error (13) | broken templates; contain the +1; deprecated infobox parameters; extraneous article titles; grammatical errors; harv/sfn citation error; inconsistencies; oaitication; OAICITE; random characters in a portion of a sentence; references that redirect; WP:AIFICTREF; WP:OAICITE |
| Circular (8) | grammar; High density of AI vocabulary words; LLM-like language; prose indicators; prose reminiscent of an essay/thesis written by a language model; vocab/syntax distro typical of LLMS; vocabulary; writing style{sup}†` |
| (no sub-band) (1) | formatting headers{sup}†` |
Table 6. Coded labels in the Textual features band (201 labels). {sup}†` = coded as pragmatic infelicity.
Table 9. Coded labels in the External, Vibes, Other bands (62 labels). † = coded as pragmatic infelicity.
| Sub-band () | Labels |
| External (28 labels) | |
| Authority (26) | AISIGNS; Contained G15 eligible user communication; Item from AI Cleanup Noticeboard; item from noticeboard; Special:AbuseLog; too many AISIGNS to list; Wikipedia:Administrators’_noticeboard; Wikipedia:AI_noticeboard; Wikipedia:Signs of AI writing; Wikipedia:WikiProject_AI_Cleanup/Noticeboard; WP:AI noticeboard; WP:AIN; WP:AINB; WP:AISIGNS; WP:AITELLS; WP:AN; WP:ANI; WP:ELBODY; WP:LLMISMS; WP:LLMN; WP:LLMPROD; WP:LLMPRV; WP:LLMSIGNS; WP:NEWLLM; WP:NOLLM; WPLLLMN |
| Tools (2) | Rates high on LLM detection tool; scanner identification |
| Vibes (30 labels) | |
| Circular (24) | ...; AI slop; AI style; AI-esque; AIspeak; and so on; Blatant AI signs; chatbot-esque; contains common tells of LLM writing; etc; etc etc; etc etc etc; has many AI traits; historical indicators of older LLM output; hodge-podge of AI generated sentences; is just straight AI chatbot output; LLM-like construction; Looks like particular version of model; Looks like signs of a particular LLM; obvious; Points to LLM of a particular year; suspicious text; textual indicators of AI output; title is obviously LLM input |
| Inappropriate (2) | odd†; odd formatting† |
| Sensory (1) | feels like AI writing |
| (no sub-band) (3) | Language and structure suggest possible LLM assistance; Suspected AI Text no specific reason; Vibes |
| Other (4 labels) | |
| (no sub-band) (4) | appears AI-edited rather than fully generated; appears only partially AI; bifty; Suspected AI translation |
Wikipedia Supplementary Figures and Tables
This section collects the supplementary figures and tables for the Wikipedia analysis. Table 10 lists the 25 most frequently applied labels with their Level-1 bands, and Figure 5 shows which labels most often appear as the sole label on a reason. Figure 8 and Figure 9 give the monthly share of reasons citing each Level-1 band and each Level-2 sub-band, with Figure 10 isolating the textual-feature versus external contrast. Figure 4 and Figure 6 break down the external warrants by target (essay, noticeboard, other) and track deferral to noticeboards over time. Figure 7 reports the share of feature head nouns used without an AI-pointing modifier, and Figure 11 compares hapax rates on raw wording versus coded labels. The remaining figures address editor effects: Figure 12 and Figure 13 describe how concentrated tagging is among editors, Figure 14 redraws four main-text trend measures with the most prolific editors removed or downsampled, and Figure 15 reports the Heaps exponent by citing group, sampling, and editor.
Figure 4. Which external warrant reasons cite, by month. Share of reasons that cite any External label and that cite the wp:AISIGNS essay (red), the AI Cleanup noticeboard or one of its shortcuts (teal), or some other external label such as an abuse-log or sockpuppet-investigation link (gray, dashed). A reason citing both the essay and the noticeboard counts in both lines. Numbers under the axis are External-citing reasons in the bin.
| Rank | Label | Uses | Level-1 band | % of reasons |
| 1 | WP:AISIGNS | 2,445 | External | 40.4 |
| 2 | etc | 2,053 | Vibes | 34.0 |
| 3 | Wikipedia:AI_noticeboard | 1,801 | External | 29.8 |
| 4 | vocab distro typical of LLMS | 1,469 | Textual features | 24.3 |
| 5 | Pattern of AI use from a particular user | 1,309 | Author | 21.7 |
| 6 | Points to LLM of a particular year | 993 | Vibes | 16.4 |
| 7 | Superficial analysis | 967 | Content | 16.0 |
| 8 | promotional tone | 798 | Textual features | 13.2 |
| 9 | undue emphasis on significance | 519 | Content | 8.6 |
| 10 | WP:WWT | 350 | Author | 5.8 |
| 11 | markdown | 308 | Textual features | 5.1 |
| 12 | promotionality | 242 | Content | 4.0 |
| 13 | AIATTR | 211 | Content | 3.5 |
| 14 | WP:AINB | 204 | External | 3.4 |
| 15 | edit summary | 184 | Author | 3.0 |
| 16 | AIVOCAB | 182 | Textual features | 3.0 |
| 17 | Suspected AI Text no specific reason | 163 | Vibes | 2.7 |
| 18 | rapid edits | 154 | Author | 2.5 |
| 19 | Looks like signs of a particular LLM | 143 | Vibes | 2.4 |
| 20 | High density of AI vocabulary words | 119 | Textual features | 2.0 |
| 21 | Special:AbuseLog | 116 | External | 1.9 |
| 22 | disclaimer about lack of sourcing | 110 | Author | 1.8 |
| 23 | WP:AIN | 89 | External | 1.5 |
| 24 | Suspected AI translation | 88 | Other | 1.5 |
| 25 | Declared AI use | 83 | Author | 1.4 |
Table 10. The 25 most frequently applied labels, with Level-1 band and the share of the 6,046 reason-carrying tags whose reason received the label. A reason usually receives several labels, so shares do not sum to 100%.
Figure 5. Labels most often used as the sole label on a reason. For each coded label with at least five uses, the share of its uses where no other label was applied to the same reason. Annotations give that count over the label’s total uses. Sorted by share; labels with few uses (e.g., 1/5) are listed for completeness but are not stable estimates.
Figure 6. Reasons that defer to a noticeboard, by month. Share of reason-carrying tags in the month whose reason links to or names a noticeboard (teal), the subset of those with no prose at all (black), and, for comparison, the share of all reasons that are link-only regardless of target (gray, dashed). Noticeboard mentions are identified by pattern match on the raw reason text.
Figure 8. Level-1 evidence bands cited in reasons, by month. Share of reason-carrying tags in the month whose reason received at least one label in each band. A reason usually receives labels from several bands, so shares do not sum to 100%. Numbers under the axis are reason-carrying tags in the bin.
Figure 9. Level-2 sub-bands within each Level-1 band, by month. Share of reason-carrying tags in the month whose reason received at least one label in the sub-band; the denominator is all reasons in the bin. Sub-bands with fewer than five label uses are omitted.
Figure 10. Textual-feature versus external warrants, by month. Share of reason-carrying tags in the month whose reason received at least one Textual features label (red) or at least one External label (teal). A reason can cite both. Numbers under the axis are reason-carrying tags in the bin. The two bands track each other through June and separate sharply in July.
Figure 12. Concentration of tagging among editors, by month. Top: share of the month’s {{AI-generated}} tags placed by the single most active editor, the three most active, and the ten most active, with ranks recomputed within each month. Bottom: number of distinct editors who placed at least one tag that month. Counts are of tags whose placing editor could be resolved from the page history; numbers under the axis give that total per month. October 2026 is omitted as a partial month.
Figure 13. Tags placed per editor, rank-size. Each point is one of the 683 editors who placed at least one resolved tag between December 2025 and September 2026, ordered by number of tags placed, on logarithmic axes.

Figure 15. Heaps exponent by citing group, sampling, and editor.
| Sub-band () | Labels |
| Inaccuracy (46) | AIREDCAT; bogus AI citation; Bogus references; content describing things that don’t exist; dab links; DOI of reference is invalid; dubious references; duplicate; fabricated source; failed verification of citations; fake citations; Fake quotes; false references; faux footnotes; Faux inline references; fictitious references; hallucinated categories; hallucinated citations; hallucinated claims; hallucination; hallucination, (main clause) , (participle modifier) structure; Inconsistency between text and sources†; Incorrect facts; ISBNs have invalid checksums; Issues with sources; link error (404); non-existent categories; nonsenical reference; poor text-source integrity; red cats; REDCAT; Redlinked categories; redlinks; ref irregularities; reference returns 404 error; references; references don’t don’t work; references don’t exist; References have access date issues; soruces citing wrong publication; Sources that are summarised incorrectly; too much citation errors†; Trailing “‘ from a Markdown codeblock; unsourced article spam; WP:AIREDCAT; WP:RSPLLM |
| Inappropriate (62) | adding unsupported content†; AIASSOCIATION†; AIATTR†; AILEGACY†; AITREND†; AIWEASEL†; bizarre usage of literal translations of hashtags†; book/journal references without page numbers†; circular references†; citation to wiki mirror†; didactic source disclaimer variant†; disclaimer on lack of sourcing (personal life variant)†; disclaimers on source usage†; graybles†; incorrect use of ref tags†; Irrelevant content†; No sources†; Overattribution†; Overemphasis†; overemphasis on arbitrary statements†; overgeneralization†; POV-pushing†; ref to grokipedia†; refs just go to the main pages of websites and not specific articles†; seems very SYNTHy†; source misrepresentation†; specific quote not explained†; speculation†; Strange commentary†; superficial editorializing†; too many inline links†; too much focus on random things†; undue emphasis on association†; undue emphasis on attribution†; undue emphasis on being cited†; undue emphasis on broader context†; undue emphasis on broader entities†; undue emphasis on broader stuff†; undue emphasis on broader trends†; undue emphasis on entity associations†; undue emphasis on existence of coverage†; undue emphasis on identification†; undue emphasis on importance†; undue emphasis on legacy†; undue emphasis on media identification†; undue emphasis on significance†; undue emphasis on superficial analyses†; undue emphasis on symbolism†; undue emphasis on theme editorializing†; undue in-text emphasis on independent coverage†; undue in-text emphasis on notability†; undue speculation†; unnecessary weight on things like FEIN†; vague overgeneralisations†; vague overgeneralization of sources†; vague unlinked citations†; WP:AITREND†; WP:AICITESTYLE†; WP:AILEGACY†; WP:AITREND†; WP:CIRCULAR†; year-per-year press coverage-style bibliography† |
| Quality (16) | AIPUFFERY†; AIPUSFERCIAL†; cruft†; dramatic headings and subheadings†; Fake Profundity†; long tracts of text with few citations†; proportionality†; puffery†; repeted information†; Superficial analysis†; typical of LLM context wavering†; vague and superficial analysis†; vague attributions of opinion†; vague generalizations†; WP:AIPUFFERY†; WP:SUPERFICIAL† |
| Remnant (24) | AI markup bug; “article title” header; byte sequences or unicode encoding issues (turn0search0); chatbot block and a specific byte sequence; Clear artifacts of the LLM response; copypasted plaintext citation markers; file name includes LLM marker; infobox params; infobox placeholders; lack of a preceding space; LLM commentary; LLM params; non-boilerplate infobox placeholder params†; Noticeably collaborative communication; Obvious self-referential AI-talk; placeholder text; plaintext footnote markers; string added twice; treating article title as surrogate search query; truncation; “turn0search8” bug; unicode characters; UTM links with a LLM referrer; WP:AIPLACEHOLDER |
Table 7. Coded labels in the Content band (148 labels). † = coded as pragmatic infelicity.
Table 8. Coded labels in the Author band (61 labels). † = coded as pragmatic infelicity.
| Sub-band () | Labels |
| Behavior (40) | account may have been shared between users; admitted LLM use; AfD bludgeoning; article is almost entirely written by one editor; chronic disruptive editing; contributed in massive chunks; copyright violations; Declared AI use; disclaimer about lack of sourcing; disclaimer about varying details; edit summary; English CIR issue; Harassment; humanized/ rewritten; LLM paste check; LLMPROD; marked difference in writing style of user; Mass AI generated articles; Mass AI generated comments; may be multiple authors; misleading description (e.g. edit summary); no edit summaries; overly smooth content; Pattern of AI use from a particular user; Persistent IDHT usage; physical signs (e.g. creation time or edit pattern); poor review; problematic page creations; rapid edits; removed when draftifying; revert; typical non-denial denials; user communication; user filter log; without edit summaries†; WP:AISUMM; WP:AISUMMARY; WP:G15; WP:LLMPRVOBJ; WP:WWT |
| Trait (21) | affiliated with OKA; blocked as AI agent; blocked user; COI; confessed AI use; created by AI agent; no acknowledgement of warnings; persistent creation of non-notable subjects; probably LLM-assisted; Repeated ECR vios in the WP:CT/CASTE space; sockpuppets; TA temporary account on wiki; translation from organization that solicits AI translating; User Experienced; User Newcomer; user or editor sounds suspicious; User Student; Wikipedia:Sockpuppet investigations/; WP:DE; WP:MEAT; WP:V |
References
- [1]Asif Agha. 1998. Stereotypes and registers of honorific language. Language in society 27, 2 (1998), 151–193.DOI
- [2]Asif Agha. 2003. The social life of cultural value. Language & Communication 23, 3-4 (2003), 231–273. doi:10.1016/S0271-5309(03)00012-0DOI
- [3]John L. Austin. 1962. How to Do Things with Words. Clarendon Press, Oxford.
- [4]R Harald Baayen. 2001. Word frequency distributions. Vol. 18. Springer Science & Business Media, Dordrecht, Netherlands.DOI
- [5]Lisanne Bainbridge. 1983. Ironies of automation. In *Analysis, design and evaluation of man–machine systems*. Elsevier, Amsterdam, Netherlands, 129–135.
- [6]blader. 2026. Humanizer: Agent skill that removes signs of AI-generated writing from text. GitHub repository. https://github.com/blader/humanizer
- [7]Richard Y Bourhis and Howard Giles. 1976. The language of generation in Wales: A field study. *Language sciences* 42, 13–16 (1976), 13–16.
- [8]E Summerson Carr. 2010. *Scripting addiction: The politics of therapeutic talk and American sobriety*. Princeton University Press, Princeton, NJ.
- [9]Joon-Beom Chu. 2020. Losing Portia’s Voice: The Metapragmatic Evaluations of “Quiet” and “Loud” Female Law Students in US Mock Trial Competitions. *Journal of Linguistic Anthropology* 30, 3 (2020), 353–368.DOI
- [10]Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. 2021. All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text. In *Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)*. Association for Computational Linguistics, Online, 7282–7296. doi:10.18653/v1/2021.acl-long.565DOI
- [11]Hillary Dawkins, Kathleen C. Fraser, and Svetlana Kiritchenko. 2025. When Detection Fails: The Power of Fine-Tuned Models to Generate Human-Like Social Media Text. In *Findings of the Association for Computational Linguistics: ACL 2025*. Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 13494–13527. doi:10.18653/v1/2025.findings-acl.695DOI
- [12]Daniel C. Dennett. 2023. The Problem With Counterfeit People. https://www.theatlantic.com/technology/archive/2023/05/problem-counterfeit-people/674075/
- [13]Jonas Dolezal, Sawood Alam, Mark Graham, and Maty Bohacek. 2026. The Impact of AI-Generated Text on the Internet. arXiv:2604.26965 [cs.CY]. doi:10.48550/arXiv.2604.26965DOI
- [14]Jie Dong. 2010. The enregisterment of Putonghua in practice. *Language & Communication* 30, 4 (2010), 265–275.DOI
- [15]Liam Dugan, Daphne Ippolito, Arun Kirubakaran, Sherry Shi, and Chris Callison-Burch. 2023. Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Text. *Proceedings of the AAAI Conference on Artificial Intelligence* 37, 11 (2023), 12763–12771. doi:10.1609/aaai.v37i11.26501DOI
- [16]Penelope Eckert. 2000. *Language variation as social practice: The linguistic construction of identity in Belten High*. John Wiley & Sons, Hoboken, NJ.
- [17]Leo Egghe. 2007. Untangling Herdan’s law and Heaps’ law: Mathematical and informetric arguments. *Journal of the American Society for Information Science and Technology* 58, 5 (2007), 702–709. doi:10.1002/asi.20524DigitalDOI
- [18]Johanna Fleckenstein, Jennifer Meyer, Thorben Jansen, Stefan D. Keller, Olaf Köller, and Jens Möller. 2024. Do Teachers Spot AI? Evaluating the Detectability of AI-Generated Texts Among Student Essays. *Computers and Education: Artificial Intelligence* 6 (2024), 100209. doi:10.1016/j.caeai.2024.100209DOI
- [19]Nils Fränzén and Andreas Stokke. 2025. A unified account of semantic and pragmatic infericity. *Synthese* 205, 3 (2025), 115.DOI
- [20]Kathleen C. Fraser, Hillary Dawkins, and Svetlana Kiritchenko. 2025. Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods. *Journal of Artificial Intelligence Research* 82 (2025), 2233–2278. doi:10.1613/jair.1.16665DOI
- [21]Yuval Noah Harari. 2023. Yuval Noah Harari Argues That AI Has Hacked the Operating System of Human Civilisation. https://www.economist.com/by-invitation/2023/04/28/yuval-noah-harari-argues-that-ai-has-hacked-the-operating-system-of-human-civilisation
- [22]Harold Stanley Heaps. 1978. *Information retrieval: Computational and theoretical aspects*. Academic Press, Inc., Cambridge, MA.
- [23]Gustav Herdan. 1960. *Type-token mathematics: A textbook of mathematical linguistics*. Mouton, The Hague, Netherlands.DOI
- [24]Dell Hymes et al. 1972. On communicative competence. *sociolinguistics* 269293 (1972), 269–293.
- [25]Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2020. Automatic Detection of Generated Text is Easiest when Humans are Fooled. In *Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*. Association for Computational Linguistics, Online, 1808–1822. doi:10.18653/v1/2020.acl-main.164DOI
- [26]Judith T. Irvine. 2002. "Style" as distinctiveness: the culture and ideology of linguistic differentiation. Cambridge University Press, Cambridge, England, 21–43.DOI
- [27]Judith T. Irvine. 2022. Revisiting theory and method in language ideology research. *Journal of Linguistic Anthropology* 32, 1 (2022), 222–236.DOI
- [28]Maurice Jakesch, Jeffrey T. Hancock, and Mor Naaman. 2023. Human Heuristics for AI-Generated Language Are Flawed. *Proceedings of the National Academy of Sciences* 120, 11 (2023), e220839120. doi:10.1073/pnas.220839120DOI
- [29]Barbara Johnstone. 2016. Enregisterment: How linguistic items become linked with ways of speaking. *Language and Linguistics Compass* 10, 11 (2016), 632–643.DOI
- [30]Barbara Johnstone and Scott F Kiesling. 2008. Indexicality and experience: Exploring the meanings of /aw/-monophthongization in Pittsburgh 1. *Journal of sociolinguistics* 12, 1 (2008), 5–33.
- [31]Cameron R. Jones, Ishika Rathi, Sydney Taylor, and Benjamin K. Bergen. 2025. People Cannot Distinguish GPT-4 from a Human in a Turing Test. In *Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’25)*. Association for Computing Machinery, New York, NY, USA, 1615–1639. doi:10.1145/3715275.3732108DOI
- [32]Scott F Kiesling. 2019. Stances of the ‘gay voice’ and ‘brospeak’. In *Oxford Handbook of Language and Sexuality (online edn.)*, ed. by Kira Hall, and Rusty Barrett. Oxford: Oxford University Press 10 1093 (2019), 1–42.
- [33]Miles Klee. 2025. Are Em Dashes Really a Sign of AI Writing? Rolling Stone. Accessed 2 November 2026. https://www.rollingstone.com/culture/culture-features/chatgpt-hypen-em-dash-ai-writing-1235314945/
- [34]Kristian Kuznetsov, Eduard Tulchinskii, Laida Kushnareva, German Magai, Serguei Barannikov, Sergey Nikolenko, and Irina Piontkovskaya. 2024. Robust AI-Generated Text Detection by Restricted Embeddings. In Findings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 17036–17055. doi:10.18653/v1/2024.findings-emnlp.992DOI
- [35]Haley Lepp and Daniel Scott Smith. 2025. “You Cannot Sound Like GPT”: Signs of language discrimination and resistance in computer science publishing. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency (Athens, Greece) (FAccT ’25). Association for Computing Machinery, New York, NY, USA, 3162–3181. doi:10.1145/3715273.3732202DOI
- [36]Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou. 2023. GPT Detectors Are Biased Against Non-Native English Writers. Patterns 4, 7 (July 2023), 100779. doi:10.1016/j.patter.2023.100779DOI
- [37]Norma Mendoza-Denton. 2002. Language and Identity. The handbook of language variation and change 11 (2002), 475.
- [38]Elizabeth Mertz. 2007. The language of law school: learning to” think like a lawyer”. Oxford University Press, Oxford, England.
- [39]Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. In Proceedings of the 40th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 202), Andreas Krause, Emma Brunskill Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.). PMLR, Cambridge, MA, 24950–24962. https://proceedings.mlr.press/v202/mitchell23a.htmlarxiv.org/abs/2301.11305
- [40]Raja Parasuraman and Victor Riley. 1997. Humans and automation: Use, misuse, disuse, abuse. Human factors 39, 2 (1997), 230–253.
- [41]Cara Penry Williams. 2019. Appeals to Semiotic Registers in Ethno-Metapragmatic Accounts of Variation. Journal of Linguistic Anthropology 29, 3 (2019), 294–313.DOI
- [42]Brian Porter and Edouard Machery. 2024. AI-Generated Poetry Is Indistinguishable from Human-Written Poetry and Is Rated More Favorably. Scientific Reports 14 (2024), 26133. doi:10.1038/s41598-024-76900-1DOI
- [43]Jenna Russell, Marzena Karpinska, and Mohit Iyyer. 2025. People Who Frequently Use ChatGPT for Writing Tasks Are Accurate and Robust Detectors of AI-Generated Text. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vienna, Austria, 5342–5373. doi:10.18653/v1/2025.acl-long.267DOI
- [44]Michael Silverstein. 1976. Shifters, Linguistic Categories, and Cultural Description. In Meaning in Anthropology, Keith H. Basso and Henry A. Selby (Eds.). University of New Mexico Press, Albuquerque, 11–55.
- [45]Linda J Skittka, Kathleen L Mosier, and Mark Burdick. 1999. Does automation bias decision-making? International Journal of Human-Computer Studies 51, 5 (1999), 991–1006.
- [46]James Slotta. 2020. Pragmatics. In The International Encyclopedia of Linguistic Anthropology, James Stanlaw (Ed.). Wiley-Blackwell, Hoboken, NJ.
- [47]Lauren Squires. 2010. Enregistering internet language. Language in society 39, 4 (2010), 457–492.DOI
- [48]Debora Weber-Wulff, Alla Anohina-Naumeca, Sonja Bjelobaba, Tomáš Foltýnek, Jean Guerrero-Dib, Olumide Popoola, Petr Šigut, and Lorna Waddington. 2023. Testing of Detection Tools for AI-Generated Text. International Journal for Educational Integrity 19, 1 (2023), 26. doi:10.1007/s40979-023-00146-z
- [49]Anna Weisselbraun. 2026. ChatGPTese and the Enregisterment of Machine Language. Non-archival conference presentation, 7–10 October 2026.
- [50]Joseph Weizenbaum. 1972. On the Impact of the Computer on Society. Science 176, 4035 (May 1972), 609–614. doi:10.1126/science.176.4035.609DOI
- [51]Wikipedia contributors. 2026. Wikipedia:Ethically researching Wikipedia. Wikipedia, The Free Encyclopedia. Accessed 2 November 2026. Permanent link: https://en.wikipedia.org/w/index.php?title=Wikipedia:Ethically_researching_Wikipedia&oldid=1366700221. https://en.wikipedia.org/wiki/Wikipedia:Ethically_researching_Wikipedia
- [52]Wikipedia contributors. 2026. Wikipedia:Researching with Wikipedia. Wikipedia, The Free Encyclopedia. Section “Special research considerations concerning Wikipedia.” Accessed 2 November 2026. Permanent link: https://en.wikipedia.org/w/index.php?title=Wikipedia:Researching_with_Wikipedia&oldid=1382728486. https://en.wikipedia.org/wiki/Wikipedia:Researching_with_Wikipedia
- [53]Wikipedia contributors. 2026. Wikipedia:Signs of AI writing. Wikipedia, The Free Encyclopedia. Essay maintained by WikiProject AI Cleanup. https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
- [54]Wikipedia contributors. 2026. Wikipedia:What Wikipedia is not. Wikipedia, The Free Encyclopedia. Policy. Section “Wikipedia is not a laboratory” (WP:NOTLAB). Accessed 2 November 2026. Permanent link: https://en.wikipedia.org/w/index.php?title=Wikipedia:What_Wikipedia_is_not&oldid=1382748100. https://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not
- [55]Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. 2025. A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions. Computational Linguistics 51, 1 (March 2025), 275–338. doi:10.1162/coli_a_00549DOI
- [56]Andi Zeisler. 2025. AI Can’t Have My Em Dash. Salon. 11 June 2025. Accessed 2 November 2026. https://www.salon.com/2025/06/11/ai-cant-have-my-em-dash
- [57]Julia Witte Zimmerman, Calla G. Beauregard, Tabia Tanzin Prama, Parisa Suchdev, Kathryn Cramer, and Elisabeth Kollrack. 2026. LLMs are not stochastic parrots: Evidence for meaning-mediated abstraction from conlang-like tasks. arXiv:2609.34187 [cs.CL]. https://arxiv.org/abs/2609.34187
- [58]Julia Witte Zimmerman, Denis Hudon, Kathryn Cramer, Alejandro J. Ruiz, Calla Beauregard, Ashley Fehr, Mikaela Irene Fudolig, Bradford Demarest, Yoshi Meke Bird, Milo Z. Trujillo, Christopher M. Danforth, and Peter Sheridan Dodds. 2024. Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning. arXiv:2412.10924 [cs.CL]. https://arxiv.org/abs/2412.10924
- [59]Julia Witte Zimmerman, Bryn Loftness, Calla Beauregard, Kristine Harootunian, June Clark, Hannah Reing, Katie Ekstrom Grenon, Michelle Morgenstern, Tabia Tanzin Prama, and Juniper L. Lovato. 2026. Vermont AI Literacy project. Ongoing open-source curriculum and materials,












