WorksComputer Science
Lookover: Scaffolding Student Participation in Social Annotation
Abstract
Social annotation platforms such as Perusall and Hypothesis are widely used to support student engagement with course readings, often through required commenting. Yet visible participation does not necessarily produce sustained or substantive peer exchange. Analyzing six weeks of Perusall activity from a graduate course (2,873 comments), we found that participation was often fragmented: 53.5% of comments received no reply, 38.3% were posted within the final 12 hours before the deadline, and over one-third of a double-coded sample went no further than repeating or agreeing with the reading or a peer. Interviews pointed to two linked difficulties: locating a discussion worth entering and contributing something that meaningfully advances it. We present Lookover, an AI-assisted annotation system that scaffolds both stages without writing for students. Discussion Overview helps students identify active themes and relevant threads, while Spark Thinking poses questions that encourage them to move beyond reaction and restatement toward questioning, extending, and substantively responding to peers. A preliminary study with five participants found Discussion Overview valuable for those who noticed it, while the usefulness of Spark Thinking depended on how much a participant already knew about the topic.
Figure 1. Lookover. Left: Discussion Overview groups comments into themes. Right: Spark Thinking poses questions when a user opens the comment editor.
CCS Concepts
• Human-centered computing → Interactive systems and tools.
ACM Reference Format:
Yubo Zhou, Xinyi Xu, Zesen Zhao, Anhong Guo, and Xinyue Chen. 2026. Lookover: Scaffolding Student Participation in Social Annotation. In The 39th Annual ACM Symposium on User Interface Software and Technology (UIST Adjunct ’26), November 02–05, 2026, Detroit, MI, USA. ACM, New York, NY, USA, 3 pages. https://doi.org/10.1145/3830397.3841903
Introduction
Social annotation platforms such as Perusall and Hypothesis allow students to comment directly on course readings and respond to peers [3]. These activities are often structured around required numbers of annotations, which can encourage reading participation [6]. However, visible comments and participation counts do not necessarily produce sustained or substantive exchange. Prior work has proposed scaffolds such as collaboration scripts to improve response quality [4] and mechanisms to help students navigate large volumes of annotations [1]. Yet we know less about how participation unfolds under routine comment requirements and whether it develops into meaningful peer exchange.
To investigate this question, we collected six weeks of Perusall logs from a 55-student graduate course at an R1 university, comprising 2,873 comments across 18 readings. We adapted the ICAP framework [2] into a codebook and two coders independently annotated a random sample of 500 comments. In the full log, 53.5% of comments received no reply and 38.3% were posted within the final 12 hours before the deadline; over one-third of a This work is licensed under a Creative Commons Attribution 4.0 International License.
UIST Adjunct ’26, Detroit, MI, USA
© 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2855-6/2026/11 https://doi.org/10.1145/3830397.3841903 sample (36.4%) went no further than restating the reading or a peer. A survey of 35 students echoed this: only 31.4% agreed that discussions felt like a real exchange of ideas rather than posting in parallel. Together, these findings point to two difficulties students face: locating where a discussion is happening and where their own contribution fits, and moving beyond restating the reading to engage constructively.
We present Lookover, which uses AI to scaffold students into discussion rather than write it for them, addressing each in turn. Discussion Overview organizes existing comments into themes so students can see what peers are discussing and where to join; Spark Thinking poses questions as a student writes, prompting them to question and extend rather than restate, while leaving the answering to them.
Understanding Perusall Participation on a Real Course
We studied a graduate course on Perusall at an R1 university over six weeks, covering 55 students, 18 assigned readings, and 2,873 comments. The course required at least three constructive comments per reading, due before class. We combined three sources: a descriptive analysis of the annotation log, a content analysis coding individual comments for engagement quality, and interviews and a survey that let students reflect on their participation experience.
Descriptive Analysis. Participation stayed close to the required minimum. The average reading drew 159.6 comments, near the expected from three comments per student, and most comments (72%) arrived in the final 24 hours before the deadline. Comments were also largely isolated: 46% sat on a passage where no other student commented, and peer endorsement was almost absent (only 4.1% received any upvote, none more than two).
Content Analysis: We adapted the ICAP framework [2] by grouping comments into two broader levels. Comments that primarily reacted to, restated, or directly applied the reading or a peer’s point were coded as passive/active, while comments that introduced something new or substantively built on a peer’s contribution were coded as constructive/interactive. Two coders independently annotated a random sample of 500 comments, considering each comment in relation to the reading and, where applicable, the preceding comments in its thread. The coders agreed that 36.4% were passive/active and 54.8% were constructive/interactive, and disagreed across this binary boundary on the remaining 8.8% ().
Interviews and Survey. During the contextual inquiry sessions, participants (N=3) described skimming the reading, stopping after meeting the required count of comments, and rarely returning to check for responses. One described the interaction as one-way, “I want to do the interaction with others, but I don’t think there will be any people doing interaction with me”, and another asked for a summary of “what people have discussed, so that I don’t need to look at too many things.”
The survey (N=35) echoed this. Replying to peers was reported as common (65.8%), yet only 31.4% felt the discussion was a real exchange of ideas rather than parallel posting, and 65.7% found it hard to keep track of comments across a reading. On AI, most preferred it to act only when asked (68.6% vs. 22.9% for AI acting on its own).
Together, these point to two unaddressed problems: comments often carried limited social meaning, restating the reading rather than building on peers; and annotations were hard to navigate as a shared space, dispersed across passages so students struggled to track the discussion.
Lookover
Lookover is a sidebar add-on that uses AI to scaffold students into discussion rather than write it for them, addressing our two findings in turn. It is built on Hypothesis, an open-source annotation platform, so it can be deployed in existing courses.
Discussion Overview
Discussion Overview groups a document’s comments into a few themes, each with a title and a short account of what students argued (Figure 1). Opening a theme shows a fuller summary with the peer comments behind it, each anchored to its passage and carrying a reply link, so a student can move from a theme into the specific exchange and reply there.
Spark Thinking
Spark Thinking targets the second problem: moving students beyond restatement toward questioning and building on peers. Questioning is a well-established scaffold for this move—instructor prompts and question frames lead students to reason more and give more elaborated explanations [4, 5]. We place AI in the role of the questioner. When a student opens the editor, Spark Thinking generates questions at three levels: Understand (what the text or a peer claims), Challenge (the reasoning behind it), and Connect (links to elsewhere in the reading), each drawn from the passage and surrounding comments, and a See what others said link ties it to specific peer comments the student can respond to. By design, it supplies the question but not the answer so students think and write their own response.
Preliminary Evaluation and Future Plan
Five participants used Lookover on a real reading without being introduced to its features. Discussion Overview was the most valued feature among those who found it, though the participant who scrolled linearly, and would have benefited most, never discovered it. Spark Thinking’s effect depended on what a participant already had to work with: one who knew the topic well found that it raised angles they had not considered, while the participant who rated it highest was unfamiliar with the topic, used it throughout, yet still produced only brief reactions.
We plan to pursue two directions. First, whether participation can extend past submission: this requires changing how the assignment is set up, not the interface, so we plan to interview instructors on why they set a fixed comment count and whether they would adopt an alternative that gives students a reason to return and respond. Second, whether better discussion means better reading: we plan to deploy the system in real classrooms and, alongside discussion quality, use an AI-free comprehension test to check whether students actually understood the reading.
Acknowledgments
We thank all participants, and Muzhe Wu, Ellie Seehorn, and the research community for their constructive feedback.
References
References
- [1]Jumana Almahmoud, Marc Faciotti, Michele Igo, Kamali Sripathi, and David Karger. 2025. Enhancing User Engagement in Large-Scale Social Annotation Platforms: Community-Based Design Interventions and Implications for Large Language Models. arXiv preprint arXiv:2501.01545. doi:10.48550/arXiv.2501.01545DOI
- [2]Michele H. T. Chi and Ruth Wylie. 2014. The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes. Educational Psychologist 49, 4 (2014), 219–243. doi:10.1080/00461520.2014.965823
- [3]Tristan Cui and Jeff Wang. 2024. Empowering active learning: A social annotation tool for improving student engagement. British Journal of Educational Technology 55, 2 (2024), 712–730. doi:10.1111/bjet.13403DOI
- [4]Vincent de Boer and Howard Spoelstra. 2021. The effects of collaboration scripts on the number and quality of student interactions in a social annotation tool. In 7th International Conference on Higher Education Advances (HEAd’21). Universitat Politècnica de València, 967–974. doi:10.4995/HEAd21.2021.12862DOI
- [5]Alison King. 1990. Enhancing Peer Interaction and Learning in the Classroom Through Reciprocal Questioning. American Educational Research Journal 27, 4 (1990), 664–687. doi:10.3102/00028312027004664DOI
- [6]Kelly Miller, Brian Lukoff, Gary King, and Eric Mazur. 2018. Use of a Social Annotation Platform for Pre-Class Reading Assignments in a Flipped Introductory Physics Class. Frontiers in Education 3 (2018), 8. doi:10.3389/feduc.2018.00008DOI
