betaFeedback

WorksComputer Science

Rubric-Based Assessment of Student Reports Using Small Language Models

Abstract

Language models can analyze and assess student texts using predefined rubrics and deliver structured feedback, reducing instructors' workload. However, existing solutions often rely on external services that are resource-intensive, costly, and raise privacy concerns, making them unsuitable for constrained environments. This study evaluates an automated assessment pipeline with three key aims: design for low-resource environments, privacy, and focus on a novel context in cybersecurity education. The pipeline integrates lightweight language models for textual evaluation and a vision-language model for image interpretation. Responses from 105 students were collected over two semesters of an undergraduate course during a cybersecurity exercise on incident response. Results across different hardware configurations show that small models in low-resource environments can achieve up to 97% accuracy in text-only and up to 94% in rich-text assessments. This rivals or even outperforms a large model while adding the benefits of locally hosted assessment. Our solution can help educators and practitioners who need grading assistance without replacing human oversight, especially in environments where cloud-based solutions may be impractical. This study contributes an open-source, locally deployable AI-assisted scoring pipeline that runs on affordable hardware and can be extended to various types of student reports in computing education. As a result, it makes automated assessment accessible to a broad range of educators and institutions.

CCS Concepts • Social and professional topics → Computing education; • Computing methodologies → Artificial intelligence.

ACM Reference Format: Sandesh Kumar Kafle, Jan Vykopal, and Valdemar Švábenský. 2027. Rubric-Based Assessment of Student Reports Using Small Language Models. In Proceedings of the 58th ACM Technical Symposium on Computer Science Education V.1 (SIGCSE TS 2027), February 17–20, 2027, Sacramento, CA, USA. ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/3857222.3860169

This work is licensed under a Creative Commons Attribution 4.0 International License. SIGCSE TS 2027, Sacramento, CA, USA

© 2027 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-3075-7/2027/02 https://doi.org/10.1145/3857222.3860169

Introduction

Generative AI (GenAI), especially Large Language Models (LLMs), is shifting learning environments from fully human-involved tutoring to automation in order to reduce instructors’ workload. Classrooms are becoming more interactive [28]; LLMs are applied across learning contexts for short-answer grading [4, 31], essay summarization, automated feedback [1, 15, 29], and programming support [9, 10].

Nevertheless, existing studies do not address the context of rich-text reports, such as those created in cybersecurity exercises. In addition, computational accessibility remains a barrier to adoption. Many instructors lack access to high-end hardware or cloud-based infrastructures [20, 21, 23]. While some institutions provide shared computing resources, there is a growing need for tools that can run effectively on moderate setups.

Data privacy is also a concern for the education community [2, 8]. However, using cloud-based models or outsourcing computation to external services often means losing control of student data. In contrast, locally deployed models help preserve privacy, as there is no need to send sensitive data to a third party. Processing data locally reduces the risk of unauthorized access or misuse.

This study evaluates the effectiveness of language models for text grading and feedback. It compares the performance and accuracy of five models across four hardware configurations, validating model output against human assessment. We analyze how small, open-source models perform in assessing student documents (both simple and rich-text) using a rubric – particularly in low-resource setups prevalent in many educational contexts across the globe [20]. Further, our work addresses the identified gap by providing a lightweight, locally deployable solution that maintains strong performance without requiring costly hardware [6].

Unlike open-ended questions and essay grading, we focus on structured student reports, aiming not only to interpret meaning or assess argumentation but also to accurately parse and evaluate predefined formatting, embedded content, and uniformity across submissions. In doing so, this study explores the potential of language models to save instructors’ time for evaluation and feedback.

Related Work

Automated Assessment Challenges

Scaling automated assessment is challenging, particularly when providing real-time, context-aware feedback to many students [5, 30]. Running large, state-of-the-art models requires specialized GPU clusters, enough memory, and disk space. If such resources are unavailable, using smaller, more computationally accessible variants is needed [27, 32]. Furthermore, privacy regulations such as FERPA and GDPR must be considered, as transmitting student data to third parties can lead to information leakage or ethical issues [7, 26]. To address these concerns, institutions have explored local deployment of open models, which can reduce costs and keep student data under institutional control [19]. However, instructors may face system failures caused by resource overuse during peak submission periods, which can disrupt grading workflows [26].

Efficient Language Models for Education

Small models, such as phi-3:3.8b, mistral:7b, and llama-3.2:3b, have expanded the possibilities of AI-assisted education in limited environments [27, 32]. These models can be deployed in resource-constrained settings with solid performance, reducing the need for larger models. CyberLLaMA, a fine-tuned llama-3.2:3b model for recognizing cybersecurity entities (e.g., malware names and IP addresses), is an example [32]. Instruction-tuned models are increasingly used for specialized tasks such as reflection assessment and multi-session dialogue understanding [11]. Prior work showed that students can receive help in mathematics, supporting the broader applicability of compact and instruction-tuned models beyond text generation [3]. Despite their computational advantages, smaller models can be less effective on complex prompts, requiring fine-tuning or careful task decomposition. For instance, TinyLlama-based study highlighted both the promise and limitations of compact models, depending on task complexity [33].

Rubric-Aligned Assessment Methodologies

Rubric-structured direct prompting can automate complex evaluations, in which language models follow expert-defined criteria and perform assessment [9, 19, 34]. These systems use forced-choice or ordinal scoring. Models respond with criterion-level results (e.g., correctness of a reasoning step, presence of a required element) and assign discrete values (e.g., 0/1/2) to reduce subjectivity and improve inter-rater consistency [12]. To enable integration into learning platforms, outputs are constrained to structured formats (e.g., JSON) that include criterion scores and feedback [19]. Few-shot grading systems, combined with guided rubrics and graded examples, enhance the accuracy and quality of grading for open-ended responses [34]. However, they are often used for grading essays or short answers rather than long, rich-text reports (such as cybersecurity incident reports, summaries, and advisories).

Contribution of This Paper

Prior studies on assessment primarily focused on structured short-answer evaluation, essay grading, and feedback generation in programming education using rubric-guided prompting [10]. Next, deployment of automated assessment often assumed access to LLMs or did not consider low-resource contexts as a constraint [12, 19]. Our study addresses this gap by investigating smaller, efficient models for rubric-aligned scoring and feedback under constrained conditions. The proposed tool performs a rubric-based assessment of reports with rich formatting and contexts that may include both textual and visual evidence. Also, it speeds up the evaluation process, enabling students to receive an assessment report much more quickly, which supports their learning and helps them improve more effectively. We release the tool as open-source software [6].

Teaching Context

The student reports analyzed in our study come from a cybersecurity simulation exercise held in two semesters (in 2024 and 2025) of an introductory undergraduate course, “Cybersecurity in an Organization,” taught at a Czech public university. The exercise was a real-time, in-person simulation of a cybersecurity incident taking place at a fictitious University. Students roleplayed as incident handlers on the University’s Computer Security Incident Response Team (CSIRT-U). Towards the end of the exercise, each student team was asked to prepare two text documents: an executive summary and a security advisory. The teams created the reports as plain text and PDF files, which were manually scored by exercise instructors according to assessment rubrics (see § 4.4). The following text details the content and structure of both documents and rubrics.

Student Report 1: Executive Summary (ES)

This document reports essential information about a phishing incident to the CSIRT-U’s management. It summarizes key developments and findings, reporting on how the incident was handled. In particular, it provides information on incident reports received, how users who visited the phishing website were contacted, and whether their passwords were reset. It also explains how the phishing site was disabled and whether a security advisory was issued. Figure 1 depicts an exemplary ES created by the instructors.

Exemplary executive summary of the incident

Figure 1. Exemplary executive summary of the incident.

Assessment Rubrics. The rubric includes four criteria, with 4 points awarded by the grader if the report contains information:

  • about incident reports CSIRT received (from an external party and an employee) – 1 pt.

  • that CSIRT contacted the users who visited their phishing website, and reset their passwords – 1 pt.

  • that CSIRT disabled the site hosting the phishing form – 1 pt.

  • that CSIRT issued a security advisory – 1 pt.

Student Report 2: Security Advisory (SA)

In the exercise, students were tasked with informing University users about the ongoing incident via a security advisory. The SA should summarize what happened, present the phishing email and the corresponding malicious website, and provide recommendations for users with additional references. Figure 2 depicts an exemplary SA, which must be a PDF document with two screenshots: a phishing email users may receive and the phishing website with a form, making it easier for recipients to recognize these threats. The expected structure of sections includes a title, date, target audience, summary, recommendations, and references.

Exemplary security advisory showing phishing email and website screenshots

Figure 2. Exemplary security advisory.

Assessment Rubrics. The rubric includes four criteria, with a total of 5 points that can be awarded by the grader:

  • Presence of all important sections (see above) and their meaningful content – 2 pts when all good, 1 pt when one section is missing or has low quality, 0 pts otherwise.

  • Screenshot of the phishing email – 1 pt.

  • Screenshot of the phishing website – 1 pt.

  • Inclusion of the phishing email sender address – 1 pt.

Research Methods for Report Assessment

While LLMs are often used for natural language processing [27], we investigated smaller models for automated assessment in low-resource, local deployment contexts. This approach aims to reduce resource consumption and make GenAI more accessible for assessment. Figure 3 outlines our data processing methods. The following subsections refer to steps marked as circled numbers in Figure 3.

Figure 3. Illustration of the data processing pipeline and methods.

Data Collection and Preprocessing

This study did not require approval from the Czech university’s institutional review board since only anonymized student texts were processed. We included only data from students who provided their informed consent to use their exercise data for research.

The exercise was conducted on the freely available INJECT Exercise Platform (further IXP) [24] (step 1). Out of 38 teams of three, 35 provided consent for this research. From the exercise logs (1a), we analyzed two reports created by each team: the executive summary (2a) and the security advisory (2b). To clean the dataset, we removed duplicate submissions (e.g., cases where a team uploaded the same report multiple times) and kept only the final version for each team and report type (step 2). This step prepared the model’s input data: email text and PDFs submitted by the students to the IXP. A Python script with widely used libraries PyPDF2 and Pillow was employed to extract text and embedded images separately (step 3). The summary consisted of text only (3a), whereas the text and images were handled separately as the advisory included both types of content (3b and 3c). Since small models may lack the capacity to handle raw and noisy inputs on limited hardware, pre-processed and cleaned inputs help the model process them accurately. With minimal modifications, the entire processing pipeline can be adapted to handle a wide range of document types and structures, making it suitable for diverse educational contexts.

Data Analysis

Evaluating executive summaries (step 4a) is straightforward since they contain only text. In contrast, security advisories mix text and images, necessitating two models: one for text and another for images, because the text models cannot process images. We first used an image model to generate textual descriptions of the images (step 3b) while verifying their inclusion and correctness as required by the rubric. These descriptions were then combined with the extracted text from the advisory (step 3c) and evaluated with the text model (step 4b). This approach allows us to use a smaller model, reducing computational load.

The analysis (step 4) was implemented in Python using the langchain_ollama package. The models were hosted locally using Ollama [17], an open-source platform for running language models on-device. Different hardware configurations with commonly available NVIDIA GPUs were used (see Table 1). S1 uses a low-cost card from 2018, S2 is equipped with a new, yet low-power, laptop GPU, and S3 has an affordable desktop GPU. In addition, we tested a system without a GPU. The text-only evaluation took an average of 30 minutes per model. However, the CPU-only setup could not complete the rich-text evaluation. Even with a smaller image model, the output was incomplete, so we excluded it from our comparisons.

SystemGPUVRAMCPURAM
System 1 (S1)RTX 2080Ti12 GBAMD EPYC 7F5230 GB
System 2 (S2)RTX 5070Ti12 GBAMD Ryzen AI 9 XH 37032 GB
System 3 (S3)RTX PRO 200016 GBIntel i7-1170064 GB
System 4 (SR)H100 NVL94 GB2× AMD EPYC 94541.5 TB

Table 1. Hardware configurations used in our study.

The models were selected based on the hardware constraints (see Table 1). We used llama3:8b and mistral:7b, which follow instructions strictly and efficiently [25], enabling real-time response generation. We also used qwen3:1.7b and gemma3:4b, whose cost-effective reasoning makes them well-suited for complex tasks in resource-constrained environments [25]. For image analysis, we used llava:7b, which generates textual descriptions of images. Prior work shows that llava:7b offers multimodal reasoning and delivers competitive accuracy across benchmarks, adapting well to both general-purpose and domain-specific image analysis tasks [14, 22]. Finally, to contextualize the possibilities of using small models on consumer-grade hardware, we also tested a larger gpt-oss-120b model, which achieves near-parity with OpenAI o4-mini [18]. This model was hosted on an on-premise AI platform with one NVIDIA H100 GPU dedicated to this model. This remote system (denoted SR) serves only as a baseline for comparison with the locally hosted models.

Rubric-Aligned Prompt Design

We tried less-structured prompt techniques (e.g., zero-shot, one-shot, context-free), and response times were quick, but accuracy was not satisfactory. Here, we present the prompts developed after multiple test runs for assessing the student reports (described in § 3). The prompts are designed for summative scoring and feedback, using language models to act as an instructor for evaluation. The model assesses student responses using a detailed rubric, then returns scores with feedback in JSON format. The prompts combine role assignment, context, assessment rubrics, an exemplary report, and output constraints (see details in [6]). Prompt templates are listed below; example output is in Figure 4.

Figure 4. Defined output format and the generated output for (1) ES and (2) SA. And, (3) image analysis prompt and output.

Executive Summary (ES) Assessment Prompt:

Role: You are an instructor for the university course Cybersecurity in an Organization, evaluating a student’s executive summary following a simulated security incident for CSIRT-U.

Context: The student must write a concise executive summary (700–1000 characters) for university management, describing CSIRT’s response actions, suitable for professional communication.

Rubrics: [copied from § 3.1]

Scoring instructions: Assess the student’s executive summary. Evaluate strictly using the rubric, awarding 0 or 1 point for each criterion; provide only brief, constructive feedback and respond in the specified JSON format.

Security Advisory (SA) Assessment Prompt:

Role: You are an instructor for the university course Cybersecurity in an Organization, evaluating a student’s security advisory following a simulated security incident for the CSIRT-U.

Context: The student is tasked with writing a concise security advisory about an ongoing phishing campaign, addressed to all university staff and students. The advisory must include all critical information, two specific images (of the phishing email and phishing website), and be suitable for rapid dissemination within the university.

Rubrics: [copied from § 3.2]

Scoring instructions: Assess the student’s security advisory and review the image analysis. Evaluate strictly using the rubric. Provide only brief, constructive feedback and respond in the specified JSON format.

Metrics for Performance Evaluation

Each student’s report had from 56 to 305 words. Across the two exercise sessions, five instructors (2024) and six instructors (2025) graded different subsets of submissions. The grades were then checked and possibly updated by the main instructor (the course guarantor). The instructors also carefully reviewed all reports to avoid potential prompt injection by students; no such attempts were present. To measure the performance (step 5), we used evaluation accuracy and processing time, computed as:

Accuracy⁡=1N∑i=1N{1if mi=hi,0if mi≠hi,Processing time (avg.)=1N∑i=1Nti\operatorname{Accuracy} = \frac{1}{N}\sum_{i=1}^{N} \begin{cases} 1 & \text{if } m_i = h_i,\\ 0 & \text{if } m_i \ne h_i \end{cases} ,\qquad \text{Processing time (avg.)} = \frac{1}{N}\sum_{i=1}^{N} t_i

Where NN is the total number of evaluated submissions, mim_i is the model score for submission ii, hih_i is the human score for submission ii, and tit_i is the time (in seconds) taken by the model to evaluate submission ii. For each model across the systems, we evaluated each student report 10 times and measured the mean accuracy and mean evaluation time across these runs (see Figure 5). This approach mitigates potential variation that could affect a single run.

Results from 10 independent runs

Figure 5. Results from 10 independent runs. Overall accuracy (match %) with 95% CIs. Evaluation time (s) with ± SD.

Results and Discussion

For each report type (ES and SA), the metrics (accuracy and time) are the mean across all teams. Figure 5 presents the metrics when evaluating a single student report across the four systems (see Table 1), with 95% confidence intervals (CIs). Table 2 and Table 3 report the per-criterion accuracy of evaluating ES and SA, respectively.

Criteriaincident_reportsphishing_site_disabledsecurity_advisoryuser_contact_and_reset
S1S2S3S1S2S3S1S2S3S1S2S3
Model
gemma368.5768.5768.5797.1497.1497.1468.5768.5768.5777.1477.1477.14
llama368.5768.5768.5791.4391.4388.5785.7185.7185.7185.7182.7180.00
mistral68.5771.4371.4385.7185.7188.5771.1482.8680.0085.7185.7188.57
qwen371.4368.5771.4397.1494.2988.5765.7165.7171.4374.2977.1480.00
gptoss

Table 2. Per-criteria Evaluation Accuracy (%) for ES (See § 3.1 and § 4.4) for a single run.

Criteriaphishing_email_addressphishing_email_picturephishing_website_picturesections
S1S2S3S1S2S3S1S2S3S1S2S3
Model
gemma391.4394.2988.5785.7174.2980.0071.4382.8677.1451.4354.2948.57
llama382.8677.1474.2982.8685.7188.5771.4377.1477.1431.4331.4331.43
mistral80.0085.7185.7182.8654.2965.7174.2980.0074.2931.4328.5731.43
qwen388.5780.0082.8660.0060.0068.5768.5771.4374.2943.8634.2934.29
gptoss82.8637.1440.00

Table 3. Per-criteria Evaluation Accuracy (%) for SA (See § 3.2 and § 4.4) for a single run.

Insights About the Automated Assessment

Assessing both ES and SA achieved high accuracy in most cases, with noticeable differences across rubric criteria (see Table 2 and Table 3). In addition, plain-text ES are processed much faster than rich-text SA across all systems (see Figure 5). In practice, the workflow was most reliable for structured, evidence-based rubric criteria and less reliable for criteria requiring broader contextual interpretation. Further, we discuss four practical observations below that are relevant for instructors considering AI-assisted grading.

Performance Analysis (Accuracy and Time)

The fastest configurations (often gemma3 on S2 or S3) achieved accuracy comparable to slower alternatives on most criteria, suggesting no clear accuracy-speed trade-off (compare Table 2 and Table 3). In contrast, the performance of gptoss does not consistently align with small models, as its fast ES performance but weaker SA accuracy suggests an accuracy-speed trade-off. In both evaluations, qwen3 is the slowest in all systems, except mistral, which was slower in SA on S2. The qwen3 is slower because it is a reasoning model and produces larger output tokens. gemma3 is among the fastest in all evaluations, except for llama3 on S2 for SA. Across tasks and systems, assessing ES required less time than SA (see Figure 5). S3 was the fastest overall, while S1 was the slowest due to an older GPU. The evaluation of SA across all models and systems took longer due to the visual elements and a more complex context (see Figure 2).

Overall Metrics Across 10 Runs

Based on Figure 5, ES consistently achieves higher accuracy than SA across all systems. Accuracy is also consistent across runs, as indicated by the narrow 95% CIs around the mean for most models. llama3 tends to lead in accuracy. Although qwen3 has similar accuracy on ES, it is noticeably slower, particularly on S2, due to reasoning. ES also has lower response times, which is beneficial for real-time usage. gptoss is fastest for ES, while gemma3 and llama3 are the fastest for SA. In contrast, gptoss shows mixed performance, with competitive accuracy on some ES criteria but lower results on others, particularly in SA. These findings align with per-criterion observations where ES tasks were faster and more accurate than SA tasks.

Strict Rubrics Cause Performance Variations

Unlike traditional evaluations of essays or open-ended questions [12, 13, 29], where models are often given the flexibility to interpret or score along a range, our study uses strict assessment rubrics (see § 3.1 and § 3.2). Most responses were graded as 0 or 1, with a single case using a 0–2 scale. In such an all-or-nothing system, partially correct answers receive 0 (which aligns with the intended grading system of the course), so small omissions or mismatches can still lead to disagreement with the ground truth score. Among different assessment criteria (see § 3), models achieved over 90% accuracy on phishing-related tasks such as phishing_site_disabled and phishing_email_address, strongly aligning with human evaluation in those rubrics. However, performance dropped substantially for context-heavy criteria such as sections, where accuracy fell below 55% (see Table 2 and Table 3).

Contextual and Integration Challenges

In our evaluation, models were given a predefined context of students’ tasks and reports. Despite embedding that context into the prompts, the models sometimes struggled to recognize it. The results of both evaluations (Table 2 and Table 3) indicate that current language models are not yet mature enough to be reliably deployed in context-dependent tasks. This highlights a key challenge for educational use: criteria such as incident_reports in ES and sections in SA require multi-step synthesis and remain laborious for human graders. These observations highlight that while there is promise in using language models for assessment, careful consideration is needed when designing evaluation rubrics and prompts. In particular, rubrics should be based on what counts as evidence and how it is scored, which helps the model follow the provided context in its judgments rather than relying on generic reasoning.

Implications for Teaching Practice

Since no substantial performance trade-off was observed between larger and smaller models, smaller models demonstrate comparable or better performance with the same pipeline design. This enables a practical approach and a toolset for integrating small language models into automatic text assessment workflows, particularly in low-resource environments.

Our study suggests that the approach is most useful for rubric criteria tied to clearly identifiable evidence in student submissions. It presents a replicable grading pipeline that uses locally run lightweight text and vision models on consumer-grade GPUs. This setup automates scoring and feedback without relying on cloud-based models. It is a cost-effective solution that reduces dependence on external services and enables offline grading. Furthermore, it demonstrates the feasibility of deploying these models in resource-constrained environments without fine-tuning, making it easier to use for instructors.

The work contributes an open-source extensible tool [6], a detailed method, and lessons learned to support other instructors in adopting AI-assisted grading in diverse educational settings. The proposed method serves as a foundation for educators to replicate and adapt it for automated text assessment of student writing, including both open-ended text and rich-text responses, although our results suggest that more interpretive and context-dependent criteria still require human review. It requires minimal adjustment to accommodate various file formats, making it applicable across multiple educational and institutional contexts. Instructors can use the generated scores and feedback as an initial triage step to highlight likely score ranges and flag responses that need closer human review, especially for criteria involving document structure, synthesis, or multimodal interpretation, which they then verify, adjust, and finalize. Moreover, it encourages educators to explore AI-assisted grading not as a replacement for human assessment but as a scalable support tool that can reduce instructor workload. Also, speeding up assessment benefits students by ensuring timely feedback and enhancing consistent grading across instructors.

Limitations and Future Work

We consider human scores to be the ground truth. There may be cases where humans made a mistake, but the model was correct, which can affect the reported accuracy. We mitigated this risk by having expert human instructors consult the lead course instructor before and during grading, and follow well-defined rubrics. The evaluation criteria were tailored to one course and need to be modified for other contexts. In addition, we observed inconsistencies in models’ scoring accuracy, particularly for criteria requiring complex structure understanding or contextual interpretation. These inconsistencies may also reflect model biases, such as over-reliance on surface-level patterns or incomplete use of the provided context, as has been observed in prior work on LLM-based assessment [16].

Future work may examine how the workflow can be adapted to other assignment types and how prompt and rubric design can better support contextual interpretation, document structure assessment, or multimodal evidence. It would also be valuable to identify which rubric dimensions are suitable for automated first-pass assessment and which continue to require closer human evaluation.

Conclusions

Prior studies on AI-assisted grading emphasized cloud-based or high-resource LLM setups, mostly overlooking locally deployable, open-source models for assessment in low-resource environments. Our study demonstrated that smaller models can be equally or even more effective than a large model for automated assessment of both text-only and rich-text documents. These accessible models can be used without fine-tuning in constrained environments, providing a practical, quick triage system for grading to support instructors. However, since our evaluation showed that some context-dependent tasks remain challenging, we emphasize that such deployment is not intended to replace human oversight. To prevent potential biases, the instructor must make the “final call”.

Our evaluation across systems and models tested the practical feasibility for instructors, especially those in under-resourced institutions with limited access to costly hardware or third-party services. This offers a viable, privacy-preserving path to adopting AI-assisted grading. To support transparency and reproducibility, all materials, including data, code, and evaluation templates, are publicly shared on GitLab [6]. The cybersecurity exercise platform we used is also available as open-source software [24] – see also https://inject.muni.cz. These resources provide a cost- and privacy-aware foundation for extending low-resource automated assessment to diverse educational domains.

Acknowledgments

This work was supported by the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (ID:90254).

References

References

  1. [1]Umair Z. Ahmed, Shubham Sahai, Ben Leong, and Amey Karkare. 2025. Feasibility Study of Augmenting Teaching Assistants with AI for CS1 Programming Feedback. In Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1 (Pittsburgh, PA, USA). Association for Computing Machinery, New York, NY, USA, 11–17. doi:10.1145/3641554.3701972DOI
  2. [2]Ryan S. Baker, Stephen Hutt, Christopher A. Brooks, Namrata Srivastava, and Caitlin Mills. 2024. Open Science and Educational Data Mining: Which Practices Matter Most? In Proceedings of the 17th International Conference on Educational Data Mining. International Educational Data Mining Society, Atlanta, Georgia, USA, 279–287. doi:10.5281/zenodo.12729816
  3. [3]Aristoteles Barros, Mateus Monteiro, Luiz Rodrigues, Diego Dermeval, Seiji Isotani, and Igbert Bittencourt. 2025. Evaluating Large Language Model Quality in Resource-Constrained Environments: An Educational Stakeholders’ Survey on Accuracy, Completeness, and Readability. In Artificial Intelligence in Education: 26th International Conference, AIED 2025, Palermo, Italy, July 22–26, 2025, Proceedings, Part II (Palermo, Italy). Springer-Verlag, Berlin, Heidelberg, 218–232. doi:10.1007/978-3-031-98417-4_16DOI
  4. [4]Hiroaki Funayama, Tasuku Sato, Yuichiro Matsubayashi, Tomoya Mizumoto, Jun Suzuki, and Kentaro Inui. 2022. Balancing Cost and Quality: An Exploration of Human-in-the-Loop Frameworks for Automated Short Answer Scoring. In Artificial Intelligence in Education: 23rd International Conference, AIED 2022, Durham, UK, July 27–31, 2022, Proceedings, Part I (Durham, United Kingdom). Springer-Verlag, Berlin, Heidelberg, 465–476. doi:10.1007/978-3-031-11644-5_38
  5. [5]Yang Jiang, Jiangang Hao, Wenju Cui, Emily Kerzabi, and Patrick Kyllonen. 2025. Uncovering Transferable Collaboration Patterns Across Tasks Using Large Language Models. In Artificial Intelligence in Education, Alexandra I. Cristea, Erin Walker, Yu Lu, Olga C. Santos, and Seiji Isotani (Eds.). Springer Nature Switzerland, Cham, 320–335. doi:10.1007/978-3-031-98417-4_23DOI
  6. [6]Sandesh K. Kafle, Jan Vykopal, and Valdemar Švábenský. 2026. Supplementary materials for this paper. Retrieved October 23, 2026 from https://gitlab.fi.muni.cz/cybersec/papers/2027-sigcse-rubric-based-assessment
  7. [7]Tanzila Kekshahan, Raja Adil Riaz, Ahmad Sami Al-Shamayleh, Adnan Akhundzada, Noman Ali, Muhammad Hamza, and Faheem Akbar. 2025. AI-generated text detection: A comprehensive review of methods, datasets, and applications. Computer Science Review 58 (2025), 100079. doi:10.1016/j.cosrev.2025.100793
  8. [8]Mark Klose, Vasvi Desai, Yang Song, and Edward Gehringer. 2020. EDM and Privacy: Ethics and Legalities of Data Collection, Usage, and Storage. In Proceedings of the 13th International Conference on Educational Data Mining. ERIC, Virtual Conference, 451–459. https://eric.ed.gov/?id=ED607820
  9. [9]Charles Koutcheme, Juliette Woodrow, and Chris Piech. 2026. Aligning Small Language Models for Programming Feedback: Towards Scalable Coding Support in a Massive Global Course. In Proceedings of the 57th ACM Technical Symposium on Computer Science Education V. 1 (USA) (SIGCSE TS 2026). Association for Computing Machinery, New York, NY, USA, 610–616. doi:10.1145/3770762.3772539DOI
  10. [10]Lorenzo Lee Solano, Charles Koutcheme, Juho Leinonen, Alexandr Vassar, and Jake Renzella. 2026. Fine-Tuning Open-Source Models as a Viable Alternative to Proprietary LLMs for Explaining Compiler Messages. In Proceedings of the 57th ACM Technical Symposium on Computer Science Education V.1 (USA) (SIGCSE TS 2026). Association for Computing Machinery, New York, NY, USA, 617–623. doi:10.1145/3770762.3772756
  11. [11]Gen Lin, Li Chen, Cheng Tang, Valdemar Švábenský, Daisuke Deguchi, Takayoshi Yamashita, and Atsushi Shimada. 2025. Single-Agent vs. Multi-Agent LLM Strategies for Reflection Assessment. In Proceedings of the 29th Pacific-Asia Conference on Knowledge Discovery and Data Mining (Sydney, Australia). Springer Nature, Singapore, 300–311. doi:10.1007/978-981-96-8186-0_24
  12. [12]Zhaohui Li, Chengming Zhang, Yumin Jin, Xuesong Cang, Sadhana Puntambekar, and Rebecca J. Passonneau. 2023. Learning When to Defer to Humans for Short Answer Grading. In Artificial Intelligence in Education, Ning Wang, Genaro Rebolledo-Mendez, Noboru Matsuda, Olga C. Santos, and Vania Dimitrova (Eds.). Springer Nature Switzerland, Cham, 414–425. doi:10.1007/978-3-031-36272-9_34
  13. [13]Zhaohui Li, Chengming Zhang, Yumin Jin, Xuesong Cang, Sadhana Puntambekar, and Rebecca J. Passonneau. 2023. Learning When to Defer to Humans for Short Answer Grading. In Artificial Intelligence in Education, Ning Wang, Genaro Rebolledo-Mendez, Noboru Matsuda, Olga C. Santos, and Vania Dimitrova (Eds.). Springer Nature Switzerland, Cham, 414–425. doi:10.1007/978-3-031-36272-9_34DOI
  14. [14]Islam Nassar, Munawar Hayat, Ehsan Abbasnejad, Hamid Rezatofighi, Mehrtash Harandi, and Gholamreza Haffari. 2023. LAVA:Label-efficient Visual Learning and Adaptation. In 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, Piscataway, NJ, USA, 147–156. doi:10.1109/WACV56688.2023.00023DOI
  15. [15]Connor Nelson, Adam Doupé, and Yan Shoshitaishvili. 2025. SENSAI: Large Language Models as Applied Cybersecurity Tutors. In Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1 (Pittsburgh, PA, USA). Association for Computing Machinery, New York, NY, USA, 833–839. doi:10.1145/3641554.3701801DOI
  16. [16]Jaclyn Oumpaugh, Xiner Liu, and Andres Felipe Zambrano. 2025. Language Models and Dialect Differences. In Proceedings of the 15th International Learning Analytics and Knowledge Conference (LAK ’25). Association for Computing Machinery, New York, NY, USA, 204–215. doi:10.1145/3706468.3706496
  17. [17]Ollama. 2023. Get up and running with large language models. https://ollama.com/
  18. [18]OpenAI. 2025. gpt-oss-120b & gpt-oss-20b Model Card. arXiv:2508.10925 [cs.CL]. https://arxiv.org/abs/2508.10925
  19. [19]Pavlin G Poličar, Martin Špendl, Tomaž Curk, and Blaž Zupan. 2025. Automated assignment grading with large language models: insights from a bioinformatics course. Bioinformatics 41 (07 2025), i21–i29. doi:10.1093/bioinformatics/btaf196
  20. [20]Maria Mercedes T. Rodrigo. 2024. Is the AI CodeNuma a First-World Problem? International Journal of Artificial Intelligence in Education 34, 1 (2024), 55–61. doi:10.1007/s40593-023-00345-2
  21. [21]David Soto, Shizuka Shirai, Mayumi Ueda, Manabu Higashida, Yuki Uranishi, and Haruo Takemura. 2024. Cloud Computing Challenges and Needs in Higher Education Institutions in Post-COVID-19 Times: A Case of a Japanese Survey. IEEE Access 12 (2024), 168043–168059. doi:10.1109/ACCESS.2024.3493422DOI
  22. [22]Fouad Trad and Ali Chehab. 2025. Evaluating the Efficacy of Prompt-Engineered Large Multimodal Models Versus Fine-Tuned Vision Transformers in Image-Based Security Applications. ACM Trans. Intell. Syst. Tech. 16 (2025), 23 pages. doi:10.1145/3735648DOI
  23. [23]Joel Antonio Trejo-Sánchez et al. 2022. Teaching High-Performance Computing in Developing Countries: A Case Study in Mexican Universities. In 2022 IEEE International Parallel and Distributed Processing Symposium Workshops. IEEE Access, Lyon, France, 338–345. doi:10.1109/IPDPSW55747.2022.00066
  24. [24]Valdemar Švábenský, Jan Vykopal, Martin Horák, Martin Hofbauer, and Pavel Čelada. 2024. From Paper to Platform: Evolution of a Novel Learning Environment for Tabletop Exercises. In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 (Milan, Italy) (ITiCSE 2024). Association for Computing Machinery, New York, NY, USA, 213–219. doi:10.1145/3649217.3653639DOI
  25. [25]Kevin Shukang Wang and Ramon Lawrence. 2025. Quantitative Evaluation of Using Large Language Models and Retrieval-Augmented Generation in Computer Science Education. In Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1 (Pittsburgh, PA, USA) (SIGCSETS 2025). Association for Computing Machinery, New York, NY, USA, 1183–1189. doi:10.1145/3641554.3701917DOI
  26. [26]Tianyu Wang, Nianjun Zhou, and Zhixiong Chen. 2025. CyberMentor: AI Powered Learning Tool Platform to Address Diverse Student Needs in Cybersecurity Education. arXiv:2501.09709 [cs.CY]. https://arxiv.org/abs/2501.09709
  27. [27]Yumou Wei, Paulo Carvalho, and John Stamper. 2025. Small but Significant: On the Promise of Small Language Models for Accessible AIED. arXiv:2505.08588 [cs.CL]. https://arxiv.org/abs/2505.08588
  28. [28]Lixiang Yan, Samuel Greiff, Ziwen Teuber, and Dragan Gašević. 2024. Promises and challenges of generative artificial intelligence for human learning. Nature Human Behaviour 8, 10 (2024), 1839–1850. doi:10.1038/s41562-024-02004-5
  29. [29]Steven Yeung. 2025. A comparative study of rule-based, machine learning and large language model approaches in automated writing evaluation (AWE). In Proceedings of the 15th International Learning Analytics and Knowledge Conference (LAK ’25). Association for Computing Machinery, New York, NY, USA, 984–991. doi:10.1145/3706468.3706566DOI
  30. [30]J. D. Zamfirescu-Pereira, Lany Qiu, Björn Hartmann, John DeNero, and Narges Norouzi. 2025. 61A Bot Report: AI Assistants in CS1 Save Students Homework Time and Reduce Demands on Staff. (Now What?). Vol. 1. Association for Computing Machinery, New York, NY, USA, 1309–1315 pages. Issue 1. doi:10.1145/3641554.3701864
  31. [31]Zijie Zeng, Lin Li, Quanlong Guan, Dragan Gašević, and Guanliang Chen. 2023. Generalizable Automatic Short Answer Scoring via Prototypical Neural Network. In Artificial Intelligence in Education: 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings (Tokyo, Japan). Springer-Verlag, Berlin, Heidelberg, 438–449. doi:10.1007/978-3-031-36272-9_36DOI
  32. [32]Hao Zhang, Tingmin Wu, Tianqing Zhu, Sheng Wen, and Yang Xiang. 2025. CyberLLaMA: A fine-tuned large language model for cybersecurity named entity recognition. Knowledge-Based Systems 328 (10 2025). doi:10.1016/j.knosys.2025.114183
  33. [33]Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. 2024. TinyLlama: An Open-Source Small Language Model. arXiv:2401.02385 [cs.CL]. https://arxiv.org/abs/2401.02385
  34. [34]Chenyan Zhao, Mariana Silva, and Seth Poulsen. 2025. Language Models are Few-Shot Graders. In Artificial Intelligence in Education, Alexandra I. Cristea, Erin Walker, Yu Lu, Olga C. Santos, and Seiji Isotani (Eds.). Springer Nature Switzerland, Cham, 3–16. doi:10.1007/978-3-031-98459-4_1DOI

Paper details

vlnc:W_wthmhrk7
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 7

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents