WorksComputer Science
LusoClin: A Synthetic Clinical Note Dataset for Medical Information Retrieval in European Portuguese
Abstract
Access to large-scale clinical text collections is crucial for advancing medical information retrieval research, including clinical note search, patient cohort identification, and domain-adapted retrieval models. However, real clinical notes are rarely available due to privacy regulations, which particularly limits progress for low-resource languages such as European Portuguese. We present LusoClin, a publicly released dataset of synthetic clinical notes in European Portuguese, created to support privacy-preserving research on clinical text retrieval. LusoClin was generated by pre-processing real clinical data, fine-tuning the open-source Llama 3.2 (3B) instruction-tuned model using Quantized Low-Rank Adaptation, and applying prompt-based clinical note generation. To facilitate semantic indexing and clinical search, the notes are automatically annotated with clinical entities using a BERT-based model. We evaluate LusoClin in terms of utility, fidelity, diversity, and privacy using automatic metrics and human evaluation. Results indicate that the dataset captures clinically relevant language patterns and can support downstream IR applications such as clinical document retrieval and domain-specific model adaptation, although annotators can often distinguish synthetic from real notes and some medical inconsistencies persist.
Introduction
Electronic health records (EHRs) contain longitudinal patient information collected during healthcare delivery [6, 15]. Much of this information is stored in clinical notes, unstructured records describing clinical encounters [33]. These notes capture contextual information often absent from structured EHR fields.
Clinical information retrieval plays a central role in enabling access to this unstructured information, useful for decision-making and clinical research [42]. Retrieval techniques, including recent dense retrieval methods, have also been applied to patient cohort identification by querying electronic health records to identify patient groups with shared criteria [18, 24, 4]. Because clinical information is often recorded as unstructured text [23], retrieval systems rely on robust text representations, often supported by information extraction and natural language processing techniques [40]. In this context, the development and evaluation of clinical IR systems increasingly depend on large-scale, high-quality textual collections, which are difficult to obtain due to privacy regulations [25].
This challenge is especially pronounced for languages other than English. While English medical corpora are increasingly available [40], many languages remain severely under-resourced [39]. Portuguese is a representative example: although some clinical text resources exist for Brazilian Portuguese [22, 35, 38], European Portuguese remains largely unsupported [27], primarily due to a lack of publicly available clinical text [26]. This scarcity limits the development, adaptation, and evaluation of clinical retrieval models for European Portuguese.
Synthetic data is computer-generated data used to augment or replace real data [17]. Recent progress in large language models (LLMs) has renewed interest in synthetic data generation showing that LLMs can generate high-quality synthetic clinical text at scale, reducing both cost and privacy concerns [25, 21, 48, 50, 51]. In this work, we introduce LusoClin, a large-scale, publicly available dataset of synthetic clinical notes in European Portuguese generated using the open-source Llama 3.2 (3B) instruction-tuned model. Although the dataset presents limitations typical of synthetic clinical text, LusoClin provides a resource for developing and evaluating clinical IR models in European Portuguese, addressing a critical gap in available retrieval resources. The dataset, along with generation prompts, annotations, and human evaluation results, is publicly released in a dedicated repository [13].
Related Work
Synthetic data has been increasingly explored as a way to address the difficulty of obtaining large-scale real-world data, particularly in sensitive domains such as healthcare. In addition to alleviating data scarcity, synthetic data generation has been shown to offer benefits such as reducing bias [21, 46], improving performance, scalability, and stronger privacy guarantees [19]. These properties make synthetic data particularly attractive.
Recent advances in LLMs have further accelerated interest in synthetic clinical text generation. Due to extensive pre-training, LLMs exhibit strong language understanding capabilities [8, 48] and can generate fluent unstructured text [48, 49]. As a result, several studies have leveraged LLMs to generate synthetic unstructured clinical datas. While methodologies vary, most approaches share elements related to data preparation, prompt design, and evaluation.
Some studies rely primarily on pre-trained LLMs [10, 44], occasionally augmented with few-shot examples [12, 28, 49], while others further adapt models through fine-tuning on clinical or task-specific corpora [34, 1, 20].
Prompt engineering plays a central role in guiding LLMs to produce clinically meaningful outputs. Prior work has explored diverse prompting strategies, ranging from highly structured prompts specifying content and format constraints [10] to iterative prompt refinement [44], few-shot prompting with annotated examples [12, 28], and frameworks that inject external medical knowledge [49]. These studies highlight the importance of balancing guidance and variability to achieve realistic and diverse synthetic clinical text.
Evaluation of synthetic clinical datasets is independent of the generation approach and remains a critical challenge. In the clinical domain, evaluation typically focuses on four key dimensions: utility, fidelity, diversity, and privacy [2]. Utility assesses whether synthetic data can effectively support downstream modelling and retrieval tasks [3], while fidelity measures the degree to which synthetic samples resemble real clinical text in terms of correctness and coherence [3, 11, 25]. Diversity reflects the ability to capture clinically meaningful variation [9, 25], and privacy ensures that synthetic data cannot be linked back to real individuals [9]. These dimensions are often in tension, requiring careful trade-offs during dataset construction and evaluation [14].
Dataset Construction
We generated LusoClin using Meta’s Llama 3.2 [45], specifically the 3-billion-parameter instruction-tuned variant (“Llama-3.2-3B-Instruct”). This model was selected to support local execution and fine-tuning using publicly available weights, avoiding the need to share sensitive data with third-party services and enabling greater control over the generation. Additional selection criteria included a context window large enough to accommodate the longest clinical documents and example prompts (with 2,048 tokens used as a safe threshold), compatibility with available GPU and storage resources, and support for Portuguese—particularly European Portuguese—within an instruction-based prompt–response framework.
To ground the generation process in realistic clinical language and structure, we leveraged existing publicly available clinical datasets. Specifically, we used SemClinBr [33] and CLINpt [26] as source data. SemClinBr consists of 1,000 Brazilian Portuguese clinical notes spanning four types—admission notes, ambulatory notes, discharge summaries, and nursing notes—and includes annotations for 87 semantic entity types, along with abbreviation and negation labels and two relation types. CLINpt comprises 281 European Portuguese neurology clinical cases with manually annotated named entities, providing complementary coverage of European Portuguese clinical language. Both datasets are anonymized. An overview of the generation process is shown in Figure 1.

Figure 1. Generation pipeline.
Fine-tuning
Following prior work [1, 34], we pre-processed both datasets by extracting document text, converting characters to lowercase, normalizing diacritics, and removing punctuation. Fine-tuning was performed using QLoRA [7] with the AdamW optimizer [29], a learning rate of , weight decay of 0.01, four epochs, and a batch size of four. Training used a virtual machine with 100 GB RAM, 100 GB storage, and one NVIDIA RTX 3090 GPU.
Prompt definition
We iteratively refined the generation prompt through empirical experimentation until the model’s outputs consistently resembled realistic clinical notes. Drawing on prior work, we progressively enriched an initial prompt that instructed the model to act as a Portuguese clinician by adding more detailed task descriptions, explicit constraints to mitigate common generation errors, and few-shot examples sampled from SemClinBr. Prompt refinement was accompanied by tuning the model’s temperature.
Prompt constraints controlled aspects such as text length, structure, and style, including avoiding section headers, signatures, lists, repetitive phrasing, and excessive use of generic terms (e.g., repeated mentions of “patient”). These constraints were combined to correct recurring issues while avoiding excessive prompt complexity. We did not explicitly describe the characteristics of each clinical note type, allowing the model to infer them from examples. To encourage lexical and semantic diversity, each prompt instance included an example containing a randomly selected Disease or Syndrome entity from SemClinBr. Attempts to use English-language prompts were unsuccessful.
All prompt templates, associated temperatures, and generated responses from this phase are publicly available in the accompanying repository [13]. For clarity, we present an English translation of the final prompt template below; the original Portuguese version was used for generation and is included in the repository.
Write, in European Portuguese, [Clinical Note Type] with 10
sentences, 130 words, and only 1 paragraph.
Clinical Note Type] must be in running text, without any subdivision.
Do not include any ‘Age: 52’-style sections. Don’t repeat the word
‘patient’ several times. Don’t repeat sentences. Your answer must
include only [Clinical Note Type].
Some examples are: [Examples with one randomly chosen ‘Disease or
Syndrome’ entity]. Don’t give any more examples. Don’t elaborate on
any of these examples.
For each generation prompt, we specified a clinical note type corresponding to those defined in SemClinBr, translated to European Portuguese with the assistance of medical students. Prompts included up to four few-shot examples selected from SemClinBr notes of the requested type that contained a randomly chosen Disease or Syndrome entity; when fewer matching notes were available, all were used. This strategy was adopted to encourage semantic diversity while maintaining note-type consistency.
To mitigate common generation errors, we introduced explicit constraints on length and structure, setting limits of 130 words and 10 sentences based on the average size of SemClinBr notes, and empirically selected a temperature of 0.6. In addition, simple rule-based post-processing was applied to generated outputs to remove instruction-related artifacts, repeated or incomplete sentences, and isolated numbered points.
To assess whether further prompt refinement was necessary before large-scale generation, we conducted a small-scale preliminary evaluation with two final-year medical students. Sixteen synthetic notes (four per note type) and eight real notes from SemClinBr (two per note type) were randomly selected and evenly split between the evaluators. The evaluators were blinded to both the origin and type of each note and completed a Turing-style assessment and qualitative classifications. Although limited in scale, this evaluation was sufficient to determine that no further prompt modifications were required, and the original prompt was retained.
Dataset generation and post-processing
After completing prompt engineering, we generated the full LusoClin dataset comprising 100,000 synthetic clinical notes, with 25,000 notes per clinical note type. Generation was performed on the same virtual machine used for fine-tuning and required a total of 17 days, 20 hours, 50 minutes, and 44 seconds.
Following generation, we applied additional rule-based post-processing to remove noisy outputs. Some generated notes contained instruction-like text, such as requests to generate notes or meta-level guidance, while others resembled transcripts of physician–patient interactions rather than clinical documentation. To filter such cases, we automatically removed notes containing a predefined set of keywords indicative of these errors, including the portuguese translations of ’to write’, ’to rewrite’, ’write’, ’rewrite’, ’to register’, ’you are’, formal version of ’you’, informal version of ’you’.
Human evaluation
Human evaluation was conducted due to its importance in assessing the realism (fidelity dimension) and utility of synthetic clinical data. After post-processing, LusoClin comprised 98,571 synthetic clinical notes: 24,759 admission notes, 24,411 ambulatory notes, 24,639 discharge summaries, and 24,762 nursing notes.
The evaluation involved medical professionals and medical students enrolled in the third to sixth years of a six-year medical degree at the University of Porto. Recruitment was conducted via institutional email lists and faculty outreach, and evaluations took place between May 15 and June 2, 2025. Participants completed the evaluation through a dedicated web application1, where they were shown a clinical note selected with a 90% probability from LusoClin and a 10% probability from the real SemClinBr dataset, without being informed of its origin.
For each note, evaluators indicated whether they believed it to be real or synthetic and optionally provided justifications and excerpt-level comments. To increase reliability while preserving coverage, we collected up to two independent evaluations per note using a controlled random assignment strategy that avoided repeated evaluations by the same user. The results of this evaluation are included in § 4.1.
Dataset improvements
Following the initial evaluation, we applied a series of refinements to improve LusoClin, focusing on reducing Brazilian Portuguese (PT-BR) influence, correcting syntactic issues, and removing irrelevant content. To identify PT-BR usage, we employed PtVId [43], a BERT-based classifier that distinguishes between European Portuguese (PT-PT) and PT-BR and provides a confidence score. In an initial pass, only 31,525 of the 98,564 notes (approximately 32%) were classified as PT-PT. To mitigate this, we manually analyzed the 1,000 most salient terms (by TF–IDF) in notes classified as PT-BR and replaced PT-BR expressions with their PT-PT equivalents. This process resulted in modifications to 71,381 notes and increased the number of notes classified as PT-PT to 56,079 (approximately 57%).
We then addressed recurrent syntactic issues, primarily the absence of diacritics, which led to corrections in 60,590 clinical notes. These corrections improved linguistic quality without altering clinical content. Finally, we removed notes deemed irrelevant. This included six notes containing no Portuguese content and 2,362 notes containing terms indicative of prompt echoing or generic descriptions rather than clinical documentation. After these filtering steps, LusoClin comprised 96,202 clinical notes, of which 54,518 (approximately 57%) were classified as PT-PT.
Given our objective of producing a strictly European Portuguese clinical dataset, we retained only notes classified as PT-PT. The final version of LusoClin contains 54,518 synthetic clinical notes: 14,253 admission notes, 13,561 ambulatory notes, 9,032 discharge summaries, and 17,672 nursing notes.
Automatic Evaluation
To complement human evaluation, we conducted an automatic assessment targeting utility, fidelity, diversity, and privacy. Evaluation included perplexity, textual and entity-level statistics, t-SNE visualizations, Central Moment Discrepancy (CMD), cosine similarity, and Hausdorff distance to compare synthetic and real clinical text representations.
Dataset Annotation
To enable entity-level evaluation, we automatically annotated LusoClin with named entities using MediAlbertina PT-PT [32], a 1.5B BERT-based model fine-tuned for NER. The model was pre-trained on European Portuguese text [37] and further adapted using medical documents from Portugal’s largest public hospital [32]. We applied the publicly available NER version to all clinical notes using the entity categories Diagnosis, Symptom, Medication/Drug, Dosage, Medical Procedure, Vital Sign, Result, and Progress.
Evaluation Results
This section reports the results of our two-stage evaluation.
Human evaluation results
We collected 149 evaluations of 78 clinical notes. Among these, 71 notes (57 LusoClin and 14 SemClinBr) received two evaluations, summarized in Table 1.
| Note type | LusoClin | SemClinBr | ||||
| 2R | 1R+1S | 2S | 2R | 1R+1S | 2S | |
| Admission notes | 0 | 2 | 6 | 0 | 1 | 0 |
| Ambulatory notes | 1 | 10 | 9 | 1 | 4 | 2 |
| Discharge summaries | 1 | 8 | 8 | 0 | 2 | 0 |
| Nursing notes | 1 | 2 | 9 | 1 | 1 | 2 |
| Total | 3 | 22 | 32 | 2 | 8 | 4 |
Table 1. Distribution of votes for clinical notes with two independent evaluations (R = Real, S = Synthetic).
We computed Cohen’s Kappa [5], obtaining a value of -0.04, indicating no meaningful inter-annotator agreement [30, 41, 47]. This may reflect the difficulty of distinguishing synthetic from real clinical notes or differences in evaluator criteria [41]. We therefore restrict the remaining analysis to the 41 notes with concordant evaluations.
Table 2 shows the human classifications. Only 9% of LusoClin notes were classified as real, while 67% of SemClinBr notes were judged synthetic. This suggests evaluators adopted strict realism criteria, potentially interpreting characteristics of genuine clinical notes as indicators of synthetic origin. Five LusoClin notes received concordant justifications, attributed to medical errors, meta-data artifacts, missing key information, or an overall synthetic appearance. Discordant justifications are summarized in Table 3. Although most justifications were provided by only one evaluator, they remain useful for identifying potential weaknesses and informing dataset improvements.
| Note type | LusoClin | SemClinBr | ||
| Real | Synthetic | Real | Synthetic | |
| Admission notes | 0 (0%) | 6 (100%) | 0 (0%) | 0 (0%) |
| Ambulatory notes | 1 (10%) | 9 (90%) | 1 (33%) | 2 (67%) |
| Discharge summaries | 1 (13%) | 8 (87%) | 0 (0%) | 0 (0%) |
| Nursing notes | 1 (10%) | 9 (90%) | 1 (33%) | 2 (67%) |
| Total | 3 (9%) | 32(91%) | 2 (33%) | 4 (67%) |
Table 2. Human classification of clinical notes.
| Justification | LusoClin N | LusoClin % | SemClinBr N | SemClinBr % |
| Feels synthetic | 7 | 17.07% | 0 | 0.00% |
| Medical errors | 7 | 17.07% | 1 | 50.00% |
| Missing key information | 6 | 14.63% | 0 | 0.00% |
| Non-sensical | 5 | 12.20% | 0 | 0.00% |
| Contains meta-data | 4 | 9.76% | 0 | 0.00% |
| Repetitive | 3 | 7.32% | 0 | 0.00% |
| Incorrect structure and/or grammar | 3 | 7.32% | 0 | 0.00% |
| Unusual terminology | 3 | 7.32% | 0 | 0.00% |
| Brazilian words | 2 | 4.88% | 1 | 50.00% |
| Non-existent medical terminology | 1 | 2.44% | 0 | 0.00% |
Table 3. Number and proportion of justifications for classifying notes as synthetic, excluding empty justifications.
Automatic evaluation results
As described in § 3.6, we combined several automatic evaluation methods targeting utility, fidelity, diversity, and privacy. Table 4 summarizes the methods and baselines used.
Table 4. Evaluation methods, assessed dimensions, and comparison baselines.
* Computed on a subset of LusoClin matched to the size of SemClinBr.
Perplexity and textual statistics were computed on the full dataset, while t-SNE, CMD, cosine similarity, and Hausdorff distance used a subset of 1,000 LusoClin notes matched to SemClinBr. These methods relied on multilingual SentenceBERT embeddings [36]. SemClinBr served as baseline for perplexity, statistics, and cosine similarity, while [49] provided the English-language baseline for t-SNE and CMD.
Perplexity We computed the average perplexity of GlórIA [27] and Cabrita [22] on LusoClin and SemClinBr. LusoClin achieved lower perplexity on both models (68.32 vs. 82.57 for GlórIA; 29.44 vs. 37.31 for Cabrita), indicating that the synthetic notes were easier to predict. Since lower perplexity has been associated with medically coherent clinical content [1, 21, 31], these results support the utility and fidelity of LusoClin.
Textual and entity statistics We compared textual and entity-level statistics between datasets, including document length, sentence count, and entity distributions. Figure 2 shows that synthetic notes are slightly longer than real notes, while Figure 3 indicates similar sentence structures, although SemClinBr displays greater variability. Table 5 reports entity type frequencies. The average absolute difference between datasets was 3.71%, indicating close alignment in medical entity distributions and supporting the realism of LusoClin. Figure 4 further shows that synthetic notes contain more entities on average, suggesting denser medical terminology.

Figure 2. Word count distributions for each note type.

Figure 3. Sentence count distributions for each note type.

Figure 4. Entity count distributions for each note type.
| Entity type | LusoClin | SemClinBr | ||
| Count | % | Count | % | |
| Diagnosis | 273,520 | 24.28% | 2,378 | 16.31% |
| Symptom | 193,152 | 17.14% | 1,915 | 13.14% |
| Medical Procedure | 189,966 | 16.86% | 2,228 | 15.28% |
| Result | 153,159 | 13.59% | 2,616 | 17.94% |
| Vital Sign | 142,933 | 12.69% | 2,617 | 17.95% |
| Medication/Drug | 85,835 | 7.62% | 1,331 | 9.13% |
| Dosage | 57,143 | 5.07% | 1,281 | 8.79% |
| Progress | 30,939 | 2.75% | 212 | 1.45% |
Table 5. Entity type frequencies.
t-SNE plots We examined t-SNE visualizations of the LusoClin subset and SemClinBr, shown in Figure 5. The plots reveal only partial overlap between the two datasets, suggesting that the distribution of LusoClin could more closely align with that of SemClinBr to achieve comparable levels of utility and fidelity. This observation is consistent with findings reported by Xu et al. [49], whose English-language baseline exhibits a greater overlap between synthetic and real data distributions. At the same time, the overall spacing and dispersion of LusoClin data points appear similar to those of SemClinBr, mirroring the behavior observed in the English baseline [49]. This indicates that LusoClin captures a comparable level of diversity, which is a positive outcome given that SemClinBr is a multi-specialty clinical dataset.

Figure 5. t-SNE plots of LusoClin and SemClinBr.
Central Moment Discrepancy We additionally computed the average CMD between the LusoClin subset and SemClinBr, obtaining a value of 0.395. Compared with the English-language baseline reported by Xu et al. [49], LusoClin achieves a lower CMD on this metric. This result indicates limited shift between the synthetic and real clinical notes, providing further evidence that LusoClin preserves key characteristics of the clinical domain and exhibits satisfactory fidelity.
Cosine similarity We computed cosine similarity using sentence-level embeddings to assess both diversity and privacy. First, we calculated the average cosine similarity over all pairwise combinations of sentences within the LusoClin subset. LusoClin exhibits a lower average pairwise cosine similarity (0.091) than SemClinBr (0.137), indicating higher internal diversity among synthetic notes. This is a positive outcome, as it suggests that LusoClin is not overly repetitive. One possible concern is that increased diversity could reflect a shift away from the clinical domain toward more general language; however, this interpretation is not supported by the human evaluation and CMD results, which indicate preservation of clinical characteristics.
For privacy, we then computed the average cosine similarity between cross-dataset pairs consisting of one LusoClin sample and one SemClinBr sample, considering (1) all note types jointly and (2) pairs matched by note type. As shown in Table 6, cross-dataset similarities are consistently low. Following the privacy preservation criterion proposed by Henrnadez et al. [16], which sets a cosine similarity threshold of 0.5, both the type-agnostic and type-matched results fall well below this threshold. This provides strong evidence that LusoClin does not exhibit excessive similarity to real clinical notes and therefore supports its privacy-preserving nature.
| Note type | CosineS | HausD |
| All types | 0.052 | 1.272 |
| Admission notes | 0.061 | 1.267 |
| Ambulatory notes | 0.049 | 1.274 |
| Discharge summaries | 0.068 | 1.275 |
| Nursing notes | 0.073 | 1.244 |
Table 6. Average cosine similarity (CosineS) and Hausdorff distance (HausD) between pairs consisting of one LusoClin note and one SemClinBr note.
Hausdorff distance. We computed the average Hausdorff distance between cross-dataset pairs consisting of one LusoClin note and one SemClinBr note, considering both all note types jointly and pairs matched by note type. These results were compared against the privacy preservation baseline proposed by Hernadez et al. [16], which sets a threshold value of 1. As shown in Table 6, all computed Hausdorff distances exceed this threshold, both in the type-agnostic and type-matched settings. This provides additional evidence that synthetic notes are sufficiently separated from real notes, supporting the privacy-preserving properties of LusoClin.
Conclusions
Based on the applied evaluation methods, we draw several conclusions regarding the quality of LusoClin. Utility and fidelity were the most critical dimensions and were primarily assessed through human evaluation. While evaluators identified issues such as nonsensical content and medical errors in a subset of notes, many synthetic notes were medically coherent and informative. Although most LusoClin notes did not pass as real, this outcome must be interpreted in light of the fact that two-thirds of the SemClinBr notes were also judged synthetic. This highlights the inherent difficulty and subjectivity of human evaluation in synthetic data settings, where evaluators tend to apply stricter realism criteria.
Automatic evaluations of utility and fidelity yielded encouraging results, with LusoClin matching baselines on t-SNE plot and textual and entity statistics and outperforming them on perplexity and CMD. Overall, these findings indicate that LusoClin provides a reasonable level of utility and fidelity, though quality issues remain in a subset of notes.
Evaluation of diversity showed that LusoClin is at least as diverse as SemClinBr, suggesting that it meets diversity requirements for downstream use. Privacy assessment results were also positive. The combination of our evaluation results and the use of professionally de-identified source data strongly suggests that LusoClin does not pose privacy risks.
Overall, the results indicate that LusoClin cannot fully replace real clinical data but can serve as a valuable complementary resource for research, particularly in low-resource settings.
Acknowledgments. This work was co-financed by Component 5 - Capitalization and Business Innovation, integrated in the Resilience Dimension of the Recovery and Resilience Plan within the scope of the Recovery and Resilience Mechanism (MRR) of the European Union (EU), framed in the Next Generation EU, for the period 2021 - 2026, within project HfPT, with reference 41.
References
- [1]Amin-Nejad, A., Ive, J., Velupillai, S.: Exploring transformer text generation for medical dataset augmentation. In: Calzolari, N., Béchet, F., Blache, P., Choukri, K., Cieri, C., De-clerck, T., Goggi, S., Isahara, H., Maegaard, B., Mariani, J., Mazo, H., Moreno, A., Odijk, J., Piperidis, S. (eds.) Proceedings of the Twelfth Language Resources and Evaluation Conference. pp. 4699–4708. European Language Resources Association, Marseille, France (May 2020), https://aclanthology.org/2020.lrec-1.578
- [2]van Breugel, B., Liu, T., Oglic, D., van der Schaar, M.: Synthetic data in biomedicine via generative artificial intelligence. Nature Reviews Bioengineering 2(12), 991–1004 (Dec 2024), https://doi.org/10.1038/s44222-024-00245-7, doi:10.1038/s44222-024-00245-7
- [3]Budu, E., Etminani, K., Soliman, A., Rögnvaldsson, T.: Evaluation of synthetic electronic health records: A systematic review and experimental assessment. Neurocomputing 603, 128253 (2024), https://www.sciencedirect.com/science/article/pii/S0925231224010245, doi: 10.1016/j.neucom.2024.128253DOI
- [4]Chamberlin, S.R., Bedrick, S.D., Cohen, A.M., Wang, Y., Wen, A., Liu, S., Liu, H., Hersh, W.R.: Evaluation of patient-level retrieval from electronic health record data for a cohort discovery task. JAMIA Open 3, 395–404 (10 2020). https://doi.org/10.1093/JAMIAOPEN/OOAA026, https://dx.doi.org/10.1093/jamiaopen/ooaa026
- [5]Cohen, J.: A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 37–46 (1960), https://doi.org/10.1177/001316446002000104, doi: 10.1177/001316446002000104
- [6]Cowie, M.R., Blomster, J.I., Curtis, L.H., Duclaux, S., Ford, I., Fritz, F., Goldman, S., Janmohamed, S., Kreuzer, J., Leaney, M., Michel, A., Ong, S., Pell, J.P., Southworth, M.R., Stough, W.G., Thoenes, M., Zannad, F., Zalewski, A.: Electronic health records to facilitate clinical research. Clinical Research in Cardiology 106(1), 1–9 (Jan 2017), https://doi.org/10.1007/s00392-016-1025-6, doi: 10.1007/s00392-016-1025-6
- [7]Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L.: Qlora: efficient finetuning of quantized llms. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. NIPS ’23, Curran Associates Inc., Red Hook, NY, USA (2023)arxiv.org/abs/2305.14314
- [8]Ding, B., Qin, C., Liu, L., Chia, Y., Joty, S., Li, B., Bing, L.: Is gpt-3 a good data annotator? (2023), https://arxiv.org/abs/2212.10450, doi: 10.48550/arXiv.2212.10450
- [9]Eigenschenk, P., Reutterer, T., Vamosi, S., Vamosi, R., Sun, C., Kalcher, K.: Deep generative models for synthetic data: A survey. IEEE Access 11, 47304–47320 (2023), doi: 10.1109/ACCESS.2023.3275134
- [10]Falis, M., Gema, A.P., Dong, H., Daines, L., Basetti, S., Holder, M., Penfold, R.S., Birch, A., Alex, B.: Can GPT-3.5 generate and code discharge summaries? Journal of the American Medical Informatics Association 31(10), 2284–2293 (09 2024), https://doi.org/10.1093/jamia/ocae132, doi: 10.1093/jamia/ocae132
- [11]Figueira, A., Vaz, B.: Survey on synthetic data generation, evaluation methods and gans. Mathematics 10(15) (2022), https://www.mdpi.com/2227-7390/10/15/2733, doi: 10.3390/math10152733DOI
- [12]Frei, J., Kramer, F.: Annotated dataset creation through large language models for non-english medical nlp. Journal of Biomedical Informatics 145, 104478 (2023), https://www.sciencedirect.com/science/article/pii/S1532046423001995, doi: 10.1016/j.jbi.2023.104478DOI
- [13]Félix, D., Teixeira Lopes, C.: Dataset of synthetic clinical notes in european portuguese generated using an open-source large language model, along with prompting and evaluation data. INESC TEC (2025), https://doi.org/10.25747/4GC6-DK48
- [14]Goncalves, A., Ray, P., Soper, B., Stevens, J., Coyle, L., Sales, A.P.: Generation and evaluation of synthetic patient data. BMC Medical Research Methodology 20(1), 108 (May 2020), https://doi.org/10.1186/s12874-020-00977-1, doi: 10.1186/s12874-020-00977-1
- [15]Gunter, T.D., Terry, N.P.: The emergence of national electronic health record architectures in the united states and australia: models, costs, and questions. J. Med. Internet Res. 7(1), e3 (Mar 2005), https://pmc.ncbi.nlm.nih.gov/articles/PMC1550638/
- [16]Hernandez, M., Epelde, G., Alberdi, A., Cilla, R., Rankin, D.: Synthetic tabular data evaluation in the health domain covering resemblance, utility, and privacy C. Teixeira Lopes and D. Félix. dimensions. Methods Inf Med 62(S 01), e19–e38 (Jan 2023), http://www.thieme-connect.com/products/ejournals/abstract/10.1055/s-0042-1760247, doi: 10.1055/s-0042-1760247
- [17]IBM: What is synthetic data? https://research.ibm.com/blog/what-is-synthetic-data, accessed: 2024-08-27
- [18]Jadhav, P.: Cohort retrieval using dense passage retrieval (2025), https://arxiv.org/abs/2507.01049
- [19]Joshi, I., Grimmer, M., Rathgeb, C., Busch, C., Bremond, F., Dantcheva, A.: Synthetic data in human analysis: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 46(7), 4957–4976 (2024), doi: 10.1109/TPAMI.2024.3362821DOI
- [20]Kumichev, G., Blinov, P., Kuzkina, Y., Goncharov, V., Zubkova, G., Zenovkin, N., Goncharov, A., Savchenko, A.: Medsyn: Lm-based synthetic medical text generation framework. In: Bifet, A., Krilavičius, T., Miliou, I., Nowaczyk, S. (eds.) Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track. pp. 215–230. Springer Nature Switzerland, Cham (2024), doi: 10.1007/978-3-031-70381-2_14DOI
- [21]Kweon, S., Kim, J., Kim, J., Im, S., Cho, E., Bae, S., Oh, J., Lee, G., Moon, J.H., You, S.C., Baek, S., Han, C.H., Jung, Y.B., Jo, Y., Choi, E.: Publicly shareable clinical large language model built on synthetic clinical notes (2024), doi: 10.48550/arXiv.2309.00237DOI
- [22]Larcher, C., Piau, M., Finardi, P., Gengo, P., Esposito, P., Caridá, V.: Cabrita: closing the gap for foreign languages (2023)arxiv.org/abs/2308.11878
- [23]Li, I., Pan, J., Goldwasser, J., Verma, N., Wong, W.P., Nuzumlal, M.Y., Rosand, B., Li, Y., Zhang, M., Chang, D., Taylor, R.A., Krumholz, H.M., Radev, D.: Neural natural language processing for unstructured data in electronic health records: A review. Computer Science Review 46, 100511 (2022). https://doi.org/10.1016/j.cosrev.2022.100511, https://www.sciencedirect.com/science/article/pii/S1574013722000454, doi: 10.1016/j.cosrev.2022.100511
- [24]Liu, S., Wang, Y., Wen, A., Wang, L., Hong, N., Shen, F., Bedrick, S., Hersh, W., Liu, H.: Implementation of a cohort retrieval system for clinical data repositories using the observational medical outcomes partnership common data model: Proof-of-concept system validation. JMIR Medical Informatics 8 (10 2020). https://doi.org/10.2196/17376
- [25]Long, L., Wang, R., Xiao, R., Zhao, J., Ding, X., Chen, G., Wang, H.: On llms-driven synthetic data generation, curation, and evaluation: A survey (2024), https://arxiv.org/abs/2406.15126, doi: 10.48550/arXiv.2406.15126
- [26]Lopes, F., Teixeira, C., Gonçalo Oliveira, H.: Contributions to clinical named entity recognition in Portuguese. In: Demner-Fushman, D., Cohen, K.B., Ananiadou, S., Tsujii, J. (eds.) Proceedings of the 18th BioNLP Workshop and Shared Task. pp. 223–233. Association for Computational Linguistics, Florence, Italy (Aug 2019), https://aclanthology.org/W19-5024, doi: 10.18653/v1/W19-5024DOI
- [27]Lopes, R., Magalhaes, J., Semedo, D.: GlórIA: A generative and open large language model for Portuguese. In: Gamallo, P., Claro, D., Teixeira, A., Real, L., Garcia, M., Oliveira, H.G., Amaro, R. (eds.) Proceedings of the 16th International Conference on Computational Processing of Portuguese. pp. 441–453. Association for Computational Linguistics, Santiago de Compostela, Galicia/Spain (Mar 2024), https://aclanthology.org/2024.propor-1.45arxiv.org/abs/2402.12969
- [28]Lorge, I., Joyce, D.W., Taylor, N., Nevado-Holgado, A., Cipriani, A., Kormilitzin, A.: Detecting the clinical features of difficult-to-treat depression using synthetic data from large language models (2024), https://arxiv.org/abs/2402.07645, doi: 10.48550/arXiv.2402.07645
- [29]Loshchilov, I., Hutter, F.: Decoupled weight decay regularization (2019), https://arxiv.org/abs/1711.05101, doi: 10.48550/arXiv.1711.05101
- [30]McHugh, M.L.: Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 22(3), 276–282 (2012), doi: 10.1177/001316446002000104
- [31]Melamud, O., Shvade, C.: Towards automatic generation of shareable synthetic clinical notes using neural language models (2019), https://arxiv.org/abs/1905.07002, doi: 10.48550/arXiv.1905.07002
- [32]Nunes, M., Boné, J., Ferreira, J.C., Chaves, P., Elvas, L.B.: Medialbertina: An european portuguese medical language model. Computers in Biology and Medicine 182, 109233 (2024), https://www.sciencedirect.com/science/article/pii/S0010482524013180, doi: 10.1016/j.compbiomed.2024.109233DOI
- [33]Oliveira, L.E.S.e., Peters, A.C., da Silva, A.M.P., Gebeluca, C.P., Gumiel, Y.B., Cintho, L.M.M., Carvalho, D.R., Al Hasan, S., Moro, C.M.C.: Semclimbr - a multi-institutional and multi-specialty semantically annotated corpus for portuguese clinical nlp tasks. Journal of Biomedical Semantics 13(1), 13 (May 2022), https://doi.org/10.1186/s13326-022-00269-1, doi: 10.1186/s13326-022-00269-1
- [34]Peng, C., Yang, X., Chen, A., Smith, K.E., PourNejatian, N., Costa, A.B., Martin, C., Flores, M.G., Zhang, Y., Magoc, T., Lipori, G., Mitchell, D.A., Ospina, N.S., Ahmed, M.M., Hogan, W.R., Shenkman, E.A., Guo, Y., Bian, J., Wu, Y.: A study of generative large language model for medical research and healthcare. npj Digital Medicine 6(1), 210 (Nov 2023), https://doi.org/10.1038/s41746-023-00958-w, doi: 10.1038/s41746-023-00958-w
- [35]Pires, R., Abonizio, H., Almeida, T.S., Nogueira, R.: Sabía: Portuguese large language models. In: Intelligent Systems. pp. 226–240. Springer Nature Switzerland, Cham (2023)DOI
- [36]Reimers, N., Gurevych, I.: Sentence-bert: Sentence embeddings using siamese bert-networks. https://fq.pkwyxx.com/default/https/aclanthology.org/D19-1410.pdf (2019)arxiv.org/abs/1908.10084
- [37]Rodrigues, J., Gomes, L., Silva, J., Branco, A., Santos, R., Cardoso, H.L., Osório, T.: Advancing neural encoding of portuguese with transformer albertina pt-*. In: Moniz, N., Vale, Z., Cascalho, J., Silva, C., Sebastião, R. (eds.) Progress in Artificial Intelligence. pp. 441–453. Springer Nature Switzerland, Cham (2023), doi: 10.1007/978-3-031-49008-8_35DOI
- [38]Schneider, E.T.R., de Souza, J.V.A., Gumiel, Y.B., Moro, C., Paraiso, E.C.: A gpt-2 language model for biomedical texts in portuguese. In: 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS). pp. 474–479 (2021), doi: 10.1109/CBMS50727.2021.00056DOI
- [39]Schneider, E.T.R., de Souza, J.V.A., Kn afou, J., e Oliveira, L.E.S., Copara, J., Gumiel, Y.B., de Oliveira, L.F.A., Paraiso, E.C., Teodoro, D., Moro, C.M.C.: Biobertpt - a portuguese neural language model for clinical named entity recognition (2020). https://doi.org/10.18653/v1/2020.clinicalnlp-1.7, https://aclanthology.org/2020.clinicalnlp-1.7/
- [40]Shi, J., Yuan, Z., Guo, W., Ma, C., Chen, J., Zhang, M.: Knowledge-graph-enabled biomedical entity linking: a survey. World Wide Web 26(5), 2593–2622 (Sep 2023), https://doi.org/10.1007/s11280-023-01144-4, doi: 10.1007/s11280-023-01144-4
- [41]Sim, J., Wright, C.C.: The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Physical Therapy 85(3), 257–268 (03 2005), https://doi.org/10.1093/ptj/85.3.257, doi: 10.1093/ptj/85.3.257
- [42]Sivarajkumar, S., Mohammad, H.A., Oniani, D., Roberts, K., Hersh, W., Liu, H., He, D., Visweswaran, S., Wang, Y.: Clinical information retrieval: A literature review. Journal of Healthcare Informatics Research 8, 313–352 (6 2024). https://doi.org/10.1007/s41666-024-00159-4, https://link.springer.com/article/10.1007/s41666-024-00159-4
- [43]Sousa, H., Almeida, R., Silvano, P., Cantante, I., Campos, R., Jorge, A.: Enhancing portuguese variety identification with cross-domain approaches. Proceedings of the AAAI Conference on Artificial Intelligence 39, 25192–25200 (4 2025), doi: 10.1609/aaai.v39i24.34705DOI
- [44]Tang, R., Han, X., Jiang, X., Hu, X.: Does synthetic data generation of llms help clinical text mining? (2023), https://arxiv.org/abs/2303.04360, doi: 10.48550/arXiv.2303.04360
- [45]Team, L.: The llama 3 herd of models (2024), https://arxiv.org/abs/2407.21783, doi: 10.48550/arXiv.2407.21783 C. Teixeira Lopes and D. Félix.
- [46]Umer, F., Adnan, N.: Generative artificial intelligence: synthetic datasets in dentistry. BDJ Open 10(1), 13 (Mar 2024), https://doi.org/10.1038/s41405-024-00198-4, doi: 10.1038/s41405-024-00198-4
- [47]Viera, A.J., Garrett, J.M., et al.: Understanding interobserver agreement: the kappa statistic. Fam med 37(5), 360–363 (2005), http://www1.cs.columbia.edu/ julia/courses/CS6998/Interrater_agreement.Kappa_statistic.pdfpubmed.ncbi.nlm.nih.gov/15883903
- [48]Waseem, H.M., Islam, S.U., Matragkas, N., Epiphaniou, G., Arvanitis, T.N., Maple, C.: Review of generative ai for synthetic data generation: a healthcare perspective. Artificial Intelligence Review 59(2), 55 (Dec 2025), https://doi.org/10.1007/s10462-025-11440-2, doi: 10.1007/s10462-025-11440-2
- [49]Xu, R., Cui, H., Yu, Y., Kan, X., Shi, W., Zhuang, Y., Wang, M.D., Jin, W., Ho, J., Yang, C.: Knowledge-infused prompting: Assessing and advancing clinical text data generation with large language models. In: Ku, L.W., Martins, A., Srikumar, V. (eds.) Findings of the Association for Computational Linguistics ACL 2024. pp. 15496–15523. Association for Computational Linguistics, Bangkok, Thailand and virtual meeting (Aug 2024), https://aclanthology.org/2024.findings-acl.916, doi: 10.18653/v1/2024.findings-acl.916DOI
- [50]Yu, Y., Zhuang, Y., Zhang, J., Meng, Y., Ratner, A., Krishna, R., Shen, J., Zhang, C.: Large language model as attributed training data generator: a tale of diversity and bias. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. NIPS ’23, Curran Associates Inc., Red Hook, NY, USA (2023)arxiv.org/abs/2306.15895
- [51]Zhang, G., Jin, Q., Zhou, Y., Wang, S., Idnay, B., Luo, Y., Park, E., Nestor, J.G., Spotnitz, M.E., Soroush, A., Campion, T.R., Lu, Z., Weng, C., Peng, Y.: Closing the gap between open source and commercial large language models for medical evidence summarization. npj Digital Medicine 7(1), 239 (Sep 2024), https://doi.org/10.1038/s41746-024-01239-w, doi: 10.1038/s41746-024-01239-w