WorksComputer Science
Learning From the Best: What Makes Popular Hugging Face Models?
Abstract
Hugging Face has become a central platform for sharing and reusing pre-trained models, yet it remains unclear which practices are associated with higher visibility and adoption. Prior studies describe model reuse and model-card quality but do not systematically compare popular and unpopular models across documentation, repository structure, platform metadata, and technical metadata. We studied 66,909 Hugging Face models collected on 2026-02-28 and extracted 31 features across four dimensions: Documentation, Models, Platforms, and Techniques. For downloads and likes, we compared the top 10% popular models with the bottom 80% using statistical tests and effect sizes, trained five classifiers to predict popularity, ranked feature importance with the best-performing model, and tested whether predictions held across model domains and contributor affiliations. Popular models consistently had longer model cards, more code examples, and more GitHub and arXiv links, while quantized models showed contrasting patterns under downloads and likes. A Random Forest reached AUC 0.861 for downloads and 0.848 for likes, with root-level file counts and model size also among the most important features. Cross-group transfer stayed mostly above random but dropped by 0.12-0.17 AUC on average, showing that prediction performance was partly group-specific. The findings remained stable after removing download outliers and under alternative popularity thresholds, and a manual review of popular model cards linked the key documentation features to concrete content. We derive recommendations for model practitioners and directions for future research on model-sharing platforms.
github.com/YinanWusomylk/What-Makes-HF-Models-Popular
Introduction
Artificial Intelligence (AI) models, especially pre-trained models (PTMs), are now reused across many tasks, including natural language processing (NLP) [8, 27], computer vision (CV) [14], speech recognition [2], and code generation [9]. This reuse is supported by model-sharing platforms, where model practitioners publish model weights, model cards, platform metadata, etc., for others to inspect and reuse. Hugging Face1 has become the largest public hub for this practice [30, 36]. On this platform, each shared model is hosted as a model repository. On our data collection date, 2026-02-28, Hugging Face hosted more than 2.6 million public model repositories across many tasks. As the number of models accumulates, visibility becomes a practical concern for practitioners:
How to make models more popular so that they are visible to more users and can have a greater impact?
Popularity matters in practice. Hugging Face provides downloads and likes as two popularity indicators. Users can sort models by either indicator when searching the Hub. Models that appear higher in these listings are easier to discover and more likely to be reused as baselines or starting points for downstream tasks [16, 36, 18, 29]. On the open-source software (OSS) side, perceived quality and community trust have all been linked to repository popularity indicators such as GitHub stars and forks [34, 11, 6]. Practitioners therefore have a clear incentive to understand which practices are associated with higher popularity on Hugging Face, but the evidence available to them is still limited.
Existing research partially fills this gap. Prior OSS studies showed that documentation quality, repository structure, activity, and owner responsiveness are associated with GitHub stars and forks [6, 5, 37, 34]. The closest prior work to ours, by Fan et al. [11], studied academic AI repositories on GitHub and found that popular repositories differed from unpopular ones in documentation, code, and reproducibility practices. However, Hugging Face model repositories differ from GitHub-hosted repositories. A Hugging Face model repository is not only a code archive or a README page; it also provides model-specific and platform-specific metadata, such as task tags, implementation libraries, weight files, and Spaces or widgets. These fields shape how models are discovered, loaded, and evaluated on the Hub. Recent Hugging Face studies have characterized model reuse, naming, packaging, and model-card completeness [16, 15, 25, 21, 22, 18]. However, they mainly provide descriptive evidence. They do not identify which individual features are associated with popularity, nor which features are most important for prediction.
In this paper, we addressed this gap with a large-scale empirical study of what makes Hugging Face models popular. We collected public Hugging Face model repositories on 2026-02-28. After removing repositories created within 180 days before data collection, repositories smaller than 300 MiB, repositories with fewer than 2 likes, and broken or restricted repositories, we retained 66,909 models for analysis. For each model, we extracted 31 features grouped into four dimensions: Documentation, Models, Platforms, and Techniques. We measured popularity using two indicators provided by Hugging Face: 30-day downloads and cumulative likes, and analyzed each indicator independently.
We organize the study around three research questions.
RQ1: Which features are associated with model popularity? We compared the top 10% (popular) and bottom 80% (unpopular) models on each indicator, with a 10% gap buffer, using Mann–Whitney U tests for continuous features and chi-square tests for binary features, with FDR correction. We reported effect sizes using Cliff’s for continuous features and Cramér’s V for binary features. Documentation features accounted for most features with at least small effect: popular models had longer model cards, more code examples, and more GitHub and arXiv links.
RQ2: Which features best predicted model popularity? We filtered redundant features with Spearman correlation, ranked the remaining features by mutual information, trained five classifiers with grid search and repeated stratified cross-validation, and reported permutation importance of the best classifier on a held-out test set. Random Forest performed best, reaching AUC 0.861 for downloads and 0.848 for likes. The most important features were mainly related to documentation depth, repository structure, model size, and quantization.
RQ3: Does popularity prediction remain reliable across model domains and contributor affiliations? We mapped models to task domains using models’ metadata and to contributor affiliations using author profile pages. We then evaluated within-group, one-versus-one, and others-versus-one transfer with bootstrap confidence intervals. Within-group AUCs remained above 0.7 across all evaluated domains and contributor affiliations, while cross-group prediction produced lower AUCs, especially across contributor affiliations. Training on all other groups together produced higher AUCs than training on a single source group.
The main contributions of this paper are as follows.
Large-scale evidence on Hugging Face popularity. We studied 66,909 Hugging Face models and analyzed downloads and likes separately, showing which documentation, repository, platform, and technical features distinguish the popular group from the unpopular group.
Predictive and cross-group analysis. We trained and interpreted classifiers for popularity prediction under both downloads and likes, and tested whether prediction performance remained reliable across model domains and contributor affiliations. We also evaluated the stability of the results under outlier removal and alternative popularity thresholds.
Practical recommendations and replication package. We translated the findings into practical recommendations for model practitioners and provided a public replication package2 for future studies of model-sharing platforms.
The rest of the paper is structured as follows. § 2 reviews related work. § 3 describes data preparation, including collection, filtering, and labelling. § 4 defines the 31 features and the extraction pipeline. § 5 to § 7 report the research questions and the results. § 8 presents the robustness analyses, the qualitative review of sampled model cards, and practical recommendations. § 9 discusses threats to validity. § 10 concludes.
Background and Related Work
Popularity in Open-Source Ecosystems
Understanding why OSS projects receive more attention than others has been a recurring topic in Mining Software Repositories (MSR) and empirical software engineering. Prior work commonly studied GitHub engagement measures, such as stars, forks, watchers, and contributor counts [6, 5, 37]. These studies connected repository popularity with project features. For example, repositories with conventional folders such as docs, tests, and examples tend to attract more forks [37], while documentation quality, maintenance activity, and owner responsiveness are associated with star growth [6]. The early growth of GitHub stars can also help predict later popularity [5].
Documentation has received particular attention because it is often the first file that potential users inspect. Aggarwal et al. [1] found that source code and README content co-evolve in mature projects. Prana et al. [26] characterized common README sections, such as purpose, usage, contribution guidelines, and license information, and reported that more complete READMEs are associated with stronger community engagement. Wang et al. [34] further studied README content features, including length, code blocks, images, and external links, and related them to GitHub repository popularity. Together, these studies suggest that repository structure and documentation provide useful information about how a project is perceived and adopted.
However, most existing popularity studies were designed for GitHub-hosted code repositories. Hugging Face model repositories provide different types of information from GitHub projects. In addition to model-card content, they include model-specific and platform-specific fields such as task labels, implementation libraries, model-file formats, and demo features.3 These fields may shape how models are discovered and reused, but they have not been studied together with documentation and repository structure in a popularity analysis of model repositories.
Sharing PTMs on Hugging Face
Hugging Face has become a major platform for sharing PTMs across NLP, CV, audio, and multimodal tasks [8, 27, 9, 30, 36, 29]. Recent empirical studies have started to examine this ecosystem. Jiang et al. studied how Hugging Face models are named and versioned [15], how users discover and reuse models across downstream tasks [16], and where PTM packaging becomes fragile in practice [35]. The PeaTMOSS dataset [17] further linked PTM repositories with downstream GitHub projects, supporting reproducible studies of model reuse at scale. Wang et al. [33] studied licensing risks in LLMware supply chains that involve OSS projects, Hugging Face models, and datasets. This line of work shows that Hugging Face model repositories are part of a broader reuse ecosystem, not just storage locations for model weights.
Researchers have also examined the documentation practices around model cards. Mitchell et al. [24] introduced model cards as a structured way to describe a model’s intended use, performance, and limitations. This format later became a common convention on Hugging Face. Pepe et al. [25] and Liang et al. [21, 22] studied how model cards are completed in practice and found large variation in sections such as intended use, evaluation, and limitations. Jones et al. [18] proposed a 0 to 5 documentation-quality scale for Hugging Face model cards and found that the top 1,000 most-downloaded models scored higher than the bottom 1,000.
These studies provide the closest Hugging Face context for our work because they characterize model repositories and their documentation and describe reuse, packaging, licensing, and model-card quality. Our study complements this line of work by examining which repository and model-card features are associated with popularity.
Feature-Based Approaches to Predicting Project Popularity
Feature-based analysis has been widely used in MSR to connect project features with project outcomes. Bao et al. [3], for example, used developer activity and working-hour features to predict developer attrition with several classifiers. For popularity, Wang et al. [34] extracted README content features from GitHub repositories and found that length, code blocks, and images were informative. The closest prior work to ours is Fan et al. [11], who studied academic AI repositories on GitHub. They extracted 21 features from documentation, code, and reproducibility dimensions, compared popular and unpopular repositories, and trained random forest classifiers to identify the features that best distinguish the two groups. Their results highlighted actionable practices such as providing richer README content, linking related repositories, including images, and adding a license.
Our study builds on this feature-based research paradigm, but shifts the setting from GitHub-hosted repositories to Hugging Face model repositories. Instead of GitHub stars, we used downloads and likes as popularity indicators. We extracted 31 features across four dimensions: Documentation, Models, Platforms, and Techniques. Several of these features are specific to model-sharing platforms, including platform metadata, task and library information, model-file formats, and file-tree structures. We also extended the analysis beyond a single classification task by testing feature-level differences (RQ1), ranking predictive features with model-based importance (RQ2), and evaluating cross-domain and cross-affiliation performance (RQ3). To the best of our knowledge, this is the first feature-based study that connected Hugging Face-specific features to model popularity and examined whether popularity prediction remains reliable across model domains and contributor affiliations.
Data Preparation
In this section, we describe how we collected the Hugging Face model repositories, filtered them for analysis, and defined the downloads- and likes-based popularity groups.
Data Collection
We collected the data on 2026-02-28. The initial crawl returned 2,669,509 public model repositories from the Hugging Face Hub. We collected the data in three steps.
First, we used the /api/models endpoint to enumerate public model repositories. The crawler started from /api/models?limit=1000, followed the pagination URL in response.links["next"], and stored the metadata returned for each model, including basic fields such as the model identifier, creation time, downloads, and likes. Second, we queried two repository-level endpoints for each model: /api/models/{model_id} for per-model details, and /api/models/{model_id}/tree/main for the repository file tree and per-file byte sizes. Third, we downloaded the raw README.md from each model repository using the /{model_id}/raw/main/README.md path.
Filtering
After removing 9 duplicate model_id entries, we obtained 2,669,500 unique model repositories. We then applied four filters to construct the final dataset. First, we removed repositories created within 180 days before data collection, i.e., on or after 2025-09-01. Recently created models may not have had enough time to accumulate stable downloads or likes, which would have introduced time-on-platform noise into the popularity labels. Second, we observed that repositories without model files were often small in size, thus we filtered out those repositories that are smaller than 300 MiB. Third, we removed repositories with fewer than 2 likes. Repositories with 0 or 1 likes provided little ranking information, and the heavy tie at the bottom of the distribution would have made the unpopular group less meaningful. Finally, we removed repositories whose Hub API responses indicated a broken or gated/private repository (if_repository or if_restricted ). This final filter did not remove additional models in our collected data. After filtering, the final dataset contained 66,909 models. Table 1 shows the number of models remaining after each step.
| Step | Models remaining |
| Raw crawl (2026-02-28, public repositories) | 2,669,509 |
After deduplicating model_id | 2,669,500 |
created_time < 2025-09-01 (180-day recency) | 1,935,844 |
model-size MiB | 967,002 |
likes | 66,909 |
| Valid, non-restricted repository | 66,909 |
| Final dataset | 66,909 |
Table 1. Filtering steps from the 2026-02-28 Hugging Face crawl to the final dataset. Each row reports the number of models remaining after the listed filter is applied; filters are applied cumulatively in the order shown.
Data Labelling
We labelled each model repository as popular or unpopular using two popularity indicators reported by the Hugging Face Hub: the cumulative number of likes and the monthly number of downloads. Likes reflect explicit user appreciation, while downloads also capture automated retrievals from downstream code paths. The two indicators were related but not interchangeable. On the final 66,909-model dataset, their Spearman rank correlation was . Under the interpretation used by Wang et al. [34] ( weak; moderate; strong), this indicates a moderate association. We therefore treated downloads and likes as two separate popularity indicators in the labeling step.
For each indicator, we sorted the 66,909 models in descending order and applied a 10-10-80 scheme: the top 10% (6,690 models) were labeled popular, the bottom 80% (53,528 models) were labeled unpopular, and the 10% in between (6,691 models) was used as a gap buffer. The gap buffer made the difference between the popular and unpopular groups more substantial, following the same rationale used in prior study [11]. Figure 1 shows the resulting partition for downloads and likes.

Figure 1. Distribution of the two popularity indicators on the 66,909-model final dataset under the 10-10-80 scheme. The horizontal axis is log-scaled.
Feature Extraction
We extracted 31 features for each model repository and grouped them into four dimensions: Documentation, Models, Platforms, and Techniques. Table 2 summarizes the feature definitions, and the following subsections describe how the features in each dimension were extracted.
| Dimension | Feature | Description |
| Documentation | length-yaml | The total number of words in the YAML part of the README. |
| Documentation | length-doc | The total number of words in the Markdown part of the README. |
| Documentation | has-video | Whether the documentation includes URLs to videos. |
| Documentation | num-inline-code | The number of inline code snippets in the documentation. |
| Documentation | num-code-block | The number of fenced code blocks in the documentation. |
| Documentation | num-static-img | The number of static images (e.g., .jpg, .png). |
| Documentation | num-animation | The number of animated images (e.g., .gif). |
| Documentation | num-lists | The number of lists used in the documentation. |
| Documentation | num-tables | The number of tables included in the documentation. |
| Documentation | num-gh-links | The number of GitHub links in the documentation. |
| Documentation | num-hf-links | The number of Hugging Face links in the documentation. |
| Documentation | num-arxiv-links | The number of arXiv identifiers declared in the metadata. |
| Documentation | has-bibtex | Whether the README contains a BibTeX entry. |
| Documentation | has-license | Whether license information is declared in the metadata. |
| Models | has-config | Whether the model repository contains a configuration file. |
| Models | has-results | Whether the metadata reports performance results (model-index). |
| Models | has-dataset | Whether the metadata specifies any dataset information. |
| Models | num-dataset | The number of datasets linked to the model. |
| Models | match-hf-dataset | Whether the linked datasets are hosted on Hugging Face. |
| Models | model-size | The total size of the model files in the repository. |
| Models | num-root-files | The number of files in the root directory. |
| Models | num-modules | The number of modules (subdirectories) in the repository. |
| Models | num-model-files | The number of model-weight files (.bin, .safetensors, etc.). |
| Models | has-quantized | Whether the model is a quantized variant. |
| Platforms | has-space | Whether the model is featured in at least one Hugging Face Space. |
| Platforms | has-safetensors | Whether the model uses SafeTensors to store model weights. |
| Platforms | has-widgets | Whether the model’s inference API contains widget data. |
| Techniques | has-pipeline | Whether the metadata specifies a named pipeline (pipeline_tag). |
| Techniques | has-impl-lib | Whether the metadata specifies a primary implementation library. |
| Techniques | has-supported-lib | Whether the model uses a library officially supported by Hugging Face. |
| Techniques | has-additional-fw | Whether the model supports additional frameworks beyond the primary one. |
Table 2. The 31 features analyzed in our study, organised into four dimensions.
Documentation Dimension
Documentation is the main way in which model practitioners explain what a model does, how it should be used, and how it should be cited. Prior studies [34, 11, 26, 1] have similarly treated README content as an important indicator of project quality and adoption. On Hugging Face, each model card is stored in the repository as README.md and rendered on the model page. It usually contains a Markdown body and, when available, a YAML metadata header.
Following Fan et al. [11], we parsed the README into HTML before extracting documentation features. We used Mistune4 as the main Markdown parser and Pandoc5 for table detection. We first separated the YAML metadata block from the Markdown body and counted words in both parts. These counts gave length-yaml and length-doc. We then counted structural elements in the rendered body: num-lists was the number of <ol> and <ul> elements, and num-tables was the number of <table> elements in the Pandoc-rendered HTML.
Runnable examples are especially important for model repositories, where unclear usage instructions are a common source of user issues [35]. We counted code by inspecting each <code> element. If its parent was <pre>, we counted it as a fenced code block (num-code-block); otherwise, we counted it as inline code (num-inline-code). We also counted images and videos. Static images were counted when an <img> source used an image extension such as .png, .jpg, or .svg; animated images were counted when the source ended in .gif. The binary feature has-video was set to 1 when a link or iframe pointed to a video source such as YouTube or a video file.
Finally, we captured links and citation-related metadata, because links to code, papers, and citation entries help users trace a model’s provenance and additional materials. We counted GitHub and Hugging Face links by inspecting the href attributes of <a> and <link> elements. We counted arXiv references from the metadata returned by /api/models rather than from README text, because Hugging Face returns them as structured arxiv:<paper-id> tags. The feature num-arxiv-links was therefore the number of such arXiv tag entries. We set has-bibtex to 1 when the README contained a BibTeX code block, either through a language-bibtex class or through a code block containing the standard fields author=, title=, and year=. We set has-license to 1 when the metadata included a license:<spdx-identifier> tag.
Models Dimension
The Models dimension captures properties of the model files and repository layout. These features are specific to model-sharing platforms and are mostly absent from prior GitHub-based popularity studies [18, 11]. We extracted them from two sources: metadata, the per-model details and repository file-tree endpoints.
We used the per-model details endpoint to describe repository structure, because the repository structure may affect how easily practitioners can inspect, load, and reuse a model. The endpoint returned a siblings list that enumerated repository files. We counted the files at the repository root and used this count as num-root-files. For files in subdirectories, we collected the first path component and reported the number of unique components as num-modules. We also counted model-weight files with common weight-file extensions, such as .bin, .safetensors, and .onnx. The feature has-config was set to 1 when the per-model details JSON contained a non-empty config field. We computed model-size by summing the per-file byte sizes returned by the repository tree endpoint, which avoided downloading the model weights.
We captured dataset and result reporting because these fields may help practitioners identify whether a model is relevant and trustworthy. The feature has-dataset was set to 1 if the structured metadata contained at least one dataset:<name> entry, and num-dataset counted those entries. We set match-hf-dataset to 1 if at least one declared dataset appeared in the set of Hugging Face dataset identifiers collected in § 3. We set has-results to 1 when the per-model details contained a non-empty model-index block with at least one entry containing a non-empty results field.
Quantized models are often smaller or faster than full-precision checkpoints [13], which may make them easier to deploy and share. We therefore extracted whether a model was quantized. Hugging Face did not expose a single canonical quantization flag, so we combined model identifiers and tag information. We searched for common quantization terms from the Hugging Face quantization documentation,6 such as gguf, gptq, and 4bit. We also accepted the exact tag quantized and the substring quantized in model_id, while excluding negated forms such as unquantized and non-quantized. A model was assigned has-quantized = 1 when any of these checks matched.
Platforms Dimension
The Platforms dimension captures features that make a model easier to inspect or use from the Hugging Face interface. We included three binary features. The feature has-safetensors was set to 1 when the structured metadata contained a safetensors-prefixed entry. SafeTensors is a model-weight format designed to avoid the code-loading risks associated with pickle-based weight files. The feature has-space was set to 1 when the metadata indicated that the model was connected to a Hugging Face Space. Spaces let users try a model in the browser and may improve discoverability [29]. The feature has-widgets was set to 1 when the model details contained a non-empty widgetData field, which enables example inputs on the model card.
Techniques Dimension
The Techniques dimension describes how a model is implemented and made available to users. All four features were extracted from structured metadata. We set has-pipeline to 1 when the model entry contained a non-empty pipeline_tag. A pipeline tag links the model to a Hugging Face task abstraction, which helps users find the model by task and load it through standard interfaces. We set has-impl-lib to 1 when the entry contained a non-empty library_name, such as transformers, diffusers, or sentence-transformers. We set has-additional-fw to 1 when the metadata listed an additional framework or library beyond the primary library_name, such as pytorch, tensorflow, or onnx.
Manual Sanity Check
To check the correctness of the extraction pipeline, we manually verified a stratified random sample of 30 models. One author compared and verified the Hugging Face model page with the extracted values for all 31 features. We excluded downloads and likes from this check because they are live Hub counters and can change after data collection.
RQ1: Features Associated with Model Popularity
Motivation
Following prior studies [11, 34], we first examined how individual features differed between popular and unpopular models. RQ1 compared the 31 extracted features under downloads and likes separately, allowing us to see whether the two popularity indicators pointed to the same feature dimensions.
Method
We compared the popular and unpopular groups defined under the main 10-10-80 scheme (§ 3.3), excluding the gap buffer. For each of the 16 continuous features, we used the Mann–Whitney U test [23] and reported the signed Cliff’s ; positive values indicate that the feature tended to be larger in the popular group. For each of the 15 binary features, we used the Chi-square test and reported Cramér’s . Since Cramér’s is non-directional, we indicated direction by comparing feature prevalence between the two groups. We applied the Benjamini–Hochberg false discovery rate procedure [4] at to the 31 tests under each indicator. We interpreted effect-size magnitudes using standard thresholds: for Cliff’s , negligible (), small ( to ), medium ( to ), and large () [28]; for Cramér’s , negligible (), small ( to ), medium ( to ), and large () [10]. The full analysis was run twice, once using downloads-based labels and once using likes-based labels.
Results
After FDR correction at , 28 of 31 features were statistically significant under the downloads indicator, and 29 of 31 features were statistically significant under the likes indicator. However, most statistically significant differences were negligible in magnitude. Under the thresholds above, 13 of 31 features reached at least small effect on downloads, and 8 of 31 features reached at least small effect on likes. Table 3 reports the union of the 15 features that reached at least small effect on at least one indicator.
Table 3. Features that reach at least small effect on at least one popularity indicator. All rows shown are FDR-significant. Continuous features report signed Cliff’s ; positive values mean larger values in the popular group. Binary features report Cramér’s , with / indicating whether the feature is more / less prevalent in the popular group. Bold = small or larger.
| Dimension | Feature | downloads | likes |
| Documentation | length-yaml | +0.152 | +0.049 (negligible) |
| Documentation | length-doc | +0.296 | +0.292 |
| Documentation | num-code-block | +0.282 | +0.252 |
| Documentation | num-inline-code | +0.170 | +0.180 |
| Documentation | num-lists | +0.124 (negligible) | +0.190 |
| Documentation | num-tables | +0.180 | +0.117 (negligible) |
| Documentation | num-gh-links | +0.264 | +0.227 |
| Documentation | num-arxiv-links | +0.268 | +0.249 |
| Documentation | has-bibtex | 0.183 | 0.152 |
| Models | has-config | 0.116 | 0.065 (negligible) |
| Models | has-quantized | 0.032 (negligible) | 0.113 |
| Platforms | has-safetensors | 0.107 | 0.069 (negligible) |
| Techniques | has-pipeline | 0.128 | 0.098 (negligible) |
| Techniques | has-impl-lib | 0.107 | 0.049 (negligible) |
| Techniques | has-additional-fw | 0.121 | 0.030 (negligible) |
Documentation accounted for most small-effect differences. Documentation contributed the largest share of small-effect features on both indicators: 8 of the 14 Documentation features reached small effect on downloads and 7 of 14 on likes. Six documentation features did so under both indicators: length-doc, num-code-block, num-inline-code, num-gh-links, num-arxiv-links, and has-bibtex. Popular model cards were longer, contained more code blocks and inline code, included more GitHub and arXiv links, and more often included a BibTeX block. The largest single difference was length-doc: popular cards had a median of 451 words versus 234 under downloads, and 483 versus 240 under likes. These differences are consistent with how practitioners inspect model cards: longer cards, runnable examples, links to code or papers, and citation blocks help users understand what a model does, try it, trace its source, and cite it. A few documentation features were indicator-specific: length-yaml and num-tables reached small effect only under downloads, while num-lists reached small effect only under likes.
Technique features reached small effects mainly under downloads. Under downloads, three of the four Technique features reached small effect: has-pipeline, has-impl-lib, and has-additional-fw. Under likes, all four Technique features remained negligible. This result is consistent with the role of technical metadata on the Hub: pipeline and library declarations can make a model easier for downstream applications and scripts to find and load, while they are less directly tied to explicit user appreciation.
Model and platform features showed fewer small-effect differences. In the Models dimension, only has-config reached small effect under downloads. This is consistent with configuration files making a repository easier for Hugging Face tooling and downstream scripts to load. Most other model-structure features remained negligible. The notable exception was has-quantized, which was more prevalent in the unpopular group under likes: 14.7% of popular models were quantized, compared with 31.1% of unpopular models (). This does not imply that quantization reduces popularity; rather, it suggests that quantized releases may follow a different popularity pattern, especially under likes. In the Platforms dimension, has-safetensors reached small effect under downloads but remained negligible under likes, which is consistent with SafeTensors being primarily a loading and model-file format feature.
Summary of RQ1. Popular and unpopular models differed on many features after FDR correction, but the non-negligible differences were concentrated in a smaller set: 13 features under downloads and 8 under likes reached at least small effect. Documentation was the most consistent dimension across both indicators. Technique metadata appeared mainly in the downloads comparison, while quantization was the only small-effect feature that was more prevalent in the unpopular group, under likes.
RQ2: Feature Importance for Popularity Prediction
Motivation
RQ1 compared popular and unpopular models one feature at a time. RQ2 examined the features jointly: we trained classifiers to distinguish popular from unpopular models and used permutation importance to identify which features contributed most to the best classifier.
Method
We ran the same four-stage procedure separately for downloads and likes, using the popular and unpopular groups defined by the 10-10-80 scheme.
Stage 1 – Correlation filtering. We computed pairwise Spearman correlation across the 31 features and, for every pair with [12, 32], removed the feature with the lower absolute correlation to the popular/unpopular outcome. This step removed five features under each indicator: has-dataset, num-dataset, num-model-files, and has-supported-lib. It also removed one member of the framework/library pair: has-impl-lib on downloads and has-additional-fw on likes. The remaining 26 features were used in the next stage.
Stage 2 – Mutual-information ranking and -selection. Using the 26 retained features, we computed mutual information [20] with the binary popular/unpopular outcome and ranked features in descending order. We then swept from 5 to 26, trained a Random Forest [7] on the top- features for each , and selected the that maximised mean 5-fold cross-validation ROC AUC. The optimal was 26 under both indicators, with 5-fold CV AUC of 0.854 on downloads and 0.841 on likes.
Stage 3 – Classifier comparison. We compared five classifiers: Random Forest, Decision Tree, Linear SVM, Gaussian Naive Bayes, and -Nearest Neighbours. Hyperparameters were tuned with GridSearchCV. Each model was evaluated with 10-fold cross-validation repeated 10 times (100 folds total). ROC AUC was the primary metric because it evaluates ranking quality and is less sensitive to the decision threshold under class imbalance [5, 3]. Accuracy, precision, recall, and F1 were reported as secondary metrics. Following prior popularity-prediction work, we treated AUC > 0.7 as the acceptability threshold [11]. Since the popular and unpopular groups had an approximately 1:8 class ratio, we used balanced class weights for classifiers that supported them.
Stage 4 – Permutation importance. For the best classifier, we computed permutation importance on a stratified 80/20 holdout test set with 30 shuffles per feature and ROC AUC as the scoring metric. The reported AUC is the mean drop in test AUC when a feature’s values are shuffled. This complements RQ1 because it estimates the importance of each feature conditional on the other features in the model.
Results
Random Forest performed best under both popularity indicators. Table 4 reports the 100-fold scores. Random Forest achieved the highest AUC on both downloads (0.861 0.007) and likes (0.848 0.006). KNN ranked second, Decision Tree third, Linear SVM fourth, and Gaussian Naive Bayes had the lowest AUC under both indicators. All classifiers except Gaussian Naive Bayes exceeded 0.7 AUC, a conventional lower bound for acceptable discrimination used in prior popularity-prediction work [11]. This result is consistent with the use of Random Forest in prior study [11]: the model can combine numeric and binary features, capture non-linear feature combinations, and reduce the instability of a single decision tree through ensembling [7].
| Classifier | AUC | Acc. | Prec. | Rec. | F1 |
| by downloads | |||||
| Always unpopular | 0.500 | 0.889 | 0.000 | 0.000 | 0.000 |
| Random Forest | 0.861 | 0.906 | 0.608 | 0.437 | 0.509 |
| k-NN | 0.807 | 0.905 | 0.618 | 0.384 | 0.473 |
| Decision Tree | 0.780 | 0.738 | 0.256 | 0.714 | 0.377 |
| Linear SVM | 0.722 | 0.659 | 0.194 | 0.658 | 0.300 |
| Gaussian Naive Bayes | 0.684 | 0.152 | 0.114 | 0.981 | 0.205 |
| by likes | |||||
| Always unpopular | 0.500 | 0.889 | 0.000 | 0.000 | 0.000 |
| Random Forest | 0.848 | 0.902 | 0.676 | 0.232 | 0.345 |
| k-NN | 0.782 | 0.898 | 0.590 | 0.258 | 0.359 |
| Decision Tree | 0.758 | 0.712 | 0.234 | 0.700 | 0.350 |
| Linear SVM | 0.724 | 0.640 | 0.190 | 0.685 | 0.297 |
| Gaussian Naive Bayes | 0.684 | 0.395 | 0.141 | 0.864 | 0.242 |
Table 4. Classifier performance (mean std across 100 folds = 10-fold 10 repeats). Best model under each indicator is in bold. “Always unpopular” is a majority-class baseline that predicts every model as unpopular; its precision, recall, and F1 for the popular class are reported as 0 because no popular instances are predicted.
The class imbalance made accuracy alone misleading. An always-unpopular classifier would obtain AUC = 0.5 and F1 = 0, but still reach 0.889 accuracy because unpopular models make up most of the evaluation set. Random Forest improved only modestly over this baseline in accuracy, but it achieved much higher AUC and non-zero recall for the popular class. On downloads, Random Forest reached precision = 0.608, recall = 0.437, and F1 = 0.508; on likes, it showed higher precision and lower recall (precision = 0.676, recall = 0.232, F1 = 0.345). Decision Tree and Linear SVM showed the opposite pattern, with higher recall but lower precision.
Permutation importance identified eight features that were stable across indicators. Table 5 lists the top-10 permutation importances on the best Random Forest under each indicator. Eight features appeared in both top-10s: length-doc, length-yaml, num-code-block, num-gh-links, num-arxiv-links, model-size, num-root-files, and has-quantized. These eight common features consisted of five Documentation features and three Models features. length-doc was the strongest feature under both indicators, with AUC = 0.035 on downloads and 0.067 on likes. This result reinforces the RQ1 pattern that model-card content remains central even after metadata, file structure, and model-format features are considered together. At the same time, RQ2 showed a result that RQ1 alone would miss: model-size had a negligible feature-level effect in RQ1, but ranked 2nd on downloads and 3rd on likes in permutation importance. This suggests that model size becomes informative when interpreted jointly with documentation and repository-structure features, rather than as a standalone difference between the two groups. Because these rankings come from the best-performing Random Forest, they should be read as model-specific importance estimates rather than model-independent causal effects.
| Rank | Feature (downloads) | AUC | Feature (likes) | AUC |
| 1 | length-doc | 0.035 | length-doc | 0.067 |
| 2 | model-size | 0.027 | has-quantized | 0.058 |
| 3 | num-gh-links | 0.024 | model-size | 0.031 |
| 4 | has-additional-fw | 0.024 | num-arxiv-links | 0.025 |
| 5 | length-yaml | 0.022 | length-yaml | 0.020 |
| 6 | num-arxiv-links | 0.019 | num-hf-links | 0.019 |
| 7 | num-code-block | 0.013 | num-code-block | 0.016 |
| 8 | num-root-files | 0.013 | num-gh-links | 0.016 |
| 9 | has-quantized | 0.011 | has-license | 0.015 |
| 10 | has-config | 0.011 | num-root-files | 0.012 |
Table 5. Top-10 permutation importance (AUC, mean std over 30 shuffles) on the best Random Forest under each popularity indicator. Bold = appears in the top-10 on both indicators.
The stronger predictive performance in RQ2 therefore does not contradict the small effect sizes in RQ1. RQ1 measured each feature separately, so a small effect means that the popular and unpopular groups still overlap substantially on that single feature. RQ2 instead evaluated combinations of features after removing highly correlated pairs. The 26 features covered different aspects of a model repository, including model-card detail, external references, file-tree structure, model size, and model format. Small differences across these aspects can accumulate, and Random Forest can capture threshold-based and non-linear combinations that are not visible in one-feature-at-a-time comparisons.
The two indicators shared a common core but differed on secondary features. The largest cross-indicator difference was has-quantized, which ranked 9th on downloads (AUC = 0.011) but 2nd on likes (AUC = 0.058). This result is consistent with RQ1: quantized models were more common in the unpopular group under likes, and the Random Forest used this feature to separate unpopular models from popular ones. One interpretation is that quantized releases often function as deployment-oriented versions of already-known base models. They can receive downloads through reuse, but they do not necessarily receive likes as independent model cards. Two other indicator-specific features also reflect the different meanings of downloads and likes: has-additional-fw ranked 4th on downloads and was removed in Stage 1 for the likes indicator in favour of has-impl-lib, while has-license appeared only in the likes top-10. In contrast, the common top features were mostly about documentation depth, repository structure, and model size, suggesting that both indicators were associated with models that are easier to inspect and reuse.
Summary of RQ2. A Random Forest trained on the 26 features retained after Spearman filtering and MI selection reached AUC = 0.861 on downloads and 0.848 on likes, outperforming both the 0.7 reference threshold and the always-unpopular baseline. The same eight features appeared in the permutation-importance top-10 under both indicators: five Documentation features and three Models features. length-doc was the strongest feature under both indicators, while has-quantized was much more important under likes than under downloads. Compared with RQ1, RQ2 also showed that some features with negligible feature-level effects, especially model-size, became important when considered jointly with the other features.
RQ3: Cross-Group Generalization of Popularity Prediction
Motivation
RQ2 evaluated prediction with all models in the main analysis pooled together, but model repositories may follow different conventions across domains and contributor affiliations. RQ3 therefore examined whether the best RQ2 classifier remained reliable across these groups. This analysis helps assess whether prediction performance is broadly transferable, or whether it depends on domain- or affiliation-specific release practices.
Method
We added two grouping variables to each model and reused the RQ2 Random Forest with the same 26 features and tuned parameters. Keeping the model setup fixed allowed us to focus on how group differences affected prediction performance.
Grouping by domain. We mapped the pipeline_tag field to six task families aligned with the Hugging Face task taxonomy:7 NLP, CV, Multimodal, Audio, Tabular, and Reinforcement Learning. Models with a missing or unmapped pipeline_tag were assigned to Other. In the current data, Other mostly consists of models without a pipeline_tag; we therefore interpret it as a mixed missing-tag group rather than as a single task domain.
Grouping by affiliation. We parsed each unique contributor’s Hugging Face profile page for the <span class="capitalize"> badge. We retained the badge categories used by the platform: company, university, non-profit, community, classroom, and government. Profiles that loaded but did not display one of these badges were assigned to organization or individual, the platform’s default category for accounts without an organization badge. Profiles that could not be fetched were assigned to unknown. These labels were used only for grouping; they were not included as model features.
Transfer experiments. For each grouping variable and popularity indicator, we ran three experiments. We included a group only if it contained at least 30 popular and 30 unpopular models. The within-group baseline trained and tested the Random Forest inside the same group using 10-fold cross-validation. One-versus-one transfer treated each group as a test target in turn: for a target group, we trained separate models on each other valid group and reported the mean AUC on the target group. Others-versus-one also treated each group as a target, but trained one model on the pooled data from all other valid groups before testing on the target group. For one-versus-one and others-versus-one transfer, we estimated test-set AUC with 1,000 bootstrap resamples [31] and reported the mean and 95% confidence interval. The within-group baseline measured prediction performance inside a group. The gap between within-group and one-versus-one AUC indicated how much performance was lost when training data came from another group, and the difference between one-versus-one and others-versus-one measured whether pooling multiple source groups improved prediction.
Results
After applying the minimum-sample filter, we retained five domain groups: NLP, CV, Multimodal, Audio, and Other. We also retained five affiliation groups: organization or individual, company, community, university, and non-profit. The classroom, government, and unknown groups were excluded because their popular groups were too small. We report the per-group results in Table 6 and Table 7.
| Domain | Within | 1-vs-1 | Others-vs-1 |
| by downloads | |||
| Audio (2,187) | 0.814 | 0.684 | 0.690 |
| CV (7,853) | 0.891 | 0.753 | 0.776 |
| Multimodal (1,913) | 0.844 | 0.712 | 0.761 |
| NLP (27,809) | 0.861 | 0.664 | 0.713 |
| Other (20,276) | 0.811 | 0.662 | 0.720 |
| by likes | |||
| Audio (2,164) | 0.824 | 0.735 | 0.781 |
| CV (7,521) | 0.852 | 0.716 | 0.756 |
| Multimodal (1,942) | 0.832 | 0.713 | 0.769 |
| NLP (28,302) | 0.848 | 0.708 | 0.754 |
| Other (20,103) | 0.816 | 0.686 | 0.748 |
Table 6. Cross-domain transfer AUC, per domain, on the two popularity indicators. “Within” trains and tests inside the row domain with 10-fold cross-validation. “1-vs-1” treats the row domain as the test target and reports the mean AUC of models trained separately on each other valid domain. “Others-vs-1” trains one model on all other valid domains pooled together and tests it on the row domain. Sample sizes are shown in parentheses.
| Affiliation | Within | 1-vs-1 | Others-vs-1 |
| by downloads | |||
| community (5,353) | 0.868 | 0.681 | 0.728 |
| company (9,700) | 0.881 | 0.692 | 0.708 |
| non-profit (2,374) | 0.817 | 0.674 | 0.708 |
| organization or individual (40,168) | 0.831 | 0.674 | 0.705 |
| university (2,403) | 0.839 | 0.664 | 0.701 |
| by likes | |||
| community (5,298) | 0.865 | 0.677 | 0.716 |
| company (9,510) | 0.867 | 0.679 | 0.707 |
| non-profit (2,317) | 0.839 | 0.697 | 0.722 |
| organization or individual (40,472) | 0.825 | 0.665 | 0.717 |
| university (2,402) | 0.765 | 0.635 | 0.645 |
Table 7. Cross-affiliation transfer AUC, per affiliation, on the two popularity indicators. Columns are defined as in Table 6: the row affiliation is always the evaluation target for “1-vs-1” and “Others-vs-1”. Sample sizes are shown in parentheses.
Cross-domain transfer. Table 6 reports the three AUC scores for each valid domain. The within-group AUC ranged from 0.811 (Other, downloads) to 0.891 (Computer Vision, downloads), and every within-group result exceeded the 0.7 reference threshold used in RQ2. One-versus-one transfer was lower than the within-group baseline for every domain. The average drop was 0.149 on downloads and 0.123 on likes. This shows that cross-domain prediction was harder than prediction within the same domain, even though AUC remained above random. The largest drop appeared for NLP under downloads, where AUC decreased from 0.861 within-group to 0.664 in one-versus-one transfer. Pooling all other domains improved over the average one-versus-one result for every domain, by 0.037 AUC on downloads and 0.050 on likes. This result suggests that cross-domain prediction benefits from seeing several source domains rather than only one.
Cross-affiliation transfer. Table 7 reports the same three AUCs for the five valid affiliation groups. Within-group prediction was again strong. The highest within-group AUCs appeared for company models (0.881 on downloads and 0.867 on likes). One possible explanation is that company releases provide the repository-level cues used by our classifier more consistently, such as structured metadata, licensing, and complete model cards. The lowest within-group AUC appeared for university models under likes (0.765), suggesting that likes for academic releases may depend more on factors outside our repository-level features, such as paper visibility or author reputation.
One-versus-one transfer dropped more under affiliation grouping than under domain grouping: the average drop was 0.171 on downloads and 0.162 on likes, compared with 0.149 and 0.123 for domains.
This difference suggests that contributor affiliation separated models by release practice more strongly than task domain did. Domains such as NLP, vision, audio, and multimodal describe what a model does, while affiliations also reflect who releases the model and under what institutional context. As a result, affiliation groups may differ more in metadata completeness, documentation style, and licensing practice, which are the types of features used by the classifier. The largest drop appeared for community under likes, where AUC decreased from 0.865 within-group to 0.677 in one-versus-one transfer. Pooling all other affiliations again improved over one-versus-one for every group, by an average of 0.033 AUC on downloads and 0.031 on likes.
Summary of RQ3. The RQ2 Random Forest retained strong within-group performance across the five valid domains and five valid affiliations, with AUCs from 0.765 to 0.891. One-versus-one prediction was consistently lower than within-group prediction, dropping by 0.12–0.17 AUC on average, showing that performance was partly group-specific. The drop was larger across contributor affiliations than across domains. Training on all other groups together improved AUC by 0.031–0.050 over average one-versus-one prediction for every target group.
Discussion
This section examines the stability and interpretation of the results. We first report two robustness checks, then use sampled model cards to interpret the documentation findings, and finally provide practical recommendations for model practitioners.
Robustness to High-Download Outliers
A concern with the downloads indicator is that it includes both user-initiated retrievals and automated calls from scripts or applications. Prior work on software repository popularity has noted that engagement metrics such as GitHub stars are skewed and heavy-tailed [5]. Empirical SE guidelines also recommend checking whether conclusions are sensitive to extreme observations [19]. We therefore tested whether the main findings depended on a small number of extreme values by removing the top 1% most-downloaded models (670 models, leaving 66,239 models) and rerunning the full analysis: RQ1 significance testing, RQ2 classifier comparison and permutation importance, and RQ3 within / 1-vs-1 / others-vs-1 transfer. The main results remained largely unchanged after removing these high-download models. For RQ1, the count of FDR-significant features changed by at most two (downloads 28 to 27, likes 29 to 27), and no feature that reached at least small effect in the main analysis fell to negligible after cleaning. For RQ2, every classifier’s AUC dropped by at most 0.017, and the model ranking was unchanged. Permutation-importance rankings on the best Random Forest correlated with the main analysis at Spearman on downloads and on likes, with the top-10 features overlapping 10/10 and 9/10. For RQ3, no within-group AUC shifted by more than 0.03, and only four of the 80 cross-group 1-vs-1 cells changed by more than 0.03, all with . The 0.12–0.17 within-to-1-vs-1 drop and the improvement from others-versus-one training also remained.
These results indicate that the main findings were not explained by a small number of high-download models. The slight reduction in classifier AUC (mean −0.01) is consistent with the cleaned data being slightly harder to classify after the most extreme popular examples were removed.
Robustness to the Popular-Group Threshold
Our main scheme compared the top 10% (popular) against the bottom 80% (unpopular), with a 10% gap buffer. This is one reasonable labelling choice, but not the only possible one. We therefore reran the analysis under three alternative schemes (5-10-85, 15-10-75, and 20-10-70) and compared them with the main 10-10-80 scheme.
Classifier AUC moved monotonically with the popular-group threshold: tightening the threshold to the top 5% raised AUC by 0.024–0.040 because the popular group became more distinct, while loosening it to the top 15% or 20% lowered AUC by 0.012–0.030 because the popular group included more boundary cases. This reflects a change in task difficulty rather than a change in the main conclusions. The model ranking was preserved across all four thresholds, and Random Forest remained the best classifier in every setting. Permutation-importance rankings correlated with the main analysis at under every threshold, and the top feature (length-doc) was unchanged. Cross-group transfer also followed the same structure: the within-to-1-vs-1 drop stayed in the 0.10–0.17 range, others-versus-one improved AUC by 0.03–0.05 over one-versus-one, and university remained the lowest within-group AUC on likes under every threshold.
Qualitative Review of Sampled Model Cards
To interpret the Documentation features identified in RQ1 and RQ2, we sampled popular model cards from the main 10-10-80 data: 10 cards from the downloads-based popular group and 10 cards from the likes-based popular group. Two cards appeared in both samples, leaving 18 unique popular model cards. We then examined how the most important documentation features appeared in these sampled model cards.
The 18 sampled popular cards had a mean length-doc of 1,129 words and a median of 1,038.5 words, compared with medians of 451 words for the overall downloads-popular group and 483 words for the overall likes-popular group. Most sampled popular cards also contained the concrete elements highlighted by RQ1 and RQ2: 17 of 18 had at least one code block, 16 of 18 had at least one GitHub link, 16 of 18 had at least one arXiv entry, and 8 of 18 contained a BibTeX block. These observations provide concrete examples of why length-doc, num-code-block, num-gh-links, num-arxiv-links, and has-bibtex appear repeatedly in the quantitative results. They correspond to model-card content that helps a practitioner understand what the model is, try it, trace it to related work or code, and cite it.
When reading these cards, we also observed one documentation practice that is not directly captured by the 31 features: explicit intended-use, limitation, risk, or misuse sections. A keyword check found such content in 12 of the 18 sampled popular cards. For example, the CompVis/stable-diffusion-v1-4 card8 contains sections on the intended use of the model with the Diffusers safety checker, misuse, out-of-scope use, limitations, and bias. Our current features capture the length of the card and the presence of links or code, but they do not distinguish a general long model card from one that explains limitations and responsible use. This suggests a possible direction for future feature extraction: model-card sections about intended use, limitations, and risks may deserve explicit features rather than being absorbed into length-doc.
Practical Recommendations for Model Practitioners
Taken together, the quantitative results and qualitative review point to three practical recommendations for Hugging Face model practitioners. These recommendations are not causal claims. Rather, they describe practices that repeatedly appear in popular models and that are directly under practitioners’ control when preparing a model repository.
Model-card content. A model card should give enough information for a practitioner to decide whether and how to try the model. In the main analysis, popular model cards had a median length-doc of 451 words under downloads and 483 words under likes, compared with about 230–240 words for unpopular model cards. The strongest documentation features point to a concrete structure: include at least one runnable code block with imports and an example invocation (num-code-block and num-inline-code), link to the underlying paper or implementation when available (num-arxiv-links and num-gh-links), and provide a BibTeX block when the model is tied to a publication (has-bibtex). The qualitative review further suggests adding a short limitations or intended-use paragraph following the standard model-card template, even though this practice is not directly measured by the 31 features.
Model files and formats. A model repository should make the model easy for Hugging Face tooling to load and inspect. In our results, has-config reaches small effect on downloads and appears among the top permutation-importance features, while has-safetensors also reaches small effect on downloads. The number of root-level files also appears in the RQ2 top-10 under both indicators, suggesting that the file layout itself carries information about how complete and inspectable a repository is. model-size is also highly ranked in RQ2, but it should not be read as a recommendation to make models larger. Instead, it shows that model scale is part of the context in which popularity is evaluated, and that repository files should make the model’s size and loading requirements clear. For quantized models, practitioners can still release quantized models, but the model card should clearly state the quantization method, the source or base model, and the expected trade-off between efficiency and quality. This is especially important because quantized models are more concentrated in the unpopular group under likes, even though quantization remains important in RQ2.
Platform metadata. Practitioners should fill the structured metadata fields used by the Hugging Face Hub, following the official model-card metadata instructions.9 In particular, they should set a pipeline_tag, declare the main implementation library through library_name, and, when applicable, list one of the officially supported libraries. In RQ1, has-pipeline, has-impl-lib, and has-supported-lib all reach small effect on downloads, while length-yaml and several metadata-related features appear in the RQ2 top features. If the model supports interactive use, adding a Space or a widgetData block can also make the repository easier to try from the Hub. Together, these features suggest a practical checklist for making model metadata more complete and easier to use. Beyond individual practitioners, these findings also suggest opportunities for platform support. Model-sharing platforms could provide lightweight checks for missing runnable examples, citation information, model-card metadata, or limitation and intended-use sections. Such checks would not predict quality directly, but they could help practitioners make model cards more complete and easier to inspect.
Threats to Validity
Construct validity. We measured popularity using two count-based indicators provided by Hugging Face: downloads and likes. Neither indicator is a perfect measure of adoption or community value. Downloads can include automated calls from scripts or applications, while likes require a logged-in user to actively click the like button. For this reason, we did not merge the two indicators. We reported their Spearman correlation (), analyzed every research question separately for downloads and likes, and removed the top 1% most-downloaded models in § 8.1 to check whether high-tail download counts affected the main conclusions. Our 31 features are extracted from model content and Hub metadata. This makes the measurements reproducible, but it also means that they miss model-card sections that require semantic interpretation, such as intended-use, misuse sections, limitation, or risk.
Internal validity. Two design choices could affect the results: the 10-10-80 popularity scheme and the classifier-evaluation design. We addressed the first issue by rerunning the analysis under three alternative threshold schemes in § 8.2. For classifier comparison, we used 10-fold cross-validation repeated 10 times and AUC as the primary score; repeated runs produced the same model ranking. The permutation-importance rankings in RQ2 depend on the best-performing Random Forest. A different classifier, or a future model with higher predictive performance, could rank the features differently. The Mann–Whitney U and Chi-square tests used in RQ1 do not assume normally distributed feature values. All RQ3 cross-group AUCs include 1,000 bootstrap resamples to account for sampling variation. We also do not control for all possible confounders, such as contributor reputation beyond the affiliation label, because our features are defined at the model-repository level rather than at the contributor level.
External validity. Our findings reflect the state of the Hugging Face Hub on the data collection date, 2026-02-28. Hugging Face is a single platform, and the practices we identify may not transfer directly to model registries with different community norms, such as ONNX Model Zoo or PyTorch Hub. Several features are platform-specific, including has-pipeline, has-safetensors, and has-widgets, so they would need to be redefined for another registry. Within Hugging Face, the results may also change as recommendation systems, default model-card templates, and community practices evolve. Finally, the qualitative review in § 8.3 draws on 18 sampled popular model cards, so it should be read as an interpretation of the main documentation features rather than as a complete account of model-card practices.
Conclusion validity. We used several checks to reduce the risk of over-interpreting the results. For RQ1, we applied the Benjamini–Hochberg false discovery rate procedure across all 31 feature tests under each indicator. For the overall pipeline, we reran the analysis after removing high-download outliers and under three alternative popular-group thresholds. These checks reduce the chance that the findings depend on one outlier treatment or one threshold scheme. The qualitative review has a narrower role: it is based on a small sample of popular model cards and is used only to interpret the documentation features identified by RQ1 and RQ2. Most importantly, our analyses are observational. They identify feature differences and predictive importance, but they do not establish causal effects between repository practices and popularity.
Conclusion
We studied 66,909 Hugging Face models collected on 2026-02-28 and extracted 31 features across four dimensions: Documentation, Models, Platforms, and Techniques. We analyzed downloads and likes separately because they capture different forms of platform engagement.
Across both indicators, Documentation features provided the most consistent differences between the popular and unpopular groups. RQ1 identified 13 features on downloads and 8 features on likes that reached at least small feature-level effect. RQ2 showed that these differences also mattered in prediction: Random Forest reached AUC 0.861 on downloads and 0.848 on likes with top permutation-importance features such as length-doc, length-yaml, and num-code-block. RQ3 showed that prediction performance was partly group-specific. Within-group AUC ranged from 0.77 to 0.89, while one-versus-one transfer dropped by 0.12–0.17 AUC on average; training on all other groups improved AUC by 0.03–0.05 over one-versus-one transfer.
The main findings remained stable after two robustness checks: removing the top 1% most-downloaded models and rerunning the pipeline under three alternative popular-group thresholds. A manual review of 18 sampled popular model cards further showed how the main documentation features appear in real model cards: the sampled cards were longer than the overall popular-group median, usually included code blocks and links to papers or code, and sometimes included BibTeX blocks. The review also pointed to one practice not captured by our 31 features: explicit limitations and intended-use sections.
These findings support a small set of recommendations for Hugging Face model practitioners. Model cards should describe a complete usage flow, include runnable code examples, link to the underlying paper or implementation when available, and provide citation information when appropriate. Model repositories should include loadable configuration files and SafeTensors weights when possible, and practitioners should fill structured metadata fields such as pipeline_tag, library_name, supported-library declarations, and Spaces or widgets when applicable. Future work could extract limitations and intended-use sections explicitly, replicate the analysis on registries beyond Hugging Face, and study how these popularity-related features evolve as the platform’s recommendation system and default model-card templates change.
Data Availability
The replication package is publicly available at: https://github.com/YinanWusoymilk/What-Makes-HF-Models-Popular.
References
- [1]Karan Aggarwal, Abram Hindle, and Eleni Stroulia. Co-evolution of project documentation and popularity within github. In Proceedings of the 11th working conference on mining software repositories, pages 360–363, 2014.DOI
- [2]Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M Tyers, and Gregor Weber. Common voice: A massively-multilingual speech corpus. arXiv preprint arXiv:1912.06670, 2019.arxiv.org/abs/1912.06670
- [3]Lingfeng Bao, Zhenchang Xing, Xin Xia, David Lo, and Shanping Li. Who will leave the company?: a large-scale industry study of developer turnover by mining monthly work report. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR), pages 170–181. IEEE, 2017.DOI
- [4]Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 57(1): 289–300, 1995.DOI
- [5]Hudson Borges, Andre Hora, and Marco Tulio Valente. Predicting the popularity of github repositories. In Proceedings of the The 12th international conference on predictive models and data analytics in software engineering, pages 1–10, 2016.DOI
- [6]Hudson Borges, Andre Hora, and Marco Tulio Valente. Understanding the factors that impact the popularity of GitHub repositories. In 2016 IEEE international conference on software maintenance and evolution (ICSME), pages 334–344. IEEE, 2016.DOI
- [7]Leo Breiman. Random forests. Machine learning, 45(1):5–32, 2001.
- [8]Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dharival, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.arxiv.org/abs/2005.14165
- [9]Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.arxiv.org/abs/2107.03374
- [10]Jacob Cohen. Statistical power analysis for the behavioral sciences. routledge, 2013.
- [11]Yuanrui Fan, Xin Xia, David Lo, Ahmed E. Hassan, and Shanping Li. What makes a popular academic AI repository? Empirical Software Engineering, 26(1), January 2021. ISSN 1573-7616. doi: 10.1007/s10664-020-09916-6. URL http://dx.doi.org/10.1007/s10664-020-09916-6.
- [12]Isabelle Guyon and André Elisseeff. An introduction to variable and feature selection. Journal of machine learning research, 3(Mar):1157–1182, 2003.
- [13]Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015.arxiv.org/abs/1510.00149
- [14]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.arxiv.org/abs/1512.03385
- [15]Wenxin Jiang, Chingwo Cheung, Mingyu Kim, Heesoo Kim, George K Thiruvathukal, and James C Davis. Naming practices of pre-trained models in hugging face. arXiv preprint arXiv:2310.01642, 2023.
- [16]Wenxin Jiang, Nicholas Synovic, Matt Hyatt, Taylor R Scholenmmer, Rohan Sethi, Yung-Hsiang Lu, George K Thiruvathukal, and James C Davis. An empirical study of pre-trained model reuse in the hugging face deep learning model registry. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 2463–2475. IEEE, 2023.
- [17]Wenxin Jiang, Jerin Yasmin, Jason Jones, Nicholas Synovic, Jiashen Kuo, Nathaniel Bielanski, Yuan Tian, George K Thiruvathukal, and James C Davis. Peat moss: A dataset and initial analysis of pre-trained models in open-source software. In Proceedings of the 21st International Conference on Mining Software Repositories, pages 431–443, 2024.arxiv.org/abs/2402.00699
- [18]Jason Jones, Wenxin Jiang, Nicholas Synovic, George Thiruvathukal, and James Davis. What do we know about Hugging Face? A systematic literature review and quantitative validation of qualitative claims. In Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pages 13–24, 2024.
- [19]Barbara Kitchenham, Lech Madeyski, David Budgen, Jacky Keung, Pearl Brereton, Stuart Charters, Shirley Gibbs, and Amnart Phothong. Robust statistical methods for empirical software engineering. Empirical Software Engineering, 22(2):579–630, 2017.DOI
- [20]Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Physical Review E—Statistical, Nonlinear, and Soft Matter Physics, 69(6):066138, 2004.pubmed.ncbi.nlm.nih.gov/15244698
- [21]Weixin Liang, Nazneen Rajani, Xinyu Yang, Ezinwanne Ozoani, Eric Wu, Yiqun Chen, Daniel Scott Smith, and James Zou. What’s documented in AI? systematic analysis of 32K AI model cards. arXiv preprint arXiv:2402.05160, 2024.
- [22]Weixin Liang, Nazneen Rajani, Xinyu Yang, Ezinwanne Ozoani, Eric Wu, Yiqun Chen, Daniel Scott Smith, and James Zou. Systematic analysis of 32,111 AI model cards characterizes documentation practice in AI. Nature Machine Intelligence, 6(7):744–753, 2024.DOI
- [23]Henry B Mann and Donald R Whitney. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics, pages 50–60, 1947.DOI
- [24]Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency, pages 220–229, 2019.DOI
- [25]Federica Pepe, Vittoria Nardone, Antonio Mastropaolo, Gabriele Bavota, Gerardo Canfora, and Massimiliano Di Penta. How do hugging face models document datasets, bias, and licenses? an empirical study. In Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension, pages 370–381, 2024.DOI
- [26]Gede Artha Azriadi Prana, Christoph Treude, Ferdian Thung, Thushari Atapattu, and David Lo. Categorizing the content of github readme files. Empirical Software Engineering, 24:1296–1327, 2019.arxiv.org/abs/1802.06997
- [27]Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
- [28]Jeanine Romano, Jeffrey D Kromrey, Jesse Coraggio, and Jeff Skowronek. Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen’s d for evaluating group differences on the NSSE and other surveys. In annual meeting of the Florida Association of Institutional Research, volume 177, 2006.
- [29]Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends in Hugging Face. 2023. arXiv preprint arXiv:2303.17580, 2023.arxiv.org/abs/2303.17580
- [30]Mina Taraghi, Gianolli Dorcelus, Armstrong Foundjem, Florian Tambon, and Foutse Khomh. Deep learning model reuse in the huggingface community: Challenges, benefit and trends. In 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 512–523. IEEE, 2024.arxiv.org/abs/2401.13177
- [31]Robert J Tibshirani and Bradley Efron. An introduction to the bootstrap. Monographs on statistics and applied probability, 57(1):1–436, 1993.
- [32]Jorge R Vergara and Pablo A Estévez. A review of feature selection methods based on mutual information. Neural computing and applications, 24:175–186, 2014.DOI
- [33]Bo Wang, Yueyang Chen, Jieke Shi, Minghui Li, Yunbo Lyu, Yinan Wu, Youfang Lin, and Zhou Yang. Hidden Licensing Risks in the LLMware Ecosystem. arXiv preprint arXiv:2602.10758, 2026.arxiv.org/abs/2602.10758
- [34]Tianlei Wang, Shaowei Wang, and Tse-Hsun Peter Chen. Study the correlation between the readme file of GitHub projects and their popularity. Journal of Systems and Software, 205:111806, 2023.
- [35]Zhou Yang, Chenyu Wang, Jieke Shi, Thong Hoang, Pavneet Kochhar, Qinghua Lu, Zhenchang Xing, and David Lo. What do users ask in open-source AI repositories? An empirical study of GitHub issues. In 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR), pages 79–91. IEEE, 2023.arxiv.org/abs/2303.09795
- [36]Zhou Yang, Jieke Shi, Prem Devanbu, and David Lo. Ecosystem of large language models for code. ACM Transactions on Software Engineering and Methodology, 2024.arxiv.org/abs/2405.16746
- [37]Jiaxin Zhu, Minghui Zhou, and Audris Mockus. Patterns of folder use and project popularity: A case study of GitHub repositories. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pages 1–4, 2014.DOI