WorksComputer Science
IRON-Meme: A Benchmark for Structured Understanding of Identities and Roles in Online Meme Narratives
Abstract
As polarization grows on social media, understanding how political memes shape online narratives has become increasingly important. Previous research has relied on either qualitative frameworks that are hard to scale or computational methods that lack theoretical grounding. Both approaches treat narrative categories in isolation and focus exclusively on moral portrayals of identity. To address these limitations, we introduce a framework grounded in social science theories, encompassing three categories: social identities (such as gender, politics, and ethnicity), viewpoints (ingroup and outgroup perspectives), and narrative roles (moral character, intelligence, appearance, and power). Building on this framework, we present IRON-Meme, a human-annotated benchmark comprising 1,110 political memes across substantially more templates than existing benchmarks. Our analysis of IRON-Meme shows that most memes assign derogatory roles and that identities, roles, and templates are systematically interconnected. Evaluation of nine vision-language models indicates that frontier-scale and reasoning models achieve the highest performance, while accuracy on social identities and narrative roles decreases for less frequent labels. Incorporating template background knowledge does not enhance narrative role classification, suggesting that meme interpretation relies on how memes are adapted for specific communicative purposes. We make IRON-Meme and our code publicly available.
Code: https://github.com/gabriellabollici/ironmeme
Introduction
In the digital age, memes have emerged as a widespread and influential form of political communication on social media platforms [2]. Memes can be defined as identity-shaping symbols that embody the culture and values of online communities [23]. They communicate group-specific political narratives by drawing on cultural knowledge and shared references, many of them encoded in templates whose conventional uses and communicative functions guide interpretation [29]. In doing so, they reinforce group belonging and collective identity [15, 3]. Political memes rarely provide balanced representations of political actors and events, often relying on exaggeration and simplification that produce selective and distorted portrayals [22]. Opposing actors are often depicted as inferior, immoral, unappealing, or incompetent, while members of the favored group are presented as superior, virtuous, attractive, or more knowledgeable [4, 18]. These asymmetric depictions are built by the implicit mapping of social identities (e.g. women/men, democrat/conservative) onto narrative roles (e.g. hero, villain), which are constructed through the distinct framing of favored or opposed actors (ingroups or outgroups). Hence, understanding meme narrative construction requires examining the interplay of these three elements: identities, viewpoints and narrative roles, considering the template conventions through which they are conveyed.
Previous studies have introduced theoretical frameworks for analyzing narratives in memes [14, 20, 18, 26] and benchmarks for evaluating the classification of narrative roles (Sharma et al., 2022b; Barrón-Cedeño et al., 2024; Sharma et al., 2023b,a; Sharma and Chakraborty, 2025). However, we find four limitations in this work. First, theoretically grounded computational methods remain scarce and highly fragmented, as current approaches tend to be either qualitative or scalable, rarely both [9, 42]. Second, existing research treats social identities, narrative roles, and viewpoints in isolation rather than as an interconnected system, obscuring the interactions among these narrative categories. Third, both theoretical and computational approaches have predominantly focused on moral portrayals of identities, resulting in narrow taxonomies that overlook other vital dimensions of identity construction, such as intellectual ability, physical appearance, and social status. Fourth, most existing benchmarks do not incorporate template knowledge, and those that do address only a limited set of templates, leaving open whether such knowledge improves narrative understanding.
Addressing these gaps requires scalable, theory-grounded frameworks that jointly identify and classify identities, viewpoints, and roles in political memes using broader taxonomies of identity construction, along with benchmarks that cover diverse templates and assess the role of template knowledge in narrative understanding, which is the goal of our paper. We make the following contributions:
A unified compositional framework that jointly structures meme understanding across three narrative categories: a set of social identities (e.g., gender, political affiliation, ideological orientation); a definition of viewpoint based on ingroup/outgroup positioning relative to the author or diffusor; and a narrative role taxonomy that extends beyond moral portrayals to four dimensions: moral, intellectual, physical, and power.
A novel human-annotated benchmark, IRON-Meme (Identities and Roles in Online Meme Narratives), comprising 1,110 template-based memes across 348 unique templates, a much wider template coverage than prior benchmarks. To our knowledge, IRON-Meme is the first dataset to systematically operationalize these narrative categories across diverse political themes and meme templates, thus directly bridging theoretical conceptualization and empirical evaluation.
An empirical evaluation of state-of-the-art vision-language models (VLMs) on IRON-Meme in zero-shot settings, assessing the ability of VLMs to scale theory-grounded narrative analysis and the impact of explicit template background knowledge on VLMs’ contextual interpretation of narrative roles.
Related work
Conceptual Frameworks Recent scholarship has increasingly applied theoretical frameworks to the study of narrative elements in internet memes. The field has moved from Dawkins’ view of memes as discrete cultural units to Shifman’s conceptualization of memes as collective sociocultural practices [13, 51, 19]. Researchers have explored gendered representations and feminist counter-narratives [34, 5], national and ideologically driven identity formation [39] (McSwinney et al. 2021), and perspectives on the climate crisis [28, 32]. A more specialized strand of research conceptualizes identities as characters occupying certain roles within broader social and political narratives. These approaches, applied to contexts such as COVID-19 [14], conspiracy theories [20], the Russian-Ukrainian war [18], and the 2016 US elections [26], have primarily examined moral roles, such as hero, villain, and victim, thus limiting the analytical depth and applicability of these frameworks for capturing broader narrative complexities.
Benchmarks Interest has grown in developing benchmarks to evaluate the classification of social identities and narrative roles in memes. Existing benchmarks vary considerably in their definition of social identity, ranging from broad demographic and social categories to open-ended identification of named or depicted entities [12, 44, 47]. For narrative role classification, the most prominent benchmarks emerged from the CON-STRAINT 2022 and CheckThat! 2024 shared tasks, which adopt the same moral taxonomy (hero, villain, victim) as the theoretical frameworks [50, 6, 48]. More recently, viewpoint identification has been explored through binary opposing perspectives generated with VLMs [25], but to our knowledge, no benchmark has yet been developed for this task. Although these benchmarks establish standardized tasks and evaluation protocols, operationalization of narrative elements is fragmented, and no unified, structured framework yet integrates them. Similarly, existing methods are isolated and task-specific: identity classifiers rely mainly on attention-based multimodal fusion [1, 24], while narrative role classifiers draw on contextual embeddings [55], external knowledge bases [49, 58], and cross-modal inference [31, 53, 46]. Finally, most benchmarks do not formalize meme template knowledge at all, and those that do cover only a limited number of templates (e.g., 39 in [55]), limiting their ability to assess models’ use of templates to interpret political memes.
Framework
We now detail our framework’s elements, which are illustrated in Figure 1.

Figure 1. Framework overview. An input meme is annotated by a human or VLM to produce a structured mapping of characters across the three dimensions of the IRON-Meme label schema: social identity, viewpoint, and narrative role. In this example, the white character is assigned the identity climate change denier with an ingroup viewpoint and sage role, while the grey character is classified as a climate activist with an outgroup viewpoint and fool role.
Social identities form the foundational level of our framework (Figure 1): the other two narrative categories, viewpoints and narrative roles, are grounded in, and interpreted through, the identities they represent. Social identities are social categories constructed through discursive practices in which groups are defined by their differences from others, leading to positive or negative differential treatment [57]. They are based on the definition of shared norms, symbolic references, group representatives, and the distinction between “us” and “them” [21]. Understanding the concept of social identity and this polarized structure is essential for meme analysis [57]. Identity-defining attributes include gender and gender orientation, ideological orientation, national and ethnic origin, economic status, religious affiliation, and others.
Viewpoints capture the perspective associated with each identity. They are grounded in a central distinction of social identity theory: that between ingroups and outgroups [11]. Ingroup
identities (“us”) are often defined in contrast to outgroup identities (“them”), through the association of traits or characteristics. From the perspective of meme creators, ingroups represent those who belong to their own social category [21], while outgroups function as antagonistic figures whose voices are systematically excluded, mocked, or targeted [45, 37]. Accordingly, the viewpoint category distinguishes whether a meme identity represents the perspective of the author or diffuser, identified as the ingroup, or the targeted perspective, identified as the outgroup.
Narrative roles capture how identities are portrayed within the narrative. They draw on a crucial insight from character theory: the concept of character can extend beyond individuals, perceived through anthropomorphic projections onto groups, organizations, nations, and similar entities, which are personified and treated as individuals [8]. Building on this, we conceptualize social identities in memes as characters that can be assigned narrative roles, increasing memes’ potential to exert political influence. Although most existing studies base their narrative role taxonomy on the Drama Triangle by Karpman [30], we argue that this framework does not sufficiently capture portrayals within social media environments, as it focuses exclusively on the moral dimension. For this reason, we expand this framework by introducing a new model based on four dimensions: moral, intellectual, physical, and power (Table 1), drawing upon recurring themes identified in research on online media narratives as well as empirical observations of internet memes:
| Evaluative dimension | Moral | Intellectual | Physical | Power |
| Positive / Superior | Hero | Sage | Charmer | Winner |
| Negative / Inferior | Villain | Fool | Monster | Loser |
| Positive / Inferior | Victim |
Table 1. Narrative role labels in our framework. This set is based on two sets of binary oppositions (positive/negative and superior/inferior), and four evaluative dimensions (moral, intellectual, physical and power).
The moral dimension encompasses the hero,
the villain, and the victim. Heroes are depicted as strong, well-intentioned individuals who protect others and restore order, while villains evoke fear and hatred and are usually expected to be punished [8, 18]. Victims are portrayed as innocent, virtuous characters who need protection or rescue [54].
The intellectual dimension contrasts the sage, who is wise, intellectually superior, and self-aware, with the fool, who is characterized as irrational, ignorant, unskilled, or prone to poor judgment [40, 20, 41, 35].
The roles in the physical dimension are the charmer, whose attractiveness aligns with prevailing beauty standards and signals desirability, and the monster, whose appearance is depicted as ugly, grotesque, or monstrous [43, 38, 4].
The power dimension differentiates winners, associated with socioeconomic success, education, stability, career achievement, social competence, and fulfilling relationships, from losers, who are defined by the absence or opposite of these attributes [52, 27].
Figure 1 presents a meme contrasting two identities based on their perspectives on climate change: the denier, framed as the ingroup the audience is invited to align with, and the activist, targeted as the outgroup whose perspective is dismissed. The denier assumes the role of the sage, using historical questioning as seemingly logical counter-evidence, while the activist is cast as the fool (via the uncritical NPC template), unable to answer and reacting with frustration. However, this is a simplified case: in practice, the same actor may adopt different roles across memes or even within a single
meme [14], and positive or negative roles may be assigned to ingroups or outgroups (e.g., self-mockery [27]).
IRON-Meme
This section outlines the construction of the IRON-Meme benchmark, detailing the process from data collection through template-based filtering.
Dataset construction
Data collection. The source datasets for IRON-Meme, whose distribution is outlined in Figure 2, were chosen for their diverse domain coverage.

Figure 2. Final origin distribution of the IRON-Meme benchmark dataset. The inner ring illustrates the overall distribution between reused benchmarks (blue) and newly scraped data (red). The outer ring details the individual datasets and subreddits contributing to each category. The legend indicates the number of memes contributed by each source.
We reuse four existing datasets introduced in prior work. HVVMemes [44] provides 3,202 memes from the US politics subset (Harm-P), collected from Google Image Search, Reddit, Facebook, and Instagram. MAMI [17] contributes 11,000 memes focused on anti-women and antifeminist content, collected from 9GAG, Imgur, Reddit, and Twitter. CliME [10] covers 2,579 multimodal posts addressing climate discourse, ranging from humorous commentary to skepticism, sourced from Twitter and climate-focused subreddits such as r/climatememes. FigMemes [33] provides 5,141 political memes sourced from 4chan’s /pol/ board, addressing topics such as racial minorities, elections, and LGBTQ+ issues.
To incorporate more recent memes and ensure that our benchmark reflects up-to-date topics and formats, we collected Reddit data via the Pull-Push API, retrieving the most recent image submissions available in April 2026. To align with the domains represented in the reused benchmarks, we selected content from r/climatememes, r/conservativememes, and r/trollxchromosomes, focusing on climate discussions, conservative US political perspectives, and feminism, respectively. We collected 5000 memes from each subreddit. We included feminist content to counterbalance the misogynistic perspectives frequently represented in MAMI and FigMemes.
Template-based filtering. We retain only memes that match established templates, reducing ambiguity in characterizing their communicative functions. Specifically, we scraped templates from Know Your Meme and Imgflip.1 We embedded both memes and templates using a DINO ViT-B/16 model, retrieved candidate matches via cosine similarity in FAISS [16], and selected the top template based on a similarity threshold. Remaining unmatched memes were grouped using dHash-based near-duplicate clustering. After manual filtering, the dataset comprised 1,410 memes. More details are available in Appendix § 11.§ 7.
Human annotation and aggregation
Annotation protocol All memes were independently labeled over a three-month period by three annotators: two female social science master’s students and one male political science bachelor’s student, all aged 24. Annotators used a custom interface (Appendix § 11.§ 1) and a comprehensive dynamic codebook containing category definitions, the analytical framework, multiple annotated examples, and detailed instructions for interface usage and annotation procedures. This process followed the open coding procedure outlined by Tracy [56]. Each meme was annotated by all annotators using a hierarchical scheme: they first assigned one or more social identities per meme, then designated a viewpoint, and one or more narrative roles (Table 1) separately for each identified identity. To establish a standardized social identity taxonomy, we conducted an initial dry-run phase in which we iteratively normalized, merged, and expanded freely recorded labels as new categories emerged. This taxonomy stabilized by week 6, when no substantively new categories emerged, producing the final set of 38 social and political identity labels (Appendix § 11.§ 2). Figure 6 visualizes agreement progress across annotation stages.

Figure 6. Pairwise Jaccard agreement across annotation stages
Derivation of ground-truth labels. Ground-truth labels were derived by retaining annotations agreed on by at least two raters. Of 1,410 annotated memes, 1,112 reached consensus, an overall consensus rate of 78.87% (30.85% perfect agreement, 48.01% majority consensus); after excluding two memes, 1,110 were retained for final analysis. Inter-annotator agreement (IAA) varied markedly across subtasks: viewpoint classification achieved the highest consensus (grand Jaccard index: 0.794), followed by social identities (0.660) and narrative roles (0.532). The lower agreement for narrative roles likely stems from subjective interpretation and implicit group assumptions rather than grounded visual/textual evidence. Detailed agreement statistics, including Fleiss’ and Cohen’s , are provided in Appendix § 11.§ 6, and examples of ground truth derivation are presented in Table 10.
| Meme example | Annotator 1 | Annotator 2 | Annotator 3 | Ground Truth |
| !Meme example | men; ingroup; victim; women; outgroup; villain | men; ingroup; victim; women; outgroup; villain | men; ingroup; victim; women; outgroup; villain | men; ingroup; victim; women; outgroup; villain; (perfect agreement) |
| !Meme example | democrat; outgroup; villain/loser | democrat; outgroup; villain; republican; ingroup; sage | democrat; outgroup; loser | democrat; outgroup; villain/loser; (pairwise agreement) |
| !Meme example | climate activist; ingroup; sage | climate denier; outgroup; fool | climate activist; ingroup; winner | no ground truth; (no agreement) |
Table 10. Meme annotated examples showing agreement types and ground truth aggregation.
Benchmark statistics and analysis
Figure 3 displays the distribution of narrative roles among the ten most frequent social identities. The data show a strong concentration of villain, loser, and fool, indicating that derogatory roles are particularly prominent. Notably, 58.92% of the memes are labeled with negative roles. Looking at the distribution within each identity’s assigned roles, corporatist and capitalist identities are mainly depicted as villains (77.4%), climate skeptics or deniers (78.9%), and climate-disengaged groups (65.8%) as fools, and women as losers (45.5%, vs. 25.3% for democrats and 25.9% for feminists) and monsters (10.7%). Conversely, men are mostly represented as victims (35.4%) and winners (20.0%), while climate activists are framed as sages (56.9%). These patterns heavily reflect source dataset sampling rather than general meme culture: 89% of women annotations and 85% of feminist annotations stem from MAMI (a misogyny-focused dataset where feminists are 91% derogatory and 136 of 140 women–loser pairs originate), whereas in female-authored communities (r/TrollXChromosomes), women appear mostly as victims (79%) and never as losers, with feminists portrayed solely as ingroup sages. Similarly, climate annotations are drawn almost exclusively from pro-climate action sources (r/ClimateMemes and CliME), which frame activists as ingroup sages and skeptics or corporate actors as fools and villains. § 7 details these sampling biases, and Appendix § 11.9 provides further statistics.

Figure 3. Identity–role correlation for the 10 most frequent social identities. The chart displays the frequency of social identity–narrative role combinations, with circle size and numerical values indicating frequency counts. Narrative roles (y-axis) are shown in descending order. Colors correspond to distinct roles.
Overall, the distribution indicates that social identities are not represented uniformly across narrative roles; instead, particular identities are systematically associated with specific evaluations. Analysis of template usage by category reveals associations suggesting that certain meme formats are repeatedly employed to fulfill specific narrative functions. When examining the distribution of narrative-role labels across meme templates, we observe that the villain role is most frequently associated with Drakeposting (16 instances), and Distracted Boyfriend (13), This Is Fine with the fool (15), and My Parents Are Dead / Batman Slapping Robin with the sage (12). Figure 11 shows the composition of the ten most frequent templates by narrative role. Examining template, narrative role, and social identity jointly reveals more specific recurring configurations. Two templates most frequently represent both men as ingroup victims and women outgroup villains: Socially Awesome Awkward Penguin and Confession Bear. In the template My Parents Are Dead / Batman Slapping Robin, climate activists are frequently constructed as ingroup sages (10), reinforcing the pattern of representing climate activists as knowledgeable or authoritative.

Figure 11. Narrative-role composition for the ten most frequently used meme templates. Stacked bars represent the number of memes assigned to each narrative role for each template. Template names correspond to their listings in Know Your Meme.
Experimental setup
Evaluation protocol. We evaluate nine VLMs following the same hierarchical structure as the human annotation (§ 4.2): models first identify the discussion domain and social identities, then assign a viewpoint to each identity, and finally infer identity’s narrative roles by selecting an evaluative dimension (moral, intellectual, physical, or power) and polarity (positive or negative) before assigning the role label. All models receive identical inputs, instructions, label inventories, and output constraints. To isolate performance at each stage, prompts are conditioned on upstream ground-truth labels. Model outputs are parsed and normalized to each task’s label inventory; unparseable responses and labels outside the inventory are counted as incorrect. Predicted viewpoints and roles are counted as correct only when linked to the correct identity. Parsing failures and discarded labels per model are detailed in Appendix § 11.§ 4, and prompt templates are listed in Appendix § 11.§ 5.
Furthermore, we conduct an ablation study using the best-performing model. Considering that meme templates have conventional uses and communicative functions, we hypothesize that knowledge of these conventions helps interpret the narrative roles implicit in a meme. In particular, template knowledge can support a model’s inference about the attitudes identities hold toward one another, thus facilitating the classification of narrative roles (Figure 11). To test this hypothesis, we introduce template knowledge as an information intervention and test whether incorporating conventional meanings and uses improves performance compared to meme-only inputs. We sourced template information for each meme from Know Your Meme.
Metrics. We report precision, recall, and F1-score, each micro- and macro-averaged. Micro-averaging reflects performance across all instances, while macro-averaging weights each class equally, making it more sensitive to performance on less frequent classes, such as the monster/charmer roles or the authoritarian and politically disillusioned identity labels. We use macro-F1 as our primary metric. See Appendix § 11.8 for details and formulas.
VLM selection. We conducted a zero-shot ICL evaluation using nine state-of-the-art VLMs that represent a range of model families, architectures, and reasoning capabilities. We also include geographically diverse models, accounting for how differing alignment procedures, training data, and development contexts affect performance in socially and narratively complex visual tasks. A complete overview of the models and their respective specifications is given in Appendix § 11.10.
Experimental results
We address four questions: (RQ) How does performance differ across models and tasks? (RQ)
What are the underlying causes of model performance? () How does the frequency of specific meme templates, social identities, and narrative roles within the dataset influence model performance? () How does explicit knowledge about a meme template’s conventional meaning and usage affect a VLM’s inference of narrative roles?
: Variations in performance across models and tasks Performance varies substantially across both models and tasks (Table 2). Viewpoint classification is the easiest task, with all VLMs exceeding 70% F1, reflecting its binary label space. Social identity and narrative role classification are considerably more challenging, with scores ranging from 5.80 to 52.78 and from 11.70 to 64.14 F1, respectively. Scores on these two tasks are highly correlated (Spearman’s ), with DeepSeek-VL2 as the main exception (13.07 vs. 34.44).
| Model | Social Identities Precision | Social Identities Recall | Social Identities F1 | Viewpoints Precision | Viewpoints Recall | Viewpoints F1 | Narrative Roles Precision | Narrative Roles Recall | Narrative Roles F1 |
| Random (stratified) | 11.45 | 11.46 | 11.45 | 64.94 | 64.96 | 64.94 | 21.09 | 21.10 | 21.09 |
| DeepSeek-VL2 | 11.25 | 39.72 | 13.07 | 67.18 | 90.89 | 77.10 | 47.00 | 41.90 | 34.44 |
| Gemini-3.6 | 45.58 | 73.55 | 51.99 | 93.55 | 92.86 | 93.20 | 59.82 | 75.75 | 64.14 |
| GLM-4.1V | 45.28 | 42.81 | 38.60 | 81.45 | 77.93 | 79.61 | 41.20 | 60.49 | 46.55 |
| Kimi-K3 | 44.99 | 77.60 | 52.78 | 92.60 | 92.62 | 92.61 | 48.11 | 81.01 | 57.88 |
| Molmo-7B | 4.16 | 27.99 | 5.80 | 60.01 | 91.25 | 71.08 | 17.31 | 15.49 | 11.70 |
| Pixtral-12B | 36.83 | 47.33 | 35.58 | 77.29 | 78.72 | 77.98 | 41.65 | 46.81 | 40.60 |
| Qwen2.5-VL | 40.58 | 37.88 | 26.41 | 79.00 | 69.18 | 72.78 | 34.86 | 32.86 | 23.52 |
| Qwen3-VL-I | 39.66 | 50.46 | 36.35 | 76.11 | 80.56 | 77.05 | 41.30 | 55.53 | 41.74 |
| Qwen3-VL-T | 47.99 | 47.33 | 43.42 | 80.51 | 75.63 | 77.93 | 41.54 | 58.57 | 45.51 |
Table 2. Zero-shot performance of VLMs across categories (macro metrics) compared against a stratified random baseline. For each meme and each label, the baseline assigns a prediction probability equal to the prevalence of that label in the ground truth. Full model names are listed in Table 9.
| Model | Developer / Region | Family | Orientation / Capability | Parameter Scale |
| DeepSeek-VL2 | DeepSeek-AI (East Asia) | DeepSeek | Instruction | 27.5B (MoE) |
| Molmo-7B | Allen Institute for AI (North America) | Molmo | Instruction | 7B |
| Qwen2.5-VL-7B-Instruct | Alibaba Cloud (East Asia) | Qwen | Instruction | 7B |
| Qwen3-VL-8B-Instruct | Alibaba Cloud (East Asia) | Qwen | Instruction | 8B |
| Qwen3-VL-8B-Thinking | Alibaba Cloud (East Asia) | Qwen | Explicit Reasoning | 8B |
| GLM-4.1V-9B-Thinking | Zhipu AI / Tsinghua (East Asia) | GLM | Explicit Reasoning | 9B |
| Pixtral-12B | Mistral AI (Europe) | Pixtral | Instruction / Standard | 12B |
| Gemini-3.6-Flash | Google (North America) | Gemini | High-Efficiency / Reasoning | Proprietary |
| Kimi-K3 | Moonshot AI (East Asia) | Kimi | High-Capacity Reference | Large-scale (MoE) |
Table 9. Overview of the VLMs selected for evaluation, covering diverse model families, architectures, reasoning orientations, parameter scales, and geographic regions.
Two models clearly lead across all three tasks: Gemini-3.6 achieves the best results on viewpoints (F1: 93.20) and narrative roles (F1: 64.14), while Kimi-K3 leads on social identities (F1 52.78) and ranks second on the other two tasks. The remaining models trail considerably: on viewpoints, they cluster between 71 and 80 F1, more than 13 points below the leaders, and on narrative roles, the next-best model, GLM-4.1V (F1: 46.55), scores 17.59 points below Gemini-3.6. Precision and recall reveal how this advantage is achieved. On social identities, the leaders’ advantage stems from recall (R: 73.55 and 77.60), while their precision (P: 45.58 and 44.99) is comparable to that of smaller models such as Qwen3-VL-T (P: 47.99). On narrative roles, the two leaders follow different strategies: Gemini-3.6 achieves the highest precision (P: 59.82), whereas Kimi-K3 achieves the highest recall (R: 81.01) at a considerably lower precision (P: 48.11). More generally, most models show higher recall than precision on both fine-grained tasks, indicating a tendency to over-assign identities and roles rather than miss them. Full results are reported in Appendix A.10, Table 13.
| Model | Micro Metrics | Macro Metrics | ||||
| Precision | Recall | F1 | Precision | Recall | F1 | |
| Social Identities | ||||||
| deepseek-vl2 | 11.65 | 33.23 | 17.26 | 11.25 | 39.72 | 13.07 |
| gemini-3.6-flash | 55.79 | 76.31 | 64.46 | 45.58 | 73.55 | 51.99 |
| glm-4.1v-9b-thinking | 57.02 | 45.97 | 50.90 | 45.28 | 42.81 | 38.60 |
| kimi-k3 | 55.19 | 77.50 | 64.47 | 44.99 | 77.60 | 52.78 |
| molmo-7b-d-0924 | 3.94 | 36.34 | 7.11 | 4.16 | 27.99 | 5.80 |
| pixtral-12b-2409 | 47.09 | 58.02 | 51.99 | 36.83 | 47.33 | 35.58 |
| qwen2.5-vl-7b-instruct | 27.49 | 30.30 | 28.82 | 40.58 | 37.88 | 26.41 |
| qwen3-vl-8b-instruct | 44.37 | 55.37 | 49.26 | 39.66 | 50.46 | 36.35 |
| qwen3-vl-8b-thinking | 59.74 | 51.07 | 55.07 | 47.99 | 47.33 | 43.42 |
| Viewpoints | ||||||
| deepseek-vl2 | 69.62 | 92.94 | 79.61 | 67.18 | 90.89 | 77.10 |
| gemini-3.6-flash | 94.13 | 93.70 | 93.91 | 93.55 | 92.86 | 93.20 |
| glm-4.1v-9b-thinking | 83.45 | 80.79 | 82.10 | 81.45 | 77.93 | 79.61 |
| kimi-k3 | 93.46 | 93.32 | 93.39 | 92.60 | 92.62 | 92.61 |
| molmo-7b-d-0924 | 60.65 | 92.33 | 73.21 | 60.01 | 91.25 | 71.08 |
| pixtral-12b-2409 | 79.73 | 80.64 | 80.18 | 77.29 | 78.72 | 77.98 |
| qwen2.5-vl-7b-instruct | 81.81 | 75.53 | 78.55 | 79.00 | 69.18 | 72.78 |
| qwen3-vl-8b-instruct | 77.04 | 78.97 | 77.99 | 76.11 | 80.56 | 77.05 |
| qwen3-vl-8b-thinking | 82.67 | 76.77 | 79.61 | 80.51 | 75.63 | 77.93 |
| Narrative Roles | ||||||
| deepseek-vl2 | 34.92 | 52.60 | 41.98 | 47.00 | 41.90 | 34.44 |
| gemini-3.6-flash | 59.98 | 76.58 | 67.27 | 59.82 | 75.75 | 64.14 |
| glm-4.1v-9b-thinking | 41.66 | 63.63 | 50.35 | 41.20 | 60.49 | 46.55 |
| kimi-k3 | 50.44 | 82.42 | 62.58 | 48.11 | 81.01 | 57.88 |
| molmo-7b-d-0924 | 20.09 | 16.80 | 18.29 | 17.31 | 15.49 | 11.70 |
| pixtral-12b-2409 | 39.17 | 49.10 | 43.58 | 41.65 | 46.81 | 40.60 |
| qwen2.5-vl-7b-instruct | 32.96 | 39.74 | 36.03 | 34.86 | 32.86 | 23.52 |
| qwen3-vl-8b-instruct | 35.68 | 65.55 | 46.21 | 41.30 | 55.53 | 41.74 |
| qwen3-vl-8b-thinking | 40.38 | 59.72 | 48.18 | 41.54 | 58.57 | 45.51 |
| Averages† | ||||||
| deepseek-vl2 | 38.73 | 59.59 | 46.28 | 41.81 | 57.50 | 41.54 |
| gemini-3.6-flash | 69.97 | 82.20 | 75.21 | 66.32 | 80.72 | 69.78 |
| glm-4.1v-9b-thinking | 60.71 | 63.46 | 61.12 | 55.98 | 60.41 | 54.92 |
| kimi-k3 | 66.36 | 84.41 | 73.48 | 61.90 | 83.74 | 67.76 |
| molmo-7b-d-0924 | 28.23 | 48.49 | 32.87 | 27.16 | 44.91 | 29.53 |
| pixtral-12b-2409 | 55.33 | 62.59 | 58.58 | 51.92 | 57.62 | 51.39 |
| qwen2.5-vl-7b-instruct | 47.42 | 48.52 | 47.80 | 51.48 | 46.64 | 40.90 |
| qwen3-vl-8b-instruct | 52.36 | 66.63 | 57.82 | 52.36 | 62.18 | 51.71 |
| qwen3-vl-8b-thinking | 60.93 | 62.52 | 60.95 | 56.68 | 60.51 | 55.62 |
Table 13. Performance evaluation of vision-language models grouped by category. † Average metrics are calculated as the arithmetic mean across Social Identity, Viewpoint, and Narrative Roles tasks.
: Underlying causes of model performance Model scale and reasoning appear to be the main factors underlying these differences, though our comparison does not control for other design choices such as training data. The two frontier-scale models, Gemini-3.6 and the sparse Mixture-of-Experts (MoE) Kimi-K3, outperform all other models on every task, with the largest margins on narrative roles (64.14 and 57.88 vs. at most 46.55). This cannot be explained by architecture alone: DeepSeek-VL2, also an MoE model, is one of the weakest models on social identities (13.07). Among the 7–12B models, viewpoint performance is relatively similar (71.08–79.61), whereas social identity performance varies widely (5.80–43.42). This suggests that smaller models differ in their ability to map memes onto large, fine-grained label spaces. Within this group, reasoning improves performance: the two reasoning models, Qwen3-VL-T and GLM-4.1V, are the strongest on social identities and narrative roles, and Qwen3-VL-T outperforms its instruct counterpart, on all three tasks, most notably on social identities (43.42 vs. 36.35). The effect of reasoning differs by task: on social identities and viewpoints it mainly increases precision (P: 47.99 vs. 39.66; 80.51 vs. 76.11), whereas on narrative roles it mainly increases recall (R: 58.57 vs. 55.53).
: Influence of frequency on performance Model performance correlates positively with label frequency for social identities and narrative roles, whereas template frequency shows no meaningful relationship with performance. For social identities, the correlation is statistically significant (Spearman’s , pooled across models), with rare identities scoring considerably lower than frequent ones (e.g., women: , F1 = 65.5; authoritarian: , F1 = 3.1). Narrative roles show a similar dependence on frequency (), with frequent roles outperforming less common ones (e.g., villain: , F1 = 62.4; hero: , F1 = 27.2). Frequency alone, however, does not explain role performance: loser achieves only 33.8 F1 despite its high frequency (), suggesting that some roles are inherently harder to interpret. In contrast, template frequency is uncorrelated with performance (). See Appendix § 11.10, Figure 12 for more details.

Figure 12. Label frequency and model performance across evaluation dimensions

Figure 4. Narrative role error analysis heatmap.
RQ4: Impact of template knowledge on performance We hypothesized that knowledge of a template’s conventional meaning and usage would help models infer narrative roles. To test this, we evaluated the best-performing model, Gemini-3.6, with and without template descriptions in the prompt (Table 3). Contrary to our hypothesis, adding template information reduced performance on all three metrics, with Macro-F1 dropping by 3.3 points (from 65.2 to 61.9). The largest decrease was in precision (4.9 points), while recall changed less (1.6 points). These findings indicate that, in practice, narrative role association frequently extends beyond conventional template uses, as illustrated by the examples provided in Appendix § 12.§ 3.
| Prompt Condition | Precision | Recall | F1 |
| Standard | 60.50 | 77.10 | 65.20 |
| Templated | 55.60 | 75.50 | 61.90 |
Table 3. Macro-averaged narrative role performance of Gemini-3.6 with and without template knowledge.
Error analysis. To compare model and human performance directly, we evaluate our best-performing model, Gemini-3.6, alongside five human evaluators on a shared subset of 214 memes
without access to ground-truth labels. In this setting, models and humans must assign viewpoints and narrative roles to their own predicted identities rather than gold identities. As shown in Table 11, Gemini-3.6 outperforms untrained human evaluators across all three tasks, achieving an overall average Macro F1 of 58.67% compared to 49.38% for humans: a gap driven by the model’s superior recall in identifying social identities and narrative roles. Qualitative analysis reveals that Gemini-3.6 shares several systematic error patterns with human annotators: both tend to default to the outgroup, the majority viewpoint class, and both generally preserve the overall polarity of a portrayal (positive vs. negative) even when assigning the wrong role. However, the model’s errors are more concentrated: it frequently collapses distinct negative roles into a single category (villain), and it swaps roles between antagonists, assigning a role to the wrong identity 15. Human annotators, by contrast, are more likely to leave a role unassigned. Appendix § 13.§ 1 provides a detailed breakdown of these error patterns.
| Model | Micro Metrics | Macro Metrics | ||||
| Precision | Recall | F1 | Precision | Recall | F1 | |
| Social Identities | ||||||
gemini-3.6-flash | 51.69 | 77.73 | 62.09 | 42.98 | 63.12 | 46.15 |
| Human annotators | 49.85 | 64.84 | 56.37 | 34.40 | 43.60 | 35.86 |
| Viewpoints | ||||||
gemini-3.6-flash | 78.09 | 87.35 | 82.46 | 74.81 | 83.41 | 78.87 |
| Human annotators | 81.17 | 76.68 | 78.86 | 77.64 | 73.11 | 75.30 |
| Narrative Roles | ||||||
gemini-3.6-flash | 46.25 | 72.35 | 56.43 | 43.89 | 71.64 | 50.99 |
| Human annotators | 44.78 | 50.38 | 47.42 | 36.64 | 42.77 | 36.97 |
| Averages{sup}†` | ||||||
gemini-3.6-flash | 58.68 | 79.14 | 66.99 | 53.89 | 72.72 | 58.67 |
| Human annotators | 58.60 | 63.97 | 60.88 | 49.56 | 53.16 | 49.38 |
Table 11. Comparison of gemini-3.6-flash and human annotators in the setting without ground-truth context, evaluated on the same subset of 214 memes.
Conclusions
We present IRON-Meme, a benchmark for evaluating VLM classification of social identities, viewpoints, and narrative roles in memes. Constructed using a novel framework, IRON-Meme encompasses these categories in three distinct classification tasks. An evaluation of nine VLMs demonstrates that scalable narrative analysis is attainable but remains challenging. The most effective models reliably distinguish ingroups from outgroups, but accurately identifying portrayed individuals and their roles remains complex, especially for less frequent labels. Reasoning-based models outperform instruction-based counterparts. Contrary to initial expectations, explicit template knowledge did not improve role inference, and instead reduced performance. This finding indicates that effective meme understanding goes far beyond templates’ conventional use. Future research should expand IRON-Meme to include a broader range of sociopolitical contexts and languages, as well as larger-scale datasets with proportional representation of different ideologies. Additionally, further investigation is warranted into methods for integrating template knowledge without compromising meme-specific meaning, thereby advancing tools for tracking the construction of meme narratives at scale.
Limitations
Despite efforts to curate a balanced corpus, the dataset exhibits specific compositional limitations across some dimensions. Specifically, we observe two constraints: 1. The data skewed toward conservative ideologies. This imbalance results from the practical challenge of identifying dedicated, highly active subreddits focused on progressive or Democratic perspectives. Additionally, the dataset exhibits a significant bias due to the over-representation of misogynistic memes, primarily sourced from MAMI, the largest subset. This bias directly influences the analysis results and may affect model performance estimations. 2. The benchmark relies mostly on English-language memes from Reddit, the study is centered on the US political and cultural ecosystem and fails to capture either non-Anglophone meme environments or the potentially more extreme narratives prevalent on alternative platforms such as 4chan. As a result of these sampling constraints, the prominence of specific identity–role combinations within the dataset reflects our collected corpus rather than online discourse as a whole. While our findings highlight meaningful patterns of identity construction within this specific context, future research should expand dataset boundaries to incorporate broader platform perspectives, non-English environments, and a more balanced representation of online political debates.
Another significant limitation of this study arises from the inherently subjective nature of political meme interpretation, which is influenced by the context-dependent nature of visual tropes and the annotators’ individual background beliefs. As the first empirical investigation of political meme interpretation at this scale, we encountered these complexities firsthand. Examples of these challenges are annotators assuming that voters perceive themselves as morally superior to non-voters, or reading a brain graphic directly as a symbol of intelligence. To address these challenges, we established comprehensive annotation guidelines to facilitate the understanding of complex contextual cues. Although some annotator disagreement persists, un-
derscoring the intrinsic difficulty of the task and indicating important directions for future research, our benchmark offers a valuable resource for evaluating computational narrative analysis of political memes.
A lower human baseline may raise concerns regarding the quality of the benchmark, as Gemini-3.6-flash consistently outperformed human evaluators. This performance gap can be attributed to two primary factors: 1. Cognitive limitations. Human evaluators are constrained by individual cognitive biases and varying interpretations. 2. Information scale. Vision-Language Models (VLMs) utilize extensive background knowledge, which enables identification of intricate intertextual references embedded in political memes. While additional empirical investigation is required to substantiate this performance gap, these findings are consistent with the Data Perspectivism paradigm. Instead of treating human variation as noise, data perspectivism advocates for benchmarks that explicitly incorporate the diverse perspectives and subjective interpretations of human annotators within the knowledge representation process ([7]).
Contrary to our initial hypothesis, incorporating template-level background knowledge did not improve narrative role classification performance. We attribute this outcome to: 1. Meme fluidity and pragmatic use: While templates establish baseline conventional expectations, political memes are inherently dynamic: meanings are continuously repurposed, inverted, or deployed ironically. Consequently, narrative roles depend primarily on instance-specific communicative intent rather than static template definitions. 2. Information scope: The background information provided by platforms such as Know Your Meme focuses largely on origins and intertextual references rather than conventional usage patterns. Future work should explore whether higher-quality or more pragmatically focused template annotations can enhance role classification.
Ethical considerations
To support social media researchers in analyzing online discourse, this study provides a benchmark for evaluating VLM capabilities in decoding complex digital narratives. The authors explicitly state that this benchmark and its underlying framework are intended strictly for academic and research purposes; we do not authorize their deployment for automated content moderation, censorship, or commercial surveillance. All meme content included in the benchmark has been carefully anonymized, with identifiable personal information removed. Our analytical focus is restricted to macro-level narrative patterns and identity construction within political memes. We do not aim to evaluate, target, or depict any individual social media user, nor do we endorse or promote any of the ideological stances represented within the dataset.
The research adhered to the Code of Ethics for Research Involving Human Participants established by the relevant university faculty. Compliance with ethical requirements was confirmed through a self-assessment process.
Acknowledgments
We gratefully acknowledge the annotators Letizia Abbate, Milan van Wordragen, and Jana Morgagni, whose contributions were essential to this project. We also thank the Societal Analytics Lab for awarding us the annotation grant that funded this aspect of the research (grant no. 258280). We appreciate the feedback provided by all PhD students from the Commonsense AI group on earlier versions of the paper, especially Mohammadhossein Khojasteh and Shaurya Gaur for their valuable contributions and insights. This research is supported by the NWO AiNed project “Human-Centric AI Agents with Common Sense” (grant no. NGF.1607.22.044).
Use of AI Assistants We employed Claude Code to support with coding and utilized Grammarly and Gemini to refine the manuscript. All methodological and analytical decisions were made by the authors.
Experimental details
Annotation interface

Figure 5. Screenshot of the annotation interface used by IRON-Meme annotators. For each annotation record, annotators select one identity, one viewpoint, and one or more narrative roles. To add another social identity to a meme, the + Add annotation button is used. The interface also provides options to flag unclear cases for later review and to add comments regarding the meme or the annotations.
Social identity labels
Table 4 details the social identity labels evaluated, spanning demographics (e.g., gender, ethnicity) as well as general and domain-specific ideological positions.
| Identity Label | ||
| Women | Men | White people |
| People of color | Citizens | National identity |
| Feminist | Anti-feminist | Sex worker |
| LGBTQ+ rights supporter | Anti-LGBTQ+ | Misogynist/male supremacist |
| Racist | Anti-racist | Nationalist/regionalist |
| Progressive | Conservative | Liberal |
| Left-wing | Right-wing | Libertarian |
| Authoritarian | Democrat | Republican |
| Anticapitalist | Corporatist/capitalist | Green Party supporter |
| Voter | Abstentionist | Politically disillusioned |
| Politicians/government | World leaders/elites | Media elites |
| Celebrity | Conspiracy theorist | Climate activist |
| Climate change skeptic/denier | Climate disengaged | Clean energy advocate |
| Nuclear energy advocate | Plant-based/vegan |
Table 4. Set of social identity labels used in the dataset.
Computational resources
All open-weight models (i.e., DeepSeek-VL2, GLM-4.1V-9B-Thinking, Molmo-7B-D-0924, Pixtral-12B-2409, Qwen2.5-VL-7B-Instruct, Qwen3-VL-8B-Instruct, and Qwen3-VL-8B-Thinking) were served locally with vLLM on a SLURM cluster partition. Each job was allocated one NVIDIA H100 GPU (94 GB), 8 CPU cores, 48 GB of RAM, and a 12-hour wall-time limit. The sole exception was Kimi-K3 (a 2.8T-parameter MoE model with 104B active parameters), which was evaluated through Moonshot’s hosted, OpenAI-compatible API, because its MXFP4-quantized weights (approximately 1.4 TB) far exceed the memory of a single GPU. Since the API only accepts a sampling temperature of 1, Kimi-K3 was decoded non-greedily, unlike the locally served models (temperature 0).
Parsing
Although prompts request JSON output, model generation is unconstrained. Responses are therefore parsed by stripping reasoning traces and code fences before extracting the outermost JSON span; unparseable responses are scored as empty predictions. Predicted labels are normalized across case, punctuation and separators, pluralization, and spelling variants, and retained only if they exist in the corresponding task’s label inventory; all other labels are discarded. Evaluation metrics are computed over each task’s fixed label inventory, so discarded labels do not introduce additional classes.
Table 5 reports parsing failures and Table 6 reports discarded labels per model.
| Model | s1 (Ident.) | s2 (View.) | s3 (Roles) |
| deepseek-vl2 | 10.2% | 1.4% | 0.0% |
| gemini-3.6-flash† | — | — | — |
| glm-4.1v-9b-think | 1.0% | 1.5% | 0.0% |
| kimi-k3 | 0.0% | 0.0% | 0.0% |
| molmo-7b-d | 52.7% | 2.3% | 1.4% |
| pixtral-12b | 0.0% | 0.0% | 0.0% |
| qwen2.5-vl-7b | 0.0% | 0.0% | 0.0% |
| qwen3-vl-8b | 0.0% | 0.0% | 0.0% |
| qwen3-vl-8b-think | 3.1% | 6.0% | 5.0% |
Table 5. Percentage of unparsable responses per model across task steps.
| Model | Identities | Viewpoints | Roles |
| deepseek-vl2 | 7.2% | 0.0% | 5.2% |
| gemini-3.6-flash† | 0.0% | 0.0% | — |
| glm-4.1v-9b-think | 1.3% | 0.0% | 0.0% |
| kimi-k3 | 0.0% | 0.0% | 0.0% |
| molmo-7b-d | 47.7% | 0.2% | 0.0% |
| pixtral-12b | 3.4% | 0.0% | 0.0% |
| qwen2.5-vl-7b | 3.4% | 0.0% | 0.6% |
| qwen3-vl-8b | 0.4% | 0.0% | < 0.1% |
| qwen3-vl-8b-think | 0.2% | — | — |
Table 6. Percentage of discarded labels outside task inventories per model.
Evaluation diagnostics identified distinct failure modes among models during parsing and label validation. Molmo (molmo-7b-d-0924) exhibited significant degradation, failing to generate parseable JSON in over half (52.7%) of its step-1 responses and having two-thirds of its predicted role labels discarded. These issues primarily resulted from producing Chinese translations, substituting viewpoint terms (outgroup/ingroup, approximately 1,700 instances), or altering dimension names (e.g., Moral). Other models primarily failed due to prompt leakage or schema mismatches. For instance, deepseek-vl2 frequently repeated prompt example mappings as identity labels (e.g., “Extinction Rebellion”, “Neoliberal”) and substituted viewpoint terms for roles. Qwen3-vl-8b-thinking copied prompt placeholder tokens (IDENTITY_1, IDENTITY_2) verbatim. Several models predicted valid domain concepts that were not included in the strict task inventory, such as qwen2.5-7b-instruct using dimension descriptors like Inferior/Positive as roles, or glm-4.1v-9b-thinking generating unmapped identities such as “Government” or “Girls”. Additionally, minor inflection mismatches, such as predicting “Woman” instead of the approved plural label “Women”, were filtered out because standard normalization rules only removed trailing “s” characters. This highlights the need to expand strict spelling aliases to accommodate valid morphological variants.
Prompts
Prompt 1: Prompt for Social Identity Extraction
SYSTEM: You are an expert in semiotics and
cultural analysis. Analyze memes to identify
evaluatively represented social identities.
Return valid JSON only. Do not include markdown,
explanations, or text outside the JSON
object.
USER: Analyze the attached meme. Return:
{"social_identities":["IDENTITY_1","IDENTITY_2"]}
Select identity labels ONLY from the approved
list below.
RULES
- Label an identity only if the meme expresses
or strongly implies an evaluative stance
toward it.
- Do not make up custom labels, sub-categories,
or descriptive phrases (e.g., do NOT output
"angry leftists", "neoliberals", or
"communists"). Use exact labels from the
approved list.
MAPPING & ABSTRACTION RULES
If a specific sub-ideology, political movement,
or granular social group is depicted or
implied, map it to its broader, general
umbrella label from the approved list.
Examples:
- Communist / Socialist / Marxist / Anarchist ->
"Left wing" or "Anticapitalist"
- Neoliberal / Classical Liberal / Capitalist ->
"Liberal" or "Corporativist/capitalist"
- Fascist / Alt-Right / Far-Right -> "Right-wing
" or "Authoritarian"
- Neoconservative / Alt-Lite -> "Conservative"
- Climate Striker / Extinction Rebellion / Eco-
activist -> "Climate activist"
- Queer / Transgender / Gay / Lesbian -> "LGTBQ+
rights supporter" or "Anti-LGTBQ+" (
depending on the stance/group evaluated)
APPROVED IDENTITY LABELS:
[
"Women",
"Men",
"White people",
"Feminist",
"Anti feminist",
"LGTBQ+ rights supporter",
"Anti LGTBQ+",
"Misogynist/male supremacist",
"Sex worker",
"Racist",
"Anti racist",
"people of color",
"National identity",
"Nationalist/regionalist",
"Progressive",
"Conservative",
"Left wing",
"Right wing",
"Liberal",
"Authoritarian",
"Republican",
"Democrat",
"Anticapitalist",
"Corporativist/capitalist",
"Voter",
"Abstentionist",
"Politicians/government",
"World leaders/elites",
"Media elites",
"Celebrity",
"Green party supporter",
"Libertarian",
"Politically disillusioned",
"Conspiracy theorist",
"Climate activist",
"Climate change skeptic/denier",
"Climate disengaged",
"Plant based/vegan",
"Clean energy advocate",
"Nuclear energy advocate",
"Citizens"
]
OUTPUT REQUIREMENTS
Output valid JSON only. Key must be "
social_identities". Value must be an array
of approved strings strictly matching the
list above.
Prompt 2: Prompt for Viewpoint Classification
SYSTEM: You are an expert in semiotics and
cultural analysis specializing in meme
interpretation. Your task is to identify the
viewpoint expressed by the meme relative to
the social identities identified in the meme
.
Return valid JSON only. Do not include markdown,
explanations, commentary, or any text
outside the JSON object.
USER: Analyze the attached meme and determine
the viewpoint expressed by the meme toward
ALL of the following identified social
identities: {social_identities}.
STRICT INSTRUCTION:
You MUST evaluate and return a viewpoint entry
for EVERY single identity listed in {
social_identities}. Do not omit any identity
.
Return JSON array where each entry specifies an
identity and its viewpoint:
[
{"identity": "IDENTITY_NAME", "viewpoint": "
ingroup"},
{"identity": "IDENTITY_NAME", "viewpoint": "
outgroup"}
]
VIEWPOINT DEFINITION
The ingroup is the social identity whose
perspective, beliefs, values, actions, or
position are expressed, supported,
normalized, defended, or endorsed by the
meme.
The outgroup is the social identity that is
criticized, mocked, blamed, rejected, attacked, negatively portrayed, or positioned as opposed to the meme's perspective.
OUTPUT REQUIREMENTS Output valid JSON array only. Length of output array MUST equal the number of input identities provided. Every entity MUST have a "viewpoint" field. Each object must have exactly two keys: "identity" and "viewpoint". Each "identity" key must have only one "viewpoint" assigned. "viewpoint" must be either "ingroup" or "outgroup".
Prompt 3: Prompt for Narrative Role Classification
SYSTEM: You are an expert in semiotics and cultural analysis specializing in meme interpretation. Your task is to identify the narrative roles assigned to the social identities present in a meme. Return valid JSON only. Do not include markdown, explanations, commentary, or any text outside the JSON object.
USER: Analyze the attached meme and assign narrative roles to the previously identified social identities and viewpoints: {viewpoints_context}
Narrative roles describe how a social identity is portrayed within the meme's message. Assign roles relative to the identity's representation, evaluation, and position in the narrative.
Return a JSON array formatted as follows: [ { "identity": "IDENTITY_NAME", "roles": ["ROLE_1", "ROLE_2"] } ]
ROLE FRAMEWORK Follow two steps: Step 1: Identify the evaluative dimension (Moral, Intellectual, Physical, Power) Step 2: Determine whether the portrayal is Superior/Positive or Inferior/Negative.
NARRATIVE ROLES Moral dimension:
Hero (Superior/Positive): Morally good, ethical, justified, virtuous, protective.
Victim (Inferior/Positive): Harmed, oppressed, unfairly treated, vulnerable, deserving sympathy. Note that victims usually correspond to ingroup viewpoints.
Villain (Inferior/Negative): Morally bad, harmful, unethical, unjustified, threatening.
Intellectual dimension:
Sage (Superior/Positive): Knowledgeable, rational, intelligent, insightful, informed.
Fool (Inferior/Negative): Ignorant, irrational, uninformed, incompetent, mistaken.
Physical dimension:
Charmer (Superior/Positive): Attractive, appealing, admirable, charismatic.
Monster (Inferior/Negative): Repulsive, threatening, undesirable, dehumanized.
Power dimension:
Winner (Superior/Positive): Successful, dominant, influential, empowered, victorious.
Loser (Inferior/Negative): Defeated, powerless, unsuccessful, marginalized.
ANNOTATION RULES
Assign roles to social identities, not individuals.
An identity must have exactly 1 viewpoint (if provided) and can receive 1 or many narrative roles (e.g., ["Hero", "Winner"]).
Assign roles only when supported by the meme's text, imagery, or framing.
If an identity has no clear narrative role, return an empty array for roles.
Agreement metrics
Complete inter-rater agreement scores across all categories are summarized in Table 7. Ground-truth labels were established by retaining annotations agreed upon by at least two raters across 1,110 final memes (78.87% retention rate out of 1,410 total annotated; 30.85% perfect agreement, 48.01% majority consensus). Social identity emerged as the most reliably annotated category, exhibiting the highest Fleiss’ and Cohen’s . The close correspondence between these two measures demonstrates that agreement is consistent across all three raters, rather than being influenced by a specific rater pair. Viewpoint annotation yielded a higher Jaccard score and rate of perfect agreement, but lower values, indicating that raters generally identified overlapping viewpoints while showing less consistent agreement beyond exact matches. Agreement was weakest for narrative-role classification, suggesting that this category is more subjective or challenging to operationalize, likely due to the need for greater contextual interpretation. Nevertheless, values of approximately 0.51 reflect moderate inter-rater agreement. In summary, agreement decreases as the annotation task becomes more relational and compositional, with the lowest agreement observed for the combined relational identity, viewpoint, and role triples.
Table 7. Inter-rater agreement metrics across annotation categories.
Figure 6 shows changes in annotator agreement across successive annotation stages. Agreement generally increases through SET 6, coinciding with the period during which the annotation guidelines and codebook were iteratively refined. After SET 6, agreement largely stabilizes, with some fluctuations across subsequent stages. This pattern suggests that the refinement of the guidelines contributed to improved annotation consistency, while the subsequent stabilization indicates increasing convergence among annotators.
Template-based filtering
Only memes corresponding to established templates are retained to reduce ambiguity in communicative functions due to their structured and formalizable nature. The filtering pipeline is detailed in Figure 10.

Figure 10. Overview of the template-based meme curation pipeline.
Data creation A template dataset of 5,177 templates was constructed by scraping Know Your Meme (4,703) and Imgflip (474).
FAISS filtering 384-dimensional visual embeddings were generated for both memes and templates using a self-supervised DINO ViT-B/16 model, with all embeddings L2-normalized to measure similarity as cosine similarity. Exact nearest-neighbor matching was performed via FAISS [16], filtering matched memes with a top similarity score .
Cluster filtering Unmatched memes were evaluated for near-duplicates using a 64-bit difference hash (dHash) and grouped via agglomerative clustering with a distance threshold of 6.
Manual review Manual inspection removed unsuccessful matches, non-memes, non-template memes, and unsuitable content. This process yields a dataset of 1,410 memes.
Metrics
Let denote the total number of classes in the evaluated task, and let represent an individual class (social identity, viewpoint, or narrative role), where . For each class , TP, FP, and FN indicate the numbers of true-positive, false-positive, and false-negative predictions, respectively. Micro-averaged metrics aggregate the contributions of all classes prior to metric calculation, thereby giving greater influence to classes with more instances. In contrast, macro-averaged metrics are computed independently for each class and then averaged across all classes, assigning equal weight to each class regardless of its frequency. The corresponding precision, recall, and F1-scores are defined as follows:
Benchmark analysis
Table 7 shows the relative proportion (%) of identified social identity labels across the different source datasets on the x-axis. Each stacked bar illustrates the domain-specific representation of identities. As expected, climate activist and climate change skeptic/denier labels are more prominent in climate-focused sub-datasets (Clime, r/ClimateMemes), women in gender-focused sub-datasets (MAMI, r/TrollXChromosomes), and corporativist and republican in politically-oriented datasets (r/ConservativeMemes and Constraint22). Less expectedly, we observe a more balanced representation of women and men identities in r/TrollXChromosomes, focused on feminist memes, compared to the MAMI dataset, containing misogynistic memes, which contains more representations of women and fewer of men, with men serving as the primary ingroup. Similarly, MAMI contains a higher proportion of the feminist identity. Finally, FigMemes, a dataset focused on politically-incorrect content, contains a higher proportion of national identities than any of the other datasets.
Figure 8 presents the distribution of viewpoints (ingroup vs. outgroup) per social identity in the IRON-Meme dataset, clarifying which identities

Figure 8. Viewpoints for the 10 most common social identities.

Figure 7. Distribution of Social Identities Across Source Sub-Datasets.
are portrayed primarily from a sympathetic ingroup perspective versus a targeted outgroup one. The dataset exhibits a stark asymmetry across identity categories: over 80% of representations for women, corporativist/capitalist, democrat, feminist, climate change skeptic/denier, and climate disengaged reflect an outgroup viewpoint, indicating that these groups serve predominantly as targets of satire, hostility, or critique. Conversely, climate activists and men are framed overwhelmingly from an ingroup perspective, serving as the core self-representing figures and ideological voices within the corpus. Mid-spectrum identities like republican and national identities show a more balanced distribution between ingroup and outgroup portrayals, pointing to active cross-ideological debate surrounding these categories. Taken together, these patterns strongly suggest that the dataset’s posters are primarily male, climate-active, and anti-Democrat. Interestingly, while climate activists and climate change skeptics/deniers occupy opposing sides of the environmental debate, the dataset treats activists as the primary ingroup while framing skeptics almost exclusively as an outgroup; this indicates that climate-skeptical content in this dataset consists mostly of outgroup caricatures posted by climate activists rather than organic self-representation by skeptics.
Table 8 shows the distribution of ingroup versus outgroup representations in the final IRON-Meme dataset. Outgroup representations significantly outnumber ingroup ones across the corpus. This predominance of outgroups directly aligns with our earlier finding that negative narrative roles are far more prevalent than positive ones, further confirming that meme discourse in this dataset primarily functions as a medium for targeting, critiquing, or satirizing opposing groups.
| Category | Count | % |
| Out-group only | 650 | 58.56 |
| In-group only | 251 | 22.61 |
| Both in- and out-group | 206 | 18.56 |
| No valid viewpoint | 3 | 0.27 |
| Total | 1,110 | 100.00 |
Table 8. Distribution of viewpoint categories.
A key rationale for hypothesizing that template information influences narrative role classification is the observed predominance of specific role categories within certain templates. As shown in Figure 11, templates such as Distracted Boyfriend, Socially Awesome Awkward Penguin, Drakeposting, and Confession Bear exhibit a strong prevalence of villain and victim roles. The This Is Fine template is almost exclusively associated with the fool role, demonstrating how a template’s communicative context shapes role assignment. Formats including Steven Crowder’s "Change My Mind" Campus Sign, My Parents Are Dead / Batman Slapping Robin, and Expanding Brain display a substantial proportion of the sage role, which can be attributed to their conventional use in political discourse, often to present a controversial or assertive viewpoint. Furthermore, Figure 9 presents three examples in which annotators assigned the relevant identity the role “the sage”, suggesting that template conventions may provide cues relevant to narrative-role classification.

Figure 9. Examples of memes using the Steven Crowder’s "Change My Mind" Campus Sign template.
Experimental results
Model selection The evaluation includes both reasoning-oriented models (GLM-4.1V-9B-Thinking, Qwen3-VL-8B-Thinking) and instruction-oriented models (Qwen2.5-VL-7B-Instruct, Qwen3-VL-8B-Instruct) to assess whether explicit reasoning capabilities confer advantages on tasks involving socially and narratively complex visual content. Models with varying parameter scales are included, ranging from compact models
in the 7–12B range (Molmo-7B, Qwen2.5-VL-7B, Qwen3-VL-8B, GLM-4.1V-9B, Pixtral-12B) to the substantially larger Kimi-K3, which serves as a high-capacity reference point for evaluating the impact of model scale on benchmark performance. Multiple model families (DeepSeek, GLM, Kimi, Molmo, Pixtral, and Qwen) were represented to prevent results from being attributed to a single architecture or training pipeline. Additionally, the selection encompasses geographic diversity, with models developed by research organizations in East Asia (Kimi-K3, GLM-4.1V, Qwen2.5-VL, Qwen3-VL), Europe (Pixtral), and North America (Molmo), a particularly relevant dimension for a socially grounded benchmark, as differences in training data, alignment procedures, and development contexts may influence model behavior.
Ground truth aggregation
Table 10 illustrates three examples of how ground truth is obtained with different agreement types. Row 1 illustrates complete consensus across all three annotators. Row 2 demonstrates pairwise agreement; consensus labels across annotators 1–2 (democrat, outgroup, villain) and 1–3 (democrat, outgroup, loser) are merged via intersectional agreement into democrat, outgroup, villain/loser. Row 3 shows an instance of zero cross-annotator consensus; no ground truth can be established, so the meme is discarded.
Aditional results
Label frequency
Scatter plots in Figure 12 show the relationship between ground-truth label frequency and model performance, pooled across evaluated models. Each point represents a model–label pair for Social Identity and Narrative Role, and a model–image pair for Meme Template. Frequencies are shown on a logarithmic scale. Performance is measured using F1 for Social Identity and Narrative Role and per-image Jaccard similarity for Meme Template. Spearman’s and corresponding -values are reported for each dimension.
Error analysis
Social identity mismatches Figure 13 shows the top social identity mismatches across the 9 VLMs. The frequency of cases in which models fail to identify a single ground-truth identity and instead predict one incorrect identity is notable. VLMs frequently demonstrate extreme antagonist substitutions, such as predicting "misogynist/male supremacist" (190 cases) or "anti feminist" (87 cases) for "women," and "climate change skeptic/denier" (82 cases) for "climate activist." Related domain-level substitutions are also common, including predictions of "conservative" (28 cases) or "right wing" (25 cases) instead of "republican." Social identity errors for VLMs involve confusing a target identity with a closely related identity within the same domain, and appear to generally be more susceptible to hallucinations than to omissions.

Figure 13. Top social identity mismatches across 9 VLMs.
Viewpoint errors Figure 14 presents the viewpoint error matrices for Gemini 3.6 Flash and human evaluators. The analysis indicates that Gemini demonstrates a slight bias toward outgroup classification, misclassifying ingroups as outgroups in 15% of cases (8 instances out of 55 total ground-truth ingroups), compared to 4% in the reverse direction (6 instances out of 148 total ground-truth outgroups).

Figure 14. Viewpoint error matrices for VLM classification.
Human annotators exhibit a similar asymmetry, misclassifying ingroups as outgroups in 24% of cases (12 instances out of 51 total ground-truth ingroups) and outgroups as ingroups in 5% of cases (6 instances out of 116 total ground-truth outgroups). Considering the approximately 2.3–2.7:1 outgroup-to-ingroup imbalance in the dataset, these results suggest that both the model and human evaluators tend to default to the majority outgroup class, consistent with the findings of RQ3.
Narrative role errors Figure 15 presents the narrative role error matrices for Gemini 3.6 Flash and human evaluators. The narrative role error analysis for Gemini 3.6 Flash and human evaluators indicates a mismatch between two clusters: negatively connotated identities (villain, fool, loser, victim, monster) are predominantly confused with one another, and a similar pattern is observed among positively connotated identities (hero, sage, winner, charmer). For Gemini, the label fool functions as the primary catch-all for negative identities, frequently attracting misclassifications from ground truth (GT) villain (38

Figure 15. Narrative role error matrices for VLM classification.
Ablation study: template knowledge as information intervention
Table 12 presents three examples of narrative role classification, both with and without template information. These examples illustrate potential factors contributing to the observed decline in model performance when template information is included. In each instance, the model accurately predicts the roles without template information but fails when template descriptions are provided. In the first example, the template specifies that one character is slapped, which likely prompts the model to assign roles associated with inferiority and submission, specifically fool and loser. In the second example, the template suggests the meme pertains to either high or low intelligence; the model interprets this as low intelligence and assigns the role of fool. In the third example, the description indicates that the meme’s function is "giving advice," which the model appears to interpret as a sign of wisdom, and assigns the role of the sage, although the meme is actually demeaning toward the woman.
| Image | Template | Template Description | Identity | GT | w/o Template | w/ Template |
| !Batman Slapping Robin image | Batman Slapping Robin | Batman Slapping Robin, also known as “My Parents Are Dead,” is an exploitable image of the DC Comics superhero Batman slapping his pro-tégé Robin in mid-conversation. Taken from a comic book published in 1965, the single-panel illustration has inspired a series of parodies featuring custom-captioned speech bubbles on 4chan, Reddit and elsewhere. | climate activist | sage | sage | fool, loser |
| !IQ Bell Curve / Midwit image | IQ Bell Curve / Midwit | IQ Bell Curve, also known as Midwit or simply Bell Curve Meme, refers to a series of memes that use the IQ distribution diagrams to mock one of the three main groups represented by the diagram, usually the largest group of people towards the center of the curve who possess average intelligence. The concept of IQ Bell Curve memes utilizes the horseshoe theory, implying that the groups with low intelligence and high intelligence often choose to follow the same goals while being guided by different reasoning. Those who possess average intelligence are referred to as “midwits.” While there were many versions of memes illustrating the concept, a variant of the meme using three different Wojaks gained particular popularity in the 2020s on X / Twitter and Reddit. | climate activist | sage | sage | fool |
| !Actual Sexual Advice Girl image | Actual Sexual Advice Girl | Actual Sexual Advice Girl (ASAG) is an advice animal image macro series featuring a photograph of a brunette woman wearing a pair of black-rimmed glasses accompanied by captions relaying sexual advice to both men and women. | women | loser | loser | sage |
Table 12. Qualitative comparison of model predictions with (w/) and without (w/o) template descriptions in the prompt.
Recruitment and payment
Three students were recruited as annotators through a recruitment platform and completed the hiring process at the academic institution, providing consent to the working conditions. They were employed for three months, working 8 to 12 hours per week at a rate of 20€ per hour. Each student received 2400€ at the end of the contract.
C.1
References
- [1]Shawly Ahsan, Efte khar Hossain, Omar Sharif, Avishek Das, Mohammed Moshiul Hoque, and M. Dewan. 2024. A Multimodal Framework to Detect Target Aware Aggression in Memes. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2487–2500, St. Julian’s, Malta. Association for Computational Linguistics.
- [2]Mohammad Awad AlAfnan. 2025. The Role of Memes in Shaping Political Discourse on Social Media. Studies in Media and Communication.DOI
- [3]Glenn Anderau and Daniel Barbarussa. 2024. The Function of Memes in Political Discourse. Topoi, 43(5):1529–1546.DOI
- [4]Maja Brandt Andreasen. 2023. A monster, a pervert, and an anti-hero: the discursive construction of Harvey Weinstein, Kevin Spacey, and Louis C.K. in humorous #MeToo memes. Feminist Media Studies.DOI
- [5]Lasse Balleby, Claus Toft-Nielsen, and Tina Thode Hougaard. 2026. Digital intimacy: how everyday meme sharing shapes identity and relationships. Popular Communication, 0(0):1–18.DOI
- [6]Alberto Barrón-Cedeño, Firoj Alam, Julia Maria Struß, Preslav Nakov, Tammoy Chakraborty, Tamer Elsay ed, Piotr Przbyła, Tommaso Caselli, Giovanni Da San Martino, Fatima Haouari, Maram Hasanain, Chengkai Li, Jakub Piskorski, Federico Ruggeri, Xingyi Song, and Reem Suwaileh. 2024. Overview of the CLEF-2024 CheckThat! Lab: Check-Worthiness, Subjectivity, Persuasion, Roles, Authorities, and Adversarial Robustness. In Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 28–52, Cham. Springer Nature Switzerland.
- [7]Valerio Basile, Federico Cabitza, Andrea Campagner, and Michael Fell. 2023. Toward a Perspectivist Turn in Ground Truthing for Predictive Computing. Proceedings of the AAAI Conference on Artificial Intelligence, 37(6):6860–6868. ArXiv:2109.04270 [cs.LG].DOI
- [8]Kelly Bergstrand and James M. Jasper. 2018. Villains, Victims, and Heroes in Character Theory and Affect Control Theory. Social Psychology Quarterly, 81(3):228–247.DOI
- [9]Gabriella Bollici, Wouter van Atteveldt, and Filip Ilievski. 2025. Political Memes as Fragmented Narratives: An Interdisciplinary Framework for Meme Analysis.DOI
- [10]Abhilekh Borah, Hasnat Md Abdullah, Kangda Wei, and Ruihong Huang. 2025. CliME: Evaluating Multimodal Climate Discourse on Social Media and the Climate Alignment Quotient (CAQ). In Proceedings of the Fourth Workshop on NLP for Positive Impact (NLP4PI), pages 43–61, Vienna, Austria. Association for Computational Linguistics.arxiv.org/abs/2504.03906
- [12]Mithun Das and Animesh Mukherjee. 2023. BanglaAbuseMeme: A Dataset for Bengali Abusive Meme Classification. arXiv preprint. ArXiv:2310.11748 [cs.CV].arxiv.org/abs/2310.11748
- [13]Richard Dawkins. 2006. The selfish gene, 30th anniversary ed edition. Oxford university press, Oxford.
- [14]Constance de Saint Laurent, Vlad P. Glăveanu, and Ioana Literat. 2021. Internet Memes as Partial Stories: Identifying Political Narratives in Coronavirus Memes. Social Media + Society, 7(1):2056305121988932.DOI
- [15]Julia R. DeCook. 2018. Memes and symbolic violence: #proudboys and the use of memes for propaganda and the construction of collective identity. Learning, Media and Technology, 43(4):485–504. _eprint: https://doi.org/10.1080/17439884.2018.1544149.
- [16]Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Silvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2025. The Faiss library. arXiv preprint. ArXiv:2401.08281 [cs.LG].arxiv.org/abs/2401.08281
- [17]Elisabetta Fersini, Francesca Gasparini, Giulia Rizzi, Aurora Saibene, Berta Chulvi, Paolo Rosso, Alyssa Lees, and Jeffrey Sorensen. 2022. SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 533–549, Seattle, United States. Association for Computational Linguistics.
- [18]Anastasiya Fiadotava. 2023. Heroes and Villains in Memes on the 2022 Russian Invasion of Ukraine. Electronic Journal of Folklore.DOI
- [19]Noam Gal, Limor Shifman, and Zohar Kampf. 2016. “It Gets Better”: Internet memes and the construction of collective identity. New Media & Society, 18(8):1698–1714.DOI
- [20]Emily Godwin, Brittany I. Davidson, Tim Hill, and Adam Joinson. 2025. Internet Memes as Stabilizers of Conspiracy Culture: A Cognitive Anthropological Analysis. Social Media + Society, 11(1):20563051214306421.DOI
- [21]Lars Guenther, Georg Ruhrmann, Jenny Bischoff, Tessa Penzel, and Antonia Weber. 2020. Strategic Framing and Social Media Engagement: Analyzing Memes Posted by the German Identitarian Movement on Facebook. Social Media + Society, 6(1):2056305119898777.DOI
- [23]Theresa Henn and Oliver Posegga. 2024. Peeking behind the Memes: Evaluating the Boundary Work of Online Communities through Shared Memes. ICIS 2024 Proceedings.
- [24]Eftekhar Hossain, Omar Sharif, Mohammed Moshfiul Hoque, and Sarah M. Preum. 2024. Deciphering Hate: Identifying Hateful Memes and Their Targets. arXiv preprint. ArXiv:2403.10829 [cs].arxiv.org/abs/2403.10829
- [25]Wenqing Hou, Hongkui Tu, Ye Wang, Yue Zhang, Yuying Liu, Dong Zhu, Liqun Gao, and Bin Zhou. 2026. Beyond Single-View Detection: A Dual-Space Reasoning Framework for Interpretable Harmful Meme Understanding. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10526–10544, San Diego, California, United States. Association for Computational Linguistics.
- [26]Kyra Hunting. 2020. The Role of Popular Media in 2016 US Presidential Election Memes. Journalism and Media Faculty Publications.DOI
- [27]Outi Immonen. 2017. Losers and Superior Overlords : Humor Strategies in Nerdy Internet Memes. Ph.D. thesis, University of Helsinki Open Repository.
- [28]Michael Johann, Lukas Höhnle, and Jana Dombrowski. 2023. Fridays for Future and Mondays for Memes: How Climate Crisis Memes Mobilize Social Media Users. Media and Communication, 11(3):226–237.DOI
- [29]Saurav Joshi, Filip Ilievski, and Luca Luceri. 2024. Contextualizing Internet Memes Across Social Media Platforms. In Companion Proceedings of the ACM Web Conference 2024, WWW ’24, pages 1831–1840, New York, NY, USA. Association for Computing Machinery.arxiv.org/abs/2311.11157
- [30]Stephen Karpman. 1968. Fairy Tales and Script Drama Analysis. Transactional Analysis Bulletin, 7.
- [31]Ludovic Kun, Jayesh Bankoti, and David Kiskovski. 2022. Logically at the Constraint 2022: Multimodal role labelling. In Proceedings of the Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situations, pages 24–34, Dublin, Ireland. Association for Computational Linguistics.
- [32]Benjamin Litherland and Rachel Wood. 2026. Memeing while the world burns: Climate change news on Reddit and the cultural politics of platform participation. European Journal of Cultural Studies, 29(1):251–268.DOI
- [33]Chen Liu, Gregor Geigle, Robin Krebs, and Iryna Gurevych. 2022. FigMemes: A Dataset for Figurative Language Identification in Politically-Opinionated Memes. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7069–7086, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.DOI
- [34]Shana MacDonald. 2025. The Art of Memes in Feminist Digital Culture. Digital Media, Feminist Resistance. Ohio State University Press, The Ohio State University Press.DOI
- [35]Tarek Mahmoud, Zhuohan Xie, Dimitar Dimitrov, Nikolaos Nikolaidis, Purificação Silvano, Roman Yangarber, Shivam Sharma, Elisa Sartori, Nicolas Stefanovitch, Giovanni Da San Martino, Jakub Piskorski, and Preslav Nakov. 2025. Entity Framing and Role Portrayal in the News. arXiv preprint. ArXiv:2502.14718 [cs].
- [37]Ryan M Milner. 2013. FCJ-156 Hacking the Social: Internet Memes, Identity Antagonism, and the Logic of Lulz. The Fibreculture Journal.
- [38]Mia Moody-Ramirez, Brianna P. Lemmons, and Stephanie Clintonia Boddie. 2024. Black Fathers Matter: Messages of Hope in Black Father-Themed Instagram Memes. Journal of Black Studies, 55(2):183–209.
- [39]Alina Mozolevska. 2024. Unveiling the War and Constructing Identities: Exploring Memes in Ukrainian and Russian Social Media during the Russian Invasion of Ukraine. Czech Journal of International Relations, 59(2):7–37.DOI
- [40]Elizabeth Nixon. 2020. The fool, the hero, and the sage: narratives of non-consumption as role distance from an urban consumer-self. Consumption Markets & Culture, 23(1):44–60. _eprint: https://doi.org/10.1080/10253866.2018.1467317.
- [41]María Antonia Paz, Ana Mayagoitia-Soria, and Juan-Manuel González-Aguilar. 2021. From Polarization to Hate: Portrait of the Spanish Political Meme. Social Media + Society, 7(4):20563051211062920.
- [42]Jia Wang Peh, Ming Shan Hee, Bryan (Chen Zhengyu) Tan, Yuriel Wang Jun Long Ryan, and Roy Ka-Wei Lee. 2026. Large Scale Narrative Analysis of Multimodal Memes. Proceedings of the International AAAI Conference on Web and Social Media, 20(1):1767–1796.DOI
- [43]Chiara Polli. 2025. A multimodal critical perspective on the challenges of automatic detection of hate speech in Internet memes: The case of simianization. Critical Approaches to Discourse Analysis Across Disciplines, 17(1):96.DOI
- [44]Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021. MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets. arXiv preprint. ArXiv:2109.05184 [cs].arxiv.org/abs/2109.05184
- [45]Ondřej Procházka. 2016. Cohesive Aspects of Humor in Internet Memes on Facebook: a Multimodal Sociolinguistic Analysis. Ostrava Journal of English Philology, 8:7–38.
- [46]Shivam Sharma, Siddhant Agarwal, Tharun Suresh, Preslav Nakov, Md Akhtar, and Tanmoy Chakraborty. 2023a. What Do You MEME? Generating Explanations for Visual Semantic Role Labelling in Memes. Proceedings of the AAAI Conference on Artificial Intelligence, 37:9763–9771.
- [47]Shivam Sharma, Md Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2022a. DISARM: Detecting the Victims Targeted by Harmful Memes. arXiv preprint. ArXiv:2205.05738 [cs].arxiv.org/abs/2205.05738
- [48]Shivam Sharma and Tanmoy Chakraborty. 2025. Decoding Memes: Benchmarking Narrative Role Classification across Multilingual and Multimodal Models. arXiv preprint. ArXiv:2506.23122 [cs.CL].arxiv.org/abs/2506.23122
- [49]Shivam Sharma, Atharva Kulkarni, Tharun Suresh, Himanshi Mathur, Preslav Nakov, Md. Shad Akhtar, and Tanmoy Chakraborty. 2023b. Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim? In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2149–2163, Dubrovnik, Croatia. Association for Computational Linguistics.arxiv.org/abs/2301.11219
- [50]Shivam Sharma, Tharun Suresh, Atharva Kulkarni, Himanshi Mathur, Preslav Nakov, Md. Shad Akhtar, and Tanmoy Chakraborty. 2022b. Findings of the CONSTRAINT 2022 Shared Task on Detecting the Hero, the Villain, and the Victim in Memes. In Proceedings of the Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situations, pages 1–11, Dublin, Ireland. Association for Computational Linguistics.
- [51]Limor Shifman. 2013. Memes in a Digital World: Reconciling with a Conceptual Troublemaker. Journal of Computer-Mediated Communication, 18(3):362–377.DOI
- [53]Pranaydeep Singh, Aaron Maladry, and Els Lefever. 2022. Combining Language Models and Linguistic Information to Label Entities in Memes. In Proceedings of the Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situations, pages 35–42, Dublin, Ireland. Association for Computational Linguistics.
- [54]Inna Skrynnikova, T. Astafurova, and Nadezhda Sytina. 2017. Power of metaphor: cultural narratives in political persuasion. Atlantis Press.DOI
- [55]Shaun Toh, Adriel Kuek, Wen-Haw Chong, and Roy Ka-Wei Lee. 2023. MERMAID: A Dataset and Framework for Multimodal Meme Semantic Understanding. In 2023 IEEE International Conference on Big Data (BigData), pages 433–442.DOI
- [56]Sarah J. Tracy. 2018. A Phonetic Iterative Approach to Data Analysis in Qualitative Research. Journal of Qualitative Research, 19(2):61–76.DOI
- [57]Bradley E. Wiggins. 2019. The Discursive Power of Memes in Digital Culture: Ideology, Semiotics, and Intertextuality. Routledge, New York.
- [58]Yi Zhou, Zhenhao Chen, and Huiyuan Yang. 2021. Multimodal Learning For Hateful Meme Detection. In 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pages 1–6.arxiv.org/abs/2011.12870