betaFeedback

WorksComputer Science

GPromptShield: Elevating Resilience in Graph Prompt Tuning against Adversarial Attacks

Abstract

The paradigm of ``pre-training and prompt-tuning", with its effectiveness and lightweight characteristics, has rapidly spread from the language field to the graph field. Several pioneering studies have designed specialized prompt functions for diverse downstream graph tasks based on various graph pre-training strategies. These prompts concentrate on the compatibility between the pre-training pretext and downstream graph tasks, aiming to bridge the gap between them. However, designing prompts blindly to adapt to downstream tasks based on this concept neglects crucial security issues. By conducting covert attacks on downstream graph data, we find that even when the downstream task data closely matches that of the pre-training tasks, it is still feasible to generate highly misleading prompts using simple deceptive techniques. In this paper, we shift the primary focus of graph prompts from compatibility to vulnerability issues in adversarial attack scenarios. We design a highly extensible shield defense system for the prompts, which enhances their robustness from two perspectives: Direct Handling and Indirect Amplification. When downstream graph data contains unreliable biases, the former directly combats invalid information by incorporating hybrid multi-defense prompts to the input graph's feature space, while the latter adopts a training strategy to bypass the invalid components and amplifies valid part. We provide a theoretical derivation that proves their feasibility, indicating that unbiased prompts exist under certain conditions on unreliable data. Extensive experiments across various scenarios of adversarial attacks indicate that the prompts within our defense system exhibit enhanced resilience and superiority. This paper explores a new perspective in graph prompt learning, offering a novel option for robust prompt tuning in downstream tasks.

Introduction

Graph neural networks (GNNs) have demonstrated impressive performance across various applications due to their unique ability to handle complex and irregular data, such as recommendation systems ([19]), traffic prediction ([26]), and social computing ([30]). With the advancement of society, real-world scenarios typically align with three characteristics: abundant data, sparse labels, and diverse task domains. Therefore, the algorithm paradigm has shifted from designing specific models for particular problems to training general models that can be fine-tuned for downstream tasks—known as “pre-training and fine-tuning”. This paradigm effectively maximizes the benefits of the large volume of data and has led to the emergence of many outstanding works ([44, 43, 33, 10, 11]).

As pre-trained models in the language field have become increasingly powerful, prompt-based tuning has gradually emerged as a new research focus. Unlike fine-tuning which adjusts the parameters of pre-trained models, prompt-based methods freeze the parameters of the pre-trained model and focus on adjusting the data space through input transformation. These transformation operations are termed prompts, whose objective is to narrow the gap between the pretext of pre-training and the objectives of various downstream tasks, while avoiding the computational cost of retraining the model from scratch. Therefore, “Pre-training, prompting, and fine-tuning” has become the new paradigm.

This new paradigm has rapidly spread to the graph field due to its popularity. Although applying prompt-based tuning strategies on GNN models poses greater challenges compared to language prompts, many pioneering studies {(citep)}(Sun et al., 2023a; Liu et al., 2023b; Fang et al., 2024; Yu et al., 2024) have attempted to propose viable prompts from different perspectives. These studies have made outstanding contributions and they share a commonality in their concentrate on the compatibility between the pre-training pretext and downstream graph tasks, aiming to bridge the gap between them. However, designing prompts blindly to adapt to downstream tasks based on this compatibility neglects crucial security issues. As shown in Figure 1, effective prompts often provide strong support for downstream tasks, but making certain interventions during the prompt tuning phase, such as simple perturbations to the graph structure, can render the prompts ineffective and lead to catastrophic consequences.

Illustration of valid and invalid prompt tuning under structural perturbations

Figure 1. The prompts can be easily misled by simple deceptive tactics. The red edges represent the edges changed by structural perturbations.

To illustrate this phenomenon more intuitively, we conduct an interesting experiment. For node classification tasks, we employ the well-known MetaAttack {(citep)}(Zügner & Günnemann, 2019) to attack the downstream graph data, and we present the classification results of existing mainstream prompt functions on clean and attacked graphs in Figure 3.

Figure 3. The performance of existing prompt functions on clean graphs and graphs attacked by MetaAttack with a 20% attack ratio. The scale of the radar chart represents classification accuracy.

we find that even when the downstream data closely matches that of the pre-training tasks, it is still feasible to generate highly misleading prompts using simple deceptive techniques. Therefore, this inspires us to consider the safety issues of graph prompts.

A recent study [42] has proposed PromptBench for language prompts, as shown in Figure 2. For instance, there is a clean prompt “As a mathematics instructor, calculate the answer to the following problem related to {}”, The pre-trained model can adapt to downstream tasks based on its key terms. However, adversarial prompts can cause the pre-trained model to focus on both the target text and the adversarial content, amplifying its sensitivity to adversarial perturbations. In word-level attacks like DeepWordBug, introducing typos or altering specific words diverts the model’s attention from key terms. In sentence-level attacks like StressTest and CheckList, attackers attempt to distract the pre-trained model by adding irrelevant or unrelated sentences to the input text, which may cause it to lose focus on the primary context. The robustness of language prompts has led us to examine the robustness of graph prompts, and we aim to equip graph prompts with certain adversarial perturbation capabilities.

Adversarial language prompts generated by different attacks

Figure 2. The adversarial language prompts generated by different attacks inspire us to design graph prompts tailored to various node scenarios.

Challenges. However, designing a robust graph prompt is a significant challenge, as graph prompts are inherently more fragile than language prompts for two main reasons. First, the cost of perturbing a graph is very low and the implementation is straightforward. Just simply adding a few irrelevant edges to the graph structure can lead to catastrophic changes due to the message-passing mechanism, reminiscent of the saying “a single spark can start a prairie fire”. Second, the design of graph prompts is complex and specialized. Prompts that are suited for a specific graph property or structure can quickly become ineffective due to even slight alterations.

Contributions. To smoothly tackle these two challenges, we design a highly extensible shield defense system for the prompts, which enhances their robustness from two perspectives: Direct Handling and Indirect Amplification. When downstream graph data contains unreliable biases, the former directly combats invalid information by incorporating hybrid multi-defense prompts to the input graph’s feature space. We do not adopt a universal learnable prompt for all node features like GPF [4], as we analyzed above, a prompt that works for one node may not be suitable for all nodes. Instead, our hybrid multi-defense prompts will customize a individual prompt for each node according to the diverse sensitive information associated with different perturbations. When attacks cause biased node distributions, customized prompts can provide context-specific guidance tailored to nodes in different situations. While the latter adopts a training strategy to bypass the invalid components and amplifies valid part. We do not need to design new prompts. Instead, we can directly build upon existing prompts by providing a supplementary tool to address their overlooked safety issues. To ensure the effectiveness of our proposed strategies, we provide a theoretical analysis demonstrating that when the downstream graph data is unreliable, there exists at least one viable prompt capable of achieving results comparable to those on clean graph under certain conditions.

Overall, the contributions of this paper can be summarized as follows:

  • We have opened up a new perspective on graph prompt tuning focused on robustness, we shift the primary focus of graph prompts from compatibility to vulnerability issues in adversarial attack scenarios.

  • We propose a highly extensible shield defense system for the prompts. This system includes a hybrid multi-defense prompt and a robust prompt tuning strategy, and we provide theoretical proof of its feasibility.

  • We conduct extensive experiments in few-shot scenarios under various adversarial attacks (including adaptive and non-adaptive attacks). The results indicate that our hybrid multi-defense prompt and robust prompt tuning strategy significantly enhance the resilience of prompt tuning in downstream biased tasks.

Related Work

This paper primarily focuses on advancements in the field of prompt tuning. Additionally, We also provide a systematic description of graph pre-training strategies in Appendix § 10.§ 1.

Graph Prompt Tuning

Prompt-based methods freeze the parameters of the pre-trained model and focus on adjusting the data space through input transformation. Due to the parameter efficiency of prompts, they have been widely used in the language domain [18, 7, 22] (Sivarakumaret al. 2024), but they are still in the early stages in the graph domain. However, some pioneering studies have already been conducted to explore this area. GPPT [28] applies the pairwise token template (the task token for downstream problem and the structure token containing the node information) to modify nodes. All-in-one [29] reformulates node-level and edge-level tasks as graph-level tasks, introducing meta-learning technique to learn a prompt. GPF/GPF-plus [4] proposes a universal prompt tuning method by introducing additional learnable parameter as a prompt in the feature space of the input graph. To achieve effective knowledge transfer from pre-training to a downstream task, GraphPrompt [20] proposes a unified framework based on subgraph similarity, aiming to retain graph properties that are compatible with the given task during pre-training. MultiGPrompt [40] argues that a single pretext task is not sufficient. Therefore, it designs a series of pretext tokens to collaboratively address different pretext tasks during pre-training. It also introduces a dual-prompt mechanism that uses both a composed prompt and an open prompt to leverage task-specific and global pre-training knowledge. However, all of these prompts focus on the compatibility between pre-training and downstream tasks, neglecting potential security issues.

Preliminaries

In this paper, we primarily discuss the robustness of graph prompt tuning in downstream node classification tasks, as most mainstream attacks are centered around this scenario. We do not focus on the compatibility between upstream and downstream tasks, as existing studies have already provided excellent discussions on this aspect. Instead, we place greater emphasis on the resilience of prompts when faced with biased data.

Notations. Define an undirected, unweighted graph G=(V,E)\mathcal{G} = (\mathcal{V}, \mathcal{E}) with NN nodes, N=∣V∣N = |\mathcal{V}|. V={v1,v2,⋯ ,vN}\mathcal{V} = \{v_1, v_2, \cdots, v_N\} and E⊆V×V\mathcal{E} \subseteq\mathcal{V} \times\mathcal{V} represent the set of nodes and edges, respectively. Its feature matrix X=[x1,x2,⋯ ,xn]∈RN×din\mathcal{X} = [x_1, x_2, \cdots, x_n] \in\mathbb{R}^{N \times d_{in}}, where xnx_n is a dind_{in}-dimensional feature vector of the nn-th node. A∈{0,1}N×N\mathcal{A} \in\{0,1\}^{N \times N} is the symmetric adjacency matrix where Aij=1A_{ij} = 1 if (vi,vi)∈E(v_i, v_i) \in\mathcal{E}. Moreover, the labels of all nodes are denoted as y\mathbf{y}. Each node is associated with a label yi∈Cy_i \in\mathcal{C}, where C={c1,c2,⋯ ,cK}\mathcal{C} = \{c_1, c_2, \cdots, c_K\}.

Fine-Tuning. Define a pre-trained GNN model ff, a learnable projection head θ\theta. Next, we reformulate the node task as a subgraph classification task and define a downstream task dataset D={(Sx1,y1),⋯ ,(Sxm,ym)}D = \{(S_{x_1}, y_1), \cdots, (S_{x_m}, y_m)\}, where Sxi=(Si,XnSi)S_{x_i} = (S_i, X_{n_{S_i}}) is the multi-hop neighbor subgraph of node ii extracted from G\mathcal{G}. SiS_i is the subgraph, nSin_{S_i} is the set of nodes contained in SiS_i. XnSiX_{n_{S_i}} represents the features of the contained nodes. We adjust the parameters of the pre-trained model ff and the projection head θ\theta to maximize the likelihood of predicting the correct labels yiy_i of the downstream local subgraph SxiS_{x_i}. Fine-Tuning aims to maximize the classification likelihoods for nodes in the graph, which can be expressed as follows:

max⁡f,θ∑i=1npf,θ(yi∣XnSi,Si)(1)\max_{f,\theta} \sum_{i=1}^{n} p_{f,\theta}(y_i \mid X_{n_{S_i}}, S_i) \tag*{(1)}

Prompt-Tuning. In prompt-tuning for downstream node classification tasks, the parameters of the pre-trained model ff are frozen, and instead introduces a lightweight graph prompt function ψ\psi. ψ\psi can be attached in the form of structure or features, transforming the input subgraph into a prompt subgraph for pre-trained model ff's input. The prompt subgraph can be expressed as follows:

Sxi∗=(Si∗,XnSi∗)=ψ(Sxi)(2)S_{x_i}^{*} = (S_i^{*}, X_{n_{S_i}^{*}}) = \psi(S_{x_i}) \tag*{(2)}

Therefore, the process of prompt-tuning can be described as:

max⁡ψ,θ∑i=1npf,θ(yi∣Sxi∗)(3)\max_{\psi,\theta} \sum_{i=1}^{n} p_{f,\theta}(y_i \mid S_{x_i}^{*}) \tag*{(3)}

In the evaluation phase after prompt-tuning, by adding a prompt ψ\psi to the subgraph of a test node, the frozen model ff can process it directly.

Effectiveness Analysis. This study explores prompt effectiveness from the perspective of data distribution. Following [16], define S^xi∗\hat{S}_{x_i}^{*} as a prompt subgraph embedding output. Assume subgraph embeddings follow p(S^xi∗∣y)p(\hat{S}_{x_i}^{*}\mid y) and are sampled from the joint distribution p(S^xi∗,y)p(\hat{S}_{x_i}^{*}, y). Existing prompts seek general templates. Their effectiveness relies on unbiased data in downstream tasks (ptrain(S^xi∗,y)=ptest(S^xi∗,y)=ptrue(S^xi∗,y)p_{train}(\hat{S}_{x_i}^{*}, y) = p_{test}(\hat{S}_{x_i}^{*}, y) = p_{true}(\hat{S}_{x_i}^{*}, y)), as analyzed in [16]. With consistent distributions, prompts work well even with few samples. But attacks cause distribution deviations (ptrain(S^xi∗,y)≠ptest(S^xi∗,y)≠ptrue(S^xi∗,y)p_{train}(\hat{S}_{x_i}^{*}, y) \ne p_{test}(\hat{S}_{x_i}^{*}, y) \ne p_{true}(\hat{S}_{x_i}^{*}, y)), leading to overfitting to erroneous data. Despite fixed pre-trained model parameters, misleading prompt harm is underestimated. As p(S^xi∗,y)=p(y)p(S^xi∗∣y)p(\hat{S}_{x_i}^{*}, y) = p(y)p(\hat{S}_{x_i}^{*}\mid y) and p(y)p(y) is consistent, inconsistent p(S^xi∗∣y)p(\hat{S}_{x_i}^{*}\mid y) causes differences. Therefore, when the prompts obtained through prompt tuning with few-shot samples on the training data are added to the test data based on the principle of distribution consistency, it will result in catastrophic performance. This phenomenon is well reflected in Figure 3.

Attacks. In this paper, we mainly adopt two attack scenarios: the commonly used gray-box global poisoning attack and the white-box adaptive attack. We focus on attacks against the graph structure. In the setting of the former scenario, attackers have visibility into the graph data and labels but lacks visibility into the details of the model. MetaAttack [47] as a classic gray-box poisoning attack, utilizes a surrogate model for the attack, which can be formulated mathematically as a bilevel optimization problem:

min⁡G^∈Φ(G)Latk(fθ∗(G^))s.t.θ∗=arg⁡min⁡θ Ltrain(fθ(G^))(4)\min_{\hat{G}\in\Phi(G)} \mathcal{L}_{atk}(f_{\theta^{*}}(\hat{G})) \quad\text{s.t.}\quad\theta^{*} = \underset{\theta}{\arg\min}\ \mathcal{L}_{train}(f_{\theta}(\hat{G})) \tag*{(4)}

Φ(G)\Phi(G) represents a set of graphs that satisfies the disturbance budget constraint Δ\Delta. Δ\Delta indicates a limit on the number of changes ∥A−A^∥≤Δ\lVert A - \hat{A}\rVert\le\Delta. Latk\mathcal{L}_{atk} is the attack loss function, could be −Ltrain-\mathcal{L}_{train} or −Lself-\mathcal{L}_{self}.

Adaptive attacks [6] are a type of white-box attack with stronger capabilities, where the attacker possesses complete information, including the defender’s model features, graph structure, labels, and all details. [6] categorizes defenses into seven types and designs adaptive attacks for the most representative method in each category.

Problem Statement. This paper explores the robustness of prompt strategies in node classification tasks. To mimic real-world situations, we pretrain a GNN model on clean graphs and freeze its parameters. Our target is to design a robust prompt and optimization strategy. When the downstream graph data is biased by attacks, our prompt strategy leverages few-shot samples for robust tuning, improving the classification accuracy of unlabelled nodes.

Methods

In this section, we will introduce a highly extensible shield defense system designed for the prompts, which enhances their robustness from two perspectives: Direct Handling and Indirect Amplification. They provide viable solutions for enhancing the resilience of prompts at two different stages: robust prompt design and robust optimization strategy.

Direct Handling

We have previously analyzed that when the data distribution is biased, prompts that are effective for certain nodes may mislead others. Therefore, when the downstream graph is attacked, we do not use a universal prompt like GPF. Instead, we designed a hybrid multi-defense prompt. Since we cannot know in advance the tactics employed by attackers, we customize a specific prompt for each node’s situation by organizing the commonalities and analyzing the focal points of different attacks.

Downstream unbiased subgraphs that share consistent properties with pre-training data often do not require complex prompts. A typical example is subgraphs with high homophily similar to the pre-training data, which inherently exhibit strong compatibility. The nodes that truly require focused prompting are the biased ones resulting from attacks. For these biased nodes, adding prompts identical to those of unbiased nodes is obviously irrational.

We aim for the prompts on these biased nodes capable of mitigating the impact of biased information. Therefore, we propose several different node filtering strategies based on various sensitive points that the attacks may target. These strategies identify potential biased nodes with different properties and apply attribute-specific prompts to each of them. For a perturbed subgraph S~xi=(S~i,XNSi)\widetilde{\mathcal{S}}_{x_i} = (\widetilde{\mathcal{S}}_i, X_{\mathcal{N}_{\mathcal{S}_i}}) in a few-shot training set, we will next demonstrate how to add a hybrid multi-defense prompt to it.

Filtering Tip 1: Degree Some indiscriminate attacks (such as DICE [35] and Random attack) apply the same attack probability to all nodes. In this context, nodes with higher degrees tend to exhibit more stable community features and maintain good consistency with the pre-training data. Under the same attack cost, low-degree nodes are more likely to generate biased community features. Therefore, we filter out the set of nodes with low degree, denoted as nld={u∣∣Nu∣<τdegree}n_{ld} = \{u \mid|\mathcal{N}_u| < \tau_{degree}\}, where Nu\mathcal{N}_u is uu’s neighbors and τdegree\tau_{degree} is the degree threshold. We provide an auxiliary degree defense prompt pd∈Rdinp_d \in\mathbb{R}^{d_{in}} applied to their feature space, where dind_{in} is the dimensionality of the node features. this process can be expressed as:

Xn~S~i∗[nld]=Xn~S~i[nld]+pd(5)X^{*}_{\widetilde{n}_{\widetilde{\mathcal{S}}_i}}[n_{ld}] = X_{\widetilde{n}_{\widetilde{\mathcal{S}}_i}}[n_{ld}] + p_d \tag*{(5)}

Filtering Tip 2: Node Centrality Similarity. Most mainstream attacks aim to disrupt the homophily assumption in graphs, where nodes with the same label and similar features are often connected. They utilize low-cost, high-reward tactics to increase the graph’s heterophily as much as possible. This change in heterophily is typically reflected in the central similarity of nodes, where the attacked biased nodes connect with nodes that have different labels and features, thus confusing the assessment of their own community characteristics. Therefore, we filter out the set of nodes with low central similarity, denoted as nls={u∣1∣Nu∣∑v∈Nusim(Xu,Xv)<τsim}n_{ls} = \left\{u \mid\frac{1}{|\mathcal{N}_u|}\sum_{v \in\mathcal{N}_u} sim(X_u, X_v) < \tau_{sim}\right\}, where τsim\tau_{sim} is the similarity threshold. We provide an auxiliary similarity defense prompt ps∈Rdinp_s \in\mathbb{R}^{d_{in}} applied to their feature space. This process can be expressed as:

Xn~S~i∗[nls]=Xn~S~i[nls]+ps(6)X^{*}_{\widetilde{n}_{\widetilde{\mathcal{S}}_i}}[n_{ls}] = X_{\widetilde{n}_{\widetilde{\mathcal{S}}_i}}[n_{ls}] + p_s \tag*{(6)}

Filtering Tip 3: Out-of-distribution nodes. (Li et al.) proposes a novel adversarial training paradigm that generates perturbations through adversarial attacks during training, using adversarial edges as out-of-distribution samples and initial edges as in-distribution samples to train multiple detectors fD={fD1,fD2,⋯ }f_D = \{f_D^1, f_D^2, \cdots\}. Using this detector, we can identify out-of-distribution edges generated by attacks when faced with a biased graph. More detailed description is provided in Appendix § 10.§ 2. By detecting these edges, the nodes at both ends of the edges become the focal points of the attack method. We add prompts to these nodes and denote them as nod={u∣∃e∈E,fD(e)=1,e∈E,u∈e}n_{od} = \{u \mid\exists e \in E, f_D(e) = 1, e \in\mathcal{E}, u \in e\}. We provide an auxiliary out-of-distribution defense prompt po∈Rdinp_o \in\mathbb{R}^{d_{in}} applied to their feature space. This process can be expressed as:

Xn~S~i∗[nod]=Xn~S~i[nod]+po(7)X^{*}_{\widetilde{n}_{\widetilde{\mathcal{S}}_i}}[n_{od}] = X_{\widetilde{n}_{\widetilde{\mathcal{S}}_i}}[n_{od}] + p_o \tag*{(7)}

In this paper, we initially propose above-mentioned three feasible solutions. As attacks continue to evolve, node sets can be selected based on a wider range of sensitive properties. After obtaining different sets of vulnerable nodes, the prompt selection for a node is actually included in set {(),(pd,ps,po),(pd,ps),(po),⋯ }\{(),(p_d,p_s,p_o),(p_d,p_s),(p_o),\cdots\}. In the experiments, we also design an enhanced prompt pe∈Rdinp_e \in\mathbb{R}^{d_{in}} for nodes that do not have any sensitive prompts added.

Since each node has a different number of prompts, integrating all the prompts that belong to each node on graph is a challenge. Defining a node’s prompt set as PSiPS_i, we propose two fusion strategies: one is the weighted average MeanMean, and the other is the self-attention mechanism SASA [32] used in language models. Therefore, a node’s hybrid muti-defense prompt can be expressed as:

Xn~S~i∗[i]=∑n=1∣PSi∣wnpnor∑n=1∣PSi∣ai,npnpn∈PSi(8)X^{*}_{\widetilde{n}_{\widetilde{\mathcal{S}}_i}}[i] = \sum_{n=1}^{|PS_i|} w_n p_n \quad\text{or} \quad\sum_{n=1}^{|PS_i|} a_{i,n}p_n \quad p_n \in PS_i \tag*{(8)}

We present the fusion strategy of SA in Figure 4. Finally, the process of adding hybrid muti-defense prompt to the perturbed subgraph S~xi\widetilde{S}_{x_i} is described as follows:

Figure 4. Robust Prompt Optimization Workflow for Hybrid Multi-Defense Prompt.

S~xi∗=(S~xi,XnS~xi+XnS~xi∗)(9)\widetilde{S}_{x_i}^{*} = \left(\widetilde{S}_{x_i}, X_{n_{\widetilde{S}_{x_i}}} + X_{n_{\widetilde{S}_{x_i}}}^{*}\right) \tag*{(9)}

Our prompt is applied in the feature space. We do not choose structure-based prompts like All-in-one [29]. Instead, feature-based prompts allow for real-time adjustments to the structure of the disturbed subgraphs based on the current state of prompt tuning.

Prompt Tuning. To optimize the learnable prompts, we propose three robust auxiliary constraints. Consider a biased node classification task with a few-shot labeled training set T\mathcal{T}. Under the few-shot setting, there are only kk labeled cases for each class in C\mathcal{C}, i.e., T={(S~x1,y1),⋯ ,(S~xk×∣C∣,yk×∣C∣)}\mathcal{T} = \left\{(\widetilde{S}_{x_1}, y_1), \cdots, (\widetilde{S}_{x_{k\times|\mathcal{C}|}}, y_{k\times|\mathcal{C}|})\right\}. We define S~xi\widetilde{S}_{x_i} with prompt added as S~xi∗\widetilde{S}_{x_i}^{*}. After inputting S~xi∗\widetilde{S}_{x_i}^{*} into the pre-trained model ff, we define the output node embeddings as H~nS~xi∗\widetilde{H}_{n_{\widetilde{S}_{x_i}^{*}}}, and the graph embedding as G~i∗\widetilde{G}_{i}^{*}. To enhance the denoising capability of our prompt, we hope that the output node embeddings better satisfy the first-order proximity. Therefore, we use hidden feature smoothness as a regularizer, as shown below:

Ls=∑i=1k×∣C∣∑m,n∈S~xi∗∥H~nS~xi∗[m]−H~nS~xi∗[n]∥2(10)\mathcal{L}_{s} = \sum_{i=1}^{k\times|\mathcal{C}|}\sum_{m,n\in\widetilde{S}_{x_i}^{*}}\left\|\widetilde{H}_{n_{\widetilde{S}_{x_i}^{*}}}[m]-\widetilde{H}_{n_{\widetilde{S}_{x_i}^{*}}}[n]\right\|_{2} \tag*{(10)}

In each filtering scenario tt, we hope to reduce the distance between the output distribution of biased nodes and unbiased nodes. To elaborate clearly, we define the nodes with prompt added in filtering scenario tt as nptn_{p}^{t} and the set of remaining nodes as nnptn_{np}^{t}, i.e., the complement nodes ∁nS~xi∗npt\complement_{n_{\widetilde{S}_{x_i}^{*}}}n_{p}^{t}. (npt,nnpt)∈NT(n_{p}^{t}, n_{np}^{t}) \in N^{T}, where NT={(nld,∁nS~xi∗nld),(nls,∁nS~xi∗nls),(nod,∁nS~xi∗nod),⋯ }N^{T} = \left\{(n_{ld}, \complement_{n_{\widetilde{S}_{x_i}^{*}}}n_{ld}), (n_{ls}, \complement_{n_{\widetilde{S}_{x_i}^{*}}}n_{ls}), (n_{od}, \complement_{n_{\widetilde{S}_{x_i}^{*}}}n_{od}), \cdots\right\}. Therefore, we utilize a distribution loss to minimize the distance between the distributions of biased nodes and unbiased nodes in each perturbed subgraph across all filtering scenarios, enabling the prompt to guide and correct the biased distributions. This can be specifically expressed as follows:

Lkl=∑ik×∣C∣1∣NT∣∑(npt,nnpt)∈NTDKL(p(H~nS~xi∗[npt])∥p(H~nS~xi∗[nnpt]))(11)\mathcal{L}_{kl} = \sum_{i}^{k\times|\mathcal{C}|}\frac{1}{|N^{T}|}\sum_{(n_{p}^{t},n_{np}^{t})\in N^{T}}D_{KL}\left(p\left(\widetilde{H}_{n_{\widetilde{S}_{x_i}^{*}}}[n_{p}^{t}]\right)\middle\|p\left(\widetilde{H}_{n_{\widetilde{S}_{x_i}^{*}}}[n_{np}^{t}]\right)\right) \tag*{(11)}

Additionally, to impose constraint on the correlation between different defense prompts, we propose a node overlap matrix EE for each S~xi∗\widetilde{S}_{x_i}^{*}. Assuming there are TT filtering scenarios corresponding to TT defense prompts, EE is an T×TT \times T symmetric matrix. EijE_{ij} represents the degree of overlap between the biased node sets of the two filtering scenarios. Take nldn_{ld} and nlsn_{ls} for example, this implies:

Eij=Eji=∣nld∩nls∣∣nld∪nls∣(12)E_{ij}=E_{ji}=\frac{\left|n_{ld}\cap n_{ls}\right|}{\left|n_{ld}\cup n_{ls}\right|} \tag*{(12)}

The matrix EE indicates that defense prompts with a higher node overlap should be more similar, while those with lower overlap should exhibit greater differences. Defining the defense prompt matrix of S~i∗\tilde{S}_i^* as Pi∈RT×dinP_i \in\mathbb{R}^{T \times d_{\mathrm{in}}}, we obtain the following multi-prompt constraint loss.

Lo=∑ik×∣C∣∥PiPiT−E∥2(13)\mathcal{L}_o = \sum_{i}^{k \times|\mathcal{C}|} \left\lVert P_i P_i^T - E \right\rVert_2 \tag*{(13)}

With these three auxiliary robust constraints, we perform robust prompt tuning by optimizing the following expression:

min⁡θ,pd,ps,po,⋯∑ik×∣C∣L(G~i∗,yi)+αLs+βLkl+γLo(14)\min_{\theta,p_d,p_s,p_o,\cdots} \sum_{i}^{k \times|\mathcal{C}|} \mathcal{L}(\tilde{G}_i^*, y_i) + \alpha\mathcal{L}_s + \beta\mathcal{L}_{kl} + \gamma\mathcal{L}_o \tag*{(14)}

L\mathcal{L} is the downstream loss function (e.g., cross entropy). During the tuning process, we prune each S~i∗\tilde{S}_i^* in training set using a tuning similarity threshold τtune\tau_{tune}, based on the intermediate state of the current H~nS~i∗\tilde{H}_{n_{\tilde{S}_i^*}}, denoted as

S^i∗={(i,j)∈S~i∗∣sim⁡(H~nS~i∗[i],H~nS~i∗[j])>τtune}.(15)\hat{S}_i^* = \left\{(i,j) \in\tilde{S}_i^* \mid\operatorname{sim}\left(\tilde{H}_{n_{\tilde{S}_i^*}}[i], \tilde{H}_{n_{\tilde{S}_i^*}}[j]\right) > \tau_{tune}\right\}. \tag*{(15)}

After the tuning process, during the evaluation phase, we add the tuned prompt to different biased nodes of a biased test graph. Subsequently, we obtain the classification result using the frozen pre-trained model ff and the projection head θ\theta. We clearly illustrate our hybrid multi-defense prompt tuning workflow in Figure 4.

Indirect Amplification

Inspired by [16], given the contaminated local structure but consistent global feature distribution in the downstream task, we commence by training a multi-layer perceptron (MLP) devoid of structural information with few-shot training samples. Subsequently, the trained MLP is employed to generate pseudo-labels for subgraphs. Through a comparison of softmax scores, we identify the most confident predictions for each class and incorporate them into a new label set VpsuV_{psu}. Compute cross-entropy loss with VpsuV_{psu} to tune prompts while freezing the pre-trained model. This method applies to any prompt template. By reducing bias focus and improving unbiased understanding, it effectively alleviates the interference of irrelevant information during prompt tuning.

Theoretical Analysis

We are inspired by the theory of GPF ([4]) to prove the existence of the robust prompt we proposed under specific conditions.

Theorem 1. Given a pre-trained GNN model ff and an unbiased input graph G=(A,X)\mathcal{G}=(A,X). An attack yields a biased graph structure A~\tilde{A}, where A~=A+C∘S\tilde{A}=A+C\circ S (C=11T−I−2AC=11^T-I-2A, and Sij=Sji=1S_{ij}=S_{ji}=1 means the edge between nodes uu and vv is modified). Taking f(A,X)f(A,X) as the output of the model on unbiased data, there exists a hybrid multi-defense prompt P=N×MP=N\times M in the feature space (N∈R∣V∣×TN \in\mathbb{R}^{|V|\times T} is the prompt selection matrix, M∈RT×dinM \in\mathbb{R}^{T\times d_{\mathrm{in}}} is the prompt matrix, TT is the number of defense prompts) satisfying:

f(A~,X+P)=f(A,X)(16)f(\tilde{A}, X+P)=f(A,X) \tag*{(16)}

The proof is in Appendix § 10.§ 3. Thus, for biased downstream tasks, under certain assumptions, a hybrid multi prompt can help the model produce unbiased outputs on biased data.

Experiments

Experiment Setup

This paper primarily investigates the robust tuning of prompts on biased data in downstream node tasks across various attack scenarios. Thus, we use several datasets that are the focus of most attacks, including Cora-ML ([21]), citation graph (Cora, Citeseer) (Sen et al., 2008). We use a wide range of attacks for experiments. For non-adaptive attacks, we select four representative attacks of different types, including gradient-based attack (MetaAttack ([47])), distribution-based attack (Heuristic attack ([16])), heuristic-based attack (DICE ([35])), and Random attack. We attack downstream graph data used for prompt tuning with relatively large perturbation ratios, where MetaAttack is 25%, while the others are 50%. For convenience, it is abbreviated as M-0.25 in the Tables. For adaptive attacks, ([23]) proposes a unit test that includes 2,700 testable graphs. We select perturbed graphs targeting four defenses, including GCNSVD ([3]), GRAND ([5]), GNNGuard ([41]), and GCNJaccard ([36]), which basically cover most of the design principles of defense models. We have also conducted a general exploration on link prediction and graph classification tasks, with only the robustness-related results presented in the main text. All detailed experimental data can be found in Appendix A.4.

Pre-training strategies and prompt tuning

To better showcase the applicability of the prompt function, we select the most representative generative method GraphMAE ([10]) and contrastive method GraphCL ([39]) in the field of graph self-supervised learning as pre-training strategies. Additionally, we select the following widely-used prompts to explore their robustness, including GPF/GPF-plus ([4]), All-in-one ([29]), GPPT ([28]), GraphPrompt ([20]) and MultiGPrompt ([40]). GraphPrompt and MultiGPrompt require specific pre-training methods, so we follow their templates. The implementations of all prompts are available in Appendix A.5.

Prompt tuning under non-adaptive attacks

We conduct experiments in both 5-shot and 10-shot scenarios. We demonstrate the robustness performance under the 5-shot setting here. 10-shot results can be found in Appendix A.6. We denote our hybrid multi-defense prompt as MDMD-PTPT and the indirect amplification prompt as IAIA-PTPT. In Table 1, we can see that our prompts offer viable solutions to the potential security issues of prompts from two different perspectives. When the data is biased due to attacks, our tuning process can effectively avoid these invalid pieces of information to prevent the generation of misleading prompts. Aside from individual cases, the data in the Table also shows that the robustness of our prompts generally remains within a range of ±10%\pm10\%, unlike other prompts, which can vary by more than 40%. Notably, on clean unbiased tuning data, our prompts still maintain a state-of-the-art performance in many cases, without sacrificing excessive accuracy for the sake of robustness.

Pre-Training StrategiesPromptsCora CleanCora M-0.25Cora D-0.5Cora R-0.5Cora H-0.5Citeseer CleanCiteseer M-0.25Citeseer D-0.5Citeseer R-0.5Citeseer H-0.5CoraML CleanCoraML M-0.25CoraML D-0.5CoraML R-0.5CoraML H-0.5
GraphPrompt57.6834.0636.0142.6835.1967.2040.2234.2434.2931.6867.1741.9636.7544.6844.32
MultiGPrompt48.3439.3732.9743.7129.5250.3741.9938.0942.3148.3456.2436.9939.7546.6826.90
GraphCLAll-in-one50.9331.0217.9134.2928.3443.9637.1837.2927.7221.3137.4328.3826.5035.3527.46
GPF54.0633.8327.6229.4315.9256.4638.7844.0714.3722.1241.9530.2628.5839.3124.46
GPF-plus66.3936.8330.5242.2226.5366.1945.3038.6831.6814.3771.3035.4338.5930.7426.30
GPPT45.2420.8624.0419.5027.7141.4130.1833.3323.8824.5741.7534.6738.5130.9032.15
*MD-PT49.3940.0640.1544.9444.6754.8653.6948.0243.7054.3867.4554.8250.4450.4459.81
*IA-PT57.8258.8251.0753.6562.2752.0850.8044.7147.9252.2470.3460.5347.8060.2562.77
GraphMAEAll-in-one44.6729.7530.2535.8329.9867.8455.3640.3844.7139.0045.2032.4313.2124.5028.82
GPF66.7138.8233.1548.5339.2773.2453.3141.4541.6129.3861.0115.7320.4210.5716.65
GPF-plus63.4039.4633.3849.8436.4675.1651.8740.3336.8128.2670.2622.6240.9545.3621.70
GPPT67.7943.1747.7142.4035.4661.7046.9049.1143.8650.0571.6241.5548.0854.8441.59
*MD-PT62.4962.4952.6148.0351.6162.8758.8150.3250.1156.8965.4568.3757.2955.2450.72
*IA-PT68.9368.3061.9067.3066.6259.7260.6357.2659.4061.2275.9057.8554.3266.4164.65

Table 1. Performance of Different Prompts in 5-shot scenario under Non-Adaptive Attacks.

Prompt tuning under adaptive attacks

We conduct experiments on adaptive poisoning attack graphs targeting classical defenses to demonstrate the versatility of our prompt tuning process. We still present the 5-shot results, while the 10-shot results can be found in Appendix § 10.§ 7. In Table 2, it can be seen that the defense capability of MD-PT appears slightly inferior to that of IA-PT when facing more targeted attacks. This is understandable, “as directly ignoring a problem is often easier than dealing with more challenging issues”. However, our two strategies still remain highly competitive compared to other prompts.

Pre-Training StrategiesPromptsCoraML GCNSVDCoraML GRANDCoraML GNNGuardCoraML GCNJaccardCiteseer GCNSVDCiteseer GRANDCiteseer GNNGuardCiteseer GCNJaccard
GraphPromptGraphPrompt47.6635.2437.6441.9048.7234.9940.7138.25
MultiGprompt24.2230.2015.5614.9725.1625.9631.4133.55
GraphCLProG37.3231.0252.7832.3823.9330.5633.9231.14
GPF30.8823.0827.3935.4229.5921.8518.4325.53
GPF-plus52.2928.9341.4138.3724.9521.8524.9518.38
GPPT35.6926.5340.1440.4543.9630.8832.0543.48
*MD-PT47.8545.5449.6741.3551.6051.6051.0153.90
*IA-PT61.2758.7353.8346.0859.3557.0562.6157.10
GraphMAEProG17.0523.2227.6223.4534.3527.1439.0544.34
GPF48.5625.1250.8441.0045.8331.4650.3744.98
GPF-plus47.2130.7548.4846.2645.5735.8450.7546.37
GPPT29.5220.3232.0629.8044.7633.0146.8536.43
*MD-PT49.1745.6054.6049.2457.2654.7053.3753.47
*IA-PT66.1763.5463.1763.4965.5464.2164.4265.60

Table 2. Performance of Different Prompts in 5-shot scenario under Adaptive Attacks.

Ablation Study

Since IA-PT is nested within any prompt, we primarily discusses MD-PT. We compare MD-PT with six prompt variants: “w/o pdp_d” is a variant that does not use degree prompt. “w/o psp_s” is a variant that does not use similarity prompt. “w/o pop_o” is a variant that does not use prompt for out-of-distribution nodes. “w/o Lkl\mathcal{L}_{kl}” indicates that no distribution alignment is used during the tuning process. “w/o Lo\mathcal{L}_o” signifies that multi-prompt constraints are not employed. “w/o Ls\mathcal{L}_s” indicates that feature smoother is not used. Table 3 illustrates the key modules of the robust prompt tuning phase across different biased environments. Additionally, we have implemented a transfer attack experiment in Appendix § 10.10 to validate the scalability and transferability of GPromptShield.

Ablation VariantsCora 5-shot Meta 0.25Cora 5-shot DICE 0.5Citeseer 5-shot Meta 0.25Citeseer 5-shot DICE 0.5Cora 10-shot Meta 0.25Cora 10-shot DICE 0.5Citeseer 10-shot Meta 0.25Citeseer 10-shot DICE 0.5
MD-PT60.12%54.5564.20%52.3971.02%62.30%67.04%55.35
MD-PT w/o pdp_d62.31%53.05%63.68%51.76%67.87%61.36%66.67%56.31%
MD-PT w/o psp_s58.46%52.02%58.97%53.21%69.92%59.48%61.57%54.39%
MD-PT w/o pop_o59.32%55.37%60.42%51.98%69.09%60.99%65.69%55.18%
MD-PT w/o Lkl\mathcal{L}_{kl}59.86%55.28%60.42%52.14%69.83%60.81%64.42%54.23%
MD-PT w/o Lo\mathcal{L}_o59.59%51.20%59.94%50.00%69.23%58.33%65.64%55.01%
MD-PT w/o Ls\mathcal{L}_s58.32%56.46%60.84%51.50%69.87%60.72%66.34%54.58%

Table 3. Performance of Hybrid Multi-Defense Prompt by Ablating Different Modules.

Conclusion

In this paper, we shift the primary focus of graph prompts from compatibility to vulnerability issues in adversarial attack scenarios. We design a highly extensible shield defense system for the prompts. Specifically, we design a hybrid multi-defense prompt based on the principle of direct handling and an adaptable tuning strategy based on the principle of indirect amplification. We theoretically prove the feasibility of our strategies and achieve outstanding robustness performance in both adaptive and non-adaptive attack scenarios.

Reproducibility Statement

The code for this paper is provided at https://github.com/GTLSysGraph/GPromptShield. All detailed experimental data can be found in Appendix A.4. The implementations of all baselines are available in Appendix A.5. The explanations of the theoretical part in the paper can be found in Appendix A.3.

Ethic Statement

The robustness of graph prompts in the face of adversarial attacks is a key research focus. In this work, we have opened up a new perspective on graph prompt tuning focused on robustness, we shift the primary focus of graph prompts from compatibility to vulnerability issues in adversarial attack scenarios. We design a shield defense system for prompts, including direct handling with a hybrid multi-defense prompt and indirect amplification through a robust tuning strategy. Our work is aimed at improving the security of graph-based applications. Overall, we believe our research can positively contribute to the field without introducing new security risks.

Acknowledgement

The research work supported by National Key Research and Development Program (Grant No.2023YFB4502305), the Beijing Natural Science Foundation (4232036).

Appendix

Graph Pre-training Strategies

The pre-training strategy aims to extract knowledge from the vast amount of data in the real world to learn a general model, thereby reducing the costs associated with task-specific training. Most graph pre-training strategies exist in a self-supervised form and can be broadly divided into contrastive strategies and generative strategies. Contrastive strategies train an encoder by maximizing the mutual information between different graph views. The representative GraphCL [39] minimizes the differences in graph representations from different views of the same graph. DGI [33] follows the InfoMax principle [17] by learning node representations through local-global mutual information maximization. GRACE [43] proposes edge removal and feature masking augmentation, GCA [44] adopts probabilistic adaptive augmentation, while MVGRL [8] employs diffusion graph as augmentation view. Generative strategies use the reconstruction of input data as a pretext, significantly lowering training costs. Currently, a popular paradigm is based on masking strategies inspired by BERT [2] in the language domain and MAE [9] in the image domain. GraphMAE [10], as a representative example, reconstructs features using masking strategies and designs scaled cosine error. GraphMAE2 [11] further extends this idea by designing the strategies of multi-view random re-mask decoding and latent representation prediction for feature reconstruction, aiming to reduce excessive reliance on feature discriminability. S2GAE [31] proposes direction-aware graph masking and cross-correlation decoder. MaskGAE [14] adopts edge-wise and path-wise random masking, also introduces a degree decoder to alleviate the problem of structural information overfitting. Pre-training strategies have been studied in great depth, with a wide range of pretext tasks extracting knowledge from multiple aspects.

Adaptive Robustness from the perspective of out-of-distribution (OOD) generalization.

Adaptive attacks are a proposed new robustness evaluation standard and are a type of white-box attack. They indicate that, with a thorough understanding of the defense model, attackers can bypass defenses using corresponding strategies. Thus, existing defenses are not as robust as they were evaluated in their papers. These defenses are vulnerable to adaptive attacks because most of them are designed based on some specific properties that can be used to differentiate adversarial edges from original edges. Adversarial modifications on graphs often violate some intrinsic properties shared by the real-world graphs (e.g., increasing heterophily [47] and focusing on the high-frequency component [1]). The adversary can easily defeat the defenses by imposing constraints on the same properties during the attack. Therefore, the key to enhancing adaptive robustness is not relying on artificially defined properties. To overcome this dependence, Li et al. proposed a new approach from the out-of-distribution perspective, arguing that adversarial edges generated through attacks are inherently out-of-distribution compared to the original edges. Thus, by modeling the entire out-of-distribution problem, an integrated out-of-distribution detector is trained using adversarially generated edges to detect and remove perturbations.

Proof for Theorem 1

Proof. Consider a pre-trained graph neural network (GNN) model ff and an unbiased input graph G=(A,X)\mathcal{G} = (A, X). An adversarial attack can induce a biased graph structure A~\widetilde{A}. Specifically, A~\widetilde{A} is formulated as A~=A+C∘S\widetilde{A} = A + C \circ S, where C=11T−I−2AC = 11^{T} - I - 2A. When Sij=Sji=1S_{ij} = S_{ji} = 1, the edge linking nodes uu and vv is either added or removed. Based on this formulation, we can simplify A~\widetilde{A} as A~=A+ΔA\widetilde{A} = A + \Delta A. Taking f(A,X)f(A, X) as the output of the model on unbiased data, there exists a prompt PP that can be incorporated into the feature space, ensuring the satisfaction of the following equation:

f(A~,X+P)=f(A,X)(17)f(\widetilde{A}, X + P) = f(A, X) \tag*{(17)}

We adopt the assumptions from GPF [4], the pre-trained GNN model ff can be represented as:

f(A,X)=H=(A+(1+ϵ)I)XW(18)f(A,X)=H=(A+(1+\epsilon)I)XW \tag*{(18)}

We substitute f(A~,X+P)f(\widetilde{A},X+P) to obtain:

f(A~,X+P)=(A+ΔA+(1+ϵ)I)(X+P)W=(A+ΔA+(1+ϵ)I)XW+(A+ΔA+(1+ϵ)I)PW=(A+(1+ϵ)I)XW+ΔAXW+(A+(1+ϵ)I)PW+ΔAPW=H+ΔAXW+(A+(1+ϵ)I)PW+ΔAPW(19)\begin{aligned} f(\widetilde{A},X+P) &= (A+\Delta A+(1+\epsilon)I)(X+P)W\\ &= (A+\Delta A+(1+\epsilon)I)XW+(A+\Delta A+(1+\epsilon)I)PW\\ &= (A+(1+\epsilon)I)XW+\Delta AXW+(A+(1+\epsilon)I)PW+\Delta APW\\ &= H+\Delta AXW+(A+(1+\epsilon)I)PW+\Delta APW \tag*{(19)} \end{aligned}

Therefore, to satisfy f(A~,X+P)=f(A,X)f(\widetilde{A},X+P)=f(A,X), we need to solve the equation:

H+ΔAXW+(A+(1+ϵ)I)PW+ΔAPW=H(20)H+\Delta AXW+(A+(1+\epsilon)I)PW+\Delta APW=H \tag*{(20)}

In our hybrid multi-defense prompt, assuming there are TT defense prompts, then PP can be represented as N×MN\times M, where N∈R∣V∣×TN\in\mathbb{R}^{|V|\times T} is the prompt selection matrix and M∈RT×dinM\in\mathbb{R}^{T\times d_{in}} is the prompt matrix. Therefore, by replacing PP with NN and MM in the equation, the problem is transformed into solving for MM. To solve the equation

H+ΔAXW+(A+(1+ϵ)I)NMW+ΔANMW=H(21)H+\Delta AXW+(A+(1+\epsilon)I)NMW+\Delta ANMW=H \tag*{(21)}

Subtracting HH from both sides of this equation, we get

ΔAXW+(A+(1+ϵ)I)NMW+ΔANMW=0(22)\Delta AXW+(A+(1+\epsilon)I)NMW+\Delta ANMW=0 \tag*{(22)}

By factoring out MM from the terms involving MM, we have

ΔAXW+((A+(1+ϵ)I)+ΔA)NMW=0(23)\Delta AXW+((A+(1+\epsilon)I)+\Delta A)NMW=0 \tag*{(23)}

Let B=(A+(1+ϵ)I)+ΔAB=(A+(1+\epsilon)I)+\Delta A, then the equation becomes

ΔAXW+BNMW=0(24)\Delta AXW+BNMW=0 \tag*{(24)}

Isolating the term with MM by subtracting ΔAXW\Delta AXW from both sides, we obtain

BNMW=−ΔAXW(25)BNMW=-\Delta AXW \tag*{(25)}

Assuming that BNBN is invertible (if BNBN is not invertible, the solution may not exist in the general sense or may have infinitely many solutions depending on the properties of the matrices), we multiply both sides of the equation on the left by (BN)−1(BN)^{-1}, resulting in

(BN)−1BNMW=(BN)−1(−ΔAXW)(26)(BN)^{-1}BNMW=(BN)^{-1}(-\Delta AXW) \tag*{(26)}

Since (BN)−1BN=I(BN)^{-1}BN=I (the identity matrix), the equation simplifies to

MW=−(BN)−1ΔAXW(27)MW=-(BN)^{-1}\Delta AXW \tag*{(27)}

Case 1: WW is invertible

we multiply both sides of the equation on the right by W−1W^{-1}, so

MWW−1=−(BN)−1ΔAXWW−1(28)MWW^{-1}=-(BN)^{-1}\Delta AXWW^{-1} \tag*{(28)}

Since WW−1=IWW^{-1}=I, we get the solution for MM as

M=−(BN)−1ΔAX(29)M = -(BN)^{-1}\Delta AX \tag*{(29)}

Substitute B=(A+(1+ϵ)I)+ΔAB = (A + (1 + \epsilon)I) + \Delta A, the solution for the matrix MM is

M=−((A+(1+ϵ)I)+ΔA)N)−1ΔAX(30)M = -((A + (1 + \epsilon)I) + \Delta A)N)^{-1}\Delta AX \tag*{(30)}

Case 2: WW is non-invertible

If WW is non-invertible, we can solve the problem by calculating the pseudoinverse of WW. In this case, we cannot multiply by W−1W^{-1} directly. We can use Singular Value Decomposition (SVD). Let W=UΣVTW = U\Sigma V^{T}, where UU is an m×rm \times r orthogonal matrix (mm is the number of rows of WW, rr is the rank of WW), satisfying UTU=IrU^{T}U = I_{r} (IrI_{r} is the rr-order identity matrix). Σ\Sigma is an r×rr \times r diagonal matrix, and its diagonal elements σi\sigma_i (i=1,⋯ ,ri = 1, \cdots, r) are the non-zero singular values of WW, that is, Σ=diag⁡(σ1,⋯ ,σr)\Sigma= \operatorname{diag}(\sigma_1, \cdots, \sigma_r). VV is an n×rn \times r orthogonal matrix (nn is the number of columns of WW), satisfying VTV=IrV^{T}V = I_{r}.

The pseudoinverse of WW, W+W^{+}, can be expressed as W+=VΣ+UTW^{+} = V\Sigma^{+}U^{T}, where Σ+\Sigma^{+} is the pseudoinverse of Σ\Sigma, also an r×rr \times r diagonal matrix, and its diagonal elements are 1σi\frac{1}{\sigma_i} (i=1,⋯ ,ri = 1, \cdots, r).

Multiply both sides of the equation MW=−(BN)−1ΔAXWMW = -(BN)^{-1}\Delta AXW by W+W^{+} on the right-hand side, and we can get:

MWW+=−(BN)−1ΔAXWW+M(WW+)=−(BN)−1ΔAX(WW+)(31)\begin{aligned} MWW^{+} &= -(BN)^{-1}\Delta AXWW^{+} \\ M(WW^{+}) &= -(BN)^{-1}\Delta AX(WW^{+}) \tag*{(31)} \end{aligned}

WW+WW^{+} is a projection matrix onto the column space of WW. If WW is a full column-rank matrix, WW+WW^{+} is the identity matrix in the column space of WW. Even if WW does not have full column-rank, WW+WW^{+} can still project vectors onto the column space of WW.

Discussion on the solution of MM

  • If WW is a full column-rank matrix, at this time WW+WW^{+} is an invertible matrix (it is the identity matrix in the dimension of the column space of WW). Multiply both sides of the equation on the left by (WW+)−1(WW^{+})^{-1}, and we can get M=−(BN)−1ΔAXM = -(BN)^{-1}\Delta AX.

  • If WW is not a full column-rank matrix, let C=−(BN)−1ΔAXC = -(BN)^{-1}\Delta AX. Although we cannot directly obtain a unique solution for MM as in the case of full column-rank, MM can still take multiple values in the complementary space of the column space of WW, that is, the solution of MM is not unique. However, if there are other conditions, such as restrictions on the norm of MM, constraints on some elements of MM, etc., we can further determine MM by combining these conditions.

Therefore, after finding the inverse-like matrix of WW (pseudoinverse in the general case), under certain conditions (such as WW having full column-rank), we can obtain a unique solution for MM; in general cases, we can obtain the constraints of MM in the column space of WW, and further solve for MM by combining other conditions. This work does not conduct a detailed analysis of some non-ideal situations. Of course, we hope that future work can provide more rigorous proofs and derivations for robust prompts in different biased environments.

Datasets and Attacks

Datasets

For the attacked data, following the settings in Nettack [46], we only consider the largest connected component (LCC). The details of the datasets are presented in Table 4.

DatasetsNLCCN_{LCC}ELCCE_{LCC}FeaturesClasses
Citeseer2110366837036
Cora2485506914337
Cora-ML2810798128797

Table 4. Details of the largest connected component (LCC) for each dataset.

For node classification and link prediction tasks, we select homophilic datasets (Cora, Citeseer, Pubmed) (Sen et al., 2008; Namata et al., 2012), heterophilic datasets (Wisconsin) [25] and explore the prompts on large graph (ogbn-arxiv) [12]. We conduct experiments under 5-shot and 10-shot settings, where the link prediction task includes 5,000 positive edges and 5,000 negative edges. The details of the datasets are presented in Table 5. We present the performance of the prompt functions on node classification and link prediction tasks for these datasets in Appendix § 10.§ 8.

DatasetsGraphsNodesEdgesFeaturesClassesTaskCatagory
Cora12708542914337N / LHomophilic
Pubmed1197178864828793N / LHomophilic
Citeseer13327910437036N / LHomophilic
Wisconsin125151517035N / LHeterophilic
ogbn-arxiv1169343116624312840N / LHomophilic & Large scale

Table 5. Statistics of all datasets for node classification and link prediction tasks.

For the graph classification task, in order to be more comprehensive, we chose the molecular dataset MUTAG [13], the social network dataset COLLAB [37], the protein dataset PROTEINS [34], and the social network dataset IMDB-BINARY [38]. We also conduct experiments under 5-shot and 10-shot settings. The details of the datasets are presented in Table 6. We present the performance of the prompt functions on graph classification tasks for these datasets in Appendix § 10.§ 9.

DatasetsGraphsAvg. NodesAvg. EdgesFeaturesClassesTaskCatagory
MUTAG18817.919.872Gsmall molecule
COLLAB500074.52457.803Gsocial network
PROTEINS111339.172.832Gproteins
IMDB-BINARY100019.896.5302Gsocial network

Table 6. Statistics of all datasets for graph classification tasks.

The data recording format for all node classification, link prediction, and graph tasks datasets follows the approach used in [45].

Through generalization experiments on these datasets, we find that our auxiliary robust system does not make an excessive sacrifice of accuracy in favor of robustness. In fact, it maintains strong competitiveness on many task datasets, which is a pleasant surprise during the expansion of the experiments. We speculate that the original datasets might already contain some inherent noise, and our robustness enhancement tool further purifies the data during the training process.

Unit Test and Attacks

To provide a more comprehensive evaluation of robustness, Mujkanovic et al. [23] presents an interesting research point: almost all defenses are evaluated against non-adaptive attacks, leading to overly optimistic robustness estimates. Therefore, they categorize 49 defense methods and select the most representative method from each category to design targeted adaptive attack methods. The adversarial graphs generated by these attacks can be bundled together to test other defenses and can be considered a minimal standard for evaluating the adaptive robustness of defense models. In the unit test, the datasets are centered around Citeseer and Cora-ML. For each representative model, there are 5 random data splits, each containing poisoning and evasion attacks. The attack budget ranges from 0% to 15%, resulting in approximately 2700 testable graphs in total. The unit test module used in this paper is adapted from the following repository: https://github.com/LoadingByte/are-gnn-defenses-robust.

The code implementation for attacking the graph using different attacks can be found at the following links.

Baselines for Prompt

The code implementation for all prompts can be found at the following links. Additionally, ProG is a library built upon PyTorch to easily conduct single or multi-task prompting for pre-trained Graph Neural Networks (GNNs). You can easily use this library to conduct various graph workflows.

Results of few-shot node classification under non-adaptive attacks

The results for 10-shot scenarios under non-adaptive attacks. Table 7 shows the robustness performance of 10-shot graph prompt tuning under different non-adaptive attacks.

Pre-Training StrategiesPromptsCora CleanCora M-0.25Cora D-0.5Cora R-0.5Cora H-0.5Citeseer CleanCiteseer M-0.25Citeseer D-0.5Citeseer R-0.5Citeseer H-0.5CoraML CleanCoraML M-0.25CoraML D-0.5CoraML R-0.5CoraML H-0.5
GraphPrompt72.1540.4839.8352.3039.7973.4449.2130.8441.7332.5764.8042.2139.0141.0839.78
MultiGPrompt52.5346.8841.0849.2233.4450.2447.5938.1642.8445.4748.9937.3133.8646.7624.98
GraphCLAll-in-one69.7845.2614.0311.6418.9567.0542.1734.0445.7517.4049.6827.0919.1032.0824.21
GPF60.9935.6535.0539.3729.8573.6641.6342.1737.4021.9044.4815.9830.4534.7124.49
GPF-plus70.6141.6337.2151.0136.8969.1642.3842.6045.5826.0265.6934.8336.0124.7031.63
GPPT40.7128.9830.8629.4432.7551.8440.3832.9532.6332.2543.5137.1033.1735.2029.40
*MD-PT53.7347.0141.7744.1147.8464.5058.8155.2856.4264.2871.2956.1262.0855.3152.76
*IA-PT65.9663.5255.3457.9662.7062.6662.8252.2057.4556.3179.0870.5264.6465.1374.74
GraphMAEAll-in-one50.5535.4630.0436.8932.2469.9759.7343.4150.7342.2846.1920.4828.7521.5721.98
GPF71.6243.5644.4355.8444.3977.0759.2445.8045.2027.2668.1734.0629.3222.9529.24
GPF-plus69.0944.4841.4955.6640.0676.5956.4840.4938.7529.7671.2134.8236.8611.6824.66
GPPT77.6945.2658.0563.7247.6566.257.5153.0657.0254.4268.5737.9644.4047.2832.20
*MD-PT72.2269.4159.3464.4058.2363.9664.9955.4558.1661.9078.2667.7262.4965.1750.16
*IA-PT73.0573.3265.2371.1671.1167.4364.6659.8462.4959.3072.4271.3767.4072.8769.95

Table 7. Performance of Different Prompts in 10-shot scenario under Non-Adaptive Attacks.

Results of few-shot node classification under adaptive attacks

The results for 10-shot scenarios under adaptive attacks. Table 8 shows the robustness performance of 10-shot graph prompt tuning under different adaptive attacks. On the CoraML dataset, the performance of most prompts vary widely. Similarly, on the Citeseer dataset, there are significant differences across various prompts. As can be seen from the Table, MD-PT and IA-PT strategies perform more prominently than other prompt functions, demonstrating their effectiveness in enhancing the robustness of prompt tuning against adaptive attacks.

Pre-Training StrategiesPromptsCoraML GCNSVDCoraML GRANDCoraML GNNGuardCoraML GCNJaccardCiteseer GCNSVDCiteseer GRANDCiteseer GNNGuardCiteseer GCNJaccard
GraphPrompt53.7738.7851.0135.6055.9932.2545.6453.77
MultiGPrompt29.9932.3819.0913.2531.4925.0434.2033.39
GraphCLAll-in-one40.4825.4436.7121.9041.2533.0637.1342.33
GPF40.8043.7916.6526.9138.5937.7818.9226.29
GPF-plus38.9636.0245.7740.3945.4722.4953.1722.49
GPPT36.1130.6835.3336.5248.8341.0344.2339.89
*MD-PT41.4550.2544.6245.7258.3757.7263.1457.72
*IA-PT63.2557.9157.9160.8165.9666.6166.0766.61
GraphMAEAll-in-one20.4724.2430.7323.9243.2028.4044.6145.15
GPF53.5937.0354.8344.2562.8234.1554.9656.75
GPF-plus53.9638.2253.7745.3558.4333.5551.3353.93
GPPT50.7431.5134.8236.5263.6931.4956.4247.48
*MD-PT44.2043.8845.7748.3460.3860.2560.7663.52
*IA-PT68.1265.9267.1167.9967.8666.2364.8265.91

Table 8. Performance of Different Prompts in 10-shot scenario under Adaptive Attacks.

The results for 5-shot scenarios under Node Classification and Link Prediction Tasks. Table 9 shows the performance of 5-shot graph prompt tuning under different node classification and link prediction tasks.

MethodsCora NodeCora LinkCiteseer NodeCiteseer LinkPubmed NodePubmed LinkWisconsin NodeWisconsin Linkogbn-arxiv Nodeogbn-arxiv Link
Pre-train & Fine-tune42.7368.7952.9255.1451.0754.0420.9545.1510.9253.34
GPPT35.9665.1349.7956.5655.4243.8816.9977.656.5273.52
All-in-one50.0674.6851.1160.1544.7664.9926.2178.854.5480.34
Gprompt68.0367.2575.2558.8359.9362.5744.9261.1321.2753.31
GPF64.5258.9252.5657.0657.0562.9535.8239.4515.8149.23
GPF-plus70.5562.4574.5158.4740.9482.2039.4545.8522.4072.43
*MD-PT71.7669.9575.3878.2464.0064.7550.4952.4918.9265.42

Table 9. Performance of Hybrid Multi-Defense Prompt and Mainstream Prompts on 5-shot Node Classification and Link Prediction Tasks.

The results for 10-shot scenarios under Node Classification and Link Prediction Tasks. Table 10 shows the performance of 10-shot graph prompt tuning under different node classification and link prediction tasks.

MethodsCora NodeCora LinkCiteseer NodeCiteseer LinkPubmed NodePubmed LinkWisconsin NodeWisconsin Linkogbn-arxiv Nodeogbn-arxiv Link
Pre-train & Fine-tune52.2367.5558.8260.5750.3559.2329.1265.4914.3855.41
GPPT47.7765.4754.0974.4848.9155.3522.4359.137.7452.86
All-in-one65.9077.2353.2465.9549.3374.0341.7682.355.2580.50
Gprompt70.8166.4576.4650.4660.1061.5843.9650.4324.3175.55
GPF69.9373.3553.8260.3739.2650.0831.7257.7510.3663.03
GPF-plus72.7988.2478.4673.0242.4588.7532.2659.5714.1281.40
*MD-PT73.4574.5575.8577.4962.3462.9835.6554.4220.4568.45

Table 10. Performance of Hybrid Multi-Defense Prompt and Mainstream Prompts on 10-shot Node Classification and Link Prediction Tasks.

Performance of Hybrid Multi-Defense Prompt on Graph Classification Tasks

The results for 5-shot and 10-shot scenarios under Graph Classification Tasks. Table 11 shows the performance of graph prompt tuning under different graph classification tasks.

MethodsDatasetsMUTAGCOLLABPROTEINSIMDB-B
5-shot10-shot5-shot10-shot5-shot10-shot5-shot10-shot
Pre-train & Fine-tune68.0068.0061.2862.7753.3765.5163.0067.00
GPPT64.0068.6750.4851.9759.5560.3351.2554.48
All-in-one70.0066.6763.6054.7062.9267.2261.1266.12
Gprompt72.6770.6764.5364.2561.2458.6560.5069.50
GPF64.6765.3361.8062.4564.6166.2960.1766.75
GPF-plus68.0072.0062.2063.7565.1763.2660.7567.12
*MD-PT74.6774.3565.0765.6367.3966.5263.1362.57

Table 11. Performance of Hybrid Multi-Defense Prompt and Mainstream Prompts on Graph Classification Tasks.

Performance of Proposed Prompts on Transfer Tasks

The results for Proposed Prompts and GPF/GPF-plus on Different Attack Transfer Scenarios.

Table 12 shows the performance of graph prompt tuning under different attack transfer scenarios. As seen, IA-PT still demonstrates excellent robustness, while MD-PT is slightly less effective, but still consistently improves the robustness of GPF/GPF-plus. This aligns well with what we describe: avoiding difficulties is often much easier than facing them head-on. This transferability experiment has been instrumental in helping us validate the scalability and transferability of the designed system.

PromptsAttacksCoraCiteseerCoraML
M-0.25D-0.5R-0.5H-0.5M-0.25D-0.5R-0.5H-0.5M-0.25D-0.5R-0.5H-0.5
GPFM-0.2531.4432.4720.2326.2620.2620.8532.8035.7928.9932.0733.0730.42
GPFD-0.533.9723.3828.2132.7036.2225.3725.3720.7536.7533.7934.7931.71
GPFR-0.536.9236.3736.2434.3725.3721.9621.9621.9634.6334.4739.3139.95
GPFH-0.529.1635.8529.3029.0231.2625.3720.1519.8631.3921.7434.1932.54
GPF-plusM-0.2535.5635.5134.9236.3325.3724.4638.7640.4833.1535.6735.1536.75
GPF-plusD-0.537.1936.8738.1937.1026.9828.7729.4927.4034.3533.9932.3132.03
GPF-plusR-0.542.2640.3540.2644.3534.2935.7135.3435.6838.1137.6736.0337.83
GPF-plusH-0.533.5630.7935.9232.7920.2033.1234.7833.0740.0340.4537.1139.95
*MD-PTM-0.2547.0642.4043.1147.8942.8139.6240.4644.8553.8243.5742.1747.51
*MD-PTD-0.547.0840.1542.8147.1342.1836.7236.3238.8843.4444.5540.0251.25
*MD-PTR-0.548.0445.8250.3453.5240.4637.8237.7738.7552.7641.5745.2450.80
*MD-PTH-0.550.1541.4544.6747.4841.8238.3935.5541.0841.3541.3439.4552.36
*IA-PTM-0.2565.2249.5060.1662.5355.6648.7655.0458.3561.6154.8959.9665.57
*IA-PTD-0.564.5452.2457.6061.5757.4155.3451.2453.6158.1952.5253.0257.56
*IA-PTR-0.557.8548.0454.6960.4656.5254.0348.9654.2162.1949.1354.0061.92
*IA-PTH-0.560.4153.6255.1362.5557.9452.7350.4758.5562.1052.7159.4858.77

Table 12. Performance of Proposed Prompts and GPF/GPF-plus on Different Attack Transfer Scenarios.

References

  1. [1]Heng Chang, Yu Rong, Tingyang Xu, Yatao Bian, Shiji Zhou, Xin Wang, Junzhou Huang, and Wenwu Zhu. Not all low-pass filters are robust in graph convolutional networks. Advances in Neural Information Processing Systems, 34:25058–25071, 2021.
  2. [2]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.arxiv.org/abs/1810.04805
  3. [3]Negin Entezari, Saba A Al-Sayouri, Amirali Darvishzadeh, and Evangelos E Papalexakis. All you need is low (rank) defending against adversarial attacks on graphs. In Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 169–177, 2020.
  4. [4]Taoran Fang, Yunchao Zhang, Yang Yang, Chunping Wang, and Lei Chen. Universal prompt tuning for graph neural networks. Advances in Neural Information Processing Systems, 36, 2024.arxiv.org/abs/2209.15240
  5. [5]Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, and Jie Tang. Graph random neural networks for semi-supervised learning on graphs. Advances in neural information processing systems, 33:22092–22103, 2020.
  6. [6]Lukas Gosch, Simon Geisler, Daniel Sturm, Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Adversarial training for graph neural networks: Pitfalls, solutions, and new directions. Advances in Neural Information Processing Systems, 36, 2024.arxiv.org/abs/2306.15427
  7. [7]Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, pp. 79–90, 2023.arxiv.org/abs/2302.12173
  8. [8]Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view representation learning on graphs. In International conference on machine learning, pp. 4116–4126. PMLR, 2020.arxiv.org/abs/2006.05582
  9. [9]Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009, 2022.arxiv.org/abs/2111.06377
  10. [10]Zhenyu Hou, Xiao Liu, Yukou Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. Graphmae: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 594–604, 2022.arxiv.org/abs/2205.10803
  11. [11]Zhenyu Hou, Yufei He, Yukou Cen, Xiao Liu, Yuxiao Dong, Evgeny Kharlamov, and Jie Tang. Graphmae2: A decoding-enhanced masked self-supervised graph learner. arXiv preprint arXiv:2304.04779, 2023.arxiv.org/abs/2304.04779
  12. [12]Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118–22133, 2020.arxiv.org/abs/2005.00687
  13. [13]Nils Kriege and Petra Mutzel. Subgraph matching kernels for attributed graphs. arXiv preprint arXiv:1206.6483, 2012.arxiv.org/abs/1206.6483
  14. [14]Jintang Li, Ruofan Wu, Wangbin Sun, Liang Chen, Sheng Tian, Liang Zhu, Changhua Meng, Zibin Zheng, and Weiqiang Wang. Maskgae: masked graph modeling meets graph autoencoders. arXiv preprint arXiv:2205.10053, 2022a.
  15. [16]Kuan Li, Yang Liu, Xiang Ao, and Qing He. Revisiting graph adversarial attack and defense from a data distribution perspective. In The Eleventh International Conference on Learning Representations, 2022b.
  16. [17]Ralph Linsker. Self-organization in a perceptual network. Computer, 21(3):105–117, 1988.DOI
  17. [18]Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1–35, 2023a.arxiv.org/abs/2107.13586
  18. [19]Yuxi Liu, Lianghao Xia, and Chao Huang. Selfgnn: Self-supervised graph neural networks for sequential recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1609–1618, 2024.arxiv.org/abs/2405.20878
  19. [20]Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. In Proceedings of the ACM Web Conference 2023, pp. 417–428, 2023b.arxiv.org/abs/2302.08043
  20. [21]Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3:127–163, 2000.
  21. [22]Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky. State of what art? a call for multi-prompt llm evaluation. Transactions of the Association for Computational Linguistics, 12:933–949, 2024.
  22. [23]Felix Mujkanovic, Simon Geisler, Stephan Günnemann, and Aleksandar Bojchevski. Are defenses for graph neural networks robust? Advances in Neural Information Processing Systems, 35:8954–8968, 2022.
  23. [25]Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gcn: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287, 2020.arxiv.org/abs/2002.05287
  24. [26]Zezhi Shao, Zhao Zhang, Wei Wei, Fei Wang, Yongjun Xu, Xin Cao, and Christian S Jensen. Decoupled dynamic spatial-temporal graph neural network for traffic forecasting. arXiv preprint arXiv:2206.09112, 2022.arxiv.org/abs/2206.09112
  25. [28]Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang, and Xin Wang. Gppt: Graph pre-training and prompt tuning to generalize graph neural networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1717–1727, 2022.DOI
  26. [29]Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, and Jihong Guan. All in one: Multi-task prompting for graph neural networks.(2023). 2023a.arxiv.org/abs/2307.01504
  27. [30]Xiangguo Sun, Hong Cheng, Bo Liu, Jia Li, Hongyang Chen, Guangdong Xu, and Hongzhi Yin. Self-supervised hypergraph representation learning for sociological analysis. IEEE Transactions on Knowledge and Data Engineering, 35(11):11860–11871, 2023b.arxiv.org/abs/2212.11440
  28. [31]Qiaoyu Tan, Ninghao Liu, Xiao Huang, Soo-Hyun Choi, Li Li, Rui Chen, and Xia Hu. S2gae: Self-supervised graph autoencoders are generalizable learners with graph masking. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pp. 787–795, 2023.DOI
  29. [32]A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.arxiv.org/abs/1706.03762
  30. [33]Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. ICLR (Poster), 2(3):4, 2019.
  31. [34]Song Wang, Yushun Dong, Xiao Huang, Chen Chen, and Jundong Li. Faith: Few-shot graph classification with hierarchical task graphs. arXiv preprint arXiv:2205.02435, 2022.arxiv.org/abs/2205.02435
  32. [35]Marcin Waniek, Tomasz P Michalak, Michael J Wooldridge, and Talal Rahwan. Hiding individuals and communities in a social network. Nature Human Behaviour, 2(2):139–147, 2018.
  33. [36]Huijun Wu, Chen Wang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, and Liming Zhu. Adversarial examples on graph data: Deep insights into attack and defense. arXiv preprint arXiv:1903.01610, 2019.arxiv.org/abs/1903.01610
  34. [37]Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1365–1374, 2015a.DOI
  35. [38]Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1365–1374, 2015b.DOI
  36. [39]Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in neural information processing systems, 33:5812–5823, 2020.arxiv.org/abs/2010.13902
  37. [40]Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Multiprompt for multi-task pre-training and prompting on graphs. In Proceedings of the ACM on Web Conference 2024, pp. 515–526, 2024.arxiv.org/abs/2312.03731
  38. [41]Xiang Zhang and Marinka Zitnik. GnnGuard: Defending graph neural networks against adversarial attacks. Advances in neural information processing systems, 33:9263–9275, 2020.arxiv.org/abs/2006.08149
  39. [42]Kaijie Zhu, Jindong Wang, Jia Heng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Yue Zhang, Neil Zhenqiang Gong, et al. Promptbench: Towards evaluating the robustness of large language models on adversarial prompts. arXiv preprint arXiv:2306.04528, 2023.arxiv.org/abs/2306.04528
  40. [43]Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131, 2020.arxiv.org/abs/2006.04131
  41. [44]Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021, pp. 2069–2080, 2021.DOI
  42. [45]Chenyi Zi, Haihong Zhao, Xiangguo Sun, Yiqing Lin, Hong Cheng, and Jia Li. Prog: A graph prompt learning benchmark. arXiv preprint arXiv:2406.05346, 2024.arxiv.org/abs/2406.05346
  43. [46]Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2847–2856, 2018.
  44. [47]Daniel Zügner and Stephan Günnemann. Adversarial attacks on graph neural networks via meta learning. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=Bylnx2O9YX.arxiv.org/abs/1902.08412

Paper details

vlnc:W_mwfgypsu
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents