WorksEngineering
Synergizing Wireless Foundation Models and LLMs: A Multi-Task Framework for MA Systems
Abstract
While movable antennas (MAs) provide enhanced spatial flexibility through dynamic repositioning, they confront critical challenges in acquiring complete channel state information (CSI) and efficiently optimizing antenna positions. Encouragingly, wireless foundation models (WFMs) capture generalizable channel priors for high-quality CSI extrapolation, while large language models (LLMs) enable parameter-efficient, token-level optimization for wireless tasks. In this paper, we synergistically integrate a WFM and an LLM into a multi-task framework for MA systems. First, to address the mismatch between existing WFM pre-training and MA channel characteristics, we develop an MA-oriented ``ExtLWM'' based on the large wireless model (LWM) architecture. Second, we design a unified ExtLWM-Connector-LLM architecture that flexibly adapts to different MA configurations without structural redesign. Third, we reformulate discrete MA positioning as a sequential decision-making problem, where candidate positions are represented by numerical tokens and optimized using group relative policy optimization (GRPO) to exploit the serial generation capability of the LLM. Finally, a multi-stage training strategy is developed to maintain multi-task performance under limited training samples while mitigating catastrophic forgetting. Simulation results show that ExtLWM can produce high-quality channel embeddings while outperforming various benchmarks in CSI extrapolation. Meanwhile, the proposed LLM-based approach achieves a favorable trade-off between performance and computational complexity.
Introduction
Movable antenna (MA) is an emerging technology for next-generation wireless communications, providing a new approach to further enhance wireless transmission capability [?, ?]. Specifically, unlike conventional fixed-position antenna (FPA) arrays, whose elements are fabricated at predetermined locations, MAs are connected to radio-frequency chains through flexible cabling and can be programmably moved via precise motor control [?, ?]. This position reconfigurability introduces additional spatial degrees of freedom (DoFs), allowing MA systems to exploit geometry gain and spatial diversity to improve communication performance without the need for substantial array-size expansion [?, ?], [?, ?].
Existing MA systems typically adopt either continuous or discrete positioning mechanisms. Although continuous-position adjustment provides greater spatial flexibility and potentially higher performance, it poses significant implementation challenges in practice [?, ?]. First, the achievable positioning step size is fundamentally restricted by the physical resolution of mechanical actuators, necessitating sophisticated, high-precision hardware to approximate continuous movement [?, ?], [?, ?]. Second, continuous position adjustment typically entails frequent repositioning, resulting in elevated actuator power consumption, prolonged movement latency, and substantial control overhead [?, ?]. Consequently, discrete-position adjustment offers a more favorable trade-off between system performance and implementation complexity.
Despite its practical advantages, discrete-position MA system will inevitably confront two issues in deployment: The difficulty of acquiring complete CSI from limited channel observations and the complexity of discrete MA position optimization under implementation constraints. Firstly, for CSI acquisition, due to the limited movement speed of MAs and the constrained pilot overhead, only a small subset of candidate positions can be explored within a given channel coherence interval, resulting in incomplete CSI observations [?, ?]. To address this issue, a feasible approach is to extrapolate the CSI of all unobserved candidate positions from incomplete CSI. In [?, ?], the authors design an extrapolation algorithm by learning a low-dimensional channel basis and estimating the corresponding latent coefficients. Nevertheless, such model-based methods heavily rely on predefined channel priors. In contrast, data-driven deep neural network (DNN)-based methods can directly learn spatial channel correlations among different candidate positions from large-scale CSI samples, thereby enabling more flexible CSI extrapolation. For instance, the authors of [?, ?] design an asymmetric architecture consisting of a self-attention-based encoder and a graph-attention-based decoder. In [?, ?], the authors augment the encoder with a mixture-of-experts module to enhance feature extraction and noise resilience. Besides, similar studies [?, ?], [?, ?] have demonstrated that masked autoencoder-based architectures can achieve superior CSI extrapolation performance compared with conventional network architectures. Although the above dedicated CSI extrapolation models can recover complete CSI to directly support downstream tasks [?, ?], [?, ?], their training typically relies on sufficient complete CSI samples and their transferability to unseen scenarios remains limited.
Wireless foundation models (WFMs), which learn generalizable channel representations through large-scale pre-training, offer a promising avenue to mitigate these challenges by delivering superior sample efficiency across diverse downstream tasks [?, ?]–[?, ?]. The large wireless model (LWM) proposed in [?, ?] achieves near-saturated line-of-sight (LoS)/non-LoS (NLoS) identification performance with less than 1% labeled samples, while attaining comparable beam prediction performance using only 40%-50% of the training data. Regarding cross-scenario transferability, the ContraWiMAE proposed in [?, ?] maintains competitive performance when transferred to unseen propagation scenarios and carrier frequencies, further demonstrating the generalizability of its pre-trained channel representations. These advantages are particularly valuable for MA systems, where acquiring sufficient complete CSI samples across candidate positions is often impractical, while channel characteristics can vary substantially across different propagation environments and position configurations. Nevertheless, the use of WFMs for CSI extrapolation in MA systems, particularly under highly incomplete observations and diverse scenarios, remains insufficiently explored.
Secondly, discrete MA position optimization requires selecting the location of each MA from a large number of candidate points while satisfying inter-MA distance constraints, giving rise to a computationally challenging combinatorial optimization problem. Conventional branch-and-bound (BnB) methods can obtain high-quality solutions but suffer from prohibitive computational complexity as the search space grows [?, ?], whereas penalty-based relaxation (PBR) methods reduce the optimization burden at the cost of non-negligible performance loss [?, ?]. Besides, network-based approaches also face inherent limitations in handling such discrete combinatorial structures, since directly formulating the problem as a classification task leads to an excessively large output space, whereas generating continuous positions followed by discretization may result in suboptimal or even infeasible solutions [?, ?], [?, ?]. Overall, existing approaches still suffer from certain limitations in terms of computational efficiency and solution quality.
To strike a more favorable balance, the combinatorial MA position optimization problem can be reformulated as a sequential optimization problem. By decomposing the high-dimensional joint decision into a sequence of lower-dimensional decisions, sequential optimization can significantly alleviate the curse of dimensionality and substantially reduce optimization complexity [?, ?]. Meanwhile, compared with parallel feedforward computation, sequential optimization can provide greater effective computational depth, and thus has the potential to achieve better performance on problems with strong sequential dependencies under comparable model sizes and training data [?, ?], [?, ?]. Notably, large language model (LLM) inherently follows an autoregressive generation mechanism, which is well aligned with the sequential decision structure of the reformulated MA positioning problem. In addition to this structural compatibility, LLMs can be adapted to wireless communication tasks through lightweight task-specific training with only a limited number of trainable parameters. Existing studies have demonstrated such parameter-efficient adaptation [?, ?]-[?, ?]. For example, the authors of [?, ?] fine-tune only about 5.95M parameters of GPT-2 and achieve sum-rate performance comparable to that of a 13.15M-parameter transformer trained from scratch. Moreover, the authors of [?, ?] propose a prompt-as-prefix strategy that converts historical optimal beam indices and angle information into discrete numerical tokens, thereby improving the average normalized beamforming gain of GPT-2 by approximately 62.4% for the beam prediction task. However, existing studies mainly focus on parameter-efficient adaptation of pre-trained LLMs to individual wireless tasks, while their intrinsic autoregressive capability for sequential optimization in wireless communications remains largely unexplored. This gap motivates the use of pre-trained LLMs as sequential decision models for discrete MA position optimization.
More importantly, CSI extrapolation and MA position optimization should not be treated as isolated tasks, but rather jointly constitute an integrated wireless communication process. Developing separate task-specific models would lead to duplicated representation learning, inefficient use of limited training samples, and limited scalability across different MA configurations. To enable unified and efficient operation throughout this entire process, synergizing WFMs and LLMs offers a promising multi-task paradigm. Specifically, WFMs can perform reliable CSI extrapolation and extract informative feature embeddings from limited training samples, while LLMs can leverage these representations to support both classification and sequential optimization. However, effectively integrating these two types of foundation models into a unified architecture remains nontrivial, and systematic studies on multi-task MA systems are still limited.
Motivated by the above analysis, this paper innovatively integrates the WFM and LLM to support multiple downstream tasks in MA systems. Specifically, we consider three representative tasks, including CSI extrapolation as a regression task, LoS/NLoS identification as a classification task, and sum-rate maximization as a sequential optimization task. Building upon this framework, this paper aims to explore the under-investigated network architectures and training paradigms for synergizing WFMs and LLMs in MA systems, and seeks to answer the following pivotal questions: Q1) Can existing WFMs developed for complete CSI in FPA systems and primarily designed for feature extraction be effectively transferred to MA systems? In other words, is it necessary to develop a dedicated WFM for MA systems under incomplete CSI? Q2) How can WFMs and LLMs be seamlessly integrated at the network architecture level to construct a unified framework with a fixed backbone? Q3) How can the potential of LLMs for sequential optimization of discrete MA positions be effectively exploited? Q4) How can a unified training paradigm be designed under limited training samples while maintaining satisfactory performance across different tasks? Correspondingly, the main contributions of this paper can be summarized as follows:
For Q1), we develop a WFM for MA systems under incomplete CSI, termed “ExtLWM”. ExtLWM can simultaneously provide high-quality extrapolated CSI and informative feature embeddings. Simulation results demonstrate that ExtLWM exhibits strong generalization across different scenarios, CSI incompleteness ratios, orthogonal frequency division multiplexing (OFDM) configurations, and array structures. Moreover, its CSI extrapolation performance outperforms the FPA-oriented LWM-1.1 and models trained from scratch, with more pronounced advantages under limited training data.
For Q2), we design a lightweight Q-Former-based connector to seamlessly integrate the WFM and LLM. Meanwhile, representative tasks in MA systems are unified within the proposed framework: CSI extrapolation is handled by the WFM, while LoS/NLoS identification and sum-rate maximization are further formulated as token-level optimization tasks for the LLM. Simulation results demonstrate that the proposed framework can effectively accommodate different MA system configurations, and also exhibits good compatibility with different LLMs.
For Q3), we formulate the sequential MA positioning problem within a Markov decision process (MDP) framework and represent MA positions using numerical tokens. Building on this formulation, we employ a group relative policy optimization (GRPO) algorithm for optimization. Simulation results demonstrate that the proposed method can effectively exploit the capability of LLMs while alleviating the credit-assignment challenge. Moreover, it achieves higher sum-rate performance than conventional PBR methods with low execution overhead.
For Q4), we propose a multi-stage training algorithm for data-limited scenarios. In particular, to preserve the performance of the classification task while optimizing the sum-rate, we incorporate experience replay (ER) into GRPO training process. Simulation results demonstrate that the proposed multi-stage training strategy achieves better performance across multiple tasks than joint training, while the incorporation of ER more effectively mitigates catastrophic forgetting.
System Model and Problem Formulation
Figure 1. Overview of the MA system under incomplete CSI, where a multimodal model simultaneously enables multiple tasks.
We consider an MA-enabled OFDM downlink communication system, where a BS is equipped with MAs to serve single-antenna users. As shown in Figure 1, the BS is equipped with a one-dimensional array featuring candidate points spaced by distance that are available for the deployment of antenna elements (). Let , and denote the sets of all candidate positions, MAs and users, respectively. Define position vector to indicate the position indices of MAs with the entries arranged in ascending order, i.e., for . Besides, to mitigate mutual coupling effect, a minimum separation distance should be maintained between MAs, i.e.,
where is the “safety spacing” with . The BS employs OFDM modulation, where , , , and represent the subcarrier set, center carrier wavelength, center carrier frequency, and subcarrier spacing, respectively. The frequency of the -th subcarrier is .
Channel Model and Incomplete Observation
The downlink channel is described by a multi-carrier geometry-based channel model to characterize multipath propagation, which is consistent with the modeling principles adopted by state-of-the-art ray-tracing-based channel simulators/datasets, such as DeepMIMO [?, ?] and Sionna [?, ?]. Specifically, define as the channel vector between the -th user and all candidate locations at subcarrier , which is modeled as
where denotes the number of effective propagation paths for user , while , , , and denote the path loss, frequency-independent complex path coefficient, propagation delay, and angle of departure (AoD) of the -th path of user , respectively. Moreover, denotes the array steering vector corresponding to the candidate MA positions, which is further expressed as
where denotes the speed of light. Therefore, the complete channel matrix of the -th user is denoted as . Subsequently, denote as the channel coefficient associated with the -th candidate position, i.e., the -th element of . Then, the downlink channel vector between MAs with positions and the -th user at the -th subcarrier can be expressed as
Consequently, the channel matrix spanning MAs across all subcarriers is denoted by , which is indeed a submatrix of the complete CSI . Conventional studies [?, ?], [?, ?] typically assume perfect knowledge of complete CSI across all candidate positions, often acquired by concatenating channel measurements across multiple time slots. However, obtaining comprehensive CSI remains practically daunting due to prohibitive pilot overhead and stringent demands on antenna mechanics for instantaneous physical relocation of antenna elements. To be practical, we define the more accessible incomplete CSI for the -th user as , where each subcarrier vector is formulated as
where represents an indicator vector with indicating whether the channel coefficient of position is observable. Furthermore, we define the CSI incompleteness ratio as , which represents the proportion of candidate positions with unobservable CSI.
CSI Extrapolation
Given that complete CSI is indispensable for MA configuration, it is highly desirable to reconstruct complete CSI matrix. Specifically, we aim to extrapolate the CSI of unobserved antenna positions based on the sparse measurements through inference, instead of conventional physical scan [?, ?]–[?, ?]. Extrapolation of the channel coefficients of unseen positions (i.e., those with ) reads
where is the extrapolation function and is a prediction of the true channel . Accordingly, let represent the extrapolated CSI matrix for user , whose accuracy can be evaluated by normalized mean square error (NMSE) defined as
LoS/NLoS Identification
LoS/NLoS identification is an important component of wireless environment awareness and facilitates context-aware tasks such as parameter estimation and positioning [?, ?]. Leveraging the acquired incomplete CSI, the identification process for the -th user is formulated as
where represents the identification algorithm/network, while and signify the classified and genuine CSI category for user , with and indicating LoS and NLoS conditions, respectively. To assess the identification capability across users, we adopt the weighted F1-score, denoted as , as the performance metric, which jointly accounts for class-wise precision and recall while weighting each class according to its sample proportion, making it suitable for evaluating performance under imbalanced channel-type distributions.
Sum-Rate Maximization
The sum-rate across all users and subcarriers is a key metric for MA-enabled multi-user communication systems. Let denote the precoder of the -th user on subcarrier . The received signal by the -th user on subcarrier can be written as
where denotes the precoding matrix for subcarrier , vector denotes uncorrelated information symbols following complex Gaussian distribution with zero-mean and unit variance, i.e., , and represents the receiving noise. Consequently, the signal-to-interference-plus-noise ratio (SINR) of the -th user on subcarrier is given as
Subsequently, the achievable communication rate of the -th user on subcarrier is given by
Recalling (4), the sum-rate of all users at all subcarriers, i.e.,, is jointly determined by the positions of MAs and beamforming. Therefore, the sum-rate maximization task under incomplete CSI can be expressed as .
Problem Formulation
In this paper, instead of relying on traditional task-specific architectures that design , , and separately, we propose a unified multimodal network with parameter to perform all tasks concurrently. Therefore, the multi-task optimization problem, centered on sum-rate maximization, is formulated as follows
where eq_constraint_pos and eq_constraint_p specify the constraints on the MA positions, eq_constraint_nmse and eq_constraint_f1 mandate performance thresholds for CSI extrapolation and LoS/NLoS identification, respectively, and eq_constraint_power imposes the power constraint with fixed transmit power .
In this paper, we synergistically integrate the WFM and LLM to construct a unified network parameterized by , which flexibly accommodates varying numbers of MAs and candidate positions. Specifically, the WFM supports the effective use of the LLM in sample-limited environments by providing high-quality channel embeddings and accurate CSI extrapolation. The LLM then leverages the wireless domain knowledge provided by the WFM to formulate different wireless tasks as token-level optimization problems. Besides supporting conventional classification tasks, the LLM can fully exploit its sequential generation capability for the MA positioning.
Unified network architecture
In this section, we introduce the unified framework that synergistically integrates the WFM and LLM via a connector, i.e., . The detailed architectures of the individual network components are introduced in the following subsections.
ExtLWM-based WFM
LWM has been demonstrated to be effective in extracting channel features for OFDM systems and capable of facilitating diverse downstream physical-layer applications. Considering the mismatch between existing WFMs and MA systems, we develop an MA-oriented WFM based on the LWM architecture, termed “Extended LWM (ExtLWM)”. ExtLWM enables high-quality CSI extrapolation from incomplete CSI while simultaneously producing informative channel embeddings to support subsequent downstream tasks. Defining as the parameters of ExtLWM, the designs of its components (as illustrated in Figure 2) are specified in the following.
Input Layer
The input to ExtLWM consists of incomplete CSI measurements collected at candidate positions whose locations are known. These measurements are organized into a formatted data matrix , which comprises position-wise patches. Specifically, for , the -th patch of is given as
where is the channel coefficient of the -th MA over the -th subcarrier, i.e., , and is a scaling parameter. Moreover, the CSI patches corresponding to vacant candidate positions are replaced with a predefined value1, i.e.,
where is the indication vector (recall (5)).
Embedding Layer
The embedding layer projects the masked CSI data into a high-dimensional space, facilitating the encoder in discovering hidden features related to spatial correlation among antenna positions as well as multipath and scattering characteristics. The signal flow of the embedding layer can be expressed as follows
where the linear mapping elevates the signal dimension from to and denotes the layer normalization operation, denotes the position embedding of patches that encodes position information into signal.
Encoder Layer
The encoder layer aims to extract high-dimensional features from incomplete CSI embeddings, thereby establishing a robust representation capable of capturing intricate intrinsic correlations. Inspired by the architecture of LWM-1.1 [?, ?], the encoder layer consists of cascaded encoder blocks with identical structures. The -th encoder block takes as the input and outputs the enhanced representation , where the initial input is given by .
Each block integrates a multi-head attention module followed by a feed-forward network (FFN) module. Specifically, within the -th block, the input is first projected onto three independent linear subspaces to generate the query, key, and value matrices, denoted as , , and , respectively. To enable parallel processing of distinct feature subspaces, each matrix is evenly partitioned into heads. The scaled dot-product attention for the -th head is computed as
where . Subsequently, the outputs of all attention heads are concatenated, linearly transformed, and further processed by the FFN, yielding the output of the -th encoder block, . After passing through encoder blocks, ExtLWM produces the final CSI representation as .
Figure 2. Proposed unified network architecture, with the multi-stage training strategy for supporting multiple tasks.
Masked Modeling Head
ExtLWM retains a masked modeling head (MMH), which leverages the high-dimensional latent embeddings extracted by the preceding modules to recover the CSI. It is worth noting that, in [?, ?], MMH is employed only during pre-training to endow LWM-1.1 with task-agnostic and generalizable feature extraction capability. In contrast, in our framework, MMH is not only used for feature learning during training, but also continuously performs CSI extrapolation for subsequent downstream tasks. Specifically, MMH employs an FFN to map the output embedding to the extrapolated CSI. Denoting the -th row of by , the corresponding output is given by
and predicts the CSI associated with the -th candidate position, i.e., -th row of .
Connector
The connector is designed to bridge the modality gap by mapping from the embeddings of ExtLWM into the linguistic embedding space of LLM. Inspired by [?, ?], the connector employs a lightweight Q-Former architecture for efficient cross-modal alignment. With trainable parameters , the connector consists of an embedding layer, a query layer, and a projection layer, which are detailed in the following.
Embedding Layer
To efficiently extract semantic information from the CSI embeddings, we employ a learnable embedding matrix to generate query tokens. The initialized tokens will serve as interface for cross-modal interaction to capture the latent manifold structure of the CSI embeddings.
Query Layer
The query layer is composed of transformer blocks to distill language-relevant representations from CSI features. Specifically, each block consists of the following three components:
Self-Attention: For the -th block, an -head self-attention module is first applied to the query embeddings to facilitate interactions among the query tokens and enhance representational diversity.
Cross-Attention: Subsequently, the queries act as “probes” to interact with the CSI features through an -head cross-attention module, where the keys and values are projected from . This enables the queries to selectively extract informative task-relevant features from the CSI embeddings.
FFN: Finally, the updated query tokens are passed through an FFN to produce the output for the -th block, denoted as .
Projection Layer
To ensure compatibility with the downstream LLM, the output of the final query block, , is linearly projected into the GPT input embedding space, yielding , where denotes the hidden dimension of GPT. The projected embeddings serve as “soft prompts”, conditioning GPT on the CSI features extracted by ExtLWM.
GPT-Based LLM
We employ a frozen LLM to unify multiple downstream wireless tasks as token-level optimization problems. In particular, for MA positioning, we exploit the pre-trained autoregressive sequential generation capability of the LLM to enable parameter-efficient adaptation. Specifically, the LLM-based cross-modal generation process can be formulated as
where denotes the pre-trained LLM parameters, denotes the aligned CSI embeddings, and denotes the ordered sequence of generated token IDs, with . Specifically, for LoS/NLoS identification, and . For sum-rate maximization, and . In this paper, we adopt the GPT-2 architecture [?, ?] as the LLM backbone due to its moderate parameter scale and practical implementation feasibility. Its main components are described as follows.
Embedding Layer
The embedding layer fuses the CSI embeddings with the textual embeddings. Specifically, GPT-2 employs the standard byte-pair encoding tokenizer and maps text tokens into -dimensional embeddings through the word-token embedding (WTE) and word-position embedding , which are then combined with the CSI embeddings to form the multimodal input:
where denotes the text embedding of the -th word token, and . Packing all textual embedding into the matrix , the final multimodal input sequence is constructed by prepending the CSI embeddings to the textual prefix
Decoder Layer
The backbone adopts a decoder-only architecture consisting of cascaded decoder blocks with identical structures, which are employed to progressively extract hierarchical latent representations. The -th decoder block takes as the input and outputs the enhanced representation , where the initial input is given by . Specifically, within each block, through normalizing and linearly projecting , query, key and value matrices , , and are obtained and then split into heads to compute multi-head self-attention, with the -th head output given as
where is the multimodal causal mask. The mask enables bidirectional attention among the CSI tokens while preserving causal attention for the textual tokens. The outputs from all attention heads are then concatenated and processed by the FFN to obtain .
Output Layer
The output layer maps the final decoder representation to vocabulary logits for next-token generation. Following GPT-2, the output projection shares the token embedding matrix , reducing the parameter count while maintaining consistency with the pre-trained token space. Specifically, the unnormalized logits for the -th token are given by:
The generation of the -th token is subsequently executed by selecting the token with the highest posterior probability:
where denotes a task-specific restricted subset of tokens tailored for .
For LoS/NLoS identification, , corresponding to word unit “” and “”, respectively, which enables GPT-2 to leverage its inherent linguistic priors to output classification results in a semantic manner.
For sum-rate maximization, is defined as the set of token IDs corresponding to the position indices . By leveraging numerical tokens with explicit stable semantics, the discrete position optimization of MAs can be naturally transformed into a sequential generation problem involving consecutive tokens2.
Multi-Stage Training Strategy
Building upon the unified framework, this section develops a complete multi-stage training algorithm to enable parameter-efficient optimization of CSI extrapolation, LoS/NLoS identification, and sum-rate maximization under data-limited conditions. Details of training stages will be specified in the following subsections.
Stage-1: Pre-Training for ExtLWM
The pre-training stage encourages ExtLWM to learn generic structural features of MA channels from a large number of diverse and complex scenarios. Its optimization objective is to train the MMH to reconstruct the original unmasked CSI in (12) based on the output in (16). Accordingly, the self-supervised training loss is formulated as:
where denotes the set of users used for pre-training, denotes the set of masked patch indices (representing candidate antenna positions) for the -th user, with its cardinality varying across different pre-training scenarios.
To further enhance the CSI extrapolation capability of ExtLWM under highly incomplete CSI, we increase the masking ratio from about in LWM-1.1 to , exposing the model to more severe CSI incompleteness during pre-training. Meanwhile, the masking indices are randomized in each epoch, encouraging ExtLWM to learn spatial priors that are more robust to different CSI observation patterns and thereby improving its ability to capture informative channel features from sparse CSI.
Stage-2: Fine-Tuning for CSI Extrapolation
Building on the paradigm of large-scale pre-training followed by task-specific fine-tuning, we perform full-parameter fine-tuning of to adapt the generalized spatial representations to scenario-specific propagation characteristics. This process is particularly effective in data-scarce regimes, where ExtLWM leverages the learned channel priors to achieve high-fidelity CSI extrapolation with limited labeled samples.
Considering (?), stage-2 aims to fully exploit the CSI extrapolation capability of ExtLWM to satisfy eq_constraint_nmse, and its training objective is formulated as
Meanwhile, the training loss for stage-2 is formulated as
where denotes the set of users used for fine-tuning with complete CSI. Upon completion of stage-2, we will freeze to preserve its channel feature extraction and CSI extrapolation capabilities.
Stage-3: Fine-Tuning for LoS/NLoS Identification
The fine-tuned ExtLWM can effectively extract and output scenario-specific channel feature embeddings, serving as a shared feature backbone for multiple downstream wireless tasks. Based on eq_constraint_f1, stage-3 aims to optimize the connector to enable the adaptation of LLM to LoS/NLoS identification. The training objective of stage-3 can be expressed as
and the training loss of stage-3 can be formulated as
where denotes the set of users used for training, for which only incomplete CSI samples and the corresponding LoS/NLoS labels are required, is the ground-truth label and is the predicted probability for class . Recall (22), and can also be interpreted as the softmax-normalized scores for the semantic tokens, i.e., “” and “”, respectively.
Stage-4: RL for Sum-Rate Maximization
In this subsection, we proceed to maximize sum-rate formulated in eq_max_rate. Distinctively, we propose a paradigm shift by analogizing the antenna positioning to token prediction in natural language processing. Just as an LLM predicts the most probable “optimal” tokens within a massive vocabulary based on linguistic context, we aim to identify the optimal antenna positions from a large number of candidates. The core philosophy is to leverage the feature extraction capability of the LLM to interpret wireless channels. By treating the previously selected antenna positions and their corresponding channel states as context, the model is empowered to predict the subsequent optimal positions.
Subsequently, we formulate MA positioning as an MDP to capture sequential dependencies and align it with LLM autoregressive generation. The key elements of the MDP are defined as follows:
State : encapsulates the multimodal context, comprising the CSI embeddings output by ExtLWM and the previously generated text tokens:
Action : represents the prediction of -th token ID . Recall that we establish a semantic mapping between the discrete space and the physical coordinate of the -th MA, ensuring eq_constraint_p is satisfied.
Action mask : We incorporate a mask vector to prevent LLM from generating invalid violating the minimum antenna spacing constraint eq_constraint_pos. Specifically, denotes the mask over that identifies the valid candidate tokens for , i.e., the feasible positions of the -th MA:
where for and for , while . After the masking process, the logits corresponding to all invalid tokens are set to .
Policy : The policy depicts a probability distribution over the candidate set with the multimodal context and physical constraints given. In the training phase, the policy gradient algorithm iteratively refines , enabling the model to learn the mapping to generate high-sum-rate antenna positions.
Reward : A scalar reward is computed after generating all actions. In data-limited scenarios, the ground-truth CSI of all users may not be available. Fortunately, the reward constructed from the high-quality extrapolated CSI provided by ExtLWM can still effectively guide policy optimization, and is defined as:
where is the -th column of . Various precoding schemes can be seamlessly incorporated to jointly construct the reward function, such as MMSE, weighted MMSE (WMMSE) [?, ?], and zero forcing (ZF). In this paper, we adopt MMSE precoder to achieve a favorable trade-off between sum-rate performance and computational efficiency. Specifically, assuming that the total transmit power is equally allocated across all subcarriers, the MMSE precoder can be expressed as
where and .
The constructed MDP provides a natural framework for deploying RL methods to optimize the sequential MA positioning policy. To efficiently preserve the basic semantic and LoS/NLoS classification capabilities of the LLM, we only train while keeping the other modules frozen. Therefore, in stage-4, the original problem (?) can be reformulated as
where denotes the roll-out trajectory. denotes the trajectory reward specified in (30). Note that the constraints in eq_constraint_pos and eq_constraint_p are guaranteed by the action space and masking mechanism, respectively, while eq_constraint_power is satisfied by the normalized precoder and eq_constraint_nmse is addressed in stage-2. Hence, only eq_constraint_f1, i.e., eq_constraint_cl, needs to be explicitly handled. For each training epoch, we denote the -th trajectory as , and define its -th transition tuple as , which is sequentially stored into buffer .
To fully exploit the potential of the LLM for sequential optimization, we adopt the GRPO-based RL algorithm, an LLM-specific and training-efficient method that does not require an additional critic network [?, ?]. Specifically, GRPO directly performs advantage estimation based on group-relative rewards, thereby effectively reducing the variance of advantage estimation and alleviating the credit-assignment difficulty in sequential decision making. The advantage value of the -th trajectory is computed based on the trajectories rolled out in the current training epoch:
where is the terminal reward of the -th trajectory, which is inherently delayed and sparse since the marginal contribution of each MA cannot be explicitly measured. The GRPO loss is then formulated using the clipped surrogate objective to constrain the update magnitude:
where denotes the mini-batch subsets randomly drawn from , represents the probability ratio between the updated and old policies.
Meanwhile, to improve the training stability of GRPO, we introduce entropy regularization to better balance exploration and exploitation. The exploration loss can be expressed as
where denotes the policy entropy.
Moreover, to preserve the LoS/NLoS classification performance in eq_constraint_cl during sum-rate maximization, we incorporate experience replay (ER), a widely used strategy in continual learning (CL) [?, ?], to mitigate catastrophic forgetting. Specifically, ER can be naturally integrated with the replay buffer of the underlying RL algorithm by additionally storing a fixed set of labeled samples , which provides stable supervision throughout training and regularizes policy updates. Since only a small subset is required to effectively preserve the classification capability, the additional memory overhead introduced by ER remains limited. Finally, within each epoch, the loss function for sum-rate maximization can be expressed as
where is defined similarly to (27), and are hyperparameters controlling the priority of exploration and ER strategy, respectively.
Summary
We obtain ExtLWM through pre-training in stage-1. For a specific downstream MA system, stages-2-4 progressively endow the multimodal model with multi-task capabilities under data-limited conditions to solve problem (?). Notably, after stage-2, ExtLWM serves as an environment-specialized model to support a self-contained framework, providing both high-quality feature embeddings and extrapolated CSI to support the token-level optimization in stages-3/4 and the construction of the reward function (including precoder) in stage-4, respectively.
We now analyze the computational complexity of the proposed multimodal model for generating MA positions. Specifically, the incomplete CSI of the users is first processed by ExtLWM. The CSI of each user is successively passed through the embedding layer and encoder blocks, resulting in a computational complexity of . The extracted CSI embeddings are then fed into the connector, where learnable query tokens and query blocks are employed to project the CSI features into the embedding space of the LLM, leading to a computational complexity of . Subsequently, GPT-2 autoregressively generates the MA position tokens based on the resulting CSI embeddings, with a computational complexity of , where . Therefore, by neglecting lower-order terms and constant factors, the dominant computational complexity of the proposed model can be expressed as .
Numerical Results
In this section, we provide comprehensive simulation results and performance analysis. We first evaluate the pre-training and CSI extrapolation performance of ExtLWM. Next, we evaluate the LoS/NLoS identification capability of the proposed network. Finally, we evaluate the LLM-based MA positioning scheme in terms of sum-rate performance. All experiments were conducted on a hardware platform equipped with two NVIDIA GeForce RTX 4090 GPUs.
Pre-Training Performance
Consistent with LWM-1.1 [?, ?], the pre-training dataset comprises 120 scenarios from DeepMIMO-V3 [?, ?] with GHz and kHz, while the ExtLWM (2.4M parameters) hyperparameters are set to , , , , and . Among all users, of the CSI samples—corresponding to —are used for training. In this paper, we train three WFMs to accommodate downstream MA systems with , as summarized in Table 1. In terms of pre-training configuration, while retaining most settings from LWM-1.1, we set epoch to more effectively capture the latent structure of CSI. The peak GPU memory consumption during pre-training is approximately GB.
| Model Name | Pre-training candidate position | Cost | |
| ExtLWM-sub16 | 40 80 120 160 200 240 280 320 | 16 | 98h |
| ExtLWM-sub32 | 30 60 90 120 160 | 32 | 78h |
| ExtLWM-sub64 | 20 40 80 | 64 | 76h |
Table 1. Training configurations for three WFMs.

Figure 3. Training and validation losses for WFMs.

Figure 4. ExtLWM’s performance under various conditions.
Figure 3 presents the loss trajectories of the entire training phase of the three foundation models, demonstrating the effectiveness and stability of the training process. In this paper, we adopt “ExtLWM-sub16” as the foundation model for all subsequent experiments, referred to simply as ExtLWM3.
Next, we separately fine-tune ExtLWM on 14 sets of out-of-distribution CSI samples, where 50% of the samples in each unseen MA dataset are used for training. Figure 4 presents the validation NMSE under and . It can be observed that all curves rapidly decrease below within 500 epochs, demonstrating the generalization capability of ExtLWM across diverse unseen channel distributions from multiple perspectives. Specifically, 1) We consider two DeepMIMO scenarios that are not included in the pre-training dataset, namely “O1-blockage” and “O2-dynamic”, to demonstrate the cross-scenario generalization capability of ExtLWM. 2) We evaluate ExtLWM using CSI generated by QuaDRiGa [?, ?] and Sionna [?, ?], thereby assessing its generalization capability across different channel-generation platforms beyond DeepMIMO. 3) We evaluate channel samples generated at GHz and GHz to investigate the cross-frequency generalization capability of ExtLWM. 4) We consider kHz and kHz to evaluate its generalization across different subcarrier spacings. 5) We further generate CSI using different candidate-position spacings, i.e., and , to validate the cross-array-spacing generalization capability of ExtLWM beyond the pre-training grid with . 6) We evaluate channel configurations with and , thereby assessing the generalization capability of ExtLWM across different numbers of subcarriers. 7) Finally, we evaluate ExtLWM using uniform planar array (UPA) channels with and , demonstrating that ExtLWM can capture spatial and propagation structures beyond a specific ULA geometry.
CSI Extrapolation Performance
In this subsection, we demonstrate the CSI extrapolation performance of fine-tuned ExtLWM in the specific downstream scenario, which is constructed on the data of 1250 users lying between row-1400 and row-1500 in “Boston5G” (, , GHz, and ). To simulate the sample-limited environment, 50% of CSI samples, i.e., , are used for fine-tuning ExtLWM. The batch size, training epochs, and learning rate are set to 128, , and , respectively. The peak GPU memory consumption and training time for stage-2 are approximately GB and h, respectively.
To provide a comprehensive performance comparison, we test the following benchmarks:
LWM (60 scenarios): The number of pre-training scenarios for ExtLWM is reduced from 120 to 60.
LWM (1 scenario): A dedicated extrapolation model is first trained on the “ASU Campus” scenario and then fully fine-tuned on the target scenario.
LWM-1.1 [?, ?]: For the case of , the original patch shape of is reconfigured to , enabling LWM-1.1 to be directly applied to CSI extrapolation.
LWM-scratch: A transformer with the same architecture as LWM is trained from scratch.
Bi-LSTM: We replace the encoder blocks of LWM with a Bi-LSTM architecture having a comparable number of parameters and train the resulting model from scratch.
BERT [?, ?]: Following [?, ?], we replace the embedding layer and MMH of the BERT with the corresponding ExtLWM modules, and fine-tune the last four encoder blocks.

Figure 5. Validation loss for CSI extrapolation.
Figure 5 shows the validation loss curves of different models. It can be observed that both ExtLWM variants start from relatively low NMSE levels and then decrease rapidly during fine-tuning, indicating that MA-specific pre-training provides effective channel representations for CSI extrapolation. In comparison, LWM-1.1 remains in the region around an NMSE of 1 for a noticeably shorter period than the model trained from scratch, suggesting that its pre-training still preserves certain generic representations of geometric wireless channels. Finally, the BERT-based model consistently exhibits a higher validation loss, indicating a mismatch between its pre-trained representations and wireless channel characteristics, which cannot be sufficiently compensated for using only a limited number of downstream CSI samples.

Figure 6. Impact of CSI incompleteness ratio.
In Figure 6, we evaluate the CSI extrapolation performance under different CSI incompleteness ratios . The results show that ExtLWM consistently achieves superior performance. When the number of pre-training scenarios is increased from 60 to 120, ExtLWM achieves an average CSI extrapolation gain of approximately dB, indicating that pre-training over more diverse scenarios facilitates the learning of more transferable spatial correlations and multipath propagation characteristics. Meanwhile, compared with the model trained from scratch, the standard ExtLWM achieves NMSE gains of more than dB and dB at CSI incompleteness ratios of and , respectively. Notably, even when pre-trained using only 60 scenarios, ExtLWM still outperforms the state-of-the-art LWM-1.1, further demonstrating the necessity of MA-specific WFM pre-training. Specifically, LWM-1.1 is primarily designed to learn spatial–frequency correlations from regular FPA channels and thus has limited capability in characterizing the distinctive spatial distribution patterns of MA channels. Moreover, LWM-1.1 adopts a masking ratio of only during pre-training, resulting in insufficient exposure to highly incomplete CSI patterns.

Figure 7. Impact of training data volume.
Figure 7 illustrates the impact of data volume for training, indicated by training ratio . Results show that ExtLWM exhibits increasingly pronounced performance gains over the model trained from scratch as the amount of training data decreases, achieving NMSE improvements of 8.9 dB, 9.1 dB, and 10 dB at training ratios of 20%, 10%, and 5%, respectively. Moreover, dedicated models typically capture only the propagation characteristics and spatial correlations of specific scenarios, making it difficult to effectively adapt to new channel distributions during cross-scenario transfer. When the amount of training data in the new scenario is further reduced, their performance also degrades more significantly. These results demonstrate that WFMs can achieve better CSI extrapolation performance in data-limited scenarios.
LoS/NLoS Identification Performance
In this subsection, we validate LoS/NLoS identification performance. The experiments are conducted using the data associated with “BS3” and 1250 users from row-500 in “O1-blockage” scenario from DeepMIMO-V3, with the system setting given by , , GHz, , . To simulate the sample-limited and label-scarce environment, 50% of the users with complete CSI () are used to fine-tune ExtLWM, while 1% of users with known labels () are used to train the connector. The LLM adopts “GPT-2-small” (124M parameters) from the GPT-2 family, with , , and . The connector is configured with and (0.4M parameters) [?, ?]. The training hyperparameters are set as and . The peak GPU memory consumption and training time for stage-3 are approximately GB and min, respectively.
To compare with the proposed multi-stage training strategy, we test other training schemes as follows:
CLS embedding [?, ?]: The LWM architecture supports the output of a single CLS embedding that efficiently aggregates information across the entire input. We replace the CSI embedding with the CLS embedding output by the fine-tuned ExtLWM for the identification task.
Joint fine-tuning [?, ?]: Based on the pre-trained ExtLWM, we jointly fine-tune all parameters, i.e., and , to optimize training loss of both extrapolation in (25) and identification in (27), i.e., .
Direct fine-tuning: We directly apply the pre-trained ExtLWM to the identification task, i.e., without first going through the extrapolation task to fine-tune .
Raw CSI: We bypass ExtLWM and use only the raw incomplete CSI for the identification task.

(a) Lab.-CSI rat.: 1%,

(b) Lab.-CSI rat.: 0.5%,
Figure 8. Performance vs. labeled CSI ratio.
In Figure 8, we plot both the F1-score of LoS/NLoS identification (on the y-axis) and the NMSE of CSI extrapolation (on the x-axis) obtained by the aforementioned training schemes using different ratios of LoS/NLoS-labeled training samples. Several key observations are elaborated in the sequel. 1) Firstly, even with very limited labeled CSI, i.e., , our multi-stage training delivers F1-score of 1.00 and 0.998, respectively, at high CSI incompleteness ratios of and , underscoring its superior robustness in extremely label-scarce regimes. 2) Secondly, as shown in Figure 8a, replacing CSI embeddings with CLS leads to F1-score degradations of 12.7% and 11%. This gap widens to 22.7% and 22.0% for fewer labels in Figure 8b. This is due to the fact that incomplete CSI renders CLS representations susceptible to capturing missing/noisy patterns over salient physical features, impairing its classification capability. 3) Finally, the Pareto frontier of joint fine-tuning is depicted in Figure 8, which is dominated by our proposed multi-stage training paradigm. This sub-optimality arises because acute label scarcity intensifies inter-task gradient conflict and exacerbates task-dominance imbalances.

(a) Raw CSI

(b) w/ CSI extrap.

(c) w/o CSI extrap.
Figure 9. Impact of CSI extrapolation on embedding.
Figure 9 provides -SNE projections of CSI embeddings comparing the post-extrapolated ExtLWM (Figure 9b) with the pre-trained counterpart (Figure 9c) to visualize the impact of extrapolation task on LoS/NLoS identification. The results reveal that integrating CSI extrapolation stimulates the model to distill latent features of MA channels more effectively and to yield more discriminative CSI representations that substantially bolster the performance of downstream tasks.
Sum-Rate Maximization Performance
In this subsection, we proceed to examine the effectiveness of RL training in resolving the sum-rate maximization task. In addition to the system settings specified in § 5.3, we set , , dBm, dBm, and . For RL algorithm, we set epoch , learning rate , , , and , while is annealed from 0.1 to 0.02. To implement ER, we set and . The average execution times of the proposed framework for CSI extrapolation, LoS/NLoS identification, and sum-rate maximization are approximately s, s, and s, respectively. The peak GPU memory consumption and training time for stage-4 are approximately GB and h, respectively.

Figure 10. Performance of ER method.
In Figure 10, we evaluate the efficacy of ER in maintaining LoS/NLoS identification precision using the validation set in § 5.3. Results show that ER sustains an F1-score of 1.00 during sum-rate optimization. In contrast, omitting ER leads to catastrophic forgetting at epoch 49, with model losing its previously held identification accuracy, which never recovers. Notably, even with reduced from 12 to 2 (“2 samp” in legend), ER still preserves identification capabilities perfectly, whereas skipping the preliminary fine-tuning on the identification task (denoted as “JFT, 2 samp.” in the legend) results in an F1-score of only 0.824. These results demonstrate the effectiveness of the ER-based strategy and its strong training-sample efficiency. Moreover, we compare the joint fine-tuning scheme (denoted as “JFT”), which skips stage-3 and directly optimizes LoS/NLoS identification during sum-rate maximization. The results show that its sum-rate performance degrades noticeably, while the maximum F1-score reaches only 0.979. Together with the results in Fig. 8, these experiments further demonstrate the effectiveness of the proposed multi-stage training strategy. Finally, we compare ER with elastic weight consolidation (EWC) algorithm [?, ?], which mitigates forgetting by constraining parameters important to previous tasks. Experimental results show that such constraints inhibit sum-rate gains, validating the superior performance trade-off achieved by ER.

Figure 11. Performance of diverse LLMs.
Figure 11 validates the effectiveness and generalizability of the proposed multi-stage training paradigm. We first deploy the proposed framework on GPT-2-large (774M) and GPT-2-xl (1.5B), which adopt different configurations of , , and . The results demonstrate that the proposed training strategy can achieve satisfactory performance across LLMs with similar architectures but different model scales. We then further consider DialoGPT (124M) [?, ?], a dialogue-oriented variant fine-tuned from GPT-2, and show that the proposed training framework can also be directly transferred to task-specialized LLMs. Finally, we further consider Qwen3 (0.6B) [?, ?] and Gemma3 (270M) [?, ?], demonstrating that the proposed modality alignment mechanism (19), modified causal mask (20), and corresponding training strategy remain applicable to LLMs with different architectures4.

Figure 12. Performance of RL method.
| Precoder | WMMSE | MMSE | ZF |
| Execution time (s) | 0.487 | 0.052 | 0.052 |
| Sum-rate (bps/Hz) | 9.684 | 6.180 | 1.443 |
Table 2. Overall comparison of different precoders.
In Figure 12, we validate the effectiveness of the proposed RL method from the following aspects: 1) The designed reward function can effectively guide the improvement of the actual sum-rate, further validating that ExtLWM can generate high-quality extrapolated CSI and feature embeddings in a self-contained manner to effectively guide MA positioning. 2) Removing the entropy loss term in (34) results in a 14.45% sum-rate degradation, demonstrating the importance of balancing exploration and exploitation during training. 3) Compared with PPO [?, ?] and the classical policy gradient (PG) method, i.e., REINFORCE [?, ?], GRPO achieves clearly better performance, mainly benefiting from its group-relative advantage estimation, which can effectively alleviate the credit-assignment difficulty. Meanwhile, replacing numerical tokens with random tokens leads to an 8.28% sum-rate degradation. These results demonstrate that the proposed design can better exploit the potential of LLMs for sequential MA positioning. 4) Replacing the frozen “GPT-2-small” with an 8.45M-parameter fully trained transformer instead results in a 6.8% sum-rate degradation. In contrast, the proposed method only trains the 0.4M parameters of the connector, demonstrating its ability to effectively exploit the trainable-parameter efficiency of LLMs. 5) Removing the pre-trained weights of ExtLWM and training it from scratch only on the downstream scenario leads to a 10.68% sum-rate degradation, a 3.84 dB degradation in CSI extrapolation performance, and a reduction of the F1-score to 96.6%. These results demonstrate that integrating the WFM with the LLM can better support data-limited MA scenarios. 6) Replacing the WTE of LLM with -class embeddings transforms MA positioning into a single-shot parallel prediction based solely on , resulting in an 18.03% degradation in sum-rate performance. This demonstrates that sequential generation increases the serial computational depth of the LLM, thereby enabling it to better handle problems with sequential dependencies under a fixed network depth.
In Table 2, we compare the sum-rate performance and execution time achieved under different precoding schemes. The results show that different precoders can effectively serve as part of the reward function to guide the RL agent. Specifically, WMMSE achieves an approximately 56.7% sum-rate gain over MMSE, but requires approximately 9.37 times the execution time of MMSE. This substantially higher computational overhead mainly arises from the multiple alternating iterations over auxiliary variables required by WMMSE, together with the one-dimensional search performed in each beamforming update to determine the Lagrange multiplier satisfying the transmit power constraint. In contrast, MMSE achieves an approximately 328.3% sum-rate improvement over ZF, demonstrating its effectiveness in suppressing multiuser interference under correlated channel conditions. Therefore, considering the overall trade-off between achievable sum-rate and computational complexity, we choose MMSE precoder in the sum-rate maximization task.
Next, we compare several MA positioning algorithms for sum-rate maximization, as summarized below. For fairness, all benchmark schemes adopt the MMSE precoder (31).
BnB [?, ?] performs a systematic tree search over the discrete antenna position space.
PBR [?, ?] relaxes binary variables to continuous ones and enforces discreteness through penalty terms.
Greedy sequentially selects the optimal MA positions based on locally maximum sum-rate.
FPA assumes a fixed antenna spacing of .

Figure 13. Impact of on sum-rate.
Figure 13 illustrates the sum-rate performance achieved by different methods. The results show that sequential optimization can achieve satisfactory performance for MA positioning. Specifically, at dBm and dBm, the proposed scheme achieves and of the sum rate obtained by the BnB method, respectively. This indicates that sequential optimization indeed incurs a certain degree of suboptimality compared with directly searching the original combinatorial space. On the other hand, it outperforms the PBR method, mainly because the relaxation and penalty-based approximation may introduce a performance gap from the original combinatorial optimization problem.

Figure 14. Impact of on sum-rate.
Next, we compare overall performance of different algorithms under different , with the results presented in Figure 14 and Table 3, respectively. The results show that, by autoregressively generating MA positions based on the learned semantic representations, the proposed method achieves approximately of the upper-bound sum-rate performance. In contrast, the search space of BnB grows combinatorially with , leading to a rapidly increasing execution time and limiting its applicability to large-scale MA configurations. Compared with PBR, the proposed method achieves an average sum-rate gain of approximately , while avoiding the substantial computational overhead caused by repeated alternating optimization. Although Greedy achieves the lowest execution time, its myopic position selection strategy may lead to accumulated suboptimal decisions, resulting in noticeable performance degradation as increases. Overall, these results demonstrate that the proposed framework can achieve a favorable trade-off between sum-rate performance and computational efficiency.
| Proposed | BnB | PBR | Greedy | |
| 6 | 0.043s | 553.659s | 35.349s | 0.016s |
| 8 | 0.052s | 0.443h | 41.553s | 0.017s |
| 12 | 0.066s | 4.193h | 55.382s | 0.018s |
| 16 | 0.083s | 58.031h | 71.264s | 0.019s |
| 20 | 0.095s | – | 82.720s | 0.021s |
| 24 | 0.113s | – | 93.366s | 0.024s |
| 28 | 0.129s | – | 106.527s | 0.025s |
| 32 | 0.144s | – | 111.746s | 0.032s |
Table 3. Impact of on execution time.
| Execution time (s) | Sum-rate (bps/Hz) | |||||
| 2-7 | Proposed | PBR | Greedy | Proposed | PBR | Greedy |
| 100 | 0.089 | 47.901 | 0.018 | 8.885 | 8.662 | 8.120 |
| 140 | 0.092 | 67.840 | 0.019 | 9.631 | 8.977 | 8.300 |
| 180 | 0.095 | 82.720 | 0.021 | 10.451 | 9.281 | 8.426 |
| 220 | 0.099 | 106.807 | 0.028 | 10.924 | 9.420 | 8.572 |
| 260 | 0.104 | 139.765 | 0.030 | 11.138 | 9.609 | 8.641 |
Table 4. Impact of on execution time and sum-rate.
Finally, Table 4 presents the overall performance of different algorithms under different , where and . The results show that, with scenario-specific fine-tuning of ExtLWM, the proposed framework can effectively learn MA channel characteristics under different candidate-position spaces. As increases, the growth in overall inference latency becomes noticeably milder than that observed with increasing , because ExtLWM, rather than GPT-2, becomes the dominant contributor to the computational overhead. In addition, the proposed multi-stage training algorithm maintains a stable average CSI extrapolation NMSE of approximately dB, while ER consistently preserves the LoS/NLoS identification performance with an F1-score of . These results further demonstrate the robustness and scalability of the proposed algorithm.
Conclusions
This paper proposes a unified ExtLWM-Connector-GPT architecture for multi-task processing in MA systems. Simulation results show that ExtLWM outperforms dedicated models and models trained from scratch in downstream CSI extrapolation. Meanwhile, the proposed approach effectively activates the sequential optimization capability of the LLM and achieves a favorable performance–complexity trade-off compared with conventional alternatives. In future work, we will investigate the integration of the proposed framework into multimodal semantic communication systems to extend its applicability. Moreover, we will explore partially unfreezing the LLM while preserving existing task performance to further investigate its potential performance gains.

Figure 15.

Figure 16.

Figure 17.

Figure 18.

Figure 19.

Figure 20.
References
No references were extracted for this paper.

