Enhancing Stock Market Information Retrieval with Retrieval-Augmented Generation

Introduction

Large Language Models (LLMs) Gemini Team et al. [11] OpenAI [21] have made remarkable progress in recent years, establishing themselves as a fundamental component in various natural language processing (NLP) applications. Their ability to understand and generate human-like text has led to significant advancements in tasks such as machine translation, sentiment analysis, question answering, and text summarization. As a result, LLMs have been widely adopted across multiple domains, including education Kasneci et al. [14], healthcare Sallam [25], finance Wu et al. (2023), code writing Chen et al. [3], cybersecurity Franco et al. [10], the labor market Eloundou et al. [8], and legal aspects Sun [27], where they enhance automation, improve decision-making, and enable more natural human-computer interactions. One of the most promising approaches in this domain is Retrieval-Augmented Generation (RAG), which combines the strengths of retrieval-based and generative models to enhance the accuracy and reliability of generated content.

Despite the impressive capabilities of existing LLMs, their development is still constrained by several challenges. One of the most significant issues is the hallucination problem Dale et al. [5], where models generate responses that are fluent and coherent but factually inaccurate. Another major limitation is the difficulty in updating their knowledge. Since LLMs store information within their internal memory Meng et al. [19], keeping them up to date requires retraining or fine-tuning with new data—an expensive and resource-intensive process. Additionally, general-purpose LLMs often lack expertise in specialized domains Colombo et al. [4] Zhang et al. [31], making them less effective for domain-specific tasks. Developing LLMs tailored to a particular field requires extensive effort in dataset collection and model training, further increasing the complexity of implementation. This makes the deployment of systems for specific tasks costly in terms of both time and resources. Furthermore, collecting and processing training data for each specialized domain can present challenges such as a lack of high-quality data, security requirements, and the complexity of fine-tuning the model to fit the specific context.

Stock markets are highly dynamic, influenced by a wide range of factors, including economic indicators, corporate financial reports, geopolitical events, and investor sentiment. Efficiently retrieving relevant financial information and generating accurate insights is crucial for investors, analysts, and decision-makers. Traditional NLP models often struggle with the complexity and ever-changing nature of financial data, making it challenging to provide real-time, contextually accurate, and explainable outputs. To solve this problem, RAG presents a potential solution by integrating retrieval mechanisms with generative capabilities, allowing models to dynamically fetch the most relevant financial data and generate informed responses.

Several recent studies Guu et al. [12] Lewis et al. [16] have proposed leveraging external knowledge bases to enhance the ability of LLMs to generate better responses, a technique known as Retrieval-Augmented Generation. (RAG). By supplying LLMs with retrieved relevant factual information, the hallucination problem can be alleviated to some extent. Moreover, the knowledge update issue can also be addressed by updating the external knowledge database, allowing LLMs to incorporate up-to-date information. RAG can also transform a general LLM into a domain-specific LLM by constructing and utilizing a specialized knowledge database. Therefore, RAG plays a crucial role in enhancing the functionality of LLMs, making them more accurate, knowledgeable, and reliable across a wide range of applications Wu et al. (2024). Figure 1 illustrates the fundamental differences in workflow between a system without RAG and a system enhanced with RAG, highlighting the impact of retrieval-augmented knowledge integration.

This study aims to explore the effectiveness of RAG models in processing stock market information. Specifically, we investigate how RAG can enhance the retrieval of information and the generation of responses regarding stock market news published on VnExpress. By leveraging external financial knowledge sources, RAG models may enhance decision-making processes, reduce misinformation, and offer more comprehensive market summaries. In summary, the structure of this paper is as follows: § 2 reviews related work in financial NLP and RAG applications. § 3 describes the proposed methodology, including data sources, model architecture, and evaluation metrics. § 4 presents experimental results and discussions. Finally, § 5 concludes the paper with key findings and potential future research directions.

Related Work

NLP in Stock Market

Financial NLP has been extensively used to analyze stock market data, predict trends, and support investment decisions. Several studies have explored the effectiveness of NLP models in processing financial iinformation. Financial NLP has been extensively used to analyze stock market data, predict trends, and support investment decisions. Several studies have explored the effective ness of NLP models in processing financial information. Akita et al. Akita et al. (2016) utilized Long Short-Term Memory (LSTM) networks to predict stock prices by combining textual and numerical data from the Nikkei newspaper. Their research highlighted the importance of cross-company event correlation in financial forecasting, demonstrating how news about one company could impact the stock prices of others within the same industry. Similarly, Sidra and Jaydip Sen & Mehtab (2021) explored multiple machine learning and deep learning techniques for weekly stock market predictions, integrating sentiment analysis from social media with historical stock prices. Their findings showed that public sentiment plays a crucial role in improving stock movement predictions.

Another notable study proposed an LSTM-based forecasting model that incorporated sentiment analysis to establish correlations between stock values and news sentiment. The results demonstrated that using NLP for feature extraction and LSTM for time-series training significantly improved prediction accuracy, reducing the mean square error to 0.062 Patel et al. [22]. Additionally, ”Smart Trader,” an AI-powered chatbot designed for the Saudi Tadawul market, showcased the practical applications of NLP and machine learning in stock trading. By leveraging real-time market data and user preferences, the chatbot provided personalized investment recommendations, filling a gap in the market for AI-driven financial advisory services Alhydary et al. [2].

Retrieval-Augmented Generation (RAG) in NLP

Retrieval-Augmented Generation (RAG) is a hybrid approach that enhances language models by integrating external knowledge retrieval mechanisms. Instead of relying solely on parametric memory, RAG retrieves relevant documents or facts from external sources and conditions its responses based on retrieved content. This method is particularly useful for knowledge-intensive NLP tasks such as open-domain question answering and fact-based text generation Lewis et al. [15]. By dynamically incorporating relevant information at inference time, RAG models can mitigate issues like hallucination and outdated knowledge, which are common challenges in traditional language models.

The architecture of RAG consists of two main components: a retriever module and a generator module. The retriever module searches for relevant external knowledge, while the generator module synthesizes a response based on the retrieved content. The retriever module usually comprises three components: an encoder for encoding inputs into embeddings, an efficient indexing system that supports approximate nearest neighbor search, and a datastore for storing external knowledge in the form of key-value pairs Wu et al. (2024). Beside, Retrieval-augmented (RA) generators and default generators are the two primary types of generator modules. Default generators encompass most pre-trained or fine-tuned large language models, including Mistral models Jiang et al. [13], Gemini-series models Gemini Team et al. [11] Reid et al. [23], and GPT-series models OpenAI [21] Zhang & Li [32]. In contrast, RA generators integrate retrieval mechanisms to enhance response generation, incorporating modules like Enc-Dec Li et al. (2022) and RETRO Wang et al. [28]. These generators refine their outputs by leveraging retrieved knowledge, allowing them to generate more contextually informed responses or predictions.

Recent studies have demonstrated the effectiveness of RAG in improving factual accuracy and reducing misinformation in AI-generated text. For example, RAG models have been used to enhance chatbot responses by retrieving real-time information from external databases Lewis et al. [15]. Additionally, advances in retrieval mechanisms, such as dense passage retrieval and contrastive learning, have further improved the quality of retrieved content, leading to more coherent and contextually relevant responses.

RAG in Financial and Stock Market Applications

Retrieval-Augmented Generation (RAG) has been increasingly applied in financial domains to enhance decision-making by integrating real-time knowledge retrieval with language model capabilities. In market sentiment analysis, fine-tuned models like FinBERT within a RAG pipeline have been used to analyze financial news and predict stock price movements, though sentiment alone shows limited predictive power Echambadi [7]. Similarly, RAG improves financial analysis by addressing challenges such as lack of interpretability in ML models and hallucinations in LLMs. The AlphaFin dataset and Stock-Chain framework have demonstrated the effectiveness of integrating retrieval mechanisms for financial reasoning tasks Li et al. (2024).

Beyond sentiment analysis, RAG plays a crucial role in multi-modal financial analytics. The FinTMMBench benchmark evaluates temporal-aware RAG systems that process diverse financial data sources, such as stock prices, news, and technical charts, enabling more comprehensive trend analysis and event detection Zhu et al. [34]. Additionally, in financial risk assessment, RAG enhances LLM performance in audit report analysis by improving faithfulness and context relevance. Experiments show that models like LLaMa3.1 and Gemini-1.5-flash significantly improve financial risk evaluations by extracting detailed insights from reports ?. These applications highlight the potential of RAG in financial markets, bridging the gap between real-time information retrieval and AI-generated financial insights.

Data Preparation

Data Collection

In this study, we collected data from VnExpress, a well-known Vietnamese online news platform, to analyze stock market trends. The dataset comprises 30 news articles related to the stock market, which were crawled from VnExpress. These articles cover various aspects of the stock market, including trading volume, stock price fluctuations, corporate financial performance, and macroeconomic factors influencing market trends.

Each category represents a key dimension of the stock market. Market Trends & Index Performance tracks overall market movement and index fluctuations. Corporate Stocks & Sectors highlights individual stock performances and sector-wide trends. Foreign Investment & Liquidity focuses on the role of international capital in the Vietnamese stock market. Lastly, Regulatory & Corporate Actions examines government policies, financial regulations, and corporate decisions that influence market behavior.

Category	Number
Market Trends & Index Performance	8
Corporate Stocks & Sectors	9
Foreign Investment & Liquidity	7
Regulatory & Corporate Actions	6
Total	30

This categorization helps in understanding the key trends and themes in the stock market as reported by VnExpress. The data will be utilized for further analysis to develop a Retrieval-Augmented Generation (RAG) system that enhances stock market news comprehension and decision-making support.

Data Preprocessing

Stop words are commonly used words in a language that carry minimal semantic meaning, such as conjunctions, prepositions, and auxiliary verbs. Removing stop words is a crucial step in text preprocessing as it helps reduce dimensionality, improve computational efficiency, and enhance the relevance of text-based features for downstream natural language processing (NLP) tasks. Figure 2 illustrates the frequency of removed stop words in the documents.

In this study, we implemented a stop word removal process specifically for Vietnamese text. Given the unique characteristics of the Vietnamese language, where words are often multi-syllabic and require tokenization before processing, we utilized the Underthesea library for word segmentation. The preprocessing pipeline consists of several key steps.

First, the input text was normalized by stripping leading and trailing whitespaces. Additionally, all URLs and special characters were removed using regular expressions to eliminate noise in the data. To ensure uniformity, all text was converted to lowercase, which helps in minimizing variations due to capitalization.

Next, since Vietnamese is not naturally space-separated like English, we applied word_tokenize from the Underthesea library to segment text into meaningful tokens. Following tokenization, we curated a list of common Vietnamese stop words and removed them from the tokenized text. This step reduces redundancy and ensures that only semantically meaningful words are retained for further analysis.

After performing these preprocessing steps, the cleaned text was stored for subsequent stages of the information retrieval and retrieval-augmented generation (RAG) pipeline. The removal of stop words significantly enhances the quality of text embeddings by focusing on content-bearing words while discarding non-informative tokens. This refinement is particularly beneficial for improving the performance of semantic similarity searches and document retrieval tasks.

Proposed Methodology

This section presents the architecture of the proposed Retrieval-Augmented Generation (RAG) system, which enhances large language model (LLM) responses by integrating an external knowledge retrieval mechanism. The system consists of three main components: text embedding, information retrieval, and response generation. Figure 3 illustrates the overall workflow of the RAG system, depicting how queries are processed through embedding, information retrieval, and response generation.

Text Embedding

The embedding process converts textual documents into numerical vectors using a pretrained embedding model. Figure 4 illustrates the process of embedding textual data into numerical vector representations. As mentioned above, the documents consist of stock market news articles crawled from VnExpress, one of Vietnam’s most popular online newspapers. VnExpress provides up-to-date financial news, market analysis, and expert opinions, making it a valuable data source for financial insights.

Specifically, we employ the LaBSE (Language-agnostic BERT Sentence Embedding) Feng et al. [9] model from the SentenceTransformers library. This model is chosen for its multilingual capabilities, allowing it to effectively encode text in various languages. Notably, LaBSE provides strong support for the Vietnamese language, ensuring high-quality semantic representations of Vietnamese financial news.

Documents are first preprocessed and tokenized using Jieba Zhang et al. [33], a widely used text segmentation tool primarily designed for Chinese. Despite its origin, jieba has been adapted for Vietnamese and is used here to segment sentences into meaningful tokens before embedding. Proper tokenization is crucial for accurately representing text semantics.

The embeddings are then computed and stored in a FAISS (Facebook AI Similarity Search) Douze et al. [6] database, which enables efficient nearest-neighbor retrieval. The generated embeddings form a high-dimensional vector space that facilitates rapid similarity comparisons. Once all textual data has been transformed into vector representations, they are systematically stored in the FAISS DB store for efficient retrieval during the next processing stages.

Information Retrieval

Given a user query, the retrieval process aims to fetch the most relevant documents from the FAISS database. The retrieval mechanism follows a hybrid approach that integrates FAISS-based vector similarity search with BM25 (Best Matching 25) Robertson [24] ranking. This combination leverages both semantic similarity and term frequency-based relevance to enhance the accuracy of retrieved documents.

To perform vector-based retrieval, the input query is first embedded using the LaBSE model, ensuring a dense numerical representation in the same vector space as the stored documents. This query embedding is then compared against the stored document embeddings using FAISS. FAISS efficiently retrieves the top-$k$ nearest neighbors by measuring the L2 distance (Euclidean distance) between the query and stored vectors. The L2 distance between two vectors $\mathbf{q}$ (query) and $\mathbf{d}$ (document) in an $n$-dimensional space is computed as follows:

where: $q_i$ and $d_i$ represent the individual components of the query and document embeddings, respectively. A smaller L2 distance indicates higher similarity between the query and document.

After retrieving an initial set of candidate documents from FAISS, a BM25 ranking step is applied to further refine the results. BM25 is a TF-IDF-based retrieval model that scores documents based on their word frequency, document length normalization, and inverse document frequency (IDF). This step ensures that retrieved documents are not only semantically similar but also contextually relevant based on keyword occurrences. The BM25 score for a document $D$ given a query $Q$ is computed as:

where:

• $f(t,D)$ is the term frequency of term $t$ in document $D$.

• $\lvert D\rvert$ is the length of document $D$ (total number of words in $D$).

• $\operatorname{avg}D$ is the average document length in the corpus.

• $k_1$ is a tuning parameter controlling term frequency saturation.

• $b$ is a tuning parameter controlling document length normalization.

The inverse document frequency (IDF) component is defined as:

where:

• $N$ is the total number of documents in the corpus.

• $n(t)$ is the number of documents containing term $t$.

The BM25 ranking model is particularly effective in handling keyword-based relevance, especially for queries where specific terms are important. By applying BM25 after FAISS retrieval, we re-rank the initially retrieved documents to prioritize those that best match the user’s intent.

This hybrid approach ensures that the final set of documents includes both semantically related and keyword-relevant results, optimizing the retrieval process for accuracy and contextual relevance.

Information Generation

The final stage of the system focuses on generating a response based on the retrieved documents. We utilize the Gemini model, specifically Gemini-1.5-Flash Jiang et al. [13] Mondillo et al. [20], due to its optimized performance for real-time applications, lower latency, and ability to process long-context inputs efficiently. This choice ensures that the model can effectively integrate retrieved knowledge while maintaining responsiveness.

To generate a well-informed response, a structured prompt is carefully crafted by incorporating the retrieved documents as contextual information. The prompt is formatted to include a clear instruction, the user’s query, and the most relevant retrieved texts, ensuring that the model leverages external knowledge effectively.

Once the structured prompt is prepared, Gemini-1.5-Flash processes it and generates a response that aligns factually with the retrieved information. The model does this by grounding its output in the provided texts, minimizing hallucinations, and maintaining consistency with real-world stocket market informations.

By leveraging this retrieval-augmented approach, the system significantly enhances the quality of responses, ensuring that they are not only coherent and contextually aware but also well-supported by factual knowledge.

Experimental Result

Evaluation Metrics

To assess the performance of our Retrieval-Augmented Generation (RAG) system in generating responses related to stock market information, we employ ROUGE (Recall-Oriented Understudy for Gisting Evaluation) Score as the primary evaluation metric. ROUGE is a widely used metric for evaluating text summarization and generation tasks by comparing the overlap of n-grams, word sequences, and longest common subsequences between the generated text and reference answers.

We calculate ROUGE-1, ROUGE-2, and ROUGE-L scores, which are defined as follows:

• ROUGE-1 measures the overlap of unigrams (single words) between the generated text and the reference text:

• ROUGE-2 extends this to bigrams (two consecutive words), capturing more contextual meaning:

• ROUGE-L evaluates the longest common subsequence (LCS) between the generated and reference text, which captures sentence-level structure and fluency:

Where:

• $\operatorname{count}_{\mathrm{gen}}(w)$ and $\operatorname{count}_{\mathrm{ref}}(w)$ are the counts of word $w$ in the generated and reference texts, respectively.

• $W$ and $B$ are the sets of all unigrams and bigrams in the reference text.

• $LCS(\mathrm{gen}, \mathrm{ref})$ is the length of the longest common subsequence between the generated and reference text.

• $\operatorname{length}(\mathrm{ref})$ is the length of the reference text.

In addition to ROUGE scores, we measure Latency, which quantifies the system’s response time. Latency is defined as the time elapsed from the moment a query is submitted to the system until a response is generated. This metric is critical for assessing the real-time usability of our RAG system, particularly in financial applications where timely information retrieval is crucial.

By combining accuracy-based ROUGE metrics with latency measurement, we obtain a comprehensive evaluation of both the quality and efficiency of our system’s responses.

Retrieval-Augmented Generation (RAG) Results

To evaluate the effectiveness of our RAG system, we computed ROUGE scores based on a sample of 100 generated responses derived from stock market-related documents. The average ROUGE scores obtained are presented in Table 2.

ROUGE Metric	Precision	Recall	F-measure
ROUGE-1	0.9616	0.9597	0.9555
ROUGE-2	0.9374	0.9356	0.9314
ROUGE-L	0.9496	0.9478	0.9436

These results indicate that the system achieves high lexical and structural similarity between the generated responses and reference texts, demonstrating the capability of the RAG model in preserving key stock market information from the original documents. The high precision and recall values across all ROUGE metrics suggest that the system generates responses that closely align with the reference texts, ensuring both completeness and correctness in retrieving stock market-related data.

Specifically, the ROUGE-1 score of 0.9555 (F-measure) reflects the system’s ability to capture important individual keywords relevant to stock market discussions, ensuring that essential financial terms and concepts are accurately represented in the generated responses. Meanwhile, the ROUGE-2 score of 0.9314 indicates that the model effectively retains key bi-gram (two-word phrase) structures, which are crucial for maintaining contextual accuracy and conveying financial insights with coherence.

Moreover, the strong ROUGE-L score of 0.9436 highlights that the system maintains sentence structure and fluency, further reinforcing its reliability for financial text generation tasks. These consistently high scores across all three ROUGE metrics confirm the robustness of our approach in providing high-quality, contextually accurate, and well-structured responses for stock market information retrieval.

Apart from accuracy, latency performance was also measured to assess system efficiency. Over 100 queries, the latency statistics are as follows Tab 3:

Latency Metric	Time (seconds)
Average Latency	1.1626
Minimum Latency	0.917
Maximum Latency	2.455

These latency results suggest that the system provides responses within an acceptable time frame, balancing both accuracy and efficiency. The combination of high ROUGE scores and low latency values demonstrates the effectiveness of our RAG model for financial information retrieval. The relatively low response time further supports its practical usability in real-world applications where timely access to accurate stock market information is critical.

Conclusion

In this study, we evaluated the effectiveness of a Retrieval-Augmented Generation (RAG) system in generating responses related to stock market information. By utilizing the ROUGE metric, we assessed the lexical and structural similarity between the generated responses and reference texts. The high ROUGE-1, ROUGE-2, and ROUGE-L scores demonstrate that the system is capable of accurately retrieving and synthesizing financial data while maintaining coherence and fluency in the responses. Additionally, our analysis of latency indicates that the system operates within an acceptable response time, ensuring efficient query processing for real-time financial applications. The combination of high accuracy and low latency confirms that the proposed RAG model is a viable approach for handling stock market-related queries, providing reliable and timely information. Future work will focus on further improving the model’s ability to handle complex financial queries, integrating external knowledge sources to enhance response accuracy, and optimizing computational efficiency to reduce latency. These enhancements will contribute to making RAG-based systems more effective for real-world financial analysis and decision-making.

CRediT authorship contribution statement

Thai Phong Huan: Conceptualization, Methodology, Software, Writing – original draft, Writing – review & editing. Le Thi Anh Thu: Methodology, Software, Writing – review & editing. Nhan Huu Tran: Methodology, Software, Writing – review & editing. Lenh Cong Phan pham: Methodology, Software, Writing – review & editing. Hong Nhung Nguyen: Supervision, Methodology, Validation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.