betaFeedback

WorksComputer Science

On the Applicability of pp-adic Numbers in Computational Modules for Neural Networks

Abstract

The applicability of pp-adic arithmetic as an intermediate computational module in an otherwise unchanged machine-learning pipeline is investigated. A one-feature linear regression on the scikit-learn Diabetes dataset serves as the test model. The fitted coefficients are represented in Qp\mathbb{Q}_p, predictions are computed in that field, and rational reconstruction is applied to the outputs. In the pyAdic experiments, rational reconstruction either returns no result or produces values that differ from the baseline predictions; changing computational precision can change both the magnitude and sign of the reconstructed value. In SageMath, forward conversions succeed, but the reconstructed predictions differ from the baseline. An alternative representation based on digit reversal fails to preserve ordinary addition because of carry propagation. Under the tested settings, none of the test predictions is reconstructed correctly. These results show that the tested conversion and reconstruction procedures do not provide a compatible intermediate module even for linear regression. Models defined directly over pp-adic data and operations offer a direction for further investigation.

Introduction

Kurt Hensel introduced pp-adic numbers at the end of the nineteenth century for the study of integers and equations in number fields. They are used in number theory and algebra and provide a metric and topology distinct from those of the real numbers [9, 10, 11].

Formally, the field Qp\mathbb{Q}_p is the completion of Q\mathbb{Q} with respect to the pp-adic norm ∥⋅∥p\|\cdot\|_p. Let pp be a prime. For a rational number aa the norm is

∥a∥p={0,a=0,p−k,a≠0, a=pkmn, p∤m, p∤n,\|a\|_p = \begin{cases} 0, & a = 0, \\ p^{-k}, & a \ne0,\ a = p^k \frac{m}{n},\ p \nmid m,\ p \nmid n, \end{cases}

that is, kk is the exponent of pp in aa. Classically, a pp-adic number is a sequence x={x1,x2,…}x = \{x_1, x_2, \ldots\} in which xnx_n is a residue modulo pnp^n. Equivalently, every element of Qp\mathbb{Q}_p can be written as a series

x=∑i=k∞aipi,k∈Z, ai∈{0,1,…,p−1}.x = \sum_{i=k}^{\infty} a_i p^i,\qquad k \in\mathbb{Z},\ a_i \in\{0,1,\ldots,p-1\}.

In the pp-adic metric, proximity is determined by agreement in the low-order digits of the pp-adic expansion, in contrast to the role of leading digits in real positional representations.

Applications of pp-adic methods include Diophantine equations, congruences, algebraic geometry, dynamical systems [3], and mathematical physics. The review by Dragovich et al. [12] surveys three decades of pp-adic mathematical physics, including applications to string theory and biological models.

Computer implementations of pp-adic arithmetic require finite representations or procedures for generating digits, together with a method of managing precision. Caruso’s lecture notes [2] compare the principal computational paradigms. Caruso, Roe and Vaccon [13] examine precision tracking, while van der Hoeven [14] develops lazy and relaxed computation with power series and pp-adic numbers. Implementations are available in computer algebra systems, including SageMath [6], and in standalone libraries, including pyAdic [5].

In machine learning, Zambrano-Luna and Zúñiga-Galindo developed pp-adic cellular neural networks and considered their application to image processing [7, 8]. Zúñiga-Galindo, He and Zambrano-Luna related pp-adic statistical field theory to deep Boltzmann machines; their construction yields convolutional networks with O(N)O(N) rather than O(N2)O(N^{2}) parameters [15].

The present study concerns a different integration scheme: a module that accepts rational or floating-point inputs, computes in Qp\mathbb{Q}_p, and returns rational outputs to an existing model. The intended benefit is reuse of the surrounding architecture, datasets, and software without redefining the entire pipeline over a pp-adic field. A further motivation is the potential application of such a module to the exploding-gradient problem.

The exploding-gradient problem arises when errors are backpropagated through many layers or time steps of a recurrent network. By the chain rule, the derivative involves a product of the Jacobian matrices of the individual transformations. Let gLg_L denote the loss gradient with respect to the final layer output and JℓJ_\ell the Jacobian of layer ℓ\ell. Then

g0=J1TJ2T⋯JLTgL.g_0 = J_1^{T}J_2^{T}\cdots J_L^{T}g_L.

In real arithmetic, this product can amplify some directions as network depth increases. Large gradients can cause excessively large parameter updates, unstable training and, in extreme cases, numerical overflow. Gradient norm clipping is a common way to limit this effect [16].

The motivation for a pp-adic approach is a different measure of the size of intermediate values. The pp-adic norm satisfies the strong triangle inequality

∥u+v∥p≤max⁡{∥u∥p,∥v∥p}.\lVert u+v\rVert_p \leq\max\{\lVert u\rVert_p,\lVert v\rVert_p\}.

Thus, addition alone cannot increase the norm beyond the largest norm of its summands. Define the vector norm ∥v∥p,∞=max⁡i∥vi∥p\lVert v\rVert_{p,\infty}=\max_i\lVert v_i\rVert_p and the matrix norm ∥A∥p,max⁡=max⁡i,j∥aij∥p\lVert A\rVert_{p,\max}=\max_{i,j}\lVert a_{ij}\rVert_p. Then

∥g0∥p,∞≤(∏ℓ=1L∥Jℓ∥p,max⁡)∥gL∥p,∞.\lVert g_0\rVert_{p,\infty}\leq\left(\prod_{\ell=1}^{L}\lVert J_\ell\rVert_{p,\max}\right)\lVert g_L\rVert_{p,\infty}.

If all Jacobian entries in a model over Qp\mathbb{Q}_p belong to Zp\mathbb{Z}_p, every factor on the right is at most one: backpropagation does not increase the pp-adic gradient norm. For example, repeated multiplication by pp produces the factor pLp^L, whose real magnitude grows while its pp-adic norm ∥pL∥p=p−L\lVert p^L\rVert_p = p^{-L} decreases. This illustrates a possible mechanism for controlling growth in the pp-adic metric.

Such control requires restrictions on derivatives and does not follow merely from changing the numerical type. Conversely, the factor 1/p1/p has norm pp and causes the pp-adic norm to grow under repeated multiplication. Boundedness in the pp-adic metric also does not imply bounded reconstructed real values: in the example above, pLp^L remains large as a real number. Applying this mechanism to training therefore requires differentiable pp-adic transformations, a loss function and a parameter-update rule, together with correct reconstruction when returning to a real-valued model. Here this possibility serves as motivation; the linear-regression experiments test arithmetic-module compatibility and do not demonstrate mitigation of exploding gradients.

The experiments in this study examined this scheme using linear regression and yielded a negative result. The problem statement is given in § 2, the computational tools in § 3, and the experimental design and results in § 4 and § 5. § 6 presents a quantitative assessment of reconstruction. Sources of error, further research directions, and the conclusions are discussed in § 7–§ 10.

Problem Statement

The proposed module receives rational or floating-point inputs, represents them in Qp\mathbb{Q}_p, performs arithmetic in that field, and reconstructs a rational output. Conversion is confined to the module boundary; the stored training data and the surrounding model retain their original representation. The module is intended to preserve compatibility with the existing pipeline.

The scheme requires both forward conversion and rational reconstruction. A rational input can be represented in Qp\mathbb{Q}_p to a specified precision. Reconstruction from a finite pp-adic approximation is not uniquely determined without additional restrictions, since multiple rational numbers can agree with the same approximation.

To map x=rs∈Qx = \frac{r}{s} \in\mathbb{Q} into Z/pkZ\mathbb{Z}/p^k\mathbb{Z}, it is enough to find the inverse q=s−1q = s^{-1} of the denominator, i.e. an element with q⋅s≡1(modpk)q \cdot s \equiv1 \pmod{p^k}. When gcd⁡(s,p)=1\gcd(s,p) = 1, this is done with the extended Euclidean algorithm or with continued fractions.

In the general case, powers of pp are first separated from the numerator and denominator:

x=rs⋅pv.(1)x = \frac{r}{s} \cdot p^v. \tag*{(1)}

If the denominator still has a common factor with pp, gcd⁡(s,p)=c≠1\gcd(s,p) = c \ne1, the following transformation is applied:

x=pv′pv′⋅cv′cv′⋅rs⋅pv=r⋅(p/c)v′s/cv′⋅p(v−v′).(2)x = \frac{p^{v'}}{p^{v'}} \cdot\frac{c^{v'}}{c^{v'}} \cdot\frac{r}{s} \cdot p^v = \frac{r \cdot(p/c)^{v'}}{s/c^{v'}} \cdot p^{(v-v')}. \tag*{(2)}

The remaining fraction is a unit of Zp\mathbb{Z}_p.

Linear regression is used as a minimal test of the conversion procedure within a model computation. Agreement with its baseline predictions is a necessary requirement before applying the same procedure to more complex neural-network models.

Tools and Implementation Paradigms

The representation of pp-adic numbers and the treatment of precision are relevant to the interpretation of library outputs. Caruso [2] distinguishes three principal computational approaches.

Zealous arithmetic tracks precision explicitly and is a pp-adic analogue of interval arithmetic. Each value is represented together with its precision as a ball

IN,a={x∈Qp:∣x−a∣≤p−N}=a+O(pN),I_{N,a} = \{x \in\mathbb{Q}_p : |x-a| \le p^{-N}\} = a + O(p^N),

and each operation updates both the computed digits and the precision bound. This finite representation allows precision losses to be tracked during computation.

Lazy and relaxed computations [14] represent a number by a procedure that generates digits as required. Lazy methods defer the computation of digits until they are requested. Relaxed methods reuse previously computed digits and progressively extend results when additional precision is required, rather than recomputing the entire approximation.

pp-adic floating-point arithmetic represents a number by a fixed number of significant pp-adic digits and an exponent. The number of stored digits is fixed, but this representation alone does not provide a certified precision bound for the result of every operation.

The choice of representation depends on the computational task. The experiments use the Python library pyAdic [5] and SageMath [6]. SageMath provides fixed modulus rings and implementations with capped absolute and capped relative precision, offering different approaches to precision management. The baseline regression is implemented in scikit-learn [4].

The pp-adic cellular networks [7, 8] are not used as experimental baselines because they address different model constructions. The observations concern only the tested intermediate arithmetic module and do not constitute an assessment of pp-adic methods in neural networks generally.

Experimental Design

The baseline experiment uses the Diabetes dataset from scikit-learn [4]. A one-feature linear regression is fitted using the original floating-point representation. Its slope and intercept are converted to pp-adic numbers, predictions are computed in Qp\mathbb{Q}_p, and rational reconstruction is applied to the results.

The comparison evaluates whether the sequence of input conversion, arithmetic, and rational reconstruction reproduces the baseline predictions. This tests the compatibility of the pp-adic computation with the input and output representations of the model.

Additional experiments examine sensitivity to pp-adic precision, an analogous computation in SageMath, and an alternative representation based on reversal of the digit order.

Results

Linear Regression with pp-adic Coefficients

The baseline regression metrics are given in § 6. In the experiment, the coefficients aa and bb were converted to pp-adic representations apa_p and bpb_p and used to evaluate the corresponding linear expression. The reconstructed outputs differed from the baseline predictions [1].

Most reconstructions in the displayed output returned False rather than a rational value. Thus, the tested reconstruction procedure failed to recover predictions compatible with the baseline model.

A reduced test isolated the conversion of a single floating-point input to a pp-adic representation and back. The value 0.04716281 was converted with p=2p = 2 and absolute precision 40, followed by the built-in rationalise function. The reconstructed value had the opposite sign and differed by an order of magnitude [1]. The conversion discrepancy therefore occurred even without a regression calculation.

Dependence on Precision

Changes in the reconstructed values with the precision setting motivated a separate examination of precision dependence.

In pyAdic, the input is first represented as a fraction ab\frac{a}{b} of integers using Python’s fractions module. The pp-adic representations apa_p and bp′=(1b)pb'_p = \left(\frac{1}{b}\right)_p are then computed and multiplied: ap⋅bp′a_p \cdot b'_p.

The reconstructed values varied substantially with precision: the same rational input produced different outputs, sometimes with opposite signs. A floating-point value such as 0.04716281 becomes an exact fraction with large numerator and denominator values, making rational reconstruction sensitive to the available precision. Under the tested settings, pyAdic did not provide stable recovery of the rational values required by the intermediate module.

SageMath

An analogous scenario was examined in SageMath [6] to assess whether discrepancies also occurred with another implementation. Its fixed modulus rings and implementations with capped absolute and capped relative precision provide alternative ways of managing the finite representations discussed in [2].

In SageMath, capped relative precision was set to 80 significant pp-adic digits. With this setting, rational-to-pp-adic conversions completed successfully, unlike the problematic pyAdic cases. However, rational reconstruction of the computed results did not reproduce the baseline predictions [1].

The reconstruction discrepancy therefore also occurred in a second computational environment. Successful forward conversion alone was insufficient to preserve compatibility with the baseline model after the complete round trip.

Reversing the Digit Order

The alternative representation reverses the digit order so that leading digits become trailing digits and conversely. Its motivation is the different role of digit positions in real and pp-adic metrics. Such a representation does not, by itself, preserve arithmetic operations.

In ordinary positional addition, carries propagate towards higher powers of the base. After digit reversal, preserving the original addition would require propagation in the opposite direction in the reversed record. The resulting discrepancy is illustrated by the relation

int⁡(inv⁡(a)+inv⁡(b))≠a+b.(3)\operatorname{int}(\operatorname{inv}(a)+\operatorname{inv}(b)) \ne a+b. \tag*{(3)}

Figure 1 reproduces the decimal example from the experiments in this study. The right-hand expression uses the reversed digit order with a corresponding shift of the decimal point. The sums differ: reversing the digits of 309579000 gives 0.00975903 rather than 0.10885902. The carry-direction problem also motivates consideration of binary positional arithmetic, including the significands used in IEEE 754 representations.

Addition in the original and digit-reversed representations

Figure 1. Addition in the original and digit-reversed representations, reproduced from the experimental materials [1]. Ordinary addition does not preserve the proposed correspondence in this example because of carry propagation.

Preserving arithmetic under this representation requires explicit definitions and implementations of the transformed operations rather than direct reuse of ordinary addition. This conflicts with the aim of inserting a module without changing the surrounding computational system. Errors in low-order digits of the reversed representation are especially relevant in the pp-adic metric, which also complicates the intended application to exploding gradients.

ApproachExperimental observationConsequence
pyAdic + linear regressionMost displayed reconstructions return FalseTested procedure does not reproduce baseline
Varying precisionReconstructed values vary, including sign changesUnstable recovery under tested settings
SageMathForward conversions complete; reconstructed values differRound trip does not preserve predictions
Reversed digitsOrdinary addition does not preserve the representationTransformed operations required

Table 1. Summary of the experiments

Quantitative Assessment of Reconstruction

The main regression experiment uses the Diabetes dataset from scikit-learn [4], feature index 2, the last 20 observations as the test set, and pyAdic [5] with p=2p = 2 and precision 50. Agreement after reconstruction is evaluated with an absolute tolerance of 10−610^{-6}.

The baseline regression gives a coefficient of 938.23786125, a mean squared error of 2548.07, and R2=0.47R^2 = 0.47. None of the test predictions is recovered correctly after the pp-adic round trip: the reconstruction outcome is 0 out of 20.

Figure 2 illustrates the regression and reconstruction results. The left panel depicts the test data and fitted line. The right panel shows the reconstruction status of each test point.

Experimental results showing baseline regression and reconstruction status

Figure 2. Experimental results. Left: the Diabetes test split and baseline regression. Right: reconstruction status (pyAdic, p=2p = 2, precision 50; 20 test points). Successful reconstruction is assessed using an absolute tolerance of 10−610^{-6}.

Rational reconstruction fails to recover the test predictions for the specified regression setup and parameter choices.

Possible Problems and Sources of Error

The experiments reveal sensitivity to precision and to small input changes. Input representation, available precision, and rational reconstruction are therefore central to the reliability of the intermediate module. The different real and pp-adic metrics make preservation of the baseline model’s predictions a substantive requirement rather than an automatic consequence of changing the number representation.

For digit reversal, the carry mechanism prevents direct reuse of ordinary addition. Transformed operations would be needed to maintain the intended correspondence, contrary to the requirement of minimal changes to the existing system.

One possible explanation of the pyAdic results is incomplete handling of special cases in conversion, such as factors pnp^n in a rational denominator. The observed failures motivate closer analysis of these cases and of the precision needed for rational reconstruction.

Next Steps

Further investigation may consider datasets defined directly over pp-adic fields. Such a formulation avoids the rational-to-pp-adic round trip that produced reconstruction errors in the experiments.

Another direction is automatic selection of the prime pp and the computational precision under an explicitly defined reconstruction or prediction criterion.

Architectures formulated over pp-adic numbers provide a further research direction. Cellular neural networks use local connections between neighboring neurons and have applications in parallel computation and image processing. In [7, 8], pp-adic representations encode neuron states, with digits corresponding to individual neurons. Further work includes specifying the architecture, training and weight-update rules, and comparison with relevant baselines.

Discussion

The experiments demonstrate that the tested conversion–computation–reconstruction pipeline does not preserve the predictions of even a linear regression. Converting the fitted coefficients to pp-adic numbers is insufficient to obtain a compatible intermediate computational module.

This result concerns the tested implementations and parameter choices, not pp-adic machine-learning methods in general. The practical difficulty lies in recovering the rational outputs expected by the surrounding model.

Two aspects of the difficulty are distinct. Reconstruction failures and precision-dependent outputs concern the numerical behavior of the tested procedures. Digit reversal raises a representation-level problem: preserving addition requires changing the carry mechanism. Both aspects conflict with the aim of retaining the surrounding computational system unchanged.

The negative result identifies a limitation of direct substitution of the number representation in an existing model. A pp-adic model instead requires a formulation in which the data and operations are pp-adic from the outset, or an intermediate module explicitly designed to preserve the required input–output correspondence.

Conclusion

The present study examined an intermediate pp-adic computational module motivated by integration into existing neural-network systems. The actual test model was a one-feature linear regression: its fitted coefficients and inputs were represented in Qp\mathbb{Q}_p, and its computed outputs were subjected to rational reconstruction.

The experiments yield unsuccessful and precision-dependent reconstructions in pyAdic, discrepancies after reconstruction in SageMath, and a digit-reversal example that does not preserve ordinary addition. Under the tested settings, 0 of the 20 test predictions are reconstructed successfully.

The tested procedures therefore do not provide a compatible intermediate pp-adic module for the baseline model. Further research includes models based on pp-adic data and operations, such as pp-adic cellular neural networks [7, 8] and related constructions [15], automatic selection of pp and precision, and architectures designed around alternative numerical representations.

References

  1. [1]Vladimir Evgrafov. A Study of p-adic Numbers Applied to Neural Networks. Experimental materials in the article repository.
  2. [2]X. Caruso. Computations with p-adic numbers. Journées Nationales de Calcul Formel. Les cours du CIRM, 2018.arxiv.org/abs/1701.06794
  3. [3]A. Yu. Khrennikov, M. Nilson. P-adic Deterministic and Random Dynamics. Springer Netherlands, 2004.DOI
  4. [4]F. Pedregosa et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.arxiv.org/abs/1201.0490
  5. [5]pyAdic. Python package. URL: https://pypi.org/project/pyadic/.
  6. [6]SageMath, the Sage Mathematics Software System (Version 10.2). The Sage Developers.
  7. [7]B. A. Zambrano-Luna, W. A. Zúñiga-Galindo. p-adic Cellular Neural Networks. Journal of Nonlinear Mathematical Physics, 30(1):34–70, 2022.
  8. [8]B. A. Zambrano-Luna, W. A. Zúñiga-Galindo. p-adic Cellular Neural Networks: Applications to Image Processing. Physica D: Nonlinear Phenomena, 446:133668, 2023.
  9. [9]V. S. Anashin. Introduction to Applied p-adic Analysis. MCCME (in Russian).
  10. [10]S. B. Katok. p-adic Analysis Compared with Real. MCCME (in Russian), 2004.
  11. [11]N. Koblitz. p-adic Numbers, p-adic Analysis, and Zeta-Functions. Mir (Russian translation), 1982.
  12. [12]B. Dragovich, A. Yu. Khrennikov, S. V. Kozyrev, I. V. Volovich, E. I. Zelenov. p-Adic mathematical physics: the first 30 years. p-Adic Numbers, Ultrametric Analysis and Applications, 9(2):87–121, 2017.
  13. [13]X. Caruso, D. Roe, T. Vaccon. Tracking p-adic precision. LMS Journal of Computation and Mathematics, 17(A), 2014. arXiv:1402.7142.DOI
  14. [14]J. van der Hoeven. Relax, but don’t be too lazy. Journal of Symbolic Computation, 34(6), 2002.
  15. [15]W. A. Zúñiga-Galindo, C. He, B. A. Zambrano-Luna. p-Adic statistical field theory and convolutional deep Boltzmann machines. Progress of Theoretical and Experimental Physics, 2023(6):063A01, 2023. arXiv:2302.03817.
  16. [16]R. Pascanu, T. Mikolov, Y. Bengio. On the difficulty of training recurrent neural networks. Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):1310–1318, 2013. https://proceedings.mlr.press/v28/pascanu13.html.arxiv.org/abs/1211.5063

Paper details

vlnc:W_jr3255gp
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents