Some classical questions and problems of Number Theory, like the Goldbach conjecture, distributions of twin- and d–primes and primes among arithmetic sequences, are addressed here from an entirely probabilistic point of view. We discuss the concept of ‘independence’ relevant to number-theoretic problems and interpret the basic concepts of divisibility of natural number in terms of probability spaces and appropriate probability distributions on classes of congruence. We analyze and demonstrate the importance of Zeta probability distribution and prove, in particular, theorems stating the equivalence of probabilistic independence of divisibility by co-prime factors, and the fact that random variables with the property of independence of co-prime factors must have Zeta probability distribution. Multiplicative and additive models with recurrent equations for generating sequences of prime numbers allow to interpret such sequences as realizations of random walks on set of natural numbers and on multiplicative semigroups generated by set of prime numbers , representing paths of stochastic dynamical systems. We discuss some limit theorems related to distribution of primes and their residuals. More specifically, we provide a continuous-time description of the distribution of counting function of primes in terms of diffusion approximation of non-Markov random walks.