A common situation in physics involves two theories, ${cal A}$ and ${cal B}$, where ${cal A}$ contains a nonzero parameter, and ${cal B}$ arises as a limit of ${cal A}$ as this parameter approaches zero or infinity. In such cases, ${cal A}$ is more general and ${cal B}$ is a degenerate case of ${cal A}$. Well-known examples include relativistic theory being more general than non-relativistic theory and quantum theory being more general than classical theory. In this short review we argue that an analogous situation holds in mathematics. Classical mathematics (CM) is based on the infinite ring of integers $Z$, whereas finite mathematics (FM) is based on the finite ring $R_p=(0,1,2,...p-1)$ of residues modulo $p$. CM has foundational difficulties (as highlighted by Gödel's incompleteness theorems) while FM does not. All attempts to construct a quantum theory of gravity within CM encounter unavoidable divergencies. The existence of elementary particles also suggests that infinitesimals do not exist in nature. Despite this, CM is usually regarded as fundamental theory, while FM merely as a tool useful only in some models. We argue instead that FM is the more general theory, with CM appearing as its degenerate limit as $ptoinfty$. The key points are: $R_pto Z$ as $ptoinfty$, and this can be proved using only potential (not actual) infinity; quantum theory based on FM is more general than quantum theory based on CM.