text:
Practical Guide to Gaussian Processes

Preface
This book gives an introduction to Gaussian processes and shows their various, but not complete, applications. The introduction is aimed for users who want to apply the technique to solve practical engineering problems. With application examples, it shows how Gaussian processes can be used for machine learning to infer from known to unknown situations. The book serves as a reference for common analytical representations of Gaussian processes and for mathematical operations and methods in specific use cases.

Introduction
A Gaussian process is a stochastic process with the property that every finite subset of its values is multivariate normally distributed (or Gaussian distributed). A stochastic process is a function whose values are random variables and which follow a given probability distribution. This allows to model functions with probabilities whose values cannot be completely determined due to a lack of information. A Gaussian process is constructed from functions of mean values, variances and covariances and thus describes the function values as a continuum of correlated random variables in the form of an infinite-dimensional normal distribution. The distribution of a Gaussian process can be imagined as a probability distribution of functions. A sample of it yields a random function with certain preferred properties of its curve shape.

Applications
Gaussian processes are used for mathematical modeling of the behavior of non-deterministic systems on the basis of stochastic quantities or observations. Gaussian processes are suitable for signal analysis and synthesis, form a powerful tool for the interpolation, extrapolation, or smoothing of arbitrarily dimensional discrete measurement points (Gaussian process regression or kriging), and find application in classification problems. Gaussian processes, which are related to kernel methods, can be used as a supervised machine learning technique for abstract modeling based on training examples. This Bayesian approach to machine learning has the advantage that it often does not require iterative training as neural networks do. Instead, Gaussian processes can be derived very efficiently with linear algebra from statistical quantities of the examples and are mathematically clearly interpretable and well controllable. Moreover, for interpolations and predictions, an associated confidence interval is computed for each individual output value, which accurately estimates its own prediction error, while correctly accounting for error propagation when the variance of the input values is known.

Definition
A Gaussian process is a special type of stochastic process $$(X_t){t \in T}$$ on any index set $$T$$, if its finite-dimensional distributions are multivariate normal distributions (also Gaussian distributions) for all $$t_1, t_2, \dotsc, t_n \in T$$. That is, the multivariate distribution of $$(X{t_1}, X_{t_2}, \dotsc, X_{t_n})$$ is given by an n-dimensional normal distribution.

Term: Even the term Gaussian process may indicate temporal or sequential processes, this restriction does not exist. In a generalized sense, process can be understood as a continuum.

Notation
In analogy to the one- and multidimensional Gaussian distribution, a Gaussian process is completely and uniquely determined by its first two moments. In the multidimensional Gaussian distribution, these are the expected value vector or mean vector $$\vec\mu$$ and the covariance matrix $$\sigma$$. For the description of Gaussian process, these are replaced by an expected value function or mean function
 * $$m(t) := \mathbb{E}(X_t),\quad t \in T$$

and a covariance function
 * $$k(t,t') := \operatorname{Cov}(X_t, X_{t'}):= \mathbb{E}\left[(X_t- m(t))\cdot(X_{t'}- m(t'))\right], \quad t,t' \in T$$.

These functions can be understood in the simplest one-dimensional case as a vector with continuous rows and as a matrix with continuous rows and columns. The following table compares one-dimensional and multidimensional Gaussian distributions with Gaussian processes. The tilde symbol $$\sim$$ can be read as "is distributed as".

The probability density function of a Gaussian process cannot be represented analytically because there is no corresponding notation for operations with continuous matrices. This gives the impression that one cannot perform computations with Gaussian processes in the same way as with finite-dimensional normal distributions. However, the essential property of the Gaussian process is not the infinity of the dimensions, but rather the assignment of the dimensions to the coordinates of a function. In practical applications, one always has to deal with a finite number of interpolation points and can therefore perform all calculations as in the finite-dimensional case. The limit to infinitely many dimensions is only needed in an intermediate step, namely if values are to be read out at new interpolated grid points. In this intermediate step, the Gaussian process, i.e. the mean function and covariance function, is represented or approximated by suitable analytical expressions. In this case the assignment to the grid points is done via the parameterized coordinates $$t$$ in the analytical expression. In the finite-dimensional case with discrete grid points the associated coordinates $$t_i$$ are assigned to the dimensions by their indices.

Example of a Gaussian process
As a simple real world example, consider a Gaussian process

 * $$(X_t)_{t \in T} \sim \mathcal {GP}(m(t),k(t,t'))$$

with a scalar variable $$t$$ (time), given by the mean function

 * $$m(t) = 5 \, \text{Volt}$$

and covariance function

 * $$k(t,t')=\begin{cases}

(1\,\text{Volt})^2 &t=t'\\ 0 &t \ne t' \end{cases} $$

This Gaussian process describes an endless temporal electrical signal with Gaussian white noise with a standard deviation of one volt centered around a mean voltage of 5 volts.

Definitions of special properties
A Gaussian process is called centered if its expected value or mean is constantly 0, that is, if $$m(t) := \mathbb{E}(X_t) = 0$$ for all $$t \in T$$.

A covariance function $$k(t,t') := \operatorname{Cov}(X_t, X_{t'})$$ is called stationary when it is translation invariant, that is, it can be described by a relative function $$k(t,t') = k(t-t')$$.

A Gaussian process is called stationary (or translation invariant) if its covariance function is stationary and its mean is constant.

A covariance function is called radial when the function $$k(t,t') = k(|t-t'|)$$ is radial symmetric with a one-dimensional parameter using the Euclidean norm $$|\cdot|$$. It is used to describe systems with isotropic model properties.

List of Common Gaussian Processes and Covariance Functions

 * Constant:  $$m(t) = 0$$ and $$k(t,t') = \sigma^2$$
 * Corresponds to a constant value from a Gaussian distribution with standard deviation $$\sigma$$.

 * Offset:  $$m(t) = c$$ and $$k(t,t') = 0$$
 * Corresponds to a constant value given by $$c$$.

 * Gaussian White noise: $$k(t,t')=\sigma^2\delta_{t,t'}$$
 * ($$\sigma$$: standard deviation, $$\delta$$: Kronecker delta)

 * Rational quadratic:  $$ k(r) = (1+r^2)^{-\alpha}, \quad \alpha \geq 0$$
 * Gamma-exponential:  $$ k(r) = \exp \left(-\left(\frac{r} \ell \right)^\gamma\right)$$
 * Ornstein-Uhlenbeck: $$ k(r) = \exp \left(-\frac{r} \ell \right)$$
 * Corresponds to a simple Gauss-Markov process and describes continuous, non-differentiable functions, as well as white noise after passing through an RC low-pass filter.

 * Squared exponential:  $$ k(r) = \exp \Big(-\frac{r^2}{2\ell^2} \Big)$$
 * Describes infinitely smooth differentiable functions.

 * Matérn:
 * $$ k_{\nu=p+1/2}(r) = \exp\left(-\frac{\sqrt{2\nu}r}{\ell}\right)\frac{\Gamma(p+1)}{\Gamma(2p+1)}\sum_{i=0}^p\frac{(p+i)!}{i!(p-i)!}\left(\frac{\sqrt{8\nu}r}{\ell}\right)^{p-i}$$
 * A highly versatile Gaussian process used to describe most typical measurement curves. The functions of the Gaussian process are $$n$$ times continuously differentiable if $$\nu>n$$. Covariance functions with $$\nu=1/2$$, $$3/2$$, $$5/2$$, etc. correspond to white noise that has passed through 1, 2, or 3 RC low-pass filters or has been convolved with the function $$\exp \left(-|x| \right)$$. Common special cases include:
 * $$k_{\nu=3/2}(r) = \left(1+\frac{\sqrt{3}r}\ell\right)\exp\left(-\frac{\sqrt{3}r}\ell\right)$$
 * $$k_{\nu=5/2}(r) =\left(1+\frac{\sqrt{5}r}\ell+\frac{5r^2}{3\ell^2}\right)\exp\left(-\frac{\sqrt{5}r}\ell\right) $$
 * $$k_{\nu=1/2}(r)$$ corresponds to the Ornstein-Uhlenbeck covariance function, and $$k_{\nu \rightarrow \infty}(r)$$ corresponds to the squared exponential function.

 * Periodic:  $$ k(r) = \exp\left(-\frac{ 2\sin^2\left(\pi\frac r {T} \right)}{\ell^2} \right)$$
 * Functions from this Gaussian process are both periodic with period $$T$$ and smooth (squared exponential). If the square around the sine is replaced by the absolute value, non-smooth periodic functions result.

 * Polynomial:  $$k(t,t') = \left(t^\top t'+\sigma_0^2\right)^p$$
 * Grows rapidly outward and is usually a poor choice for regression problems, but can be useful in high-dimensional classification problems. It is positive semidefinite and does not necessarily generate invertible covariance matrices.

 * Brownian bridge: $$m(t)=0$$ and $$k(t,t')=\min(t,t') - t t'$$
 * Wiener process: $$m(t)=0$$ and $$k(t,t')=\min(t,t')$$
 * Corresponds to the Brownian motion or integral over Gaussian white noise.

 * Itō process: If $$T=\mathbb{R}_{+} $$ and $$f$$, $$g$$ are two integrable real-valued functions and $$(W_t)$$ is a Wiener process, then the Ito process
 * $$X_t= \int_0^t f(s) \, \mathrm ds + \int_0^t g(s) \, \mathrm dW_s $$
 * is a Gaussian process with $$m(t) = \int_0^t f(s)\, \mathrm ds $$ and $$ k(t,t') = \int_0^{\min(t,{t'})} g^2(s) \, \mathrm ds $$.

Remarks:
 * $$r := \|t-t'\|$$ is the distance for stationary and radial covariance functions $$k(t,t')=k(r)$$.
 * $$\ell$$ is the characteristic length scale of the covariance function where the correlation has decayed to about $$e^{-1}$$.
 * Most stationary covariance functions $$k(r)$$ are normalized to $$k(0)=1$$ and are therefore equivalent to correlation functions. For use as covariance functions, they are multiplied by a variance $$\sigma^2$$, which assigns the variables a scaling and/or physical unit.
 * Covariance functions cannot be arbitrary functions $$k(r)$$ or $$k(t,t')$$, as it must be ensured that they are positive definite. Positive semidefinite functions are also valid covariance functions, but it should be noted that these do not necessarily result in invertible covariance matrices and are therefore usually combined with a positive definite function.

Mathematical operations with Gaussian processes
Gaussian processes (or normal distributions) can be used to perform various stochastic operations that allow different functions with normally distributed errors to be joined or extracted from each other. If there are cross-correlations between the functions, it is assumed that they follow a joint normal distribution. In signal processing, for example, the operations are used to handle temporal signals and their measurement uncertainties. The distributions of these functions are described in the following operations in vector and matrix notation for finitely many interpolation points $$y \sim \mathcal N\left(\mu, \Sigma \right)$$, which analogously applies to arbitrary mean functions $$m(t)$$ and covariance functions $$k(t,t')$$. The normally distributed vectors ($$y_1$$, $$y_2$$ etc.) are described as functions accordingly.

Addition: uncorrelated functions
If the sum of two independent (and especially uncorrelated) functions is built, then their mean functions and their covariance functions add up:
 * $$y_1+y_2 \sim \mathcal N\left(\mu_1, \Sigma_1 \right) + \mathcal N\left(\mu_2, \Sigma_2 \right) = \mathcal N\left(\mu_1+\mu_2, \Sigma_1+\Sigma_2 \right)$$.

The associated probability density functions thereby undergo a convolution.

Addition: correlated functions
Correlated functions can in an extreme case be identical or differ only by constant factors. The sum then corresponds to a multiplication with the added factors. If both functions are identical, the result is $$y+y=2y \sim \mathcal N\left(2\mu, 4\Sigma \right)$$.

Difference: uncorrelated functions
If the difference of two independent uncorrelated functions is built, then their mean functions are subtracting while their covariance functions are adding:
 * $$y_1-y_2 \sim \mathcal N\left(\mu_1, \Sigma_1 \right) - \mathcal N\left(\mu_2, \Sigma_2 \right) = \mathcal N\left(\mu_1-\mu_2, \Sigma_1+\Sigma_2 \right)$$.

Subtraction of a Correlated Component
If the function y2 of a Gaussian process describes a additive component y1 of another Gaussian process, then subtracting this component results in the subtraction of the mean function and covariance function:
 * $$y_1-y_2 \sim \mathcal N\left(\mu_1, \Sigma_1 \right) \setminus \mathcal N\left(\mu_2, \Sigma_2 \right) = \mathcal N\left(\mu_1-\mu_2, \Sigma_1-\Sigma_2 \right)$$

The backslash operator $$\setminus$$ was symbolically used here in the sense of "without the contained component".

Multiplication
The following multiplication with an arbitrary matrix $$F$$ also includes the special cases of the product with a function (diagonal matrix $$F$$) or with a scalar ($$F=c\cdot\mathbb{I}$$):
 * $$Fy \sim F\cdot\mathcal N\left(\mu, \Sigma \right) = \mathcal N\left(F\mu, F\Sigma F^\top \right)$$

It should be noted here that the product of the functions of two Gaussian processes with each other would not result in another Gaussian process, since the resulting probability distribution would have lost the property of being Gaussian or normal.

General linear transformation
All previously shown operations are special cases of the general linear transformation:
 * $$A\cdot\mathcal N\left(\mu_1, \Sigma_1 \right) + B\cdot\mathcal N\left(\mu_2, \Sigma_2 \right) = \mathcal N\left(A\mu_1+B\mu_2, A\Sigma_1 A^\top+B\Sigma_2 B^\top + A\Sigma_{12} B^\top+B\Sigma_{12}^\top A^\top \right)$$

This relation describes the sum $$A\cdot y_1 + B\cdot y_2$$ with constant matrices $$A$$ and $$B$$ and the support point vectors $$y_1$$ and $$y_2$$ of the functions of two Gaussian processes with $$y_1 \sim \mathcal N\left(\mu_1, \Sigma_1 \right)$$ and $$y_2 \sim \mathcal N\left(\mu_2, \Sigma_2 \right)$$. For partially correlated functions $$y_1$$ and $$y_2$$, the cross-covariance matrix $$\Sigma_{12}$$ must be given and all variables must be jointly normal (i.e. the must follow a common multivariate normal distribution) as a precondition. In such case the sum $$A\cdot y_1 + B\cdot y_2$$ is correlated with $$y_1$$ by the cross-covariance matrix $$A\Sigma_{1}+B\Sigma_{12}$$ and with $$y_2$$ by $$A\Sigma_{12}+B\Sigma_{2}$$. A cross-covariance matrix $$\Sigma_{XY}$$ between two functions $$X$$ and $$Y$$ can be converted into a cross-correlation matrix $$C_{XY}$$ using their covariance matrices $$\Sigma_{X}$$ and $$\Sigma_{Y}$$ through the relation $$\left[C_{XY}\right]{ij} = \left[\Sigma{XY}\right]{ij}/\sqrt{\left[\Sigma_X\right]{ii}\left[\Sigma_Y\right]_{jj}}$$. In the case of two partially correlated Gaussian processes, it should be noted that special dependencies may exist where the sum does not result in a normal distribution and the equation accordingly loses its validity, although both input quantities are normally distributed.

Fusion
If the same unknown function is described by two different Gaussian processes with uncorrelated errors to each other, then a union or fusion (also Sensor fusion) of the two parts of partial information can be formed to achieve a reduction of the error or variance. For example, in signal processing, the same waveform is measured by two different sensors (such as the trajectory of an aircraft by an inertial sensor and independently by a GNSS location determination), which add up two different independent noise or error signals. The joint distribution
 * $$\Sigma_\text{Fusion} = \left(\Sigma_1^{-1} + \Sigma_2^{-1}\right)^{-1}$$
 * $$\mu_\text{Fusion} = \Sigma_\text{Fusion}\Sigma_1^{-1}\mu_1 + \Sigma_\text{Fusion}\Sigma_2^{-1}\mu_2$$

corresponds to the overlap or the normalized product of the two probability density functions and describes the most likely Gaussian process taking into account both parts of information (see also Inverse-variance weighting). The expressions can also be rearranged, such that only one matrix inversion needs to be performed:
 * $$\mu_\text{Fusion} = \mu_1 - \Sigma_1\left(\Sigma_1 + \Sigma_2\right)^{-1}\left(\mu_1 - \mu_2\right) = \Sigma_2\left(\Sigma_1 + \Sigma_2\right)^{-1}\mu_1 + \Sigma_1\left(\Sigma_1 + \Sigma_2\right)^{-1}\mu_2$$
 * $$\Sigma_\text{Fusion} = \Sigma_1 - \Sigma_1\left(\Sigma_1 + \Sigma_2\right)^{-1}\Sigma_1 = \Sigma_1\left(\Sigma_1 + \Sigma_2\right)^{-1}\Sigma_2$$

The validity of the formula requires function pairs with entirely uncorrelated errors. However, if there is partial correlation with cross-covariance $$\Sigma_{12}$$, then the extended and generalized formula, the so-called Bar-Shalom-Campo fusion, applies, where the correlated part is temporarily subtracted and then added back after fusion:
 * $$\mu_\text{Fusion} = \mu_1 - (\Sigma_1 - \Sigma_{12})(\Sigma_1 + \Sigma_2 - \Sigma_{12} - \Sigma_{21})^{-1}(\mu_1-\mu_2)$$
 * $$\Sigma_\text{Fusion} = \Sigma_1 - (\Sigma_1 - \Sigma_{12})(\Sigma_1 + \Sigma_2 - \Sigma_{12} - \Sigma_{21})^{-1}(\Sigma_1 - \Sigma_{21})$$

Decomposition
A given function $$y_\text{sum}$$ can be approximately decomposed into its additive components when the prior distributions of the entire function and the components are given. According to the addition rule, the Gaussian process of the entire function
 * $$\mu_\text{sum} = \mu_1 + \ldots + \mu_n$$
 * $$\Sigma_\text{sum}=\Sigma_1 + \ldots + \Sigma_n$$

is composed of the prior distributions of the components. The individual components $$y_i$$ can then be estimated by the posterior Gaussian processes
 * $$\mu_{\text{post,}i} = \mu_i + \Sigma_i\Sigma_\text{sum}^{-1} \left(y_\text{sum} - \mu_\text{sum}\right)$$
 * $$\Sigma_{\text{post,}i} = \Sigma_i - \Sigma_i \Sigma_\text{sum}^{-1} \Sigma_i^\top$$

which are correlated to each other by the cross covariances
 * $$\Sigma_{\text{post,}i,j} = -\Sigma_i \Sigma_\text{sum}^{-1} \Sigma_j^\top$$.

Apart from very specific cases, this decomposition is ambiguous. The components are therefore coupled probability distributions of possible solutions around the most likely components (see also Example: Signal Decomposition).

The decomposition is based on the equations for fusion in the previous section, which are applied to the specific distributions $$\mathcal N\left(\mu_\text{sum}, \Sigma_\text{sum}\right)$$ and $$\mathcal N\left(\mu_i, \Sigma_i\right)$$. The density product or overlap extracts the corresponding component in this case.

Introduction
Gaussian processes can be used to interpolate, extrapolate, or smooth discrete measurement data of a mapping $$\mathbb{R}^n \to \mathbb{R}$$. This application of Gaussian processes is called Gaussian process regression. The method is often called kriging for historical reasons, especially in the spatial domain. It is particularly suitable for problems for which no specific model function is known. Its property as a machine learning method allows automatic model building based on observations. In this application, a Gaussian process captures the typical behavior of the system, which can be used to derive the optimal interpolation for the problem. The result is a probability distribution of possible interpolation functions and the solution with the highest probability.

Overview of the individual steps
The calculation of a Gaussian process regression can be performed by the following steps:

 * 1) Prior mean function: If there is a consistent trend in the measured values, a prior mean function is constructed to equalize the trend.
 * 2) Prior covariance function: The covariance function is selected according to certain qualitative properties of the system or composed from covariance functions of different properties according to certain rules.
 * 3) Fine-tuning of parameters: to obtain quantitatively correct covariances, the selected covariance function is adjusted to the available measured values either targeted or by an optimization procedure until the covariance function reflects the empirical covariances.
 * 4) Conditional distribution: By considering known measured values, the conditional posterior Gaussian process is calculated from the prior Gaussian process for new support points with still unknown values.
 * 5) Interpretation: Finally, from the posterior Gaussian process, the mean function is taken as the best possible interpolation and, if required, the diagonal of the covariance function is taken as the location-dependent variance.

Step 2: Prior covariance function
In practical applications, a Gaussian process must be determined from finitely many discrete measured values or finitely many sample curves. In analogy to the one-dimensional Gaussian distribution, which is completely determined by the mean and standard deviation of discrete measured values, one would expect several single but complete functions $$f_i(t)$$ in order to calculate the mean function

m(t) = \frac{1}{N} \sum_{i=1}^{N} f_i(t) $$

and the (empirical) covariance function

k(t,t') = \frac{1}{N - 1} \sum_{i=1}^{N} \left[f_i(t) - m(t)\right] \cdot \left[f_i(t') - m(t')\right] $$.

Regression problem and stationary covariance
Often, however, no such distribution of exemplary functions is available. In the regression problem instead only discrete interpolation points of a single function are known, which are to be interpolated or smoothed. Also in such a case a Gaussian process can be determined. For this purpose, instead of this single function, a set of many copies of the function shifted to each other is considered. This distribution can now be described with the help of a covariance function. Usually it can be expressed as a relative function of this shift by $$k(t,t') = k(t'-t)$$. It is then called stationary covariance function and applies equally to all locations of the function and describes the everywhere equal (thus stationary) correlation of each point to its neighborhood, as well as the correlation of neighboring points among each other.

The covariance function is represented analytically and determined heuristically or looked up in the literature. The free parameters of the analytical covariance functions are fitted to the measured values. Many physical systems have a similar form of the stationary covariance function, so that with a few tabulated analytical covariance functions most applications can be described. For example, there are covariance functions for abstract properties such as smoothness, roughness (lack of differentiability), periodicity or noise, which can be combined and fitted according to certain rules to reproduce the properties of the measured values.

Examples of stationary covariance
The following table shows examples of covariance functions with such abstract properties. The example curves are random samples of the respective Gaussian process and represent typical function shapes. They were generated with the corresponding covariance matrix $$\Sigma_{ij}=k(t_i, t_j)$$ and a random generator for multidimensional normal distributions as correlated random vector. The stationary covariance functions $$k(t,t')$$ are abbreviated here as one-dimensional functions $$k(r)$$ with $$r := |t-t'| $$.

Construction of new covariance functions
The properties can be combined according to certain computational rules. The basic goal in constructing a covariance function is to reproduce the true covariances as precisely as possible, while at the same time satisfying the condition of positive definiteness. The examples shown, except for the constant, have the latter property, and the additions and multiplications of such functions also remain positive definite. The constant covariance function is only positive semidefinite and must be combined with at least one positive definite function. The lowest covariance function in the table shows a possible mixture of different properties. The functions in this example are periodic over a certain distance, have a relatively smooth behavior and are overlaid with a certain measurement noise.

For mixed properties, the following rules applies:
 * In the case of additive effects, the covariances are added, as for example in the superposition of measurement noise.
 * For reinforcing or mitigating effects to each other, the covariances are multiplied, such as in case of the slow decay of periodicity.

Multidimensional functions
What is shown here with one-dimensional functions can be transferred analogously also to multi-dimensional systems, by simply replacing the distance $$r$$ by a corresponding n-dimensional distance norm. The support points in the higher dimensions are unrolled in an arbitrary order and represented by vectors, so that they can be processed in the same way as in the one-dimensional case. The following two figures show two examples with two-dimensional Gaussian processes and different stationary and radial covariance functions. In the respective right figure a random draw of the Gaussian process is shown.

Non-stationary covariance functions
Gaussian processes can also have non-stationary properties of the covariance function, that is, relative covariance functions that change as a function of location. The literature describes how nonstationary covariance functions can be constructed so that positive definiteness is ensured here as well. A simple possibility is, for example, an interpolation of different covariance functions over the location with the inverse distance weighting.

Step 3: Fine tuning of parameters
The qualitatively constructed covariance functions contain parameters, called hyperparameters, which must be tuned (or calibrated) to the system in order to obtain quantitatively correct results. This can be done by direct knowledge about the system, e.g., the known value of the standard deviation of the measurement noise or the prior standard deviation of the overall system (sigma prior, the square corresponds to the diagonal elements of the covariance matrix).

However, the parameters can also be adjusted automatically. For this purpose, one uses the marginal likelihood, i.e., the probability density for a given measured curve as a metric for the agreement between the assumed Gaussian process and the existing measured curve. The parameters are then optimized to maximize this agreement. Since the exponential function is strictly monotone, it is sufficient to maximize the exponent of the probability density function, the so-called log-marginal likelihood function

 * $$\log p(\mathbf{y}) = - \frac12 \mathbf{y}^\top \Sigma^{-1}\mathbf{y}-\frac12 \log|\Sigma|-\frac n2 \log(2\pi)$$

with the measurement vector $$\mathbf{y}$$ of length $$n$$ and the hyperparameter-dependent covariance matrix $$\Sigma$$. Mathematically, maximizing the marginal likelihood causes an optimal tradeoff between accuracy (minimizing the residuals) and simplicity of the theory. A simple theory is characterized by large non-diagonal elements, describing a high correlation in the system. This means that there are few degrees of freedom in the system and thus, in some sense, the theory can cope with few rules to explain all correlations. If these rules are chosen too simple, the measurements would not be reproduced sufficiently well and the residual errors grow too much. At a maximum marginal likelihood, the equilibrium of an optimal theory is found, provided that sufficiently many measurement data were available for good conditioning. This implicit property of the maximum likelihood estimation can also be understood as Ockham's parsimony principle.

Step 4: Conditional Gaussian process with known support points
If the Gaussian process of a system has been determined as described above, i.e. if the prior mean function and covariance function are known, a prediction of arbitrary interpolated intermediate values can be computed with the Gaussian process, when only a few support points of the desired function are known by measurements. The prediction is done by the conditional probability of a multidimensional Gaussian distribution given a partial information. The dimensions of the multidimensional Gaussian distribution

 * $${X} = \binom{{X}_\text{U}}{{X}_\text{K}} \sim \mathcal N\left(\binom{{\mu}_\text{U}}{{\mu}_\text{K}}, \begin{pmatrix}{\Sigma}_\text{UU} & {\Sigma}_\text{UK} \\ {\Sigma}_\text{KU} & {\Sigma}_\text{KK}\end{pmatrix}\right)$$

are divided into unknown values to be predicted (index U for unknown) and known measured values (index K for known). Vectors thereby decompose into two parts. The covariance matrix is accordingly divided into four blocks: Covariances within the unknown values (UU), within the known measured values (KK) and covariances between the unknown and known values (UK and KU). The values of the covariance matrix are taken at discrete points of the covariance function and the mean vector at corresponding points of the mean function: $$\Sigma_{ij}=k(t_i, t_j)$$ or $$\mu_{i}=m(t_i)$$.

By considering the known measured values $$X_\text{K}$$, the distribution changes to the conditional or posterior normal distribution
 * $$X_\text{U} \mid X_\text{K} \sim \mathcal N \left(\mu_\text{U} + \Sigma_\text{UK}\Sigma_\text{KK}^{-1} (X_\text{K} - \mu_\text{K}), \Sigma_\text{UU} - \Sigma_\text{UK}\Sigma_\text{KK}^{-1}\Sigma_\text{KU}\right) $$,

where $$X_\text{U}$$ are the unknown variables to be determined. The notation $$\mid X_\text{K}$$ reads as "given $$X_\text{K}$$", which means under the condition that $$X_\text{K}$$ is given.

The first parameter of the resulting Gaussian distribution describes the new mean vector we are looking for, which now corresponds to the most likely function values of the interpolation. In addition, the entire predicted new covariance matrix is given in the second parameter. In particular, this contains the confidence intervals of the predicted mean values, given by the root of the main diagonal elements.

Measurement noise and other interfering signals
White measurement noise of variance $$\sigma_\text{noise}^2$$ can be modeled as part of the prior covariance model by adding appropriate terms to the diagonal of $$\Sigma_\text{KK}$$. If the same covariance function is also used to form the matrix $$\Sigma_\text{UU}$$, the predicted distribution would also describe a white noise of variance $$\sigma_\text{noise}^2$$. To obtain a prediction of an noise free signal, in the posterior distribution

 * $$X_\text{U} \mid X_\text{K} \sim \mathcal N \left(\mu_\text{U} + \Sigma_\text{UK}\left[\Sigma_\text{KK} + \mathbb{I}\sigma_\text{noise}^2\right]^{-1} (X_\text{K} - \mu_\text{K}), \Sigma_\text{UU} - \Sigma_\text{UK}\left[\Sigma_\text{KK} + \mathbb{I}\sigma_\text{noise}^2\right]^{-1}\Sigma_\text{KU}\right)$$

the corresponding terms are omitted at $$\Sigma_\text{UU}$$ and if applicable in $$\Sigma_\text{UK}$$ and $$\Sigma_\text{KU}$$. This averages out the measurement noise as good as possible, which is also correctly accounted for in the predicted confidence interval. In the same way, any unwanted additive noise signal can be removed from the measurement data (see also arithmetic operation decomposition), provided that it can be described by a covariance function and is sufficiently well distinguishable from the desired signal component. For this purpose, instead of the diagonal matrix $$\mathbb{I}\sigma_\text{noise}^2$$, the corresponding covariance matrix of the interference $$\Sigma_\text{noise}$$ is used. Measurements with noisy signals thus require two covariance models: $$k(t,t')$$ for the desired signal component to be estimated and $$k(t,t')+k_\text{noise}(t,t')$$ for the raw signal.

Derivation of the conditional distribution
The derivation can be done via the Bayes formula by substituting the two probability densities for known and unknown support points and the composite probability density. The resulting conditional posterior normal distribution corresponds to the overlap or intersection of the Gaussian distribution with the subvector space spanned by the known values.

For noisy measurements that are themselves a multidimensional normal distribution, the overlap to the prior distribution is obtained by multiplying the two probability densities. The product of the probability densities of two multidimensional normal distributions corresponds to the arithmetic operation Fusion, which can be used to derive the distribution where the noise is suppressed.

Posterior Gaussian process
In the full notation as a Gaussian process, the posterior Gaussian process yields
 * $$(X_t) \sim \mathcal {GP}(m,k)$$

and the $$n$$ known measurements $$\mathbf{x}=(x_1,x_2,\ldots,x_n)$$ at the coordinates $$\mathbf{t}=(t_1,t_2,\ldots,t_n)$$ a new distribution, given by the conditional posterior Gaussian process
 * $$(X_t\mid \mathbf{t},\mathbf{x}) \sim \mathcal {GP}\left(m_{\mathrm{post}}, k_{\mathrm{post}}\right)$$

Here, $$K$$ is a covariance matrix obtained by evaluating the covariance function $$k$$ at discrete rows $$t_i$$ and columns $$t_j$$. Moreover, $$\mathbf{k}$$ was appropriately formed as a vector of functions by evaluating $$k$$ only at discrete rows or only at discrete columns.

In practical numerical calculations with finite numbers of support points, only the equation of the conditional multivariate normal distribution is used. The notation of the posterior Gaussian process serves here only the theoretical understanding, in order to describe the limit towards the continuum in the form of functions and thus to depict the assignment of the values to the coordinates.

Step 5: Interpretation
From the prior Gaussian process, the measured values are used to obtain a posterior Gaussian process, which takes into account the known partial information. This result of the Gaussian process regression represents not only one solution, but the entirety of all possible solution functions of the interpolation weighted with different probabilities. The indecision expressed in this way is not a weakness of the method. It does perfect justice to the problem, since in the case of a theory which is not completely known or in the case of noisy measurements, the solution, in principle, cannot be determined unambiguously. Mostly, however, we are specifically interested in at least the solution with the highest probability. This is given by the mean function $$m_{\mathrm{post}}(t)$$ in the first parameter of the posterior Gaussian process. From the conditional covariance function in the second parameter, it is possible to obtain the scatter around this solution. The diagonal $$k_{\mathrm{post}}(t,t)$$ of the covariance function gives a function with the variances of the predicted most likely function. The confidence interval is then given by the bounds $$m_{\mathrm{post}}(t) \pm \sqrt{k_{\mathrm{post}}(t,t)}$$.

The Python code for the examples can be found on the respective image description page.

Underdetermined measurments
In some cases of conditional Gaussian processes, groups of linearly related measured values are completely indeterminate. E.g., this is the case for indirect measurements following from underdetermined equations, such as with a noninvertible positive semidefinite matrix of the form $$A^\top\Sigma^{-1}A$$. The grid points then cannot be easily partitioned into known and unknown values, and the associated covariance matrix would be singular due to infinite uncertainties. This would correspond to a normal distribution that is infinitely stretched in certain spatial directions transverse to the coordinate axes. To account for the relationships between the undetermined variables, in such a case, the inverse matrix $$\Sigma_\text{2}^{-1}$$, called the precision matrix, must be used. This can describe completely undetermined measurements, which is expressed by zeros in the diagonal. For such a singular distribution $$\mathcal N\left(\mu_\text{2}, \Sigma_\text{2}\right)$$ with partially unknown measurements $$\mu_\text{2}$$ and singular measurement uncertainties $$\Sigma_\text{2}$$, the wanted posterior distribution is obtained by the overlap to the prior Gaussian process model $$\mathcal N\left(\mu_\text{1}, \Sigma_\text{1}\right)$$ calculated by multiplying the probability densities. The union of the two normal distributions

 * $$\Sigma_\text{Fusion} = \left(\mathbb{I} + \Sigma_\text{1}\Sigma_\text{2}^{-1}\right)^{-1}\Sigma_\text{1}$$
 * $$\mu_\text{Fusion} = \left(\mathbb{I} + \Sigma_\text{1}\Sigma_\text{2}^{-1}\right)^{-1}\mu_\text{1} + \Sigma_\text{Fusion}\Sigma_\text{2}^{-1}\mu_\text{2}$$

is obtained by the Fusion operation after appropriate transformation, so that the singular of the two matrices remains inverse. The result is always a finite distribution, since the finite matrix dominates. If both are finite, the equation can be put into the form of the posterior Gaussian process as in the section on the conditional distribution.

Linear combination to a Gaussian process
From given basis functions $$\varphi_j(t)$$ a linear combination is to be formed, which has maximum overlap with the distribution $$\mathcal N(\mu, \Sigma)$$ of an associated Gaussian process $$\mathcal {GP}(m,k)$$. Or measured values $$\mu$$ are to be approximated, while the interfering signal $$\mathcal N(0, \Sigma)$$, contained within, is ignored as far as possible. In both cases, the wanted coefficients can be calculated using generalized least squares estimation

 * $$c = \left( A^\top \Sigma^{-1} A \right)^{-1} A^\top \Sigma^{-1}\mu$$
 * $$\Sigma_c = \left( A^\top \Sigma^{-1} A \right)^{-1} $$

The matrix $$A_{ij}=\varphi_j(t_i)$$ contains the function values of the basis functions $$\varphi_j(t)$$ at the interpolation points $$t_i$$. The resulting coefficients c with the associated covariance matrix $$\Sigma_c$$ describe the linear combination with the largest possible probability density in the distribution $$\mathcal N(\mu, \Sigma)$$. The linear combination thereby approximates the mean function or the measured values $$\mu$$ in such a way that the residuals are best described by the covariance matrix $$\Sigma$$. The method is used, for example, in the program library Scikit-learn to empirically estimate a polynomial mean function of a Gaussian process.

Approximation of an empirical Gaussian process
An empirically determined Gaussian process

 * $$m(t) = \frac{1}{N} \sum_{p=1}^{N} f_p(t)$$
 * $$k(t, t') = \frac{1}{N - 1} \sum_{p=1}^{N} \left[f_p(t) - m(t)\right] \cdot \left[f_p(t') - m(t')\right]$$

from example functions $$f_p(t)$$ with few distinct degrees of freedom can be approximated and simplified by means of the eigenvalue decomposition or singular value decomposition

 * $$\Sigma = VSV^\top$$

of the covariance matrix $$\Sigma_{ij} = k(t_i, t_j)$$. This is done by choosing the $$n$$ largest eigenvalues or singular values $$\lambda_p=\sigma_p^2$$ from the diagonal matrix $$S$$. The corresponding columns $$v_p$$ of $$V$$ are the principal components of the Gaussian process (see Principal Component Analysis). If the columns are represented as functions $$v_p(t)$$, then the original Gaussian process is represented by the mean function $$m(t)$$ and the covariance function

 * $$k(t, t') \approx \sum_{p=1}^{n} \sigma_p^2 v_p(t) v_p(t')$$

This Gaussian process describes exclusively functions of the linear combination

 * $$f(t) = m(t) + \sum_p c_p v_p(t)$$,

where each coefficient $$c_p$$ is scattered around zero mean as an independent random variable of variance $$\sigma_p^2 = \lambda_p$$.

Such a simplification is positively semidefinite and it usually lacks the properties to describe small-scale variations. These properties can be added to the covariance function in the form of a stationary covariance function fitted to the residuals:

k(t, t') \approx \sum_{p=1}^{n} \sigma_p^2 v_p(t) v_p(t') + k_\text{stat}(t'-t) $$

Example: Trend prediction
In a hypothetical application example from market research, the future demand for the topic "snowboard" is to be predicted. For this purpose, an extrapolation of the number of Google searches on this term is to be calculated.

In the past data, one can see a periodic, but non-sinusoidal seasonal dependence, which can be explained by the winter in the northern hemisphere. Moreover, the trend decreased continuously over the last decade. In addition, one recognizes a recurring increase in search queries during the Olympic Games every four years. The covariance function was therefore modeled with a slow trend and a one- and four-year period:

The trend also appears to have a significant asymmetry. This can be the case if the underlying random effects do not add up but reinforce each other, resulting in a Log-Normal Distribution. However, the logarithm of such values describes a normal distribution, to which Gaussian process regression can be applied.

The figure shows an extrapolation of the curve (to the right of the dashed line). Since the results here were transformed back from the logarithmic plot using an exponential function, the predicted confidence intervals are correspondingly asymmetrical (gray area). The extrapolation plausibly shows the seasonal patterns and also the increase in searches for the Olympic Games every four years. The example with mixed properties demonstrates very well the versatile modeling possibilities of the Gaussian process regression, which are unified in a single interpolation method.

Example: Sensor calibration
In an application example from industry, sensors are to be calibrated using Gaussian processes. Due to tolerances during manufacturing, the characteristic curves $$f(x)$$ of the sensors show large individual differences. This causes high costs in calibration, since a complete characteristic curve would have to be measured for each sensor. However, the effort can be minimized by learning the exact behavior of the scattering by a Gaussian process. For this purpose, the complete characteristic curves $$f_i(x)$$ of $$N$$ randomly selected representative sensors are measured and thus the Gaussian process $$\mathcal{GP}(m,k)$$ of the scattering is calculated by

m(x) = \frac{1}{N} \sum_{i=1}^{N} f_i(x) $$

k(x,x') = \frac{1}{N - 1} \sum_{i=1}^{N} \left[f_i(x) - m(x)\right] \cdot \left[f_i(x') - m(x')\right] $$

In the example shown, 15 representative characteristic curves are given. The resulting Gaussian process is represented by the mean function $$m(x)$$ and the confidence interval $$m(x) \pm \sqrt{k(x,x)}$$.

With the conditional Gaussian process $$\mathcal{GP}(m_\text{post},k_\text{post})$$ with

 * $$m_{\mathrm{post}}(x) = m(x) + \mathbf{k}^\top(x,\mathbf{x})K(\mathbf{x},\mathbf{x})^{-1} (\mathbf{y} - m(\mathbf{x}))$$

k_{\mathrm{post}}(x,x') = k(x,x') - \mathbf{k}^\top(x,\mathbf{x})K(\mathbf{x}, \mathbf{x})^{-1}\mathbf{k}(\mathbf{x},x') $$

the complete characteristic map can now be reconstructed for each new sensor with a few individual measured values $$\mathbf{y}$$ at the coordinates $$\mathbf{x}$$. The number of measured values must correspond at least to the number of degrees of freedom of the tolerances, which have an independent linear influence on the shape of the characteristic curve.

In the example shown, a single measured value is not yet sufficient to determine the characteristic curve unambiguously and precisely. The confidence interval shows the region of the curve which is not yet sufficiently accurate. With another measured value in this range, the remaining uncertainty can finally be completely eliminated. The exemplary fluctuations of the very differently acting sensors in this example thus seem to be caused by the tolerances of only two relevant inner degrees of freedom.

Example: Signal decomposition
In a signal processing application example, a temporal signal is to be decomposed into its components. Let it be known about the system that the signal consists of three components following the three covariance functions
 * $$k_1(r) = 2{,}7^2 \exp(-r^2)$$
 * $$k_2(r) = 2{,}7^2 \exp(-0{,}4|\sin(r/2{,}5)|)$$
 * $$k_3(r) = 0{,}6^2 \delta_r$$

The sum signal then follows the addition rule of the covariance function
 * $$k_\text{sum}(r) = k_1(r) + k_2(r) + k_3(r)$$.

The following two figures show three random signals which were generated and added for demonstration with these covariance functions. In the sum of the signals one can hardly recognize the periodic signal hidden in it with the naked eye, since its spectral range overlaps with that of the two other components.

The example shows how this method can be used to separate very different signals in one step. In contrast, other filtering methods such as moving averaging, Fourier filtering, polynomial regression, or spline approximation are optimized for specific signal characteristics and provide neither accurate error estimates nor cross-correlations.

If the Gaussian processes of the individual components for a given signal are not precisely known, then in some cases hypothesis testing can be performed using the log-marginal likelihood function, provided sufficient data are available to well-condition the function. Via its maximization, the parameters of the conjectured covariance functions can be fitted to the measured data.

Literature

 * In: Olivier Bousquet (publisher): Advanced Lectures on Machine Learning. ML 2003. (= Lecture Notes in Computer Science. vol. 3176). Springer, Berlin/Heidelberg 2004.(, pdf)
 * C. E. Rasmussen, C. K. I. Williams: Gaussian Processes for Machine Learning. MIT Press, 2006, ISBN 0-262-18253-X. (gaussianprocess.org, pdf)
 * R. M. Dudley: Real Analysis and Probability. Wadsworth and Brooks/Cole, 1989.
 * B. Simon: Functional Integration and Quantum Physics. Academic Press, 1979.
 * M. L. Stein: Interpolation of Spatial Data: Some Theory for Kriging. Springer, 1999.

Educational material

 * Gaussian Processes Web Site (Textbook, tutorials, code, etc.)
 * Interactive demo on Gaussian process regression
 * The Kernel Cookbook (Guide to the construction of covariance functions)

Software

 * GPy – A Gaussian Process-Framework in Python
 * Scikit Learn Gaussian Process – Gaussian process module of the machine learning library Scikit-learn for Python
 * Gaussian process library written in C++11

title: Practical Guide to Gaussian Processes


text:
Talk:Practical Guide to Gaussian Processes

On this new book
This book is a translation of the German Wikipedia article of Gaussian Process (see Gaußprozess. I am the main author of this German article. Since I am no english native speaker I used for the translation the tools deepl.com and in some cases chatGPT. I hope the text reads smoothly. Physikinger (discuss • contribs) 22:27, 21 March 2023 (UTC)

 * Hi @Physikinger! Unfortunately, encyclopedic articles like this do not fall under the Wikibooks scope. Unless it can be turned into a book, it will likely need to be deleted. For more information on what Wikibooks allows, please see What is Wikibooks. Thanks! —Kittycataclysm (discuss • contribs) 00:25, 22 March 2023 (UTC)

 * I was often asked for an English introduction to this topic. At first I wanted to integrate it as a chapter to an already existing book, e.g. Statistics. But after I have checked how similar topics are written there, such as Statistics/Distributions/Normal_(Gaussian), I got the feeling that my text is much closer to what I expect from a textbook on a specific topic, with an introduction, detailed didactic explanations and application examples. It is actually much more than a Wikipedia article. After I could not find an suitable existing book, I finally decided to make it a stand-alone book here. But what makes a book a book? Should it become just bigger? Or part of a comprehensive superordinate set of topics? I don't see any comparable wikibook on statistics or machine learning that seems to develop towards the the same level of depth and didactics where I could integrate this as a chapter. Although this "small book" does not fulfill the goal of wikibooks for comprehensive big textbooks, it still fulfills the educational purpose of wikibooks and many people will appreciate the introduction to this complex advanced mathematical topic.--Physikinger (discuss • contribs) 08:32, 22 March 2023 (UTC)
 * @Physikinger Based on your description, it seems like it could be turned into a book. Based on policy, other books hosted here, and my own thoughts, I personally would recommend a few things to start making it less encyclopedic and more instructional:
 * Reformat it into multiple pages, with each one being a distinct chapter
 * Include a chapter with introduction, scope, and goals of the book
 * Include clear applications
 * Include questions and practice problems
 * Definitely reach out if you need help with any of this! —Kittycataclysm (discuss • contribs) 12:13, 22 March 2023 (UTC)
 * Some comments to your points:
 * "Reformat it into multiple pages, with each one being a distinct chapter"
 * This would not support the readability, since the size is actually not so huge. I see also the drawback in Wikibooks in general that there is not even a link at the end of each chapter to the beginning of the next chapter. This interrupts the flow of reading. I also don't think that the book will grow much further. It's a quite specific topic. I wrote 10 years on the article and got almost no help by other authors. I would more prefer to make it in future part of a book on machine learning, as a chapter.
 * "Include a chapter with introduction, scope, and goals of the book"
 * Ok, I can do that.
 * "Include clear applications"
 * There are already 3 applications at the end. One is a real application I used in industry.
 * "Include questions and practice problems"
 * There is already a free scientific textbook about Gaussian Processes which is "the bible" for this topic. There are more applications in this book. I don't want to replace or repeat this book. It should be just a more gentle introduction and practical guide.
 * Physikinger (discuss • contribs) 23:02, 22 March 2023 (UTC)
 * @Kittycataclysm I can't see a clear strategy behind the suggested split. What is the purpose? For a page of this size scolling with a scrollbar seems to me more convenient than an annoying click track. Physikinger (discuss • contribs) 21:06, 27 March 2024 (UTC)
 * Hi @Physikinger. I would like to split it for accessibility reasons. For some people, it is more accessible to navigate and edit by chapter, and for some it is easier to work with one big scroll. We have the printable template option that combines separate chapters into one long scroll, but the reverse is not possible. Cheers —Kittycataclysm (discuss • contribs) 22:26, 27 March 2024 (UTC)
 * There is an auto-generated table of content where you can click to jump to each chapter. You can also edit each section separately. The whole text is actually not so big that there would be an avantage of splitting. There are also several cross-links inside the document e.g. to examples or to the operations. I don't know if this would work with the printable template. There is also not even an auto-generated link at the end of each chapter to the next section. You would have to go back to the first page with the main table of contents in order to go to the next section. This would not feel like a book you can read continuosly from the beginning to the end. So I would rather keep it together for accessibility reasons. Physikinger (discuss • contribs) 00:23, 28 March 2024 (UTC)

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Template:Chset-left1

style="background:#F8F8F8;height:2em;font-size:small;height:22pt;line-height:1" |

title: Template:Chset-left1


text:
Template:Chset-table-header1

class="wikitable nounderlines nowrap" border="1" style="border-collapse:collapse;text-align:center;background:#FFFFFF;font-size:large"
 * colspan="17" style="background:#F8F8F8;font-size:80%;line-height:1.5" |
 * - style="background:#F8F8F8;font-size:small"
 * style="width:20pt" | 0
 * style="width:20pt" | 1
 * style="width:20pt" | 2
 * style="width:20pt" | 3
 * style="width:20pt" | 4
 * style="width:20pt" | 5
 * style="width:20pt" | 6
 * style="width:20pt" | 7
 * style="width:20pt" | 8
 * style="width:20pt" | 9
 * style="width:20pt" | A
 * style="width:20pt" | B
 * style="width:20pt" | C
 * style="width:20pt" | D
 * style="width:20pt" | E
 * style="width:20pt" | F
 * style="width:20pt" | F

title: Template:Chset-table-header1


text: Template:Char

title: Template:Char


text:
Character Encodings/Code Tables/EBCDIC

 * EBCDIC 001
 * EBCDIC 002
 * EBCDIC 003
 * EBCDIC 004
 * EBCDIC 005
 * EBCDIC 006
 * EBCDIC 007
 * EBCDIC 008
 * EBCDIC 009
 * EBCDIC 010
 * EBCDIC 011
 * EBCDIC 012
 * EBCDIC 013
 * EBCDIC 014
 * EBCDIC 015
 * EBCDIC 016
 * EBCDIC 017
 * EBCDIC 018
 * EBCDIC 019
 * EBCDIC 020
 * EBCDIC 021
 * EBCDIC 022
 * EBCDIC 023
 * EBCDIC 024
 * EBCDIC 025
 * EBCDIC 026
 * EBCDIC 027
 * EBCDIC 028 not known
 * EBCDIC 029
 * EBCDIC 030
 * EBCDIC 031
 * EBCDIC 032
 * EBCDIC 033
 * EBCDIC 034
 * EBCDIC 035
 * EBCDIC 036
 * EBCDIC 037
 * EBCDIC 037-2
 * EBCDIC 038
 * EBCDIC 039
 * EBCDIC 040
 * EBCDIC 251
 * EBCDIC 252
 * EBCDIC 254
 * EBCDIC 256
 * EBCDIC 257
 * EBCDIC 258
 * Code page 259
 * EBCDIC 260
 * EBCDIC 264
 * EBCDIC 273
 * EBCDIC 274
 * EBCDIC 275
 * EBCDIC 276
 * EBCDIC 277
 * EBCDIC 278
 * EBCDIC 279
 * EBCDIC 280
 * EBCDIC 281
 * EBCDIC 282
 * EBCDIC 283
 * EBCDIC 284
 * EBCDIC 285
 * EBCDIC 286
 * EBCDIC 287
 * EBCDIC 288
 * EBCDIC 289
 * EBCDIC 290
 * EBCDIC 297
 * EBCDIC 298
 * EBCDIC 320
 * EBCDIC 321
 * EBCDIC 322
 * EBCDIC 330
 * EBCDIC 340
 * EBCDIC 352
 * EBCDIC 361
 * EBCDIC 363
 * EBCDIC 382
 * EBCDIC 383
 * EBCDIC 384
 * EBCDIC 385
 * EBCDIC 386
 * EBCDIC 387
 * EBCDIC 388
 * EBCDIC 389
 * EBCDIC 390
 * EBCDIC 391
 * EBCDIC 392
 * EBCDIC 393
 * EBCDIC 394
 * EBCDIC 395
 * EBCDIC 396
 * EBCDIC 410
 * EBCDIC 420
 * EBCDIC 421
 * EBCDIC 423
 * EBCDIC 424
 * EBCDIC 425
 * EBCDIC 435
 * EBCDIC 500
 * EBCDIC 803
 * EBCDIC 829
 * EBCDIC 830
 * EBCDIC 831 is the same as EBCDIC 037.
 * EBCDIC 833
 * EBCDIC 836
 * EBCDIC 838
 * EBCDIC 870
 * EBCDIC 871
 * Code page 875
 * EBCDIC 880
 * EBCDIC 881
 * EBCDIC 882
 * EBCDIC 883
 * EBCDIC 884
 * EBCDIC 885
 * EBCDIC 886
 * EBCDIC 887
 * EBCDIC 888
 * EBCDIC 889
 * EBCDIC 890
 * EBCDIC 892
 * EBCDIC 893
 * EBCDIC 905
 * EBCDIC 918
 * EBCDIC 924
 * Code page 930
 * EBCDIC 1001
 * EBCDIC 1002
 * EBCDIC 1003
 * EBCDIC 1005
 * EBCDIC 1007
 * EBCDIC 1024
 * EBCDIC 1025
 * EBCDIC 1026
 * EBCDIC 1027
 * EBCDIC 1028
 * EBCDIC 1030
 * EBCDIC 1031
 * EBCDIC 1032
 * EBCDIC 1033
 * EBCDIC 1037
 * EBCDIC 1039
 * EBCDIC 1047
 * EBCDIC 1068
 * EBCDIC 1069
 * EBCDIC 1070
 * EBCDIC 1071
 * EBCDIC 1072
 * EBCDIC 1073
 * EBCDIC 1074
 * EBCDIC 1075
 * EBCDIC 1076
 * EBCDIC 1077
 * EBCDIC 1078
 * EBCDIC 1079
 * EBCDIC 1080
 * EBCDIC 1081
 * EBCDIC 1082
 * EBCDIC 1083
 * EBCDIC 1084
 * EBCDIC 1085
 * EBCDIC 1087
 * Code page 1091
 * EBCDIC 1093
 * EBCDIC 1097
 * EBCDIC 1110
 * EBCDIC 1112
 * EBCDIC 1113
 * EBCDIC 1122
 * EBCDIC 1123
 * EBCDIC 1130
 * EBCDIC 1132
 * EBCDIC 1136
 * EBCDIC 1137
 * EBCDIC 1140
 * EBCDIC 1141
 * EBCDIC 1142
 * EBCDIC 1143
 * EBCDIC 1144
 * EBCDIC 1145
 * EBCDIC 1146
 * EBCDIC 1147
 * EBCDIC 1148
 * EBCDIC 1149
 * EBCDIC 1150
 * EBCDIC 1151
 * EBCDIC 1152
 * EBCDIC 1153
 * EBCDIC 1154
 * EBCDIC 1155
 * EBCDIC 1156
 * EBCDIC 1157
 * EBCDIC 1158
 * EBCDIC 1159
 * EBCDIC 1160
 * EBCDIC 1164
 * EBCDIC 1165
 * EBCDIC 1166
 * EBCDIC 1175
 * EBCDIC 1278
 * KEIS (EBCDIC 1279)
 * EBCDIC 1300
 * EBCDIC 1301
 * EBCDIC 1302
 * EBCDIC 1303
 * JEF codepage

title: Character Encodings/Code Tables/EBCDIC


text:
Character Encodings/Code Tables

 * EBCDIC
 * MS-DOS
 * Windows
 * ISO
 * HP

title: Character Encodings/Code Tables


text:
Template:Chset-table-header1/doc

Formats and draws the headers for a 16-column table of characters, in a design matching the hundreds of templates for Unicode code blocks, such as.

Usage
See ISO 8859-1, Windows-1252, and EBCDIC for examples of usage.

result:

title: Template:Chset-table-header1/doc


text:
Template:Char/doc

For showing examples of typographic symbols, using a box designed to look a bit like a metal typeface. This is to replace many uses of parenthesis, quotes, etc being used to show this, for consistent style and to avoid confusion as to what glyphs are actually part of the symbol.

After much discussion it was decided that it is preferable to use nothing around the character, or angbr. This template should only be used when it is necessary to clearly show what part of a glyph is surrounding whitespace, for very tiny glyphs, for when angle brackets are confusing (such as when the character is itself a bracket). This template is considered preferable to using code or key or quotation marks or any brackets other than angle brackets.

Usage
x → x

This purposely does not change the font, though it removes underscores if the contents are linked. It is recommended that any desired font markup be put inside the template so that if it changed to print surrounding characters instead of a box the markup will not apply to those surrounding characters.

title: Template:Char/doc


text:
User:063271&6-52&& ww

'''GOOD MORNING 🌅 MY NAME IS CARABO AND MY SURNAME IS LEONARD. I USUALLY SPEAK ZULU AND SESOTHO FOR A EXAMPLE OF ZULU: SAWBONA-HELLO AND A EXAMPLE OF SESOTHO:DITWEBA-BIRDS SO I ALSO KNOW XHOSA FOR A EXAMPLE OF XHOSA:MOLO- HELLO '''

title: User:063271&6-52&& ww


text:
Professionalism/Albert Costa, Janet Geipel, Ethics, and Language

Albert Costa, and Janet Geipel -- both psychology researchers -- have conducted several significant researches on the effect of language on ethics.

Research and Studies
Alert Costa's paper, Your Morals Depend on Language, reported "evidence that people using a foreign language make substantially more utilitarian decisions when faced with such moral dilemmas." Costa conducted an experiment in which a variety of bilingual subjects were asked to choose to "sacrifice one to save five" in both their native and second language "learned later in life." The "utilitarian solution" was chosen more frequently when comprehended in the second language across all sets of bilingual participants.

The work of Giepel and Costa suggest that morals play out differently when the same objective scenario is interpreted across different languages. In one study, Geipel's researched the implications of "the foreign language effect on moral judgement." In judging identical stories in native and foreign languages, study participants experienced inhibited morality when processing more emotional scenarios in a foreign language. Geipel proposes that morals are processed differently due to "different processing routes" used when switching languages. This is supported by the work of Caldwell-Harris, who claims that judgements of "what is right and what is wrong are not absolute" when it comes to processing the same situation or dilemma in different languages. For one, sub-par fluency limits "engagement of the brain's emotional circuitry." Moreover, the foreign language effect is influenced by the customs of the culture from which it originates. This notion is due to the "experiences and certain emotions" that are "coded in the language" in which they were perceived. Costa references the "brain-drain model", which halves cognitive responsibilities, with the first system governing unconscious thought processes and the second ruling over "controlled" and "effortful" contemplation. The former is susceptible to bias, while the latter is better at monitoring and revising. He proposes that the "time-course of information processing" is crucial to the interplay of cognitive resource expenditure for foreign language processing, and has a role in bias reduction. Indeed, foreign language processing was found to be associated with an increased likelihood of calculated risks.

Looking forward, these findings may substantiate further motivation to include foreign language study as an integral part of professional continuing education. Re-interpreting the same situation in a second language can act as a self-check on moral intuition and gut instincts.

At Work
José Luis Peñarredonda is an author for BBC Worklife that worked in London for 4 months. He is bilingual in Spanish and English, with Spanish as his primary language. During his time in London, he described working in his second language as “trying to eat soup with a fork”. He felt it was possible to communicate successfully with his colleagues, but it required more mental focus. This extra time to think is known as the Foreign Language Effect and is in line with the research by Costa and Geipel. Peñarredonda identified four major benefits of working in your second language: emotional distance, negotiation skills, coworker’s perception and more succinct answers.

Working in your second language provides emotional distance to the work and constructive feedback you receive. Peñarredonda states that he interpreted harsh commentary on his work with more analytical steps forward and felt less offended by what his coworkers could have said. Peñarredonda identified that negotiating deals could be easier in your second language; by thinking through problems slower, he was less likely to read into how the offer was presented. This leads to having an advantage when getting what he wanted, especially if the negotiator underestimates how well you understand their prose. He felt that the extra time that he was allowed to take to respond allowed him to present offers more in his favor. Finally, by speaking in your second language, he felt he could be blunt. This enabled transparent communication that resolved the negotiations quicker with less time for his original goals to be obscured.

Healthcare
A 2021 study by Cornell and Northwestern University investigated how bilingual patients choose healthcare paths depending on which language the scenario was presented in. The different choices varied in likelihood of adverse outcomes and participants decided their willingness to accept preventative treatment. This study relates to research by Costa and Geipel as scenarios in their second language were predicted to be judged more analytically. This was affirmed; participants judged both symptoms and potential side effects as less negative when presented in their secondary language. They were more sensitive to risks on both preventative care and medical conditions and evaluated them more similarly than those presented with scenarios in their primary language. This is an important finding as nearly 30% of physicians and 20% of patients in the US are multilingual and could be making decisions for themselves and others in their second language.

Financial Decisions
In 2013, Costa, Foucart and Arnon investigated whether the Foreign Language Effect was applicable to decisions relating to money. This is important to develop their findings beyond life-and-death decisions which many people do not make daily. They sought to understand the psychological accounting of outcomes through a case study deciding to replace either money or goods. The example was presented as a woman going to a theater, but once she gets there she discovers she lost her way of getting in: either the money to buy tickets or tickets she had purchased earlier. Both scenarios have an equivalent monetary loss. Participants were asked to select which scenario the woman was more likely to buy new tickets, either by replacing money to buy tickets or rebuying tickets again. It was found that participants were less likely to purchase the tickets when the tickets were lost in their primary language, showing they had more emotional connection to the story.

This study shows how participants have less intuitive bias when scenarios are presented in a foreign language and make more rational decisions based on facts presented. This supports the findings of Costa and Geipel as participants’ decisions were less emotional in second languages. Emotionality is a key factor when reducing heuristic bias, or the bias from relation to previous personal experience and the mental shortcuts these create when processing a decision. This study is important to daily life as people make decisions about finances often, and if these are processed in a second language, a more rational outcome is possible.

Potential Implications of the Studies
The research of Albert Costa and Janet Geipel on ethics and language has significant real-life implications.

First, their work helps people further understanding of the role languages play in multilingual situations, such as international business or diplomacy. The advent of globalization since the late 20th century has strengthened the tie between nations, and increased the international trade. Since 1995, global trade volume has increases more than three folds, and nations have started collaborating on a wide range of issues including climate change, and economic instability. And in such context of increasingly globalization, business people and diplomats often need to communicate in multiple foreign languages in their work. To help international organizations collaborate better, it is important for them to understand how language affects their decision-making, especially when the tasks involve complex ethical issues. Studies of Costa and Geipel help them gain insight of the effects, and could help the communicators make fewer mistakes. This is especially helpful when they deal with high-stake issues, and the outcome can impact a large number of people.

Secondly, Costa and Geipel's research helped provide insights to the psychology of millions of people who are bilingual, and may experience bilingualism's effect on moral judgement. The number of immigrants from other countries in the United States alone is over 40 millions. And because nearly all of them use English as a second language, they could experience the effects discussed in Costa and Geipel's studies, such as the potential impact in moral judgements (e.g., making more utilitarian decisions) when they navigate complex ethical decisions at research, school, or workplaces settings.

Lastly, Geipel and Costa's work has impact on countries with diverse ethnicity populations speaking different languages. For example, people living in different regions of Egypt may use at lease three languages -- Arabic, Coptic, and English -- in their daily lives. Similarly, Canada adopts two languages, English and French, as its official language. Therefore, in these countries, residents may need to communicate with others using a second language, which means the effects detailed by Geipel and Costa could have a real impact in their decision making, and conflict resolution.


text:
User talk:150.129.206.92

--SHB2000 (discuss • contribs) 10:34, 23 March 2023 (UTC)

title: User talk:150.129.206.92


text:
Wikibooks:Reading room/Administrative Assistance/Archives/2023/February

Please block MighyHeroHard
Wmf banned user; sock of MightyHeroStrikesBackHard Xeverything11 (discuss • contribs) 13:35, 1 February 2023 (UTC)

Please block Featured Books
Edit war in Unbooks:Main Page (3RR) Xeverything11 (discuss • contribs) 16:10, 1 February 2023 (UTC)

 * Oh, this user is obviously a LTA. Xeverything11 (discuss • contribs) 16:15, 1 February 2023 (UTC)
 * ✅ by Leaderboard. SHB2000 (discuss • contribs) 20:26, 1 February 2023 (UTC)

Please block Networthon938
Wmf banned user  Xeverything11 (discuss • contribs) 08:05, 2 February 2023 (UTC)

 * ✅ SHB2000 (discuss • contribs) 09:12, 2 February 2023 (UTC)

Please block 61.220.170.133
LTA Xeverything11 (discuss • contribs) 08:11, 2 February 2023 (UTC)

 * I second this. Cross-wiki harasser. LTA. SVTCobra 08:25, 2 February 2023 (UTC)
 * ✅ SHB2000 (discuss • contribs) 09:12, 2 February 2023 (UTC)

== Please block 2A01:E11:1010:4210:E540:FA79:F701:BCB4 (talk · contribs) ==

Reason: Wmf banned user Xeverything11 (discuss • contribs) 20:59, 2 February 2023 (UTC)

Protection request
Can User:Xeverything11 be semi-protected indefinitely because LTAs are vandalising the page? Xeverything11 (discuss • contribs) 21:24, 2 February 2023 (UTC)

 * ✅. SHB2000 (discuss • contribs) 19:48, 3 February 2023 (UTC)

Please block 190.6.16.108 (talk · contribs)
Reason: LTA Xeverything11 (discuss • contribs) 12:32, 3 February 2023 (UTC)

 * ✅ —Kittycataclysm (discuss • contribs) 13:01, 3 February 2023 (UTC)

Please block B2223222 (talk · contribs)
Reason: LTA Xeverything11 (discuss • contribs) 19:55, 7 February 2023 (UTC)

 * ✅ --SHB2000 (discuss • contribs) 07:49, 8 February 2023 (UTC)

Please block JIMPOKERS (talk · contribs)
Reason: LTA, see filter log Xeverything11 (discuss • contribs) 07:42, 8 February 2023 (UTC)

 * ❌ – not an LTA. Their spamming mostly seems to have been handled by the filters. Please give me a ping if they make an edit, though. --SHB2000 (discuss • contribs) 07:49, 8 February 2023 (UTC)

User:GetWellDr.
Username sounds very suspicious, IDK if this is the LTA from above, but I'm watching them and hoping they don't do anything bad. Garfieldcat1978 (discuss • contribs) 19:43, 22 February 2023 (UTC)

 * ❌ what makes you think this username is that of an LTA? SHB2000 (discuss • contribs) 20:28, 22 February 2023 (UTC)

Please block Modrkcom
Reason: self-promotion

Money-lover-12345 (discuss • contribs) 21:36, 25 February 2023 (UTC)

 * ❌ a single spammy edit isn't enough to warrant a block. SHB2000 (discuss • contribs) 11:33, 26 February 2023 (UTC)

Block for user:Shitter Destroyer 330
Reason: Inappropriate username. Leonidlednev (discuss • contribs) 20:24, 27 February 2023 (UTC)

 * Already handled before this request was made. SHB2000 (discuss • contribs) 09:14, 28 February 2023 (UTC)


text:
Piano Solo Music: An Encyclopedia/Composers born in the 19th century/Tansman, Alexandre

Compositions for piano solo by Alexandre Tansman

 * Danses miniatures 1 - 4 [YouTube] [YouTube] [YouTube] [YouTube]
 * Etude-Scherzo [YouTube]
 * Impromptus 1 - 5 [YouTube] [YouTube] [YouTube] [YouTube] [YouTube]
 * Preludes (vol. 1) 1 - 3 [YouTube] [YouTube] [YouTube]
 * Preludes (vol. 2) 1 - 4 [YouTube] [YouTube] [YouTube] [YouTube]
 * Recueil de Mazurkas (à Albert Roussel) 1 - 10 [YouTube] [YouTube] [YouTube] [YouTube] [YouTube] [YouTube] [YouTube] [YouTube] [YouTube] [YouTube]
 * Sonata rustica [YouTube] [YouTube] [YouTube]
 * Sonatina no. 1 [YouTube] [YouTube] [YouTube]


text:
Supplementary mathematics/Mathematics

Mathematics is the art of calculating numbers and also studies topics such as quantity (number theory), structure (algebra), space (geometry) and variation (mathematical analysis). In fact, there is no universal definition of mathematics that everyone agrees on.

Most mathematical activities involve discovering and proving the properties of abstract objects by pure reasoning. These objects are either abstractions of nature, such as natural numbers or lines, or – in modern mathematics – entities that have certain properties called axioms. An argument consists of a set of applications of some deductive rules to already known results, including previously proven theorems, axioms, and (if abstracted from nature) some basic properties that serve as the actual starting point of the theory under consideration. is considered are taken, the result of an argument is called a theorem.

Mathematics is widely used in science to model phenomena. This allows the derivation of quantitative predictions from empirical laws. For example, the motion of the planets can be accurately predicted using Newton's law of gravitation combined with mathematical calculations. The independence of mathematical truth from any experiment shows that the accuracy of such predictions depends only on the adequacy of the model to describe reality. Incorrect predictions indicate that the mathematical models need to be improved or changed, not that the mathematics in the models themselves are wrong. For example, the precession of Mercury's perihelion cannot be explained by Newton's law of gravitation, but it is precisely explained by Newton's law of gravitation. Einstein's General Relativity - This experimental confirmation of Einstein's theory shows that Newton's law of gravitation is only an approximation, although it is accurate in everyday applications.

Mathematics is essential in many fields including natural sciences, engineering, medicine, finance, computer science and social sciences. Some areas of mathematics, such as statistics and game theory, have been developed in close connection with their applications and are often grouped under applied mathematics. Other areas of mathematics develop independently of any application (and are therefore called pure mathematics), but practical applications are often discovered later. A good example is the problem of factoring integers, which dates back to the time of Euclid, but had no practical application before being used in the RSA cryptosystem (for computer network security).

Historically, the concept of proof and the mathematical precision associated with it first appeared in Greek mathematics, particularly in Euclid's Elements. From the beginning, mathematics was basically divided into geometry and calculus (manipulation of numbers and natural fractions) until in the 16th and 17th centuries, algebra and infinitesimal calculus were introduced as new disciplines. Since then, the interaction between mathematical innovations and scientific discoveries has led to the rapid growth of mathematics. At the end of the 19th century, the fundamental crisis of mathematics led to the systematization of the axiomatic method. This in turn caused a significant increase in the number of mathematical disciplines and their applied fields. An example of this classification is mathematics courses, which lists more than sixty areas of first level mathematics.

History
The history of mathematics can be seen as a sequence of increasing abstractions. The first abstraction capability shared by many animals is probably the concept of number: the understanding that a set of two apples and a set of two oranges (for example) have in common, and that quantity is their number.

Prehistoric people could count both physical objects and abstract objects such as days, seasons, and years, as evidenced by woodcuts.

Evidence for more complex mathematics is not seen until 3000 BCE, when the Babylonians and Egyptians began using arithmetic, algebra, and geometry for calculations related to taxation and other economic concepts, and construction or astronomy. The oldest mathematical texts are from Mesopotamia and Egypt, dating back to 2000-1800 BC.Many early texts mention Pythagorean triads, so it seems that the Pythagorean theorem is one of the most important methods for inventing trigonometry.

This theorem has been proven many times by different geometric and algebraic methods, some of these proofs go back thousands of years. It is the oldest and most extensive mathematical development after elementary arithmetic and geometry. In historical documents, it was in Babylonian mathematics that elementary arithmetic (addition, subtraction, multiplication and division) first appeared. The Babylonians also used a place-value device that implemented a base 60 number device, which is still used to measure angles and time.

With the beginning of the 6th century BC, the Greek mathematicians with the Pythagoreans began a systematic study of mathematics, with the aim of knowing more about mathematics itself, which was the beginning of Greek mathematics. Around 300 BC, Euclid introduced the method of thematic principles still used in mathematics, which included definitions, principles, theorems, and proofs. His reference book known as Euclid's Principles is widely regarded as the most successful and influential reference book of all time. The greatest ancient mathematicians are often considered to be Archimedes (287-212 BC) from Syracuse. He found formulas for calculating the area and volume of rotating objects and used Afna's method to calculate the area under a parabolic curve using the sum of an infinite series. In a way that is not dissimilar to modern calculus. Other notable achievements in Greek mathematics were conic sections (Apollonius of Perga, 3rd century BC), trigonometry (Hiparchus of Nicaea (2nd century BC)), and the beginning of algebra (Diophantus, 3rd century AD).

The Indo-Arabic number system and the rules for using its operations, which are used all over the world today, were developed in India during the first millennium AD and then transferred to the Western world through Islamic mathematics. Other developments related to Indian mathematics include the modern definition of sine and cosine and the early form of infinite series.

During the Golden Age of Islam, which took place in the 9th and 10th centuries AD, mathematics saw important innovations that were based on the mathematics of the Greeks. The most important achievements of Islamic mathematics was the development of algebra. Other important achievements of mathematics in the Islamic period were the progress in spherical trigonometry and the addition of decimals to the Arabic numerical system. Many mathematicians of this period were Persian-speaking, such as Khwarazmi, Khayyam and Sharafuddin Tosi.

During the early modern era, mathematics began to develop rapidly in Western Europe. The development of calculus by Newton and Leibniz in the 17th century revolutionized mathematics. Leonard Euler was the most important mathematician of the 18th century who added several theorems and discoveries to mathematics. Perhaps the most important mathematicians of the 19th century was the German mathematician Carl Friedrich Gauss, who provided many services to various branches of mathematics such as algebra, analysis, differential geometry, matrix theory, number theory, and statistics. In the early 20th century, Kurt Gödel revolutionized mathematics by publishing his incompleteness theorems. These theorems showed that any system of principles of compatibility includes unprovable propositions.

Since then, mathematics has been widely developed, and fruitful interactions between mathematics and science have been created that benefit both. Mathematical discoveries continue to this day. According to Mikhail Suriuk, published in the January 2006 Bulletin of the American Mathematical Society, "the number of articles and books in the Mathematical Review database since 1940 (MR's first year of operation) now stands at 1.9 million, an annual increase of over 75 Thousands of items will be added to this database.The vast majority of works in this ocean contain new mathematical theorems and their proofs.

Areas of mathematics
Before the Renaissance, mathematics was divided into two main areas: arithmeticregarding the manipulation of numbers, and geometry, regarding the study of shapes. Some types of pseudoscience, such as numerology and astrology, were not then clearly distinguished from mathematics.

During the Renaissance, two more areas appeared. Mathematical notation led to algebra which, roughly speaking, consists of the study and the manipulation of formulas. Calculus, consisting of the two subfields differential calculus and integral calculus, is the study of continuous functions, which model the typically nonlinear relationships between varying quantities, as represented by variables. This division into four main areasarithmetic, geometry, algebra, calculusendured until the end of the 19th century. Areas such as celestial mechanics and solid mechanics were then studied by mathematicians, but now are considered as belonging to physics. The subject of combinatorics has been studied for much of recorded history, yet did not become a separate branch of mathematics until the seventeenth century.

At the end of the 19th century, the foundational crisis in mathematics and the resulting systematization of the axiomatic method led to an explosion of new areas of mathematics. The 2020 Mathematics Subject Classification contains no less than first-level areas. Some of these areas correspond to the older division, as is true regarding number theory (the modern name for higher arithmetic) and geometry. Several other first-level areas have "geometry" in their names or are otherwise commonly considered part of geometry. Algebra and calculus do not appear as first-level areas but are respectively split into several first-level areas. Other first-level areas emerged during the 20th century or had not previously been considered as mathematics, such as mathematical logic and foundations.

Number theory
Number theory began with the manipulation of numbers, that is, natural numbers $$(\mathbb{N}),$$ and later expanded to integers $$(\Z)$$ and rational numbers $$(\Q).$$ Number theory was once called arithmetic, but nowadays this term is mostly used for numerical calculations. Number theory dates back to ancient Babylon and probably China. Two prominent early number theorists were Euclid of ancient Greece and Diophantus of Alexandria. The modern study of number theory in its abstract form is largely attributed to Pierre de Fermat and Leonhard Euler. The field came to full fruition with the contributions of Adrien-Marie Legendre and Carl Friedrich Gauss.

Many easily stated number problems have solutions that require sophisticated methods, often from across mathematics. A prominent example is Fermat's Last Theorem. This conjecture was stated in 1637 by Pierre de Fermat, but it was proved only in 1994 by Andrew Wiles, who used tools including scheme theory from algebraic geometry, category theory, and homological algebra. Another example is Goldbach's conjecture, which asserts that every even integer greater than 2 is the sum of two prime numbers. Stated in 1742 by Christian Goldbach, it remains unproven despite considerable effort.

Number theory includes several subareas, including analytic number theory, algebraic number theory, geometry of numbers (method oriented), diophantine equations, and transcendence theory (problem oriented).

Geometry
Geometry is one of the oldest branches of mathematics. It started with empirical recipes concerning shapes, such as lines, angles and circles, which were developed mainly for the needs of surveying and architecture, but has since blossomed out into many other subfields.

A fundamental innovation was the ancient Greeks' introduction of the concept of proofs, which require that every assertion must be proved. For example, it is not sufficient to verify by measurement that, say, two lengths are equal; their equality must be proven via reasoning from previously accepted results (theorems) and a few basic statements. The basic statements are not subject to proof because they are self-evident (postulates), or are part of the definition of the subject of study (axioms). This principle, foundational for all mathematics, was first elaborated for geometry, and was systematized by Euclid around 300 BC in his book Elements.

The resulting Euclidean geometry is the study of shapes and their arrangements constructed from lines, planes and circles in the Euclidean plane (plane geometry) and the three-dimensional Euclidean space.

Euclidean geometry was developed without change of methods or scope until the 17th century, when René Descartes introduced what is now called Cartesian coordinates. This constituted a major change of paradigm: Instead of defining real numbers as lengths of line segments (see number line), it allowed the representation of points using their coordinates, which are numbers. Algebra (and later, calculus) can thus be used to solve geometrical problems. Geometry was split into two new subfields: synthetic geometry, which uses purely geometrical methods, and analytic geometry, which uses coordinates systemically.

Analytic geometry allows the study of curves unrelated to circles and lines. Such curves can be defined as the graph of functions, the study of which led to differential geometry. They can also be defined as implicit equations, often polynomial equations (which spawned algebraic geometry). Analytic geometry also makes it possible to consider Euclidean spaces of higher than three dimensions.

In the 19th century, mathematicians discovered non-Euclidean geometries, which do not follow the parallel postulate. By questioning that postulate's truth, this discovery has been viewed as joining Russell's paradox in revealing the foundational crisis of mathematics. This aspect of the crisis was solved by systematizing the axiomatic method, and adopting that the truth of the chosen axioms is not a mathematical problem. In turn, the axiomatic method allows for the study of various geometries obtained either by changing the axioms or by considering properties that do not change under specific transformations of the space.

Today's subareas of geometry include:

 * Projective geometry, introduced in the 16th century by Girard Desargues, extends Euclidean geometry by adding points at infinity at which parallel lines intersect. This simplifies many aspects of classical geometry by unifying the treatments for intersecting and parallel lines.
 * Affine geometry, the study of properties relative to parallelism and independent from the concept of length.
 * Differential geometry, the study of curves, surfaces, and their generalizations, which are defined using differentiable functions.
 * Manifold theory, the study of shapes that are not necessarily embedded in a larger space.
 * Riemannian geometry, the study of distance properties in curved spaces.
 * Algebraic geometry, the study of curves, surfaces, and their generalizations, which are defined using polynomials.
 * Topology, the study of properties that are kept under continuous deformations.
 * Algebraic topology, the use in topology of algebraic methods, mainly homological algebra.
 * Discrete geometry, the study of finite configurations in geometry.
 * Convex geometry, the study of convex sets, which takes its importance from its applications in optimization.
 * Complex geometry, the geometry obtained by replacing real numbers with complex numbers.

Algebra
Algebra is the art of manipulating equations and formulas. Diophantus (3rd century) and al-Khwarizmi (9th century) were the two main precursors of algebra. Diophantus solved some equations involving unknown natural numbers by deducing new relations until he obtained the solution. Al-Khwarizmi introduced systematic methods for transforming equations, such as moving a term from one side of an equation into the other side. The term algebra is derived from the Arabic word al-jabr meaning 'the reunion of broken parts' that he used for naming one of these methods in the title of his main treatise.

Algebra became an area in its own right only with François Viète (1540–1603), who introduced the use of variables for representing unknown or unspecified numbers. Variables allow mathematicians to describe the operations that have to be done on the numbers represented using mathematical formulas.

Until the 19th century, algebra consisted mainly of the study of linear equations (presently linear algebra), and polynomial equations in a single unknown, which were called algebraic equations (a ter m still in use, although it may be ambiguous). During the 19th century, mathematicians began to use variables to represent things other than numbers (such as matrices, modular integers, and geometric transformations), on which generalizations of arithmetic operations are often valid. The concept of algebraic structure addresses this, consisting of a set whose elements are unspecified, of operations acting on the elements of the set, and rules that these operations must follow. The scope of algebra thus grew to include the study of algebraic structures. This object of algebra was called modern algebra or abstract algebra, as established by the influence and works of Emmy Noether. (The latter term appears mainly in an educational context, in opposition to elementary algebra, which is concerned with the older way of manipulating formulas.)

Some types of algebraic structures have useful and often fundamental properties, in many areas of mathematics. Their study became autonomous parts of algebra, and include:

 * group theory;
 * field theory;
 * vector spaces, whose study is essentially the same as linear algebra;
 * ring theory;
 * commutative algebra, which is the study of commutative rings, includes the study of polynomials, and is a foundational part of algebraic geometry;
 * homological algebra;
 * Lie algebra and Lie group theory;
 * Boolean algebra, which is widely used for the study of the logical structure of computers.

The study of types of algebraic structures as mathematical objects is the purpose of universal algebra and category theory. The latter applies to every mathematical structure (not only algebraic ones). At its origin, it was introduced, together with homological algebra for allowing the algebraic study of non-algebraic objects such as topological spaces; this particular area of application is called algebraic topology.

Calculus and analysis
Calculus, formerly called infinitesimal calculus, was introduced independently and simultaneously by 17th-century mathematicians Newton and Leibniz. It is fundamentally the study of the relationship of variables that depend on each other. Calculus was expanded in the 18th century by Euler with the introduction of the concept of a function and many other results. Presently, "calculus" refers mainly to the elementary part of this theory, and "analysis" is commonly used for advanced parts.

Analysis is further subdivided into real analysis, where variables represent real numbers, and complex analysis, where variables represent complex numbers. Analysis includes many subareas shared by other areas of mathematics which include:

 * Multivariable calculus
 * Functional analysis, where variables represent varying functions;
 * Integration, measure theory and potential theory, all strongly related with probability theory on a continuum;
 * Ordinary differential equations;
 * Partial differential equations;
 * Numerical analysis, mainly devoted to the computation on computers of solutions of ordinary and partial differential equations that arise in many applications.

Discrete mathematics
Discrete mathematics, broadly speaking, is the study of individual, countable mathematical objects. An example is the set of all integers. Because the objects of study here are discrete, the methods of calculus and mathematical analysis do not directly apply. Algorithmsespecially their implementation and computational complexity play a major role in discrete mathematics.

The four color theorem and optimal sphere packing were two major problems of discrete mathematics solved in the second half of the 20th century. The P versus NP problem, which remains open to this day, is also important for discrete mathematics, since its solution would potentially impact a large number of computationally difficult problems.

Discrete mathematics includes:

 * Combinatorics, the art of enumerating mathematical objects that satisfy some given constraints. Originally, these objects were elements

or subsets of a given set; this has been extended to various objects, which establishes a strong link between combinatorics and other parts of discrete mathematics. For example, discrete geometry includes counting configurations of geometric shapes

 * Graph theory and hypergraphs
 * Coding theory, including error correcting codes and a part of cryptography
 * Matroid theory
 * Discrete geometry
 * Discrete probability distributions
 * Game theory (although continuous games are also studied, most common games, such as chess and poker are discrete)
 * Discrete optimization, including combinatorial optimization, integer programming, constraint programming

Mathematical logic and set theory

The two subjects of mathematical logic and set theory have belonged to mathematics since the end of the 19th century. Before this period, sets were not considered to be mathematical objects, and logic, although used for mathematical proofs, belonged to philosophy and was not specifically studied by mathematicians.

Before Cantor's study of infinite sets, mathematicians were reluctant to consider actually infinite collections, and considered infinity to be the result of endless enumeration. Cantor's work offended many mathematicians not only by considering actually infinite sets but by showing that this implies different sizes of infinity, per Cantor's diagonal argument. This led to the controversy over Cantor's set theory.

In the same period, various areas of mathematics concluded the former intuitive definitions of the basic mathematical objects were insufficient for ensuring mathematical rigour. Examples of such intuitive definitions are "a set is a collection of objects", "natural number is what is used for counting", "a point is a shape with a zero length in every direction", "a curve is a trace left by a moving point", etc.

This became the foundational crisis of mathematics. It was eventually solved in mainstream mathematics by systematizing the axiomatic method inside a formalized set theory. Roughly speaking, each mathematical object is defined by the set of all similar objects and the properties that these objects must have. For example, in Peano arithmetic, the natural numbers are defined by "zero is a number", "each number has a unique successor", "each number but zero has a unique predecessor", and some rules of reasoning. This mathematical abstraction from reality is embodied in the modern philosophy of formalism, as founded by David Hilbert around 1910.

The "nature" of the objects defined this way is a philosophical problem that mathematicians leave to philosophers, even if many mathematicians have opinions on this nature, and use their opinionsometimes called "intuition"to guide their study and proofs. The approach allows considering "logics" (that is, sets of allowed deducing rules), theorems, proofs, etc. as mathematical objects, and to prove theorems about them. For example, Gödel's incompleteness theorems assert, roughly speaking that, in every consistent formal system that contains the natural numbers, there are theorems that are true (that is provable in a stronger system), but not provable inside the system. This approach to the foundations of mathematics was challenged during the first half of the 20th century by mathematicians led by Brouwer, who promoted intuitionistic logic, which explicitly lacks the law of excluded middle.

These problems and debates led to a wide expansion of mathematical logic, with subareas such as model theory (modeling some logical theories inside other theories), proof theory, type theory, computability theory and computational complexity theory. Although these aspects of mathematical logic were introduced before the rise of computers, their use in compiler design, program certification, proof assistants and other aspects of computer science, contributed in turn to the expansion of these logical theories.

Statistics and other decision sciences
The field of statistics is a mathematical application that is employed for the collection and processing of data samples, using procedures based on mathematical methods especially probability theory. Statisticians generate data with random sampling or randomized experiments. The design of a statistical sample or experiment determines the analytical methods that will be used. Analysis of data from observational studies is done using statistical models and the theory of infer ence, using model selection and estimation. The models and consequential predictions should then be tested against new data.

Statistical theory studies decision problems such as minimizing the risk (expected loss) of a statistical action, such as using a procedure in, for example, parameter estimation, hypothesis testing, and selecting the best. In these traditional areas of mathematical statistics, a statistical-decision problem is formulated by minimizing an objective function, like expected loss or cost, under specific constraints. For example, designing a survey often involves minimizing the cost of estimating a population mean with a given level of confidence. Because of its use of optimization, the mathematical theory of statistics overlaps with other decision sciences, such as operations research, control theory, and mathematical economics.

Computational mathematics
Computational mathematics is the study of mathematical problems that are typically too large for human, numerical capacity. Numerical analysis studies methods for problems in analysis using functional analysis and approximation theory; numerical analysis broadly includes the study of approximation and discretization with special focus on rounding errors. Numerical analysis and, more broadly, scientific computing also study non-analytic topics of mathematical science, especially algorithmic-matrix-and-graph theory. Other areas of computational mathematics include computer algebra and symbolic computation.

title: Supplementary mathematics/Mathematics