# Geographical distance

Geodesy
Fundamentals
Concepts
Technologies
Standards
History

Geographical distance is the distance measured along the surface of the earth. The formulae in this article calculate distances between points which are defined by geographical coordinates in terms of latitude and longitude. This distance is an element in solving the second (inverse) geodetic problem.

## An abstraction

Calculating the distance between geographical coordinates is based on some level of abstraction; it does not provide an exact distance, which is unattainable if one attempted to account for every irregularity in the surface of the earth.[1] Common abstractions for the surface between two geographic points are:

• Flat surface;
• Spherical surface;
• Ellipsoidal surface.

All abstractions above ignore changes in elevation. Calculation of distances which account for changes in elevation relative to the idealized surface are not discussed in this article.

## Nomenclature

Distance, $D,\,\!$ is calculated between two points, $P_1\,\!$ and $P_2\,\!$. The geographical coordinates of the two points, as (latitude, longitude) pairs, are $(\phi_1,\lambda_1)\,\!$ and $(\phi_2,\lambda_2),\,\!$ respectively. Which of the two points is designated as $P_1\,\!$ is not important for the calculation of distance.

Latitude and longitude coordinates on maps are usually expressed in degrees. In the given forms of the formulae below, one or more values must be expressed in the specified units to obtain the correct result. Where geographic coordinates are used as the argument of a trigonometric function, the values may be expressed in any angular units compatible with the method used to determine the value of the trigonometric function. Many electronic calculators allow calculations of trigonometric functions in either degrees or radians. The calculator mode must be compatible with the units used for geometric coordinates.

Differences in latitude and longitude are labeled and calculated as follows:

$\begin{align} \Delta\phi&=\phi_2-\phi_1;\\ \Delta\lambda&=\lambda_2-\lambda_1. \end{align} \,\!$

It is not important whether the result is positive or negative when used in the formulae below.

"Mean latitude" is labeled and calculated as follows:

$\phi_m=\frac{\phi_1+\phi_2}{2}.\,\!$

Colatitude is labeled and calculated as follows:

For latitudes expressed in radians:
$\theta=\frac{\pi}{2}-\phi;\,\!$
For latitudes expressed in degrees:
$\theta=90^\circ-\phi.\,\!$

Unless specified otherwise, the radius of the earth for the calculations below is:

$R\,\!$ = 6,371.009 kilometers = 3,958.761 statute miles = 3,440.069 nautical miles.

$D_\,\!$ = Distance between the two points, as measured along the surface of the earth and in the same units as the value used for radius unless specified otherwise.

## Singularities and discontinuity of latitude/longitude

Longitude has singularities at the Poles (longitude is undefined) and a discontinuity at the ±180° meridian. Also, planar projections of the circles of constant latitude are highly curved near the Poles. Hence, the above equations for delta latitude/longitude ($\Delta\phi\!$, $\Delta\lambda\!$) and mean latitude ($\phi_m\!$) may not give the expected answer for positions near the Poles or the ±180° meridian. Consider e.g. the value of $\Delta\lambda\!$ (“east displacement”) when $\lambda_1\!$ and $\lambda_2\!$ are on either side of the ±180° meridian, or the value of $\phi_m\!$ (“mean latitude”) for the two positions ($\phi_1\!$=89°, $\lambda_1\!$=45°) and ($\phi_2\!$=89°, $\lambda_2\!$=−135°).

If a calculation based on latitude/longitude should be valid for all Earth positions, it should be verified that the discontinuity and the Poles are handled correctly. Another solution is to use n-vector instead of latitude/longitude, since this representation does not have discontinuities or singularities.

## Flat-surface formulae

A planar approximation for the surface of the earth may be useful over small distances. The accuracy of distance calculations using this approximation become increasingly inaccurate as:

• The separation between the points becomes greater;
• A point becomes closer to a geographic pole.

The shortest distance between two points in plane is a straight line. The Pythagorean theorem is used to calculate the distance between points in a plane.

Even over short distances, the accuracy of geographic distance calculations which assume a flat Earth depend on the method by which the latitude and longitude coordinates have been projected onto the plane. The projection of global latitude and longitude coordinates onto a plane is the realm of cartography.

The formulae presented in this section provide varying degrees of accuracy.

### Spherical Earth projected to a plane

This formula takes into account the variation in distance between meridians with latitude:

$D=R\sqrt{(\Delta\phi)^2+(\cos(\phi_m)\Delta\lambda)^2}{\color{white}\frac{\big|}{.}}\,\!$
where:
$\Delta\phi\,\!$ and $\Delta\lambda\,\!$ are in radians;
$\phi_m\,\!$ must be in units compatible with the method used for determining $\cos(\phi_m).\,\!$
To convert latitude or longitude to radians use
$1^\circ = (\pi/180)\,\mathrm{radians}.$
Note: This approximation is very fast and produces fairly accurate result for small distances[citation needed]. Also, when ordering locations by distance, such as in a database query, it is much faster to order by squared distance, eliminating the need for computing the square root.

### Ellipsoidal Earth projected to a plane

The FCC prescribes essentially the following formulae in 47 CFR 73.208 for distances not exceeding 475 km /295 miles:[2]

$D=\sqrt{(K_1\Delta\phi)^2+(K_2\Delta\lambda)^2};{\color{white}\frac{\big|}{.}}\,\!$
where
$D\,\!$ = Distance in kilometers;
$\Delta\phi\,\!$ and $\Delta\lambda\,\!$ are in degrees;
$\phi_m\,\!$ must be in units compatible with the method used for determining $\cos(\phi_m);\,\!$
$\begin{align} K_1&=111.13209-0.56605\cos(2\phi_m)+0.00120\cos(4\phi_m);\\ K_2&=111.41513\cos(\phi_m)-0.09455\cos(3\phi_m)+0.00012\cos(5\phi_m).\end{align}\,\!$
It may be interesting to note that:
$K_1=M\frac{\pi}{180}\,\!$ = kilometers per degree of latitude difference;
$K_2=\cos(\phi_m)N\frac{\pi}{180}\,\!$ = kilometers per degree of longitude difference;
where $M\,\!$ and $N\,\!$ are the meridional and its perpendicular, or "normal", radii of curvature (the expressions in the FCC formula are derived from the binomial series expansion form of $M\,\!$ and $N\,\!$, set to the Clarke 1866 reference ellipsoid).

### Polar coordinate flat-Earth formula

$D=R\sqrt{\theta^2_1\;\boldsymbol{+}\;\theta^2_2\;\mathbf{-}\;2\theta_1\theta_2\cos(\Delta\lambda)};{\color{white}\frac{\big|}{.}}\,\!$
where the colatitude values are in radians. For a latitude measured in degrees, the colatitude in radians may be calculated as follows: $\theta=\frac{\pi}{180}(90^\circ-\phi).\,\!$

## Spherical-surface formulae

If we are willing to accept a possible error of 0.5%, we can use formulas of spherical trigonometry on the sphere that best approximates the surface of the earth.

The shortest distance along the surface of a sphere between two points on the surface is along the great-circle which contains the two points.

### Tunnel distance

A tunnel between points on Earth is defined by a line through three-dimensional space between the points of interest. For a spherical Earth, this line is also the chord of the great circle between the points. For points near each other, the tunnel distance is only slightly less than the great-circle distance.[3]

The great circle chord length may be calculated as follows for the corresponding unit sphere:

$\begin{align} &\Delta{X}=\cos(\theta_2)\cos(\lambda_2) - \cos(\theta_1)\cos(\lambda_1);\\ &\Delta{Y}=\cos(\theta_2)\sin(\lambda_2) - \cos(\theta_1)\sin(\lambda_1);\\ &\Delta{Z}=\sin(\theta_2) - \sin(\theta_1);\\ &C_h=\sqrt{(\Delta{X})^2 + (\Delta{Y})^2 + (\Delta{Z})^2}.\end{align}\,\!$

The tunnel distance between points on the surface of a spherical Earth is: $D = R C_h$

### Great-circle distance

The great-circle distance article gives the formula for calculating the distance along a great-circle on a sphere about the size of the Earth. The article includes an example of the calculation.

## Ellipsoidal-surface formulae

An ellipsoidal approximation for the surface of the earth may be useful over great distances. The shortest distance along the surface of an ellipsoid between two points on the surface is along the geodesic.

### Exact method

Clairaut[4] derived a conserved quantity for geodesics on an ellipsoid of revolution, Clairaut's relation. This enabled Legendre[5] to transfer the ellipsoidal geodesics onto an "auxiliary sphere". On this sphere, the geographic latitude is replaced by the reduced latitude, the azimuth of the geodesic is preserved, and the distance along the geodesic and the geographic longitude are related to the spherical arc length and spherical longitude by simple one dimensional integrals. A succinct derivation of these results is given by Bessel.[6] The problem was further elaborated by Helmert,[7] and Rapp[8] provides a good modern summary of the method.

Vincenty[9][10] formulated an algorithm based the solution using the auxiliary sphere. This is accurate to third order in the flattening of the ellipsoid, i.e., about 0.5 mm; however, the algorithm fails to converge for points that are nearly anti-podal. For details, see Vincenty's formulae.

Karney[11] tackled the inverse geodesic problem using Newton's method to accelerate the convergence. In addition, he employs series which are accurate to sixth order in the flattening. This results in an algorithm which is accurate to full double precision and which converges for arbitrary pairs of points on the earth. This algorithm is implemented in GeographicLib.[12]

### Approximate methods

The exact methods described above are feasible when carrying out calculations on a computer. Before the advent of computers, several authors derived approximate formulas suitable for hand calculations.

If we limit distance to 100–150 km we can get millimeter accuracy with equations given by Bowring.[13][14] In the linked pdf "e' 2" is the second eccentricity squared, a constant for the chosen spheroid

$e'^2 = \frac {2r - 1}{(r - 1)^2}$

where r is the reciprocal of the flattening (r = 298.257223563 for the WGS84 spheroid). $\scriptstyle \phi_1$ and $\scriptstyle \phi_2$ are the latitudes of the two points, $\scriptstyle \lambda_1$ and $\scriptstyle \lambda_2$ are the longitudes; $\scriptstyle \Delta \phi$ is the difference in latitude (which at one point must be in radians). Calculate A, B, C and w on the first page of the pdf, then skip to "Inverse Problem" on the second page.

Lambert's formulae[15] give accuracy on the order of 10 meters over thousands of kilometers. First convert the latitudes $\scriptstyle \phi_1$ , $\scriptstyle \phi_2$ of the two points to reduced latitudes $\scriptstyle \psi_1$ , $\scriptstyle \psi_2$

$\tan \psi \; = \; \left ( \frac {r - 1}{r} \right ) \tan \phi.$

Then calculate the central angle $\sigma$ in radians between two points $(\psi_1 , \; \lambda_1)$ and $(\psi_2 , \; \lambda_2)$ on a sphere in the usual way (law of cosines or haversine formula), with longitudes $\lambda_1 \;$ and $\lambda_2 \;$ being the same on the sphere as on the spheroid.

$P = \frac { \psi_1 + \psi_2 }{2}$

$Q = \frac {\psi_2 - \psi_1}{2}$

$X = ( \sigma - \sin \sigma) \frac {\sin^2 P \cos^2 Q}{ \cos^2 \frac { \sigma}{2}}$

$Y = ( \sigma + \sin \sigma) \frac {\cos^2 P \sin^2 Q}{ \sin^2 \frac { \sigma}{2}}$

$distance = a \left ( \sigma - \frac {X + Y}{2r} \right )$ where $a$ is the equatorial radius of the chosen spheroid.

On the GRS 80 spheroid Lambert's formula is off by

0 North 0 West to 40 North 120 West, 12.6 meters
0N 0W to 40N 60W, 6.6 meters
40N 0W to 40N 60W, 0.85 meter

## References

1. Clairaut, A. C. (1735). "Détermination géometrique de la perpendiculaire à la méridienne tracée par M. Cassini" [Geometrical determination of the perpendicular to the meridian drawn by Jacques Cassini]. Mém. de l'Acad. Roy. des Sciences de Paris 1733 (in French): 406–416.
2. Legendre, A. M. (1806). "Analyse des triangles tracės sur la surface d'un sphėroïde". Mém. de l'Inst. Nat. de France (1st sem.): 130–161. Retrieved 2011-07-30.
3. Bessel, F. W. (2010). "The calculation of longitude and latitude from geodesic measurements (1825)". Astron. Nachr. 331 (8): 852–861. arXiv:0908.1824. doi:10.1002/asna.201011352. English translation of Astron. Nachr. 4, 241–254 (1825).
4. Helmert, F. R. (1964). Mathematical and Physical Theories of Higher Geodesy, Part 1 (1880). St. Louis: Aeronautical Chart and Information Center. Retrieved 2011-07-30. English translation of Die Mathematischen und Physikalischen Theorieen der Höheren Geodäsie, Vol. 1 (Teubner, Leipzig, 1880).
5. Rapp, R. H. (March 1993). Geometric Geodesy, Part II. Ohio State University. Retrieved 2011-08-01.
6. Vincenty, T. (April 1975a). "Direct and Inverse Solutions of Geodesics on the Ellipsoid with application of nested equations". Survey Review. XXIII (misprinted as XXII) (176): 88–93. Retrieved 2009-07-11.
7. Vincenty, T. (April 1976). "Correspondence". Survey Review. XXIII (180): 294.
8. Karney, C. F. F. (2013). "Algorithms for geodesics". J. Geodesy 87 (1): 43–55. doi:10.1007/s00190-012-0578-z. Retrieved 2012-07-14.
9.
10. Bowring, B. R. (1981). "The direct and inverse problems for short geodesics lines on the ellipsoid". Surveying and Mapping 41 (2): 135–141.
11. Bowring's formulae at the Wayback Machine (archived March 19, 2012)
12. Lambert, W. D (1942). "The distance between two widely separated points on the surface of the earth". J. Washington Academy of Sciences 32 (5): 125–130.
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 68, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.3, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.8277013897895813, "perplexity_flag": "middle"}