Re: formula for great circle computation

From:         Alexandre GRIES <>
Organization: Pressimage, France
Date:         13 Oct 95 01:30:21 
References:   1 2 3
Next article
View raw article
  or MIME structure

Dear Captain Barnett,

As I understood you're looking for the formulae to calculate great circle
course and distance while having departure and destination coordinates.

We, in France, call it "orthodromie" computation.

The formulae following are giving the distance and the departure azimuth
for the great circle ride.


A is the departure point
B is the destination point
M is the distance
LA is the departure latitude
LB is the destination latitude
GA is the departure longitude
GB is the destination longitude
P is the longitude difference
ZA is the departure azimuth

1. Distance computation.

Cos M = Sin(LA).SIN(LB) + Cos(LA).Cos(LB).Cos(P)

Cos M =        a        +        b

a has a negative value when latitudes have a different sign
b has a negative value when P>90°

if a+b is positive M<90°
if a+b is negative M>90°

Use the acos function.

M is given in degree and minute of arc. It has to be converted in
nautical miles.

2. Departure azimuth.

The azimuth is the the angle between the local meridian and the great
circle computed.

The two formulae are efficient:

Sin(ZA) = (Cos(LB).Sin(P))/Sin(M)

Cotg(ZA) = (((Tg(LB).Cos(LA))/Sin(P)) - Sin(LA).Cotg(P)

Cotg(ZA) =               a            -        b

a has the LB sign.
b is positive when Sin(LA) and Cotg(P) have the same sign.

use the asin or acotg functions.

3. Example.

LA = 33.00 S  GA = 91.55 W
LB = 14.30 N  GB = 100.39 E

Distance computation.

P = 167.26
Cos(M) = 0.92887
a is negative, b is negative, a+b is negative => M>90°
M = -21°44 = 180 - 21.44 = 158°16 = 9496 Nm

Departure azimuth.

Sin(ZA) = 0.56888
ZA = 34.40 = 180 + 34.40 = 214.40 (215°)

Cotg(ZA) = 1.44640
ZA = 214.40 (215°)

The destination azimuth can be found by replacing LA by LB in the

I hope that these few elements will be interesting for your demand. Sorry
for my "french pilot" way of speeking english (I am an Airbus 320 Air
France pilot, ATR 42/72 Captain and French ATPL instructor)

Alexandre GRIES - - Tel/Fax 33 (1) 42 57 15 80
38 Avenue JUNOt - F75018 PARIS - FRANCE