But now an electrical engineer with an interest in archaeology thinks he has the answer.
The near-perfect alignment along the four points of the compass might be the result of the ancient Egyptians using the autumn equinox - when the Earth's equator passes through the centre of the Sun's disk.
Glen Dash, an amateur archaeologist, has published a paper in The Journal of Ancient Egyptian Architecture that tries to explain the impressive accuracy of this ancient feat of engineering.
The Great Pyramid, only 0.067 degrees counter-clockwise from perfect cardinal alignment, could have been achieved by tracking the point of a shadow at regular intervals on the day of the autumn equinox.
To prove this theory, Mr Dash made his own experiment using a tall rod, commonly known as a gnomon, and a piece of string.
On the day of the equinox, he placed the rod in the ground, marked the tips of the shadows and formed a smooth curve of points.
He then intercepted a couple of the points on the curve and was able to draw a line running almost perfectly east to west.
Although Mr Dash's experiment was conducted in Pomfret, Connecticut, the engineer believes that, on a clear and sunny day, the same thing should work in Egypt.
He argues that the Egyptians may have used the same technique to build Pharaoh Khufu’s iconic pyramids because the degree of error measured in the experiment was slightly counter-clockwise, just like the slight error found in the alignment of the 4,500-year-old monuments in Giza.
Although the paper offers a possible explanation for the long-standing mystery of how the Giza pyramids were aligned so accurately, nothing is certain.
"The Egyptians, unfortunately, left us few clues," Mr Dash wrote.
"No engineering documents or architectural plans have been found that give technical explanations demonstrating how the ancient Egyptians aligned any of their temples or pyramids."
Some archaeologists have argued that the Egyptians used the pole star, while others say they used the sun's shadow to align the pyramids.
It is even possible, says Mr Dash, that the ancient civilisation used multiple methods.