He was born on a farm in Gloucester Township, Canada West in an area now known as Glen Ogilvie. Ogilvie articled as a surveyor with Robert Sparks, qualifying to practice as a Provincial Land Surveyor in 1869. He married Sparks' daughter Mary, a school teacher, in 1872. He worked locally as a land surveyor, qualified as a Dominion Land Surveyor in 1872 and was first hired by the Dominion government in 1875.
He was responsible for numerous surveys from the 1870s to the 1890s, mainly in the Prairie Provinces. From 1887 to 1889, Ogilvie was involved in George Mercer Dawson's exploration and survey expedition in what later became the Yukon Territory. He surveyed the Chilkoot Pass, the Yukon and Porcupine rivers. Ogilvie established the location of the boundary between the Yukon and Alaska on the 141st meridian.
During the Klondike Gold Rush, he surveyed the townsite of Dawson City and was responsible for settling many disputes between miners. Ogilvie became the Yukon's second Commissioner in 1898 at the height of the gold rush, and resigned because of ill-health in 1901.
He was the author of Early Days on the Yukon (1913), which is still available in facsimile reprints. The Ogilvie Mountains in the northern Yukon Territory are named after him.