He was born in Brudenell River, Prince Edward Island, the son of Henry Gordon, and was educated in Georgetown. Gordon taught school and then opened a general store in Georgetown, also establishing an import-export business. His firm eventually owned at least 15 ships. In 1851, he was named justice of the peace. He also served as chair of the local board of school trustees. In 1854, he married Bridget E. Kearne. In 1863, Gordon was appointed sheriff for Kings County. He served in the provincial cabinet as part of the Protestant coalition formed in 1877 but resigned with other Protestant Conservatives in 1878 which led to the failure of the coalition. In 1894, after the death of his first wife, he married Matilda McGougan. Gordon was leader of the provincial Conservative party from 1894 to 1903. He retired from politics in 1904 and died at his home in Georgetown three years later.