He was the son of Phineas Sheldon (1717–1807) and Ruth Harmon Sheldon (1733–1805). He graduated from Yale College in 1787. He was a member of the New York State Assembly from 1800 to 1808, in 1812 and in 1826, and was Speaker in 1804, 1805, 1806, 1808 and 1812. He was the last of the Speakers that wore the Cocked Hat as the badge of office.
He was married to Miriam King (b. 1770 Suffield). Their son, Smith Sheldon (1811–1884), established the publishing-house of Sheldon and Company. Their daughter Delia Sheldon was the mother of the Presbyterian missionary, Sheldon Jackson, who established more than a hundred churches, mostly in the Western United States.