Born October 6, 1743 (Old Style) / October 17, 1743 (New Style) in Southampton, he attended the public schools and studied medicine at Elizabethtown, New Jersey (later Elizabeth.) He returned to Southampton and practiced medicine from 1764 to 1776; he then resided three years in Killingworth, Connecticut during the American Revolutionary War, when he again returned to Southampton. He was undersheriff of Suffolk County from 1784 to 1787, and was sheriff from 1787 to 1792. He moved to Herkimer County in 1793 and settled in what is now the town of Lodi and continued the practice of medicine. He also erected and operated a grist mill.
Halsey was supervisor of the Town of Ovid from 1794 to 1804, and was a member of the New York State Assembly from Onondaga County in 1797 and 1798 and from Cayuga County in 1800, 1801, 1803, and 1804. He was a member of the State constitutional convention in 1801, and was clerk of Seneca County from 1804 to 1813 and in 1815. Halsey was elected as a Democratic-Republican to the Ninth Congress, holding office from March 4, 1805 to March 4, 1807. He was a member of the New York State Senate in 1808 and 1809, and engaged in farming. He died at Lodi in 1832; interment was in Old Halsey Cemetery, South Lodi.