He was born Edward Rolt, the grandson of Sir Thomas Rolt and Anne Bayntun, daughter of Henry Bayntun, of Spye Park, Calne, Wiltshire. In 1717 he assumed by royal licence the additional surname of Bayntun after inheriting the estates of his great-uncle, John Bayntun. He was succeeded by his only legitimate son, the second Baronet, who sat as Member of Parliament for Weobly, Herefordshire. He had no surviving male issue and the title became extinct on his death in 1816.