Carroll was elected to the Massachusetts Senate in 1932. During his tenure in the Senate, Carroll fought to have police strikers reinstated and March 17 made a holiday in Suffolk County, Massachusetts.[5][6] From 1935 to 1936 he was the Democratic floor leader in the Senate. In 1935 he was indicted on charges on forging and passing out Civil Works Administration work cards.[7]
In 1938, Carroll was appointed superintendent of buildings at the Quabbin Reservoir.[8] In 1940, he was an unsuccessful candidate for Suffolk County Sheriff.[9] In 1942 he attempted to reclaim his old Senate seat, but lost to Leo J. Sullivan.[10] During the 1945 Boston mayoral election, Carroll served as a radio spokesman for John E. Kerrigan.[11] From 1948 to 1950 he was the city assessor of Boston.[3] In 1950 he ran in the special election for the Ward 7 seat on the Boston City Council.[12] Carroll was a candidate for Lieutenant Governor of Massachusetts in 1952. He finished fourth in the five-candidate Democratic primary with 10% of the vote.[13] His final position in government was with the Massachusetts Division of Motor Boats.[3]