An American Roman Catholic churchman, Carroll continually ministered to Catholics and labored to counter anti-Catholic discrimination in Maryland, where Catholic churches and schools were forbidden. He led in petitioning Rome for the appointment of a priest in America with some episcopal powers, freeing Americans from the Catholic leadership of England. In 1784, Father Carroll was made superior of the missions in the United States. In the same year he published a controversial pamphlet, "An Address to the Roman Catholics of the United States of America", to combat a paper impugning the loyalty of Catholics. In 1790 he was consecrated bishop of Baltimore.

John Carroll started three seminaries and Georgetown University, and was committed to the American Revolution and ideals. A good friend of Benjamin Franklin, Carroll stressed Catholic commitment to democratic ideas. He also was honored by the American Congress for his missionary work among the American Indians.