Bailey was born in Rowley, Massachusetts, and was educated at Harvard College, ranked at the bottom of the class of 1755, which notably also included John Adams. He started his career in the ministry as a Congregational preacher in New Hampshire but converted and became an Anglican clergyman in 1760, through his connection with Dr. Silvester Gardiner, a prominent Massachusetts physician, businessman, and landowner. Gardiner, also Anglican, sought a minister to serve the rural parish of Frankfort or Pownalborough, which was part of the lands his business partnership, the Kennebec Proprietors, oversaw. Bailey's congregation was a mix of mainly German Lutheran and French Huguenot immigrants, who were often at odds on matters of religion with the area's Yankee Congregationalists. Despite continuing friction with the local Congregationalists, Bailey was able to builda church and parsonage in 1770-71. Hostilities towards Bailey continued, especially when he continued to profess Loyalty to the British Crown, and he moved with his family to Nova Scotia in 1779, after American Patriots twice made attempts on his life. He served in the parish of Cornwallis for a period and then moved to Annapolis Royal where he remained for the rest of his life. In 1780, Rev. Bailey was appointed the Deputy Chaplain to the 84th Regiment.
Poet
It is through his writings that Bailey's place in Canadian history was assured. His poetry was widely known and his verse satire was considered to be styled like that of the English poet, Samuel Butler. He wrote a considerable amount of prose as well and much of this can contribute to historians' studies of those times. "Behold the vaunting hero," Royal Gazette and the Nova-Scotia Advertiser, 11 Dec. 1798, and "Observations and conjectures on the antiquities of America," Mass. Hist. Soc., Coll., 1st ser., 4 : 100–5. Three of Bailey's poems are printed and discussed in Narrative verse satire in Maritime Canada, 1779–1814, ed. T. B. Vincent. Bailey is buried in the Garrison Cemetery.
