In 1839 the Anglican Church had founded the See of Newfoundland, its second bishop, Edward Feild, appealed in 1864 for additional clergy to minister in the diocese, thus offering Kelly an opportunity in North America which he quickly seized, leaving the Isle of Man two years before it achieved Home Rule. In June, Kelly found himself appointed incumbent of the cathedral in St. John's, Newfoundland, and archdeacon. 1867 was a momentous year for Kelly. His ministry had proven a success, and Feild named Kelly his assistant bishop. Charles Thomas Longley, the Archbishop of Canterbury, performed the ceremony of consecration on 16 August. Kelly, thirty-five years old, received his Doctor of Divinity degree from Cambridge University, and attended the first Pan-Anglican conference of British, colonial and foreign bishops conference in Lambeth. The conference had been lobbied for by, among others, the Canadian synod, but much of the agenda focused on the controversy over John William Colenso, a missionary to the Zulu. Eleven advisory resolutions regarding colonial district administration were nevertheless passed. The Anglican diocese of Newfoundland also included the island of Bermuda. Kelly and Feild travelled aboard the diocesan ship The Star, and both nearly died in 1871 in a sailing accident that destroyed the ship. This was not the last occasion in which sea travel proved a hardship to Kelly; he became seriously ill in 1874 when travelling to coastal communities on Newfoundland and Labrador. Kelly's logs of his travels on The Star have been published as Journal of a visitation by the Right Reverend J. B. K. Kelly, DD, coadjutor bishop of Newfoundland, in the church ship Star, July–October, 1869,, and The voyage the churchship Star, 1870. Kelly's trips did not distract him from projects at home. On 9 October 1871, he married Louisa Bliss, daughter of William Blowers Bliss, a prominent judge from Nova Scotia. Kelly took on leadership of the completion of the Cathedral of St. John the Baptist. The Cathedral's nave, finished in 1850, had been serving as the entire church. The cathedral was finished in 1885, but burned down in the Great Fire of 1892. Kelly organized its reconstruction. The Cathedral bears a memorial window dedicated to his memory. Kelly became diocesan bishop in 1876, after Feild's death. His understandable aversion to sea travel, however, led him to resign the position in 1877 after a fruitless search for a coadjutor who could relieve him of its necessity.
In 1877 Kelly returned to his country of birth, where he became vicar of Kirkby, Lancashire from 1877 to 1880, Bishop-commissary for the Bishop of Chester and Archdeacon of Macclesfield from 1880 to 1884. He later became Provost of Inverness Cathedral and Assistant Bishop of Moray, Ross and Caithness in 1885, becoming diocesan bishop in 1886 after Robert Eden. In 1901, he was unanimously elected Primus of the Scottish Episcopal Church and served in that capacity for three years until his retirement. He died in Inverness in 1907.
