Bolton was born in Bath, Somerset, England and was the second son of a wealthy merchant, Robert Bolton, of Savannah, Georgia, who later became an ordained minister and the founder of Christ Church in Pelham, New York. Robert Bolton married Anne Jay in 1810. She was the daughter of the Evangelical minister William Jay. When Bolton was growing up he often went with his grandfather on summer trips through the countryside making pencil drawings of the stained glass windows of the churches they visited. Bolton probably learned his early appreciation of art due to the guidance by his grandfather, William Jay. Jay was a mason's apprentice during the construction of Fonthill Abbey. He often went with his aunt Arabella Jay to King's College Chapel in Cambridge. Here he saw early sixteenth-century stained glass windows of the church. Bolton attended an Evangelical school in Mill Hill near London until he was 16. He then attended Cambridge University. He also briefly joined an engineering firm in Bath where he began painting in earnest. He first painted portraits of his family and friends. Bolton's parents and grandparents knew many famous artists including William Etty. Etty made two chalk drawings of baby Bolton. He also painted other Bolton family portraits. In 1836 the family moved to Westchester County, New York, first to the present Village of Bronxville and then to the town of Pelham. In Pelham Washington Irving, a neighbour and family friend, introduced Bolton to Samuel F. B. Morse. Around 1839 Bolton became a student of Morse at the National Academy of Design. In 1840 Bolton received first prize for his drawing of the Venus de' Medici from the Academy. In 1841 Bolton went to Europe where he saw masterpieces of art, mostly in Italy. Here he did many drawings and sketches of these and bought art for the family home. In 1842 Bolton returned to Pelham where he earnestly began his work in stained glass. Here he had a small shop at the rear of a house near the "Pelham Priory," the Gothic Revival family home in Pelham. The equipment he had to work with was sparse. He had a muffle kiln, which was used for firing his work after he painted details on the coloured glass. Bolton's younger brother John assisted him in making and designing stained glass. Bolton's first attempts at stained glass were windows for "The Priory", the family home. His first major window, a large Adoration of the Magi, was for the family church where his father was rector, Christ Church at Pelham. It was completed by the time of its consecration on 15 September 1843 and represents the first figural stained glass window made in the United States. It is similar in design to a subject in a sixteenth-century window at St. Martin's Church in Liege, Belgium, which Bolton most likely had seen in his travels some two years earlier. This work lead up to his finest work, at the Holy Trinity Church in Brooklyn, New York. Bolton went to England around 1845 and opened a glass staining studio in Cambridge. After the death of his first wife, he ceased his stained glass work and became an ordained minister in an Anglican church around 1853. He was vicar of Stratford East church in London from 1866 to 1881. He was also associated with St. James Church in Bath from 1881 to 1884. He died in Bath in 1884.