Herald was born in Manhattan in 1939, to an Armenian born poet father Leon Serabian Herald. It was through him that Herald was first exposed to live performances by blues and folk legendsLead Belly and Woody Guthrie. While at a summer camp in 1954, Herald was inspired by a performance by Pete Seeger. During his Manumit School days, he became a regular listener of Don Larkin's bluegrass radio show, and began attending open guitar jams with the likes of Bob Dylan and Rory Block. In 1958, Herald formed The Greenbriar Boys, along with Bob Yellin and Paul Prestopino. The following year, Eric Weissberg, replaced Prestopino, and Weissberg was soon replaced by Ralph Rinzler to form their most successful combination. Herald was lead guitarist and vocalist. The trio often played the Greenwich Village scene, but were notable enough to be the first Northern group to win the likes of the Union Grove Fiddler's Convention competition, where Yellin also took top honors for banjo. Shortly after backing Joan Baez on her second LP, The Greenbriar Boys were signed to Vanguard Records, for whom they released three records. In 1969, Linda Ronstadt recorded Herald's "High Muddy Water." Two years previously, she had recreated his vocal of Mike Nesmith's "Different Drum," which became a hit for her band the Stone Poneys. After the trio split up, Herald played sessions for Vanguard. In 1972, he recorded an eponymous solo album for Paramount Records, then went "electric country bluegrass" on a 1978 disc featuring the John Herald Band. Reviewing the solo album in , Robert Christgau wrote: "This casually joyous solo debut by the former Greenbriar Boy gives in at times to such folky vices as mere flash, mere lyricism, and mere whimsy. But 'Fire Song,' a casually joyous ditty about how his house burned down, and 'Brother Sam,' unpresumptuous compassion for a returned Vietnam vet, should inspire Paul Simon to work real hard on the follow-up. And his high notes should inspire Art Garfunkelto go back to architecture school." Herald's last recording was Roll On John in 2000. He was working on new material in 2005 when, on July 19, his body was found in his home in West Hurley, New York. The state police suspected suicide, although no official cause was released.