Growing up in the town of Benoni in the Gauteng province of South Africa, his first passion for music came while watching cowboy movies and Fourie wanted to imitate the sound of the musicians. After hearing the George Shearing Quintet in 1949, he focused on jazz music. At the age of 15, he left his parents' house in order to pursue his career in jazz guitar playing. His first gigs were with Boeremusiek bands, and he quickly gained recognition as a great guitarist in the Johannesburg music scene of the 1950s. In 1961, Fourie took a boat to London. In the first two weeks, his money ran out and his wife was becoming desperate. Fourie then got his first gig playing with an Eastern European violinist at the Blue Boar Inn. He described the situation: "What I saw in Soho forced me to leave in November on a boat destined for London with my wife and baby and about two hundred Rand". A short while after this, Fourie auditioned for the Ray Ellington Quartet of The Goon Show fame, and worked in London with jazz greats like Ellington and Tubby Hayes. Following the end of apartheid, he became a professor of jazz at the Technikon University in Pretoria, and told his students about the Ellington audition: "This was the biggest test of my life. It was a make or break period. I could not read the notes and initially I was turned down, but when they heard me play they told me that they would like me to perform. I was very exhilarated, but had no ability to read music. I had two weeks to memorize four years of work. It was the moment of truth and I passed!" After this, Fourie become the resident guitarist for the esteemed jazz club, Ronnie Scott's. Here, he was exposed to Bill Evans, Jim Hall, René Thomas, Freddie Hubbard, Stan Getz, Roland Kirk and Sonny Rollins, among others. After London, he returned to South Africa to go and practice the new style of jazz fusion, which began its growth during the 1970s. After this, he briefly settled in New York, where he worked with John McLaughlin, the famous British jazz/fusion guitarist. He then taught students the guitar in South Africa for the remainder of his career. Fourie formed a band with friends Johnny Boshoff and Hennie Becker and played covers of John Mclaughlin's Mahavishnu Orchestra at The Branch Office club in Johannesburg. McLaughlin said of him: "Johnny Fourie is one of the greatest guitar players of our époque."
