A goalkeeper, Baker began his career as an apprentice at First Division club Fulham in 1965. He played for the Cottagers' youth and reserve teams, but failed to be awarded a professional contract. Baker left the club to return to his native Isle of Wight for a short spell with Ryde County Old Boys.
Bournemouth
Baker returned to the mainland to sign for Third Division club Bournemouth & Boscombe Athletic in July 1967. He had to wait until the 1969–70 season to make his debut and made 17 league appearances in a disastrous season which saw the Cherries relegated to Fourth Division. A return to Third Division football in 1971 saw Baker make just 13 league appearances over the following three seasons, but he finally broke through into the first team during the 1973–74 season, when he made 33 league appearances. He was a regular in goal until the end of the 1977–78 season and missed just five league games in each of his final two seasons at Dean Court. Baker departed Bournemouth in June 1978, having made 248 appearances in 11 years for the club. That summer, he was awarded a testimonial for his service to the club.
Out of favour at Bournemouth, Baker joined struggling Third Division club Brentford on a one-month loan in February 1973, which was later extended until the end of the 1972–73 season. He made six appearances for the Bees before being recalled by Bournemouth.
In a surprise move, Baker signed for First Division club Ipswich Town in a £20,000 deal in the summer of 1978. He failed to make a single first team appearance and instead made 18 appearances for the reserves. An injury forced Baker to retire from football at the end of the 1978–79 season.
Baker joined Ipswich's East Anglian rivals Norwich City on loan during the 1978–79 season, but failed to make an appearance.
Phoenix Fire
Baker moved to the United States to sign for new American Soccer League club Phoenix Fire in early 1980, linking up with former Bournemouth teammate Harry Redknapp. The club folded due to financial irregularities before the beginning of the 1980 season.
Baker returned to England and linked up again with Harry Redknapp at Isthmian League First Division club Oxford City in 1980. He made 38 appearances for the club.
Baker moved back to the south coast to sign for Alliance Premier League club Weymouth in June 1981. He established himself in the first team and made 110 appearances before departing in 1983.
Baker followed Weymouth teammates Trevor Finnegan, Gary Borthwick and Billy Elliott to Alliance Premier League club Yeovil Town in 1983. He made just 21 appearances before retiring in November 1983.
