Potter came through the youth team ranks at Peterborough United, after being released by Wimbledon at the age of 14, then by Millwall aged 17. Before joining Peterborough he spent a few months at the Barking Abbey Centre for Football Excellence, where he made several first-team appearances for Waltham Forest in the Isthmian League. He signed his first professional contract with Peterborough on 1 June 2007, signing a one-year deal. He joined Kvinesdal of the Norwegian 4th tier on loanin the summer of 2007, but failed to make any appearances. In September 2007 he had a month's loan spell at Grays Athletic. He made just one appearance, coming on as a substitute against Exeter City. In November 2007 he made a loan move to Conference South side Havant & Waterlooville, for the rest of the season. He played in Havant's 5–2 fourth-round FA Cup defeat to Premier League side Liverpool on 26 January 2008. Potter played a big part in Havant's second goal to put Havant 2–1 ahead when his shot took a deflection off Martin Škrtel. For this feat, Potter was named "Player of the Round". In December 2007, he was offered at new two-and-a-half-year deal by Peterborough, and, on 29 January 2008, he was recalled from his loan spell at Havant & Waterlooville. On 29 February 2008, Potter was loaned to Isthmian League Premier Division side AFC Wimbledon, for a month. He made his first appearances for the Posh in the League upon his return. In July 2008, Potter was close to signing on loan for Conference National newcomers Kettering Town. However, Peterborough initially stalled the deal because of injury worries, but he finally completed his loan move to Kettering on 1 August 2008. In May 2009, it was revealed that Lincoln City had enquired about Potter and his teammate Liam Hatch.
On 3 June 2009, Potter signed for Oxford United on a season-long loan. He scored five goals in 20 Conference National games as Oxford finished third and went on to win the playoff final 3–1 against York City, his last-minute goal at Wembley Stadium securing Oxford's return to the Football League after a four-year exile. At the end of the season he joined Oxford permanently for an undisclosed fee. His debut league goal, a "fine strike", came in a 2–2 draw against Wycombe on 9 April 2011, one of two goals he scored in the 2010–11 season. He scored twice more in the first half of the following season, before breaking his ankle in February 2012 and missing the rest of the season. He returned for the start of the 2012–13 season, and on 5 September 2012 came off the bench to score an 88th-minute winner against local rivals Swindon Town in a first-round Football League Trophy match. He finished the season with a career-best haul of 11 goals. On 16 January 2015, Potter left the club by mutual consent, having spent five-and-a-half years with the U's. Later that day he returned to AFC Wimbledon.
On 20 May 2015, it was announced that Potter would be reunited with former manager Chris Wilder by joining Northampton Town on a two-year contract. He scored his first and what turned out to be only goal for Northampton in a 2–1 win over Stevenage on 24 October 2015.
Potter signed for Mansfield Town in January 2017. He scored his first goal for the club in a 1–1 draw with Luton Town on 17 April 2017. He was released by Mansfield at the end of the 2017–18 season.
