Spencer originally joined the academy as a seven-year-old in 1998 and progressed through the club's youth system. He was first offered professional terms with the club during the 2009–10 season after being loaned out to Northwich Victoria. Spencer was then loaned out to Football League Two side Morecambe. Originally a six-month deal, it was extended to a season-long loan in the January transfer window. He made 32 league appearances, scoring eight goals. He signed a new two-year deal at The Galpharm Stadium in May 2011. On 22 July, Huddersfield Town confirmed that he was set to join League Two side Cheltenham Town on a half-season loan for the 2011–12 season. He scored six goals in 18 matches and an agreement was reached on 28 December 2011, for him to remain on loan for the rest of the season.
Huddersfield Return 2012–13
Spencer made his Huddersfield Town first team début in the League Cup first round 2–0 away defeat at Preston North End on 13 August 2012, playing 75 minutes on his début before being replaced by more experienced striker Alan Lee. He made his league début in the 1–0 defeat by Cardiff City at the Cardiff City Stadium on 17 August, as a late substitute. Spencer signed a one-year contract extension, tying him to the club until 2014 in August 2012. On 16 October 2012, he joined Football League One side Brentford on a one-month loan deal. He returned on 21 November, after a period on the sidelines. On 10 January 2013, he joined League One side Crawley Town on loan for the remainder of the season, but his loan was cancelled on 6 February, after he failed to shake off a knee injury, which meant he made no appearances for the Red Devils. On 22 August 2013, Spencer joined Football League Two side Scunthorpe United on loan until 5 January 2014. The loan was ended on 30 December by the new Iron manager Russ Wilcox. His only goal in his 13 appearances for the club came in the 4–0 win over Rochdale at Spotland on 26 November. On 31 January 2014, Spencer left Huddersfield after having his contract terminated.
Notts County
On 3 February, Spencer joined Notts County on a short-term contract until the end of the 2013–14 season. On 14 May 2014, Spencer signed a new two-year contract with the Magpies.
Cambridge United
On 29 January 2016, Spencer had his contract with Notts County terminated by mutual consent. On 30 January he joined Cambridge United on an initial six-month deal. That afternoon he came on as a substitute against Leyton Orient and scored with his first touch of the ball.
Plymouth Argyle
On 29 June 2016, Spencer signed for Football League Two club Plymouth Argyle
Mansfield Town
Spencer joined Mansfield Town in July 2017. He was released by Mansfield at the end of the 2017–18 season.
