After originally being offered as a makeweight in a rejected bid to take Sam Parkin to Watford, Smith signed with Swindon Town a free transfer in July 2005, signing a one-year deal. Despite being right-footed, Smith made his debut on the left side of defence on the opening day of the season, in a 2–0 defeat at Barnsley. Doing a steady job without being outstanding, he remained the first choice in this position until retiring with a groin strain during a 0–0 draw at Brentford in October – a result that brought to an end a club record run of eight successive defeats, but an injury that kept Smith out of action for almost a month. When he returned to fitness, he at first struggled to regain his place – starting at right-back for a 2–1 derby victory over Bristol City and appearing in an unfamiliar central midfield role in a terrible 4–1 defeat at Boston United in the FA Cup – before being relegated to the bench for the next three games. A series of suspensions for key defensive players saw Smith drafted back into the side for an important relegation battle at Rotherham United, again on the right side – and when Swindon won 1–0, Smith kept this position until the end of the year, until another injury led to him hobbling off the field in the New Year's Eve clash with Swansea City. Once fit, Smith soon slotted back into his right back position, and then, when manager Iffy Onuora began to experiment with a five-man defence, Smith was utilised as a wing back. Onuora soon offered Smith an extension to his deal, and he signed a new year's contract in March. Smith was an integral part of Swindon's 2006–07 promotion winning squad, making more appearances than any other player and scoring three goals. Smith signed a one-year contract extension with Swindon in June 2007. Smith did not play as regularly in the 2007–08 season and as such delayed the signing of his new contract, though he ended up staying on for the 2008–09 season. However, following the arrival of Danny Wilson as new manager, come the end of the 2008–09 season, Jack Smith was released along with 12 other players. Out of the 17 out of contract at the end of the season, only four were offered new contracts following a season of disarray for Swindon Town.
Smith joined Millwall on a contract to the end of 2009 on 2 August 2009, after impressing manager Kenny Jackett in pre-season training. Due to injuries, Smith made his debut in the season opener at Southampton live on Sky Sports. Since then he has played regularly and in all positions across the back line. Consequently, he was rewarded with an extension to his contract to June 2011. He scored his first goal for the club in a 4–0 hammering of Staines Town in an FA Cup Replay at The Den on 9 December 2009. On 22 January 2011 he scored his first league goal for Millwall in a game against Leicester City. He was released by Millwall on 10 May 2014.
Smith was then signed by League Two side AFC Wimbledon and made 21 appearances for them, scoring 3 goals, during the 2014–15 season. He was released in the summer of 2015.
After remaining without a club, Smith signed for Conference South side Hemel Hempstead Town in September 2015. He scored his first goal, a free-kick, on his home debut in a win against Hayes & Yeading United on 15 September 2015. However, he left after only three months.
Wealdstone
In December 2015, Smith joined Conference South team Wealdstone.
