Bayne joined Inverness Caley Thistle from local rivals Ross County, having previously played for Dundee, Arbroath and his local side Raith Rovers having been born in Kirkcaldy. Bayne is a striker, and wore the much-coveted number 9 shirt for the Jags In May 2008, Bayne was linked with a possible move away from the Caledonian Stadium after manager Craig Brewster revealed he was interested in Andrew Barrowman. First Division sides St Johnstone, Dunfermline Athletic and Ross County are all thought to be after the striker. After Caley Thistle signed Barrowman, Craig Brewster confirmed that a move to Dunfermline was very likely. The same day, it was reported that a fee had been agreed by Bayne's club and the East End Park side. On 27 June it was confirmed that Bayne had signed for Dunfermline for £30,000. Although proving a success at the Pars his time with the club was cut short due to an ongoing problem with Plantar fasciitis. With an unknown recovery period it was agreed that he would leave the club when his contract expired at the end of July 2010. In February 2011 he played several games as a trialist for both Dundee and Dunfermline Athletic, before returning to Dunfermline for the rest of the 2010/11 season by signing as an amateur. On 30 May, Graham signed a one-year deal with First Division club, Dundee. On 30 May 2011 it was announced that Bayne had signed for Dundee FC on a one-year deal. Bayne made 20 appearances for the side but was released by Dundee at the end of the 2011–12 season. Having left Dundee he gave up on fully professional football to become a Firefighter. On 4 January 2013, Bayne signed a short-term contract with Highland League side, Wick Academy. Bayne came on as a substitute against Turriff United in the SHFL on 5 Jan 2013 and to mark his debut he scored within 5 minutes of his introduction. Bayne marked his 300th competitive appearance with a late goal in the 4–0 win at struggling Lossiemouth on 12 January 2013 in his second and final appearance for Wick. Bayne returned to the SFL after signing for Arbroath on 14 January 2013 A double on his second début for Arbroath saw his new side defeat Angus rivals Forfar Athletic and took his team into the play-off position in the Second Division. On 31 January 2014, Bayne was released by Arbroath. Soon after his release from Arbroath, Bayne was appointed assistant manager to friend and former teammate Barry Wilson at Elgin City. Bayne has also registered as a player. Incredibly, Elgin City proved to be Bayne's most fruitful club with an astonishing 47 goals in 21 games.
