Kavanagh began his career at Home Farm, before joining Middlesbrough in 1991 but struggled to ever hold down a first team place and was sold to Stoke City in 1996, after a short loan spell at the club, for £250,000. In 1996–97 he made 41 appearances scoring four goals including the final goal at the Victoria Ground against West Bromwich Albion. Kavanagh also had the honour of scoring Stoke's first goal at the Britannia Stadium in a League Cup match against Rochdale. He scored 10 goals in 1997–98 as Stoke suffered relegation to the third tier. In 1998–99 he top-scored with 13 goals as failed mount a sustained promotion challenge under Brian Little in 1998–99. In 1999–2000 Kavanagh scored 10 goals as Stoke reached the play-offs where they lost to Gillingham. He also played and scored in the 2000 Football League Trophy final as Stoke beat Bristol City 2–1. He was again a regular in 2000–01 as Stoke again failed in the play-offs this time losing to Walsall. He left Stoke in the summer of 2001 after making 244 appearances scoring 45 goals. He joined Cardiff City in July 2001 for £1 million, and went on to score fifteen times in his first season at the club and help them to promotion the following year when they beat Queens Park Rangers in the Second Division play-off final. He scored one of Cardiff's goals as they memorably knocked out then Premiership Leeds United in the FA Cup third round in 2002. During the 2004–05 season Cardiff chairman Sam Hammam revealed that the club was in serious financial trouble and as such a number of players left the club to attempt to regain losses, including Kavanagh who signed for Wigan Athletic for a fee close to £400,000. Whilst at Wigan he started in the 2006 Football League Cup Final. On 31 August 2006, he signed a three-year deal with Roy Keane's Sunderland for £500,000. During that season he played 14 games, scoring one goal against Leeds, before he was ruled out for the majority of games due to lingering injury problems. He joined Sheffield Wednesday on 21 September 2007 on a loan deal to regain his fitness where he played seven games, scoring one goal against Watford on 2 October. On 29 December, Leicester City made a bid for Kavanagh, together with Márton Fülöp. However, on 31 January 2008 Kavanagh again joined Wednesday on a loan deal until the end of the season. Kavanagh joined League One side Carlisle United on a month's loan on 10 October 2008. This loan was extended for a further month on 7 November. It was again extended on 18 December and would expire on 3 January 2009.
Managerial career
On 9 January 2009, Kavanagh was released by Sunderland and returned to Carlisle on a permanent basis as a player-coach. In April 2013, Kavanagh continued his post as Assistant Manager at the end of the 2012–13 season, after signing a new one-year deal. In September 2013 manager Greg Abbott was sacked and Kavanagh was installed as caretaker manager. Kavanagh was then appointed manager on a permanent basis on 30 September 2013 signing a two-year contract. Carlisle were 22nd on the ladder at the time of Kavanagh's caretaker appointment, and finished the 2013-14 season in the same position, to be relegated to League Two. After a poor start to the 2014–15 season Kavanagh left Carlisle on 1 September 2014.
