Kemp made his International debut in an ODI against Sri Lanka on 14 January 2001. He made his Test debut a week later also against Sri Lanka. He was taken on the tour of the West Indies in the 2000/2001 season but he did not score many runs and was involved in a controversy when he admitted smoking marijuana with 6 South African players. He was selected for eight more One day games but again he disappointed and was dropped. He was out of the South African side for nearly three years until he was selected to play in the home series against England. In this series Kemp impressed with his big hitting with him smashing 80 runs from 50 balls in an innings that included 7 sixes. Following his performances he was picked for the series against Zimbabwe and in the match at Durban he hit 53 from 21 balls in an innings that included 5 sixes. He was then taken on his second tour of the West Indies where he made an impressive 65 at Trinidad. He was then selected to represent Africa in the Afro-Asian Cup although he batted lower down the order and struggled. In the South African season of 2005/2006 he showed that he is a match winner when he blasted 73 from 64 at Bloemfontein. Kemp then went on his first tour of India where he showed he was able to bat on slower pitches. He went on the South African tour of Australia where he made a rare appearance in a Test Match when he and Jacques Rudolph formed a partnership that rescued a draw for South Africa at Perth. He was not picked for the second Test and has not been picked in a Test Match since. He struggled in the VB Series that involved Sri Lanka, Australia and South Africa with him averaging just 17.00. He quickly returned to form when Australia toured South Africa as he made 51 from 41 balls at Newlands Cape Town. He was selected in South Africa's squad for the 2006 ICC Champions Trophy in India where he showed he is more than just a big hitter when he made 64 from 110 balls in a partnership with Mark Boucher where he along with Mark Boucher set the record for the highest 6th wicket stand inICC Champions Trophy26 November 2006, in the third ODI of the series between India and South Africa, his side were at 137 for 7 when Kemp came in and smashed his way to his maiden ODI century with the help of Andrew Hall. Together they put on a world record 8th wicket stand of 138 and took their score to 274, with Kemp named Man of the Match. He also showed that he is a useful medium pace bowler when he took three wickets at St George's Park Port Elizabeth. He was selected in the 30 man provisional squad for the 2007 Cricket World Cup.
In 2008, Kemp joined the rebel Indian Cricket League, which was not recognised by any cricket board or the ICC, thereby ending his international career with South Africa. In the ICL, Kemp played for the Hyderabad Heroes team.
County Cricket
In 2003, Kemp played in England for Worcestershire, with a best bowling return of 5/48 in the County Championship against Glamorgan in June, and a top score of 90 against Hampshire in the same competition in September. On 20 March 2008, the ECB banned Kemp and four other Kolpak players because of their involvement in the ICL. The decision was, however, overruled and Kemp was cleared to play for Kent, with whom he has a two-year deal. However, changes to the Kolpak regulations meant Kemp could not return to the county for 2010.