Sharma was elected to the position of secretary-general over Michael Frendo, foreign minister of Malta, during the biennial Commonwealth summit in Kampala, Uganda held from 22 to 24 November 2007. He took over from Sir Don McKinnon of New Zealand on 1 April 2008. Sharma was re-elected on 30 October 2011 at the 2011 CHOGM. He was unopposed, having been proposed by India and seconded by Pakistan. His second and final four-year term began on 1 April 2012 and ended 30 March 2016. Sharma has been criticised as a "decent but ineffective" Secretary-General by Hugh Segal, Canada's former special envoy to the Commonwealth, who commented that under Sharma's tenure, the organization has been "missing in action on Sri Lankan human rights, vicious anti-gay laws in some parts of Africa and continued weakness in the promotion of judicial independence and democracy." Geoffrey Robertson QC described Sharma's tenure in the following terms: "...for eight years before 2016 it was led by an Indian diplomat who would not have known a human right if he fell over it. Sharma responded to such criticism stating: “The most important point about the Commonwealth is that it engages with member states to advance the values template. I made five visits to Sri Lanka, but you can’t keep on talking about it in public for the reason that work has to be done below the radar to carry political conviction." During his eight-year tenure as Commonwealth Secretary-General, Sharma focused on the empowerment of young people, the advancement of women's political and economic rights and raising international awareness of challenges facing small states as pressing priorities. Sharma was described by Arif Zaman, Executive Director of the Commonwealth Businesswomen's Network as "someone who has been a passionate, active and dedicated champion for women, combining advocacy with practical steps". On Sharma's tenure, The Rt HonHugo Swire, UK Minister of State, Foreign and Commonwealth Office commented that Sharma "helped to guide the Commonwealth through a period of significant challenges and he can be rightly proud of the important developments that have taken place under his leadership, such as the introduction of the Commonwealth Charter". At the opening ceremony of the Commonwealth Heads of Government Meeting in Malta, 27 November 2015, Joseph Muscat, Prime Minister of Malta stated that Sharma had "done a very good job at providing leadership and introducing new initiatives during his tenure." He gave thanks to Sharma "for his unflinching commitment to the Commonwealth", adding that "his legacy will undoubtedly be a positive one".
On 9 July 2009, Sharma was appointed Chancellor of Queen's University Belfast after the retirement of Senator George J. Mitchell. Queen's says, he was responsible for cementing relations between Northern Ireland and India, which led to the country's investment in Northern Ireland businesses. The position is a largely honorary title and Sharma said he was enormously proud to be given the job at Queen's.
