In 1988, he became COSATU's regional secretary for the Western Transvaal. In 1992 he took up the position of national organizing secretary. He served as COSATU's deputy general secretary from 1993 to 1999. Vavi took centre stage in the four-month negotiation that preceded the 1997 Jobs Summit. He served on the International Labour Organization's commission on globalisation. In 1999 he succeeded Mbhazima Shilowa as general secretary of COSATU. He addressed the 27 June 2002 XIV International AIDS Conference, where he pushed mass education and prevention campaigns. Vavi has spoken out against the expulsion of NUMSA, the metalworkers union, which used to be the largest union in the COSATU federation. This is one of the major factors contributing to Vavi's expulsion from COSATU. On 30 March 2015 the Central Executive Committee voted to expel Vavi. The vote was 31–1 in favour of expelling Vavi.
As general secretary he has expressed disappointment with political progress in Zimbabwe, blaming mismanagement by government, an ill-conceived structural adjustment programme, and factors beyond its control like sanctions against neighbouring South Africa during apartheid years. He led two fact-finding missions, in October 2004 and February 2005, to assess conditions for a fair poll during the planned Zimbabwe parliamentary elections of 2005. The members of the mission were deported during the first mission, and were turned back at Harare airport at the start of the second. Talking of the alliance which governs South Africa between the ANC, COSATU and the SACP after a failed trip to see the Zimbabwe Congress of Trade Unions in 2004, he is critical of the ANC's approach to Zimbabwe, privatisation, and corruption in government.
ANC
He took particular issue with the Public Investment Corporation's Telkom share purchase, involving former communications director-general Andile Ngcaba and the former head of the presidency Smuts Ngonyama. COSATU accuses both men of taking advantage of their government posts to access privileged information. In May 2010 the ANC threatened to charge Vavi with "ill-discipline". Vavi has argued that under Jacob Zuma's government "We are heading rapidly in the direction of a full-blown predator state in which a powerful corrupt and demagogic elite of political hyenas increasingly controls the state as a vehicle of accumulation and warned of a possible Tunisia style revolution.
Controversies
Vavi was suspended from the Cosatu secretary-general position after a rape scandal.