Smith was first elected in 1984 as the MP for Kaipara. He represented this electorate until it was abolished in 1996 during the shift to mixed-member proportional representation. Following the election he was the representative for Rodney until the election where he stood as a list-only candidate and was elected as a List MP.
Smith served as Minister of Education from 1990 until 1996 in the Fourth National Government of New Zealand. During this period he implemented a number of changes to the tertiary education sector. One high-profile change involved a radical increase in student fees, as recommended by the Todd Report, which the government had commissioned to address issues of funding. As opposition education spokesman in 1990, Smith promised to remove the Labour Government's tertiary tuition fee of $1250, if elected. Once in office, he kept this promise on a technicality: he shifted the burden of charging fees for courses from the government to the institutions, who then had to charge even higher tuition fees due to decreased government funding. Smith's term as Education Minister also saw the introduction of means-testing for student allowances, with the effect that students of middle-class parents became ineligible for allowances until they reached 25 years of age. In 1996 Smith took up the Agriculture and Trade Negotiation portfolios: Wyatt Creech succeeded him as Education Minister. Smith also became Minister for International Trade and for Tourism, as well as holding responsibilities as Associate Minister of Finance, Associate Minister of Immigration, and Minister Responsible for Contact Energy Ltd. As Trade Minister, Smith spearheaded New Zealand's efforts at the 1999 APEC negotiations. He successfully negotiated New Zealand's free-trade agreement with Singapore, which became the NZ – Singapore Closer Economic Partnership. At the WTO Ministerial in Seattle, he took part in efforts which later lead to the Doha Development Round.
Opposition, 1999–2008
In opposition, Smith held a number of spokesperson roles for the National Party, including those of Foreign Affairs, Commerce, and Immigration. In his role as Immigration spokesman, Smith challenged Mangere MP Taito Phillip Field over alleged impropriety in Field's dealings with constituents. During the 2008 election campaign, on 22 October 2008, Smith made some comments in an interview with The Marlborough Express concerning seasonal Asian and Pacific workers that caused controversy. Regarding Pacific workers he said that some employers "are having to teach them things like how to use a toilet or shower..." And he said that for pruning trees: "some of the Asian workers have been more productive... because their hands are smaller." Maori Party co-leader Tariana Turia described these remarks as "racist", and the Prime Minister Helen Clark characterised them as "absolutely daft". Smith later stated that the media had presented his comments out of context, and that he had repeated the views of employers whom he had talked to; he expressed regret at any unintended offence taken. The parliamentary leader of the National Party, John Key, subsequently referred to this statement as an apology.
Following the National Party's successes in the 2008 election, Members of Parliament unanimously elected Smith as Speaker of the House. Smith took a rather different approach from his predecessor, being more active in requiring ministers to provide answers to oral questions. Smith was re-elected as Speaker of the House again on 20 December 2011. Smith was expected to retire from Parliament and to be appointed High Commissioner of New Zealand to the United Kingdom later in 2012, but stayed on until February 2013. He gave his valedictory speech on 13 February 2013; this was in fact his first speech in Parliament in four years, as Speakers perform an apolitical role. Reflecting on his nearly 30 years in Parliament, he listed voting against the Homosexual Law Reform Bill in 1986 as his biggest regret:
I faced the classic dilemma of voting according to my own judgement or the opinion of those I was elected to represent. As a new member, I opted for the latter and I've always regretted it.
Smith began his term as High Commissioner of New Zealand to the United Kingdom on 25 March 2013, with a powhiri at New Zealand House in London. He stepped down from the role on 25 March 2017. He was replaced by Sir Jerry Mateparae, the former Governor-General.