Smith joined the Department of External Affairs in 1969 and served in New Delhi, Tel Aviv and Manila. Smith held a number of senior positions in DFAT before being appointed as an assistant secretary in 1985, and consul-general in Honolulu in 1987. In 1992 he was appointed Deputy Secretary, and was also acting Secretary of DFAT for much of 1992–1993.
Secondment to the Department of Defence
In 1994–1995, Smith was seconded to the Department of Defence as Deputy Secretary of Strategy and Intelligence.
Ambassadorial positions
Smith served as the Australian Ambassador to the People's Republic of China. In the 1998 Australia Day Honours, he was appointed an Officer of the Order of Australia "for service to international relations and to the development of policy approaches to sanctions, human rights, defence and trade issues". He was awarded the Centenary Medal on 1 January 2001 "for a significant contribution over many years to international relations". He served as the Australian Ambassador to the Republic of Indonesia, and extended his period of office when he learnt of the bombings in Bali on 12 October 2002. "For outstanding public service as Australian Ambassador to Indonesia in managing and leading Australia's response in Indonesia following the bombings which occurred in Bali on 12 October 2002", he was awarded the Public Service Medal on 17 October 2003.
Secretary of the Department of Defence
Returning to Australia in November 2002, Smith served as the Secretary of the Department of Defence until his retirement from the public service in December 2006; his initial 3-year term was extended for a further two years when he expressed his desire to retire "in two years' time". In 2004 he was awarded an Honorary Doctorate of Letters by the University of Western Australia.
Retirement
When Smith retired from the Australian Public Service on 1 December 2006, he was given the honour of delivering the first valedictory address on 29 November 2006. In 2007, Smith spent four months as a Distinguished Fellow at the S. Rajaratnam School of International Studies in Singapore. In 2008, Smith led a review of "Homeland and Border Security" in Australia for the Rudd Government. He also became a Visiting Fellow at the Lowy Institute for International Policy in Sydney, publishing his first report for them in October 2008. From March to June 2009, Smith was the Australian Scholar at the Woodrow Wilson Center for International Scholars in Washington DC. In April 2009, Smith was appointed as Australia's Special Envoy for Afghanistan and Pakistan. Smith is also a board member of the Energy and Minerals Institute, and was one of the Conference Chairs at The Reluctant Super Power Conference.
