Rochussen was born in the town of Etten, North Brabant, on 23 October 1797 to Jan Rochussen, a member of the provincial government, and his wife. In 1815 Rochussen served in a volunteer corps against Napoleon's armies. Rochussen found employment as a tax collector in Schiedam, South Holland beginning in 1814. Over the next twelve years he served as tax collector in 's-Hertogenbosch, Rotterdam, and later Amsterdam. He was then selected as secretary at the Chamber of Commerce and Industry, serving from 22 January until 10 August 1826. Leaving the chamber, he worked at an entrepôt – a trading post for the import and export of goods without paying duties – and eventually managed it from 18 July 1828 until 31 July 1840. It was during his time at the entrepôt that Rochussen found the favour of King William I of the Netherlands|; Rochussen later served as the king's adviser and became close to him. Rochussen was selected to become Minister of Finance by in the king's last year of his reign, taking office on 31 July 1840. Rochussen worked to straighten the country's public finances, which were incredibly disordered. However, after a proposal to convert the nation's debt failed 30 to 24 in the Second Chamber, Rochussen was forced to resign. He was later selected as a special envoy to Brussels, a post which he held for nearly two years and which resulted in him settling inheritance and property issues to the satisfaction of both sides. In February 1845, Rochussen was reassigned as Governor-General of the Dutch East Indies with Royal Decree of 5 February 1845. He arrived in the colonial capital of Batavia on 30 September of that year. Before his departure, former governor general Jean Chrétien Baud recommended that Rochussen take an interest in the Javanese people. Although Rochussen initially intended to build greater infrastructure in the Indies, he later felt that a full autocracy was necessary and as such instituted severe limitations to press freedom. He also instituted reforms in the nation's monetary system. On 12 May 1851 Rochussen was re-deployed to the Netherlands. On 20 September, he began a term as a member of the House of Representatives representing Alkmaar. He served in this capacity until 28 January 1857. A year later, he became the Chairman of the Council of Ministers after fifteen days forming the cabinet in collaboration with Jan Karel van Goltstein. He served from 18 March 1858 until 1 January 1861, on which date he resigned due to his proposed budget being failed at the house meeting on 14 December 1860. He then helped Schelto van Heemstra form the new cabinet. After his term as Chairman of the Council of Ministers, Rochussen returned to the House as a representative of Amsterdam, serving three consecutive terms from 14 November 1864 until 20 September 1869. Rochussen died in The Hague on 21 January 1871.
After several years of living together, Rochussen married Anna Sara Velsberg in Amsterdam on 14 December 1831; they had nine children together before Velsberg's death on 18 June 1841. His second marriage was to a Creole woman, Elisabeth Charlotta Vincent, on 2 September 1848 while he was in Batavia; the couple produced two daughters, with Vincent dying in childbirth for the second. Rochussen's son Willem Frederik Rochussen went on to be Minister of Foreign Affairs from 15 September 1881 until 23 April 1883. He also had several sons-in-law and nephews who entered politics, such as Jacob van Zuylen van Nijevelt, Frederik s'Jacob, and representative GA de Raadt. His grandson, Herman Adriaan van Karnebeek, became mayor of The Hague, Minister of Foreign Affairs, and later President of the League of Nations from 1921 to 1922.