The rather austere neoclassical palace and its stables were built between 1823 and 1828 for Prince William II of Orange in recognition of his brilliant action on the battlefield at Waterloo, from funds granted by the nation. It was the joint work of two architects, Charles Vander Straeten and Tilman-François Suys, at a total cost of 1,215,000 florins. The princely family of William of Orange and his princess, Anna Pavlovna, sister of tsars Alexander I and Nicholas I, occupied the palace a scant two years before the Belgian Revolution of September 1830 forced them to flee to the Netherlands. From 1830 to 1839, the palace was under sequestration by the newborn Belgian state, and a detailed inventory was drawn up. The public was allowed to tour the palace, and its interiors were considered the most sumptuous that had ever been seen in Belgium. An agreement on 5 November 1842 ceded the structure to the Belgian State, while its contents, adjudged the personal goods of William, were shipped to his Palace of Soestdijk in the Netherlands. After housing the 1st Regiment of Chasseurs-Carabiniers in 1848–1852, and having been refused by the Duke of Brabant when offered to him in 1853, the palace remained in use for public festivities. The architect Gustave De Man, a member of the Académie Royale de Belgique, was entrusted with transformations, finished in 1862, which fitted the building to house the Musée Moderne.
By a royal decree of 30 April 1876, the palace was put to the disposal of the two existing Belgian academies, the Académie royale des Sciences, des Lettres et des Beaux-Arts de Belgique, which had been founded in 1772, and the Académie royale de Médecine de Belgique founded in 1841. Three further academies came to share the space: the Académie royale de langue et de littérature françaises de Belgique, founded in 1920; the Koninklijke Academie voor Wetenschappen, Letteren en Schone Kunsten van België founded in 1938, called since 1999 the Koninklijke Vlaamse Academie van België voor Wetenschappen en Kunsten ; and the Koninklijke Academie voor Geneeskunde van België also founded in 1938. Since 2001, the palace has been listed as a protected monument.
