The earliest certain documentary reference to Lavant House dates from 1762–3. Architectural analysis of the building, its photographic depictions and an eighteenth-century watercolour suggest that it was built between 1707 and 1725. Richard Budgen's map of Sussex shows two houses considered worthy of note in the manor of West Lavant, of which one is undoubtedly West Lavant Farm, while the other is believed to be Lavant House.
Its first definite owner was Sir John Miller, 4th baronet, whose family had been prominent in Chichester first as justices of the peace and mayors then becoming members of parliament from the later 17th century. Sir John had enclosed the park at West Lavant by 1740, after which he may have lived in the first incarnation of Lavant House. The Duke of Newcastle's brother, Henry Pelham, wrote to the Duke saying, "you know my thoughts on Sir John, he is as friendly and honest as the day is long". This has to be balanced with the fact that the 2nd Duke of Richmond could relate to the Duke of Newcastle that Sir John had stated that he did not care a fart for his father-in-law, Dr. Combs. Sir John is believed to be depicted as a lesser figure in a 1759 painting by George Stubbs showing the 3rd Duke of Richmond out hunting. He died in 1772 and his relict Dame Susanna remained at Lavant House until her death in 1788 when the house was put onthe market by her eldest son, Sir Thomas Miller, who had by then moved to live in Hampshire.
In 1791 the house was bought by the 3rd Duke of Richmond, who let it to Lord Bathurst. In 1798 the Duke started work on the house. Apparently this was when the house was reoriented to face north. He appears to have died while the project was underway. The next inhabitant was Henriette Ann Le Clerc, thought to have been the Duke’s illegitimate daughter. It was in Henriette's bedroom in Richmond House in London where in 1791 the fire started which destroyed that building. Henriette was brought from France by his sister, Lady Louisa Conolly, in 1778 at age 5 to live at her Goodwood estate. His will named her the first beneficiary, receiving life tenure of West Lavant House and Park and other lands and farms. She married Colonel John Dorrien on 28 March 1808. John and Henriette’s only child, Charles, was born in Lavant in January 1809. Following John’s death in 1825, Henriette ran her estate, which she mentions in a series of letters to the 5th Duke. She remained much of the time at Lavant. Forty years after she had moved into Lavant House, the Agricultural Gazette announced her death on 6 January 1846. The house then returned to the Goodwood Estate and was rented to a series of tenants.
Twentieth century
In 1907 the house was sold to Mr Morrison and family. He was probably responsible for the dining room decoration. The house was then sold to Major Henry Frederic Low and family. In 1921 his widow sold it to Major Julian Day and Isabella. She then sold the building to the independent Lavant House School, a member of the Girls' Schools Association. The school closed in the summer of 2016.
