The manor is first recorded in the thirteenth century, when it was held by the de Scathebury family. In 1424 it was purchased by Thomas Walsingham a wealthy wine and cloth merchant in London and a Member of parliament. He married Margaret Bamme, daughter of Henry Bamme, of the City of London, a member of the Worshipful Company of Goldsmiths. Walsingham added additional land to the estate in 1433. His son, Thomas Walsingham married Constance Dryland, a daughter of James Dryland, of Davington, by whom he had a son, James Walsingham. Constance survived him and remarried to John Green, who in 1476 was Sheriff of Kent in right of his wife. James Walsingham married Eleanor Writtle, the daughter and heiress of Walter Writtle of Bobbingworth, Essex, by whom, according to a monumental brass formerly in the church at Scadbury, he had four sons and seven daughters. His second son was William Walsingham, of Foots Cray in Kent, who was the father of Sir Francis Walsingham, Principal Secretary to Queen Elizabeth I. James' eldest son and heir was Sir Edmund Walsingham, a soldier, Member of Parliament, and Lieutenant of the Tower of London during the reign of King Henry VIII. His son, Thomas Walsingham, inherited and married Dorothy Guildford, the daughter of Sir John Guildford. Thomas and Dorothy had five sons. The oldest, Guldeford, predeceased his father and the estate had passed to the second son, Edmund, who died in 1589, following which the third son, Sir Thomas Walsingham inherited. He was an MP and was patron of Christopher Marlowe, who was known to have been staying at Scadbury just before his violent death in 1593. Sir Thomas' son and heir, also Sir Thomas Walsingham was Vice-Admiral of Kent. He sold Scadbury in 1660. Many of the Walsingham family's marriages are represented heraldically in stained glass escutcheons dated 1562 now forming the east window of MereworthChurch in Kent. During the 1670s Scadbury was the seat of Richard Betenson. In 1736 the estate of Scadbury was purchased by Col. John Selwyn who passed it on in 1742 to Thomas Townshend, 1st Viscount Sydney, after whom the city of Sydney, Australia was named. The estate was purchased by the London Borough of Bromley in 1983 and opened to the public in 1985.