It was one of three mansions built on the large Ravensworth land grant; the other two were Ossian Hall and Oak Hill. William Fitzhugh had also purchased a townhouse in Alexandria at 607 Oronoco Street in 1799, which his family – in 1818 – lent to their cousin, Anne Hill Carter Lee, widow of Henry "Light Horse Harry" Lee, and her eleven-year-old son, Robert Edward. Eleven years later, on 26 July 1829, Anne Hill Carter Lee died at Ravensworth. William Fitzhugh died and was buried there in 1809. Ravensworth then passed to Fitzhugh's son William Henry Fitzhugh, who died in 1830. William Henry Fitzhugh's childless widow, Anna Maria Sarah Goldsborough Fitzhugh, ran the estate until her death in 1874. William Fitzhugh and Ann Bolling Randolph's daughter Mary Lee Fitzhugh married George Washington Parke Custis and became the mistress of Arlington House. Their grandson, Confederate general William Henry Fitzhugh "Rooney" Lee, inherited Ravensworth after the death of his great-aunt and lived there from 1874 until his death in 1891. In 1897 George Washington Custis Lee moved to Ravensworth after resigning as president of Washington and Lee University and lived there until his death in 1913. When Mary Anna Custis Lee fled Arlington House in May 1861 after the outbreak of the Civil War, she stayed at Ravensworth briefly, but then moved further south for fear of inviting damage to the home. Both Union and Confederate forces took advantage of resources and location at Ravensworth; during 1863, in addition to Union forces foraging hay, partisan forces commanded by John S. Mosby once slept in a haystack there and at daybreak discovered they were in full view of a Union encampment. All three of the Fitzhugh estates were protected by orders from both sides throughout the war. The house mysteriously burned on 1 August 1926. In 1957, Dr. George Bolling Lee's widow sold the estate for development. That same year the remains from the Fitzhugh family cemetery, including those of William Fitzhugh and his wife, were removed and reinterred at the cemetery of Pohick Church in Lorton. The grounds later became the Ravensworth Farm subdivision, which today is a census-designated place also called Ravensworth. The locality's population as of the 2010 census was 2,466.
