The house is a 14th-century Wealden hall house. Although the name reflects the fact that the parish priest and his housekeeper used it, the house was originally built as a farmer's house. It is a very modest property — not at all like the grand rectories that many Church of England clergy occupied by the 19th century. It is a low-ceilinged, two-storey, timber-framed building with a thatched roof. Part of the house was rebuilt in the 17th century. It is commonly said that a detail on a cornice wood carving of an oak leaf, may have inspired the National Trust's emblem, but there is no evidence to prove that claim. It has a rare chalk and sour milk floor. Outside there is a small but well-planted cottage garden, which was designed by Graham Stuart Thomas. The house is open to the public.
The 1895 decision by the National Trust about the approach to adopt to the repair and presentation of the Clergy House was critical in shaping its subsequent way of dealing with almost all its properties, which continues to this day. It was also influential in promoting more widely a much more conservative attitude to the restoration of historic buildings than had been common practice in the 19th century. This was a direct result of the close links between the newly formed National Trust and the Society for the Protection of Ancient Buildings. The SPAB had been founded by William Morris 18 years earlier as a protest against excessive restoration which robbed buildings of their true value and interest. The early Founders of the Trust knew William Morris and were equally horrified by the damage that had been done, mostly to churches, in the name of restoration. The Clergy House was severely derelict in 1890 when the new vicar, the Reverend F. W. Beynon, sought advice on its repair from a London architect, Owen Fleming. In 1891 Beynon contacted the SPAB for advice and after £124 was raised by appeal some work was undertaken. But the project got into difficulties. In July 1894 Beynon asked the SPAB for its view of the nascent, but unknown, National Trust to see if it might be interested in taking on the Clergy House. The SPAB put him in touch with Canon Rawnsley, one of the National Trust's founders. Before the Trust decided to acquire the building another founder of the Trust, Octavia Hill, contacted the Society for advice. In her letter she said that they "will be asked to 'restore' it, so far as that odious word means preserve from decay of course we should wish to do so." This dislike of damaging 19th century restoration practices was confirmed by the National Trust committee in March 1895 when it resolved that "restoration means such work as may be necessary to the preservation of the building with as little new work as possible." This set the pattern for all its later acquisitions. In June the Ecclesiastical Commissioners agreed to sell the building for £10. Octavia Hill then asked the SPAB to recommend an architect "who could be there himself whose heart was in the matter and who could decide point by point on the spot what to do and see it being done, with knowledge of art and craft." This was entirely in line with SPAB's views about the best and most careful way to repair an old building. The SPAB suggested Alfred Powell, who undertook the repairs, though the National Trust's first ever appeal for funds proved disappointing. As a result of this early co-operation the National Trust and SPAB developed a close working relationship which continues to this day. Over the years many of its properties from country houses to cottages have been repaired by architects trained by the SPAB, and working to the conservation principles put into practice at the Clergy House.
