Castlereagh Street once contained the Australia Hotel, whose foundation stone was laid by Sir Henry Parkes, and the Theatre Royal. Both of these buildings were demolished during the 1970s. The Circular Quay to Central station line was an important part of the Sydney tram network. It was an extremely busy service for passengers transferring from suburban trains, particularly prior to the opening of the City Circleunderground railway line in 1926. Trams operated in a loop from Central station, running north along Pitt Street to Circular Quay returning south via Castlereagh Street. These tracks were also used by some eastern and south-western routes during busy periods. The line closed on 27 September 1957 with four tram services replaced with buses and three others diverted to operate via Elizabeth Street. The line made use of the sandstone viaduct onto the colonnade above Eddy Avenue at Central station, which since 1997 has formed part of the Inner West Light Rail. Having previously operated as a one-way street in a northerly direction, in February 1987 Castlereagh Street was changed to operate in a southerly direction. In 1997, a bus lane was introduced between Hunter and Bathurst Streets. Significant heritage buildings located on Castlreagh Street include, from north to south:
The Trust Building 72-72a Castlereagh Street, located on the corner of King Street, and constructed between 1914-16. The building is an example of the Interwar Commercial Palazzo architectural style and was the site of the former Daily Telegraph offices and is one of four surviving newspaper offices built between 1900 and 1930. The building is listed on the NSW State Heritage Register.
Metropolitan Fire Station 211-217 Castlereagh Street, located south of the junction with Bathurst Street, and constructed from 1887. Partially designed by James Barnet, the building is an example of the Victorian Free Classical architectural style and has been used continuously since 1888 as a fire station by the NSW Fire Brigades. The building is listed on the NSW State Heritage Register.
Downing Centre 302 Castlereagh Street, located with an entire city block bounded by Castlereagh, Liverpool, Elizabeth and Goulburn Streets, and constructed from 1908. Designed by Arthur Anderson, the building is an example of the Interwar Stripped Classical architectural style and was the site of the former Mark Foy'semporium. The building is listed on the NSW State Heritage Register.