Erected, or more probably rebuilt, by John Sempill, 1st Lord Sempill, he changed its name from Castletoun to Castle-Semple. In Willem Blaeu's Atlas Maior, published in 1654, the castle is represented by a mark denoting the largest size of castles. In George Crawford's History of Renfrewshire, he noted, "Upon the brink of the loch stands the castle of Sempill, the principal messuage of a fair lordship of the same denomination, which consists of a large court, part of which seems to be a very ancient building, adorned with pleasant orchards and gardens." According to Gardner, Castleton was built in 1492-3, but Millar states that it was built closer to 1550, as a successor to Elliston Castle. The estate was sold by Hugh Sempill, 12th Lord Sempill, in 1727, to Colonel William M'Dowall, a younger son of M'Dowall of Garthland. In 1735, M'Dowall demolished the house and erected a new one on its site. The new Castle Semple was an elaborate gothick mansion which became renowned for its extensive gardens which formed the centrepiece of the estate. In 1818, M'Dowall sold Castle-Semple to John Harvey, Esquire, of Jamaica. Some workmen repairing drains in 1830 found part of the foundations of the castle still existing below ground. It burned down in 1924 and the ruins were finally demolished in 1960. The billiard room and stables wing have been re-roofed and are used as farm buildings. Impressive gate-arches and a small temple folly on a knoll to the northern end of the loch are all that remains of the estate.
Estate grounds
A copper cannon, having the arms of Scotland, and J.R.S. engraved on it, was found in the loch near the peel. This relique was preserved at Castle Semple. At one time, Glasgow and Ayr Railway passed through the Castle Semple estate and in the immediate neighbourhood of the loch. Castle Semple had its own railway station which never opened to the general public. A simple platform existed here, used mostly by the workers on the estate. Had the station opened to the public, it would likely have been called Howwood or St Bryde's. The trackbed is now part of National Cycle Route 7. Lord Semple founded a collegiate church near the lake; a stone in the outer wall of Castle Semple Collegiate Church bears the letters R. L. S., and the arms of Sempill and Montgomery. Eastward of the lake, and on the south side, are the remains of the old tower of Elliston Castle, the residence of the Semple family prior to 1550. The 'Temple' on Kenmuir Hill by Castle Semple is a Category B listed building.