Structures have been on the site as far back as the late 10th century, when Theobald I, Count of Blois, built a dungeon into the mound. Around 1140, Ginon de Mehun built a fortified castle on the site. In 1212, Robert de Courtenay of the Capetian House of Courtenay built a new castle with three corner towers and a gatehouse. In the second half of the 16th century, the north and south pavilions and a south gatehouse were built, most likely for the de Trémouille family, lords of Selles. The pavilions, main entrance and interior of the 13th-century gatehouse were redesigned in 1612 for Philippe de Béthune, count of Selles, by architect Jacques Androuet II du Cerceau. In 1719, Anne-Marie-Louise de Béthune, sister of the queen of Poland, sold the castle to Cardin Lebret, count of Selles. In 1810, Armand-Louis-Jean de Jehannot, Marquis de Bartillat, owned the castle and began significant renovations. However, in 1813 the Bande noire destroyed the west gallery. Starting in 1880, the castle underwent significant restoration work under the du Moulinet d'Hardemare family. Architect Pierre Chauvallon led a major restoration of the château in 1913.
Château today
The castle had been abandoned by 2002. It was purchased by Michel Guyot and his wife, Noémi Brunet. Guyot has done restoration work on previous castles, including Château de Saint-Fargeau, and is behind the Guédelon Castle project. In 2012, it was opened to the public. Later that year, they sold it to Nicolas Mazzesi and his wife Katherine Wu, who plan to open a winery on the site. The château hosts festivals and other events, and is the first tourist site in France to offer a virtual reality tour. In early 2016, Georg Kabierske, a German history of art student from the University of Heidelberg, who had viewed the châteaux of the Loire Valley via Google Earth, discovered a previously unknown map of Château de Selles-sur-Cher drawn by Swedish architect Carl Johan Cronstedt. The map of the castle and its elaborate French garden had been digitized by the National Museum in Stockholm and was classified as being of unknown origin, until the German student recognized it as Selles-sur-Cher. In April 2016, a 300-year-old Lebanon Cedar on the grounds of the castle was named a "Remarkable Tree of France". The tree stands high with a circumference of.