Tutbury Castle became the headquarters of Henry de Ferrers and was the centre of the wapentake of Appletree, which included Duffield Frith. With his wife Bertha, he endowed Tutbury Priory with two manors in about 1080. It would seem that Tutbury at that time was a dependency of the Norman abbey of St Pierre‑sur‑Dives.
Medieval era
The castle was "nearly destroyed" by Prince Edward in 1264 after the rebellion of Robert de Ferrers, 6th Earl of Derby. In 1269, after a further rebellion the lands were given to Edmund Crouchback and have remained part of the Duchy of Lancaster. By the 1300s the castle was largely repaired, though as a residential castle rather than a military strong-point.
Some time before 1370 the castle was even more substantially rebuilt again by John of Gaunt. From 1372 until her death in 1394, the castle was the permanent residence of Constance, the wife of John of Gaunt. Her court lavishly patronised music and the arts, and was said to have "somewhat similar to a modern Windsor" in its ambition.
In June 1568, Mary, Queen of Scots was at Carlisle Castle after fleeing from Scotland after her defeat at the battle of Langside, and the English Privy Council decided to lodge her either at Nottingham Castle, Fotheringhay Castle or Tutbury, so that she was further from the Scottish border or Yorkshire. In January 1569 the Scottish Queen was taken from Bolton Castle, going first to Ripon and staying a night nearby, then to Wetherby staying a night with John Vavasour at Hazlewood Castle, then on to Pontefract, Rotherham, Chesterfield, and Wingfield Manor. George Talbot, 6th Earl of Shrewsbury was given detailed instructions for keeping Mary, which permitted him to lodge her at Sheffield from time to time, and briefed him on what he should say to her regarding her political predicament. Tapestries, furniture, and bedding were sent from the Royal Wardrobe at the Tower of London to furnish Tutbury, but because of delays caused by poor weather Bess of Hardwick was asked to send her things from Sheffield. This left the Earl's houses at Sheffield unfurnished, so Mary had to go to Tutbury. She arrived on 4 February 1569. She noted the castle was like a hunting lodge, with its enclosure on a slight hill reminiscent of the Bois de Vincennes, and complained of the damp, the wet plaster, and draughty ill-fitting old carpentry. The British Library has a sketch plan of her accommodation at Tutbury. In March the Earl of Shrewsbury described how Mary sat with Bess, Countess of Shrewsbury in the countess' chamber at Tutbury with Lady Livingston and Mary Seton working at embroidery. She moved to Wingfield Manor in April. Mary was returned to Tutbury in September with increased guard and a reduced household. In November 1569, following the Catholic Rising of the North, Mary was rushed south to Coventry where she stayed in an inn. Elizabeth sent orders for her return to Tutbury on Christmas Eve. The "pauvre prisonniere", as she described herself, was back at Tutbury on 2 January 1570. Mary was sent to Chatsworth in May 1570. In January 1585 she once again returned to Tutbury via Wingfield Manor, and on the way stayed a night in Derby in the house of a widowed Mrs Beaumont. Her keepers Ralph Sadler and John Somer were made busy providing satisfactory hangings for Mary's bedchamber. Hangings brought from Lord Paget's nearby houses at Burton and Beaudesert were unsatisfactory. The queen's bedchamber was located at the top of a lodging, which was built against the castle wall. This room was of timber construction, and had no windows looking out through the castle wall, its two windows looked into the castle court. To increase the warmth of her bedchamber Mary improvised a tent of tapestry over her bed. Mary also had a billiard table at Tutbury. Ralph Sadler sometimes took Mary hunting with his hawks on the river Dove, no farther than three miles from the castle, with a guard of 40 or 50 men on horseback. Queen Elizabeth disapproved of this freedom and forbade it. On 19 April 1585 Mary was placed in the care of Sir Amyas Paulet, who took down Mary's cloth of estate from the chamber where he ate, and stopped Mary's servants using the wall walk near the gate. Paulet also curtailed her charitable activities in the town, and disarmed her Scottish servants, many of whom had pistols. In July she was allowed to hunt her greyhound after deer at Stockley Park near Anslow. By August Mary was anxious to move temporarily to another house so her lodgings could be swept and cleaned. Mary pointed out that the Earl of Shrewsbury followed this routine. However, Lord Paget's nearby houses and the house of Henry Cavendish at Doveridge were not suitable to house all her train of servants. Paulet considered modifying her lodging or letting the queen use another part of the castle which he occupied. Security was difficult for this other lodging, where Mary had stayed in 1569. It had two chambers twenty-four feet above ground level, with windows looking over the dykes towards Tutbury town. The windows and a "house of office", a latrine, could have been used for escape. On Christmas Eve 1585, she finally moved to Chartley Castle.
Destruction
By the late sixteenth century the fabric of the castle began to decay, although James I stayed there a number of times between 1619 and 1624. During the English Civil War the castle was held by Royalist forces and re-fortified for the Crown. In 1643 Parliamentary forces attempted unsuccessfully to dislodge the garrison, and in April 1646 the garrison surrendered after a three week siege. Following the siege, a treaty of surrender was agreed upon, and the conditions were drawn up by Sir Andrew Kniveton, the Governor of Tutbury Castle – with the agreement being signed on 20 April 1646. However in the following on 19 July 1647, it was determined "that the castle was untenable", following the siege and damages caused to the castle. Finally in control of the castle, Parliament ordered the fortification to be destroyed; demolition work being carried out from 1647 to 1648. A folly was built on top of the motte in 1780.