Mordaunt was an active campaigner against the Warwickshire Agricultural Labourers Union formed by Joseph Arch. He was quick to throw any labourers who joined the union out of their cottages. On 30 March 1872, the Leamington and Warwickshire Chronicle reported that "Sir Charles Mordaunt....has served notices upon all his union tenants to give up possession of their cottages". Mordaunt was also behind a meeting where 100 farmers agreed to discharge any labourers that so much as joined the union, reported on 13 April 1872 in the Leamington and Warwickshire Chronicle.
Mordaunt was married on 7 December 1866 to Harriet Moncreiffe, daughter of a Scottish baronet. Mordaunt was a stolid country squire with no interests beyond hunting and shooting, and his wife was a giddy young beauty. She had already caught the eye of the Prince of Wales. Lady Mordaunt had licence, or thought she did, to carry on affairs with other men. While Mordaunt hunted, fished, or sat in Parliament, she entertained numerous lovers, including the Prince and several of his aristocratic friends. In 1869, Lady Mordaunt gave birth to an illegitimate daughter, and then confessed everything to her husband, who was enraged. His offended servants had compiled diaries of the affairs. He sued for divorce. The historical record shows that the Prince of Wales was never named as a co-respondent in the divorce case, but Mordaunt threatened to do so. Lady Mordaunt's father, who had several other daughters to marry off, announced that she was mad. That would prevent a divorce trial and save the family reputation. She was incarcerated in various rented houses, and after some weeks either broke down or agreed to feign madness: smashing plates, eating coal, howling and crawling. The case was brought to court and the Prince of Wales was called as a witness; he admitted visiting Lady Mordaunt but nothing further was proved. In 1875, Mordaunt sued again. Viscount Cole, father of Lady Mordaunt's child, pled guilty to adultery with her, so Mordaunt got his divorce. She was kept in asylums for the rest of her life. Her daughter Violet was married to the 5th Marquess of Bath.
Second marriage
In his early forties Mordaunt married again, on 24 April 1878, to Mary Louisa Cholmondeley, a parson's daughter.
