Lord Wilson's father was Roderick Peter Garrett Wilson, a former naval officer, who taught languages at Dartmouth Naval College after resigning his commission due to ill-health. In 1942 Peter married Anne Dorothy Anne Chenevix Trench, daughter of an officer in the Royal Engineers. As her husband's health continued to decline, Anne studied at Trinity College of Music, qualifying to teach the piano, to earn an income. The Wilsons lived in Fittleworth, Sussex for many years, purchasing Three Chimneys in the village in 1958. His paternal grandfather was Sir Roderick Roy Wilson, a banker and politician who was Conservative MP for Lichfield. He was knighted in 1929 and Chairman of the British Guiana Parliamentary Commission, 1926. Lord Wilson was educated at Bryanston School, Dorset and Worcester College, Oxford.
Lord Wilson married Margaret Higgins, daughter of Reginald Francis Higgins, in 1974. After reading law at Oxford, Lady Wilson was called to the Bar in July 1966 by Middle Temple; admitted to Inner Temple in 1968 entitled to practice as a Barrister-at-Law. She worked at 3 Hare Court, London from 1967 to 1986 and ceased practice in 1987. For twenty years she was a chairman of the London leasehold valuation tribunal, and latterly a Judge of the successor First-tier Tribunal, until she retired in April 2015 and became a mediator at Tanfield Chambers. Lord and Lady Wilson have two children, Matthew Roderick Benjamin Wilson, born in 1977, and Camilla Jessica Wilson, born in 1981. Matthew attended Eton College where he edited the Eton Chronicle. Lord Wilson has told the story of asking his former client Rolling Stone Bill Wyman to do him favour and give Matthew an interview. "My son Matthew was once editor of the Eton Chronicle, and I said I have a real favour of you Bill, would you be prepared for my son to interview you on behalf of the Eton Chronicle – he said yes, come down next week. So Matthew went down and came back that evening – I asked him how it went and he said fine – I said "let's hear it, come on" – and none of it had taken! And so I rang up Bill Wyman who said don't worry, let him come down again tomorrow and we’ll do it all over again – wasn't that wonderful?". He has been a successful owner of hurdlers and steeplechasers, most trained by John Upson at Maidford in Northamptonshire. Winners in Lord Wilson's colours include Nick The Beak, Young Radical and Gritti Palace. One of the best horses trained by Upson was Nick The Brief, winner of 11 races, including the Hennessy Irish Gold Cup at Leopardstown in 1990 and 1991 and Peter Marsh Handicap Chase at Haydock Park in 1990 On 2 February 1991, Nick The Brief was narrowly beaten by the great Desert Orchid in a tremendous duel up the Sandown Park hill in the Agfa Diamond Handicap Chase. In the 2016/17 season, Lord Wilson's colours have been carried by Issac's Warrior, trained at Maidford by Tracey Leeson, and winner of a point to point at Ballybunion in 2012