At the general election in March 1820, James was persuaded by the lesser tradesmen of Carlisle to stand for the borough of Carlisle. He polled poorly on the first day, and withdrew. However, one of the two winners, John Curwen, was also elected for the county seat of Cumberland, and chose to sit for the county. The resulting by-election in May was a heated two-way contest between James and Sir Philip Musgrave, Bt. Polling lasted for seven days, with troops called after disturbances. At the end, James won with 468 votes to Musgrave's 382. James's expenses totalled £17,000, including £8,000 on bribes and treating; the defeated Musgrave had spent £23,000. In Parliament, James made many interventions but no major speeches. He focused on political reform, repeatedly calling for universal suffrage, and he supported protesters imprisoned after the 1819 Peterloo massacre in Manchester. He claimed to support an end to slavery, but did little to promote the cause. James voted consistently with the radicals until 1825, when he began to moderate his stances. The following year with falling returns from his Jamaican estates, James decided not to defend his seat at the 1826 general election. He was nonetheless nominated, but did not attend, and finished in a poor third place. James served as High Sheriff of Cumberland in 1826–27. He did not contest the two Carlisle by-elections in the 1820s, nor the 1831 general election, but he was re-elected for Carlisle in 1831 with the support of campaigners for parliamentary reform. He retreated from his earlier support for universal suffrage, and under pressure from Carlisle he backed the more modest electoral reforms proposed in the bill which became the Reform Act 1832. He was returned at the 1832 general election, and in July 1833, during the committee stage of the Abolition of Slavery Bill, he told the Irish nationalist leader Daniel O'Connell that: "in spite of slavery, the slaves in the West Indies were better off than the labourers of this country. If the peasants of Ireland were as well off as the negroes of the West Indies, the hon. and learned member for Dublin might indeed give up his agitation, for it would be useless." At the 1835 general election, James stood down in Carlisle. In 1836, he was elected unopposed in a by-election for East Cumberland. He was returned again in contested elections in 1837 and 1841, but stood down at the 1847 general election when he decided that a contest would be too expensive.
Death
James died at Barrock Lodge on 4 May 1861. Fanny had died ten months earlier, on 6 July 1860.