Lowther first entered Parliament in 1865, as Member of Parliament for York. His maiden speech was against the Reform Bill of 1866, the failure of which brought down the ministry of Lord Russell. He also vehemently opposed the Reform Act 1867, brought forward by Disraeli and Lord Derby, but Disraeli nonetheless appointed him Parliamentary Secretary to the Poor Law Board the same year. He was a strong voice in the opposition to Gladstone, particularly to the Irish Land Bill of 1870. With the return to power of Disraeli in 1874, he was appointed Under-Secretary of State for the Colonies. In 1878, he was further advanced to become Chief Secretary for Ireland and sworn of the Privy Council of the United Kingdom and of Ireland. However, he was beset with difficulties during his tenure in the secretaryship. His opposition to the Land Bill was held against him, and he treated with contempt the agitation of the Land League, which would soon break out in the Land War. He went out of office with Disraeli's government in 1880, and lost his seat at York as well. Lowther cultivated many interests outside politics, and had time to cultivate them while attempting to return to Parliament. He served on several public bodies in Yorkshire and County Durham, he began to breed racehorses in 1873, and regularly ran them at races in the north of England. However, he did not bet on them, and was highly scrupulous in his conduct, becoming a member of the Jockey Club in 1877. While Lowther was defeated at a by-election in Cumberland East in February 1881, he successfully captured North Lincolnshirein September of that year. Upon his return to the House of Commons, he became known for his arch-conservatism and protectionism. Upon the abolition of the North Lincolnshire constituency in 1885, he stood for Louth, but was defeated, and again in 1886 in Eskdale. He re-entered the house in 1888 at a by-election for the Isle of Thanet. Despite his uncompromising views, he enjoyed general popularity in the House of Commons, and had an excellent command of parliamentary procedure. However, by 1903, he had been forced to give over active Parliamentary work, and sold off his racehorses.
Lowther estates
In 1882, Lowther's third cousin once removed, Hugh Lowther, succeeded as Earl of Lonsdale and to the Lowther estates, of which James was senior trustee. Lonsdale's habits were extravagant, and James was to have great difficulty in restraining his spending, which would ultimately ruin the estate. James himself inherited Wilton Castle upon his father's death in 1894, and took great interest in managing the estate.
Personal life
He died at Wilton on 12 September 1904, and left the castle to his nephew Colonel John George Lowther.