He was a partner with William Penn in the settlement of East New Jersey in 1681. As one of 24 proprietors of a large parcel of property that took up much of what is now the State of New Jersey, Perth sponsored an expedition in 1684 to establish a settlement there. The City of Perth Amboy, New Jersey, which sits on the waterfront facing Staten Island, New York, and which was once a port city in its own right, is named in his honor — a statue of Lord Perth stands in front of City Hall.
Roman Catholicism
In 1685 the 4th Earl of Perth converted to Roman Catholicism, along with his brother, the 1st Earl of Melfort, which brought him into high favour with King James II. In 1686, Perth and his brother opened a Catholic chapel in Edinburgh, and their public attendance there resulted in a riot. Perth and his brother suggested to James II that the Scottish Parliament would approve a repeal of the Test Act and penal laws against Catholics and Protestant dissenters, but this was later proved untrue. On 29 May 1687, nonetheless, he was made a Knight of the Thistle, being one of the eight original knights of that Order. He is said to have had a novodamus, 17 December 1687, of his dignities, creating him Earl of Perth, Lord Drummond, Stobhall, and Montefex, with remainder, failing heirs-male of his and of his brother's body, to the heirs-male of the second Earl. When James retreated from Salisbury before William, the people, in the absence of the troops, whom Perth had disbanded, rose in Edinburgh. Perth departed on 8 December under a strong escort to his seat of Drummond Castle, Perthshire. Finding himself unsafe there, he fled in disguise over the Ochil mountains to Burntisland, where he gained a vessel about to sail to France. He had, however, been recognised, and a boatful of watermen from Kirkcaldy pursued the vessel, which, as it was almost a dead calm, was overtaken at the mouth of the Forth. Perth was dragged from the hold in woman's clothes, stripped of all he had, and thrown into the common prison of Kirkcaldy. Thence he was taken to Stirling Castle, and not released until 4 August 1693 on condition of a bond to leave the kingdom under penalty of £5000.
St. Germain
He joined the exiled King at St Germain, by whom he was, on 19 August 1696, appointed Governor to the young Prince of Wales, and by whose testamentary directions he was, before 17 October 1701, created by King James III and VIII, Duke of Perth, Marquess of Drummond, Earl of Stobhall, Viscount Cargill, and Baron Concraig, all proclaimed as in the Peerage of Scotland, with remainder to his heirs-male whatsoever. In 1701 he was recognised as a Duke in France by King Louis XIV. On 14 February 1703, he was made a Gentleman of the Bedchamber, and on 21 June 1706 he was appointed a Knight of the Garter. He was afterwards Chamberlain to Queen Mary of Modena, and is said to have been created a Knight of the Golden Fleece by the King of Spain. During James's exile, the Duke of Perth acted as his ambassador to Rome, and after the King's death, Perth erected a monument to him in Paris.