During the early part of his reign, the office of Yamtuan Muda was held by the powerful Bugis chief, Daeng Kemboja. Mahmud Shah III came of age at a time when Bugis-Dutch trade rivalry was intensifying. He exploited the rivalry by concluding a treaty of protection with VOC on board the Utrecht on 10 November 1784 in which he was accorded the style of 'Most Serene Prince'. The treaty called for the end of the Bugis monopoly over the office of Yamtuan Muda and prohibited other Bugis from holding office within Johor's administration. It also demanded the expulsion of all Bugis not born or bred in Riau. Additionally, the treaty allowed the Dutch to post a Resident in Johor. Shortly thereafter. Mahmud Shah III retreated to Pahang. All-out conflict soon erupted between the Dutch and the Bugis. Hostilities between the two powers continued until 1795, when the Dutch finally succeeded in ousting the Bugis chief, Raja Ali from Riau, allowing Mahmud Shah III to return to his capital.
The same year, however, the Netherlands came under French occupation and the Dutch allowed the British to temporarily take over their territories in the Malay world. This enabled Raja Ali to make a comeback. After the Bugis ousted Tengku Muda, the Malay Yamtuan Muda of Johor, Mahmud Shah III had no choice but to accept Raja Ali's return in 1803. To appease both the Bugis and Tengku Muda, the Sultan married his son, Tengku Hussein, to Tengku Muda's daughter, while his other son, Tengku Abdul Rahman, was made Raja Ali's ward. To maintain his distance from the Bugis, Mahmud Shah III established his capital at Daik, Lingga.
Dissolution of the empire
By the early 19th century, the Johor-Pahang-Riau-Lingga empire had begun to break up. Based in Lingga, Mahmud Shah III exercised little power over his empire. This enabled the emergence of a number of powerful chiefs from the same family, such as Temenggong Abdul Rahman and Tun Abdul Majid. Installed in 1806, the Temenggong had de facto control over Johor mainland, Singapore and the islands in the Riau Archipelago. While in Pahang, Bendahara Tun Abdul Majid became increasingly independent and began carrying the title 'Raja Bendahara' of the dominion, following the weakening control from the capital. This development had resulted in the empire's constituent parts effectively became principalities, and the cultural unity that had hitherto existed between the Malay Peninsula and the islands of Riau-Lingga was gradually destroyed.
Death
Mahmud Shah III died at Fort Tanna, Bukit Chengah, Lingga on 12 January 1811 and was buried at Masjid Jamie', Daik, Lingga without having named a successor and having had issue, two sons by commoners, and two daughters. A succession dispute arose between his sons, later ended when the Bugis seized the throne for his younger son, Tengku Abdul Rahman and crowned him in Riau as the next sultan.