In 1763, Prince Edward Island was ceded to the British by the French, and it became a British colony. In 1764, Patterson requested grants to own land on the island, and he and his brother, John Patterson, were awarded Lot 19, near the present-day town of Kensington, through the 1767 land lottery. On 30 May 1769, the British Privy Council declared St. John's Island a colony with its own government, separating it from Nova Scotia. Patterson was appointed the island's first Governor on 19 July 1769 and arrived on the island on 30 August 1770. Soon after taking the Oath of Officein September 1770, Patterson had already formed an Executive Council, and one of his and the council's first ordinances was to enforce the payment of Quit-Rent. Following the first Assembly elections in 1773, Patterson acquired over of land from proprietors who had failed to pay their quit-rent. The land was to be sold off. The former proprietors wrote to the British government asking that their land be returned, and when this failed, they asked that Patterson be removed from office. This was to the first of several incidents which would cause tension between the British government and Governor Patterson. Soon after the petitions were sorted through in England, the Secretary of State from the British government wrote to Patterson, on 24 July 1783, ordering him to present a bill to the Assembly, one which Patterson was told he had to support. The bill was for the annulment of the land sales. Going against the British government, Patterson opposed the bill and managed to receive a majority from the Executive Council, supporting his decision. The British government retaliated by removing Walter Patterson from office, on 17 June 1786. He officially left office on 4 November. In 1789, Patterson returned to England where he died nine years later. The issue of absentee landowners and quit-rent in Prince Edward Island would not be resolved until the passage of the Land Purchase Act in 1875.
Spouse
Walter Patterson was married to Hester Warren in 1770. Unhappy on St. John's Island, she returned to England in 1771. They had two children, John and Charlotte. Walter Patterson's had two additional children on St. John's Island with companion Margaret Hyde. The Hyde family arrived in Canada from England in 1770, on the same boat as Governor Patterson.