Born in Augusta, Maine, Potter attended common schools and Phillips Exeter Academy. He married Frances Elizabeth Lewis Fox Potter of Portland, Maine. Her father George Fox and unmarried sisters would move to Wisconsin and farm alongside the Potters. Their son John Kendall Potter barely survived his mother. However, their children Rebecca, Alfred and Frances did survive and have children.
Wisconsin voters elected Potter to the United States House of Representatives in 1856 and he won re-election twice. Thus, Potter served in the 35th through the 37th Congresses. Potter received his nickname in 1860, as a result of an aborted duel with Virginia Congressman Roger A. Pryor after Illinois Congressman Owen Lovejoy's remarks concerning the 1837 murder of his brother Elijah Lovejoy. Pryor had edited Potter's follow-up remarks to eliminate a mention of the Republican Party, to which Potter had objected, then Pryor challenged Potter to a duel, but his seconds objected when Potter chose bowie knives as the prospective weapon, decrying his selection of weapon as "vulgar, barbarous, and inhuman." The incident received considerable press, and Potter's friends afterward often accompanied him when on Washington's streets, lest he be accosted again to test his mettle. Potter served as chairman of the Committee on Revolutionary Pensions from 1859 to 1861 and of the Committee on Public Lands from 1861 to 1863. In this latter role, his committee handled the Homestead Act of 1862. In 1861, Potter was one of the participants in the Peace Conference of 1861, which failed to avert the American Civil War. He was defeated in his race for reelection in 1862 by fellow Maine-born lawyer James S. Brown, a Democrat who had been Milwaukee prosecutor and mayor, and who would defeated the following year by a Republican general. During the campaign, his son Alfred C. Potter had enlisted in the 28th Wisconsin Infantry in August 1862 as a sergeant, but would be mustered out the following April, and began receiving a pension in 1896.
Later career
After Potter's congressional term ended in early 1863, he declined appointment as governor of Dakota Territory, and his wife died in May 1863 in Washington, D.C., leaving Potter a widower with a ten year old son. The Lincoln administration then appointed Potter as Consul General of the United States in the British-controlled Province of Canada from 1863 to 1866. Thus Potter resided in what was then the Canadian capital of Montreal, Lower Canada. In 1866, Potter returned to East Troy, Wisconsin, where he resumed his legal practice.
