Born in Canandaigua, New York, as a young man he began selling bicycles in his hometown and within a few years eventually expanded into manufacturing his own line of bicycles. In 1897 he married Isabel Van Wie and a few years later entered the automobile retailing business in Elmira, New York. His very successful car dealership sold the Overland brand of automobiles. However, in 1907 supply problems with the Indianapolis, Indiana, Overland factory led to John Willys acquiring the company. He proved an astute operator and quickly turned the company's sagging fortunes around. In 1909 he acquired the Marion Motor Car Co. of Indianapolis, Indiana, and a few years later shifted operations to a production facility bought from the bankrupt Pope Motor Car Co. in Toledo, Ohio.
After changing the name to the Willys-Overland Motor Company in 1912, the next year John Willys acquired the Edwards Motor Co of New York which gave him a license to manufacture the patented Knight "sleeve valve" engine. Success saw his car company become the second largest carmaker in the United States and in 1915 he built a seven-story headquarters in Toledo, Ohio, that was the most modern of its day. Before the end of the decade, one-third of the city of Toledo's workforce was employed either at Willys-Overland or at one of the numerous small businesses providing parts and supplies. His automobile empire offered the consumer the choice of an Overland, Willys or Willys-Knight vehicle, each relative to a specific type of engine or price range. Through his holding company, in 1918 John Willys acquired the Moline Plow Company of Moline, Illinois, which manufactured the "Universal" brand of farm tractor and a line of Stephens cars. The following year he acquired control of the Duesenberg company primarily to get his hands on Duesenberg brothers' factory in Elizabeth, New Jersey where he planned to produce a new six-cylinder car. Labor difficulties began to emerge at the Willys-Overland Toledo plant that resulted in a violent strike in 1919, shutting down the plant for several months. Willys hired General Motors vice-president Walter Chrysler to run the Willys-Overland operation at the then astonishing salary of $1 million a year. However, Chrysler tried to oust John Willys with an attempted takeover bid that backfired when the shareholders resisted his move and Chrysler left in 1921 to go into business for himself. Although very profitable, John Willys' businesses were highly leveraged, expanded and/or acquired through massive borrowings. In 1921, Willys' nervous bankers forced him to consolidate in order to limit their exposure. To raise cash for debt reduction, the Willys-Overland plant in New Jersey was sold at auction to William C. Durant as was Willys' "New Process Gear Company," in Syracuse, New York. With debt under control, Willys once again began expanding and in 1925 bought the F.J. Stearns Co. of Cleveland, Ohio that made a line of luxury vehicles. In 1926 Willys introduced the "Whippet" model line that sold in the U.S., Canada, and Australia. Well respected in the business community, John Willys was a strong supporter of the United States Republican Party who had been an Ohio delegate to the 1916 Republican National Convention. Following the election of Herbert Hoover to the Presidency of the United States, in March 1930 Willys was appointed the first United States Ambassador to Poland, serving until May 1932. The Great Depression of the 1930s saw numerous carmakers go out of business and the Willys enterprises went into bankruptcy reorganization in 1933. The following year, John Willys and his wife of thirty-seven years divorced. He soon remarried. He died on August 26, 1935, of a stroke after recovering completely from a previous heart attack that he had in May, at his home in The Bronx, New York City. He was interred in the Kensico Cemetery in Valhalla, New York. Willys and his wife had at least one daughter, Virginia, who married a rancher, Luis Marcelino de Aguirre, in 1929 when she was 18.