A native of Mandan, North Dakota, Erhardt graduated from Jamestown College in 1953, then spent the next two years serving in the military. After leaving the service, he was hired in 1956 as an assistant coach at Williston High School in Williston, North Dakota. The following year, he began a six-year run as a head coach at two North Dakota Catholic high schools: from 1957 to 1959, his teams at St. Mary's of New England compiled a mark of 25–3–1, followed by another three-year run at Bishop Ryan High School in Minot, where he was 20–6–1 from 1960 to 1962.
Early career
Erhardt's successes elevated him to the collegiate level, where he served as an assistant at North Dakota State University for three years beginning in 1963. On February 10, 1966, he became head coach at the school, setting the stage for another strong tenure, in which he put together a record of 61–7–1 in his seven years. Erhardt also served as the Bison athletic director, winning a pair of college national championships. In 1970, he was awarded Blue Key National Honor Society's Distinguished Educator award for his dedication to the University.
On March 14, 1973, Erhardt moved on to become backfield coach of the New England Patriots, a post he served in for four years before being promoted to offensive coordinator on February 1, 1977, following the departure of Red Miller. After missing the playoffs that year, the Patriots seemed ready to challenge for a Super Bowl berth in 1978. However, prior to the last regular season game on December 18, head coach Chuck Fairbanks announced he was leaving the team to accept a contract offer from the University of Colorado. In response, the team suspended Fairbanks and made Erhardt and fellow assistant Hank Bullough co-coaches for the final game. The Patriots dropped that contest, then lost their opening round matchup to the Houston Oilers. Fairbanks was allowed to leave for Colorado, with Erhardt officially taking the reins of the team on April 6, 1979. Noting his.890 winning percentage at North Dakota State upon taking the position, Erhardt said, "I've never been a loser in football and I don't intend to start now." The Patriots underachieved to a 19–13 record in the 1979 and 1980 seasons, missing the playoffs by one game each year; the 441 points scored in 1980 was a club record not broken until 2007. But then the Patriots finished with a 2–14 record in 1981. Citing the fact that Erhardt "was just too nice a guy", owner Billy Sullivan dismissed him on December 22, two days after the conclusion of the season.
Just over a month later, in January 1982, Erhardt was hired as offensive coordinator of the New York Giants by head coach, and former fellow Patriots assistant Ray Perkins, the first since Joe Walton filled the position under head coach Alex Webster in 1973. During the latter portion of that first season, Perkins announced he was leaving to become head coach at the University of Alabama, with Bill Parcells replacing him. Parcells had been Erhardt's linebackers coach with the Patriots in 1980, and kept his former boss in the same position. That decision would prove to be the right one as the Giants went on to win two Super Bowls, with Erhardt working around a key injury during the run toward Super Bowl XXV. Quarterback Phil Simms had suffered a season-ending injury late in the season, but his replacement, Jeff Hostetler, navigated the team to the title. Parcells retired after that game, with Ray Handley being promoted to head coach.
Handley then demoted Erhardt, which resulted in the latter's departure for a similar position with the Pittsburgh Steelers in 1992. Emphasizing the run, Erhardt ran the offense for four years, helping the team reach Super Bowl XXX in 1996. However, after the season, the conflict over the offensive philosophy between Erhardt and Steelers' head coach Bill Cowher resulted in Erhardt's resignation.
Erhardt then became offensive coordinator of the New York Jets under Rich Kotite. A disastrous 1–15 mark marked the end of Kotite's brief era, with Parcells replacing Kotite. Erhardt announced his retirement on January 12, 1998.
