In his final season at Rochester, he was a player-coach, and then became minor league pitching instructor for the Red Wings' parent club, the Orioles, from 1974–77. At the close of the 1977 season, Miller agreed to join the coaching staff of the Texas Rangers, whose manager was former Baltimore third-base coach Billy Hunter. But in January 1978, the Orioles' pitching coach position opened unexpectedly when George Bamberger was named skipper of the Milwaukee Brewers. Miller was let out of his Ranger contract and succeeded Bamberger as mound tutor of the pennant-contending Orioles. He worked under managers Earl Weaver and Joe Altobelli and coached for O's teams that won the 1979 American League championship and the 1983 world title. Miller tutored 20-game-winning pitchers such as Jim Palmer, Mike Boddicker, Mike Flanagan, Steve Stone and Scott McGregor during that period. The success of the Orioles' pitching staff made Miller a sought-after managerial candidate and on June 21, 1985, he received his first opportunity. Billy Gardner, who had led the Twins to a disappointing 27–35 record, was fired and Miller took control of the young Minnesota ballclub. Although the Twins improved to 50–50 over the remainder of the season, they performed so poorly in 1986, Miller was replaced as skipper by Tom Kelly on September 12. He then returned to the coaching ranks, spending ten seasons as pitching mentor of the Pittsburgh Pirates working for Jim Leyland and one back in Baltimore under Davey Johnson. When Johnson resigned at the close of the Orioles’ AL East Division championship season, Miller replaced him as manager. But over two seasons, the Orioles played ten games under the.500, and he was fired in favor of Mike Hargrove in the autumn of 1999. Miller returned as pitching coach of the Orioles in 2004-05 and the Baltimore mound staff showed improvement under his tutelage. But Miller was forced to the sidelines by successful surgery to repair an aneurysm and was succeeded in that role by Leo Mazzone in 2006.
Miller's induction into the Baltimore Orioles Hall of Fame was announced on March 22, 2010. According to the press release from mlb.com: "Miller will be enshrined for his work as a pitching coach with the Orioles. He had three stints with the Birds in that role—1978–85, 1997 and 2004–05. During Miller's tenure, the Orioles won the 1979 American League Championship and the 1983 World Series. Five Orioles pitchers won at least 20 games under his tutelage—Mike Boddicker in 1984, Scott McGregor and Steve Stone in 1980, Mike Flanagan in 1979 and Jim Palmer in 1978. Miller also served as manager for the Orioles during the 1998 and 1999 seasons, compiling a 157–167 record."
