In addition to being the first Little League player to advance to the major leagues, Jay was one of the first "bonus baby" players in the major leagues. This resulted when he signed a significant contract with the Braves, which forced the Braves to keep Jay on their major league roster for two seasons because of the contract's amount. On September 20, 1953, at the age of 17, making his first career start, he pitched a seven-inning complete game shutout, but generally was unremarkable in his two years with the team. Following the end of his two years, he was sent to the minors to gain experience on a staff that already was loaded with Hall of FamerWarren Spahn, Bob Buhl and Lew Burdette. Jay went 7–5 with an ERA of 2.14 in 18 games for the Braves in his best season, becoming the first pitcher to win the NL Player of the Month award in July but a broken finger kept him out of the World Series.
The Braves traded Jay to the Cincinnati Reds after the 1960 season for infielder Roy McMillan. Braves General Manager at the time, John McHale, reportedly made the deal based on the feeling Carl Willey could do a better job for Milwaukee than Jay. Jay took full advantage of the trade, as he became a key figure in the Reds' stunning revival in 1961. Jay won 21 games, tied for the league lead in wins and shutouts, and won his second NL Player of the Month award in May as the Reds surged to their first National League pennant since 1940. However, the Reds faced a powerful New York Yankees club which won 109 games and featured Roger Maris, Mickey Mantle and Whitey Ford, as the Reds lost in five games. However, the lone Reds win occurred in Game 2, a 6-2 victory as Jay threw a complete-game four-hitter at Yankee Stadium, being Jay's single-game career highlight. On May 1, 1962, at the Polo Grounds, off New York Mets pitcher Sherman Jones, Jay hit a three-run HR, in the 6th inning, for his first MLB home run. At Crosley Field, on May 28, 1962, Jay hit his only other career home run, off Houston Colt.45s pitcher Bobby Tiefenauer, in the 5th inning, a two-run blast. Jay also won 21 games in 1962 as the Reds won 98 games to finish in third-place behind the Giants and Dodgers. Jay's heavy workload in 1961 and 1962 took a toll the following year where he struggled to a 7–18 record. Jay posted an mark in 1964 as the Reds finished a single game behind the eventual World Series champion St. Louis Cardinals. He would finish his career by returning to the Braves for their initial season in Atlanta in 1966.
