The Cleveland Indians selected Frazier with the fifth overall selection of the 2013 MLB Draft. He signed with the Indians instead of enrolling at the University of Georgia, receiving a $3.5 million signing bonus. The Indians assigned him to the Arizona Indians of the Rookie-levelArizona League for his professional debut. He hit a home run and a triple in his first professional game. In 44 games he batted.297/.392/.506 with five home runs and 28 RBIs. In 2014, Frazier played for the Lake County Captains of the Class AMidwest League where he batted.266 with 13 home runs and 50 RBIs in 120 games. In 2015, Frazier played for the Lynchburg Hillcats of the Class A-AdvancedCarolina League where he compiled a.285 batting average with 16 home runs and 72 RBIs in 133 games. Frazier began the 2016 season with the Akron RubberDucks of the Class AAEastern League. He was named to appear in the 2016 All Star Futures Game, where he went 2-for-3 with an RBI double and a stolen base. He was then promoted to the Columbus Clippers of the Class AAAInternational League on July 25.
On July 31, 2016, the Indians traded Frazier along with Justus Sheffield, Ben Heller and J. P. Feyereisen to the New York Yankees for Andrew Miller. The Yankees assigned Frazier to the Scranton/Wilkes-Barre RailRiders of the International League. Frazier finished 2016 with a combined.263 batting average, 16 home runs and 55 RBIs in 119 games between the three clubs. Frazier began the 2017 season with Scranton/Wilkes-Barre. He played 73 games for the RailRiders before getting called up, posting a.257 average with 12 home runs, 42 RBIs and 9 steals. The Yankees promoted Frazier to the major leagues on July 1, 2017. He doubled for his first major league hit that day and hit his first career home run in his next at-bat. On July 8, Frazier hit his first career walk-off home run off of Corey Knebel, giving the Yankees a 5-3 win over the Milwaukee Brewers. On July 19, Frazier changed his jersey number from No. 30 to No. 77, giving No. 30 back to the recently acquired David Robertson, who wore the number during his tenure with the Yankees from 2008 to 2014. Frazier stated that he appreciated the parallel between his No. 77 in left field and teammate Aaron Judge's No. 99 in right field. Frazier became the second Yankee ever with nine extra base hits before his 15th career game, after Joe DiMaggio. On August 10, Frazier was placed on the 10-day disabled list due to a left oblique strain. Frazier did not play during the post-season when the Yankees lost in Game 7 of the 2017 ALCS. Frazier began the 2018 on the disabled list with a concussion. He was reactivated in early May, and assigned to Scranton/Wilkes-Barre. On May 15, he was called up to the Major Leagues. On May 21, he was optioned to Triple-A. On June 4, he was called up again before being optioned to Triple-A on the next day. On June 18, he was called up for the 3rd time of the season. In a game against the Rays at Tropicana Field on June 22, Frazier entered the game in the ninth inning as a pinch-hitter and appeared to have hit a home run that could have given the Yankees the lead. But the ball hit a speaker and dropped down for a pop-out. He was sent down on the next day. On July 8, Frazier was once again called up to the Major Leagues as starting center fielder Aaron Hicks was banged up. On September 5, manager Aaron Boone said that Frazier would miss the rest of the season to get treatment for ongoing symptoms caused by the concussion he suffered earlier in the year. During 2019 spring training, manager Aaron Boone announced that Frazier would likely start the season in the minors. Frazier was first called up on April 2 in response to the Yankees having three injured outfielders at once: Giancarlo Stanton, Aaron Hicks, and Jacoby Ellsbury. On April 6, Frazier pinch hit in the 6th inning against the Baltimore Orioles. He hit a go-ahead three-run home run to the left field seats that ultimately gave the Yankees the win. The following day, Frazier hit two home runs in a 15-3 win against the Orioles. Despite his productive hitting, Frazier was optioned back to the minors on June 16 when the Yankees acquired Edwin Encarnación.