Phillips played high school football, basketball, and baseball at Bath County High School in Hot Springs, Virginia, earning many awards in the district and region, besides receiving All-state honors at defensive end and tight end. He also was an All-state baseball player. He accepted a scholarship from the University of Virginia and blossomed as a senior, becoming team captain and a first-team All-ACC selection. He also had a 48-reception season, second-best by a tight end in school history. He finished his college career with 69 receptions for 670 yards and five touchdowns.
Phillips was selected by the Dallas Cowboys in the sixth round of the 2009NFL Draft and surprised observers by making the team. In 2009, he had 7 catches for 62 yards, while mostly being used as a blocking tight end. In the 2010preseason, he was impressing the coaching staff with his skills as a receiver and his ability to take different assignments, including the H-Back role. His performance was starting to push Martellus Bennett for the backup tight end job, but he was lost for the year in the preseason opener, when he suffered a season-ending ACL tear. It has been speculated that he was never the same player following this injury. The next year, he returned in a backup role and posted career-highs with 15 receptions for 101 yards. After Bennett left in free agency, Phillips had a solid season as the backup tight end for the Cowboys in 2012. He had 8 receptions for 55 yards, one touchdown and started 9 games, serving mainly as an H-Back or blocking tight end. At the end of the year, the Cowboys didn't make an offer to re-sign him.
Phillips signed as a free agent with the San Diego Chargers on March 12, 2013. He was used as a blocking specialist until being placed on the injured reserve list, after suffering ACL and MCL tears in his right knee during the sixteenth game against the Oakland Raiders. He came back to play in all of the 16 games of the 2014 season. In 2015, he was released on September 5, only to be re-signed 5 days later, after Ladarius Green suffered his second concussion of the year and Antonio Gates was still serving his suspension. He was the starter in the third and fourth game of the season, making a critical touchdown reception against the Cleveland Browns. After Gates returned in the fifth game, Phillips was used mainly for blocking purposes. He played in 16 games, while posting 10 receptions for 69 yards and one touchdown.
Denver Broncos
On July 28, 2016, Phillips was signed as a free agent by the Denver Broncos. He overcame a serious ankle injury he suffered in his second day in training camp to make the team. On November 5, 2016, he was released by the Broncos. He appeared in 8 games with 3 starts, registering 5 receptions for 40 yards and one touchdown.
On November 7, 2016, he was claimed off waivers by the New Orleans Saints. He appeared in 8 games, making 5 receptions for 32 yards, while contributing as a blocking tight end and on special teams. On March 29, 2017, Phillips re-signed with the Saints on a one-year contract. On August 29, 2017, Phillips was placed on injured reserve, and later released on September 8, 2017. He was re-signed on December 19, 2017, but was once again released on December 29, 2017, and re-signed again on January 3, 2018. On August 5, 2018, Phillips re-signed with the Saints. He was released on September 1, 2018.
Arizona Cardinals
On October 30, 2018, Phillips signed with the Arizona Cardinals.