A native of New Orleans, Richardson attended L. B. Landry High School in Algiers as a freshman on the junior varsity team. Richardson left New Orleans with his parents, Albert Joseph and Anita Richardson, during Hurricane Katrina in 2005. They moved to the suburbs of Baton Rouge, Louisiana, where Richardson attended Baker High School for his sophomore year. A late-season addition to the Baker football team, he rarely played before the family moved to Fort Worth, Texas, in October 2006. In Fort Worth, Richardson enrolled at North Crowley High School, where he joined the football varsity but did not play during his junior season for academic reasons. Finally returning to the field as a senior, Richardson was named first-team All-District 3-5A in 2008. North Crowley advanced to the first round of the UIL playoffs, losing 21–0 to Luke Joeckel's Arlington. Regarded as a three-star recruit by Rivals.com, Richardson was ranked as the No. 90 offensive tackle prospect in the class of 2009, which was headed by D. J. Fluker and Mason Walters. He drew limited interest from several Big 12 schools, and eventually signed with Baylor.
College career
After being redshirted as a freshman in 2009, Richardson played in 12 games in 2010, with four starts on an offensive line that also included Danny Watkins at left tackle and Robert Griffin at right tackle. Richardson totaled 63 knockdowns and averaged coach's grade of 80.5. As a sophomore in 2011, Richardson replaced Watkins at left tackle, starting all 13 games of the season to protect quarterback Robert Griffin III's blindside. As a junior in 2012, Richardson moved back inside to left guard and started 12 of 13 games, as redshirt freshmanSpencer Drango took over as left tackle. Richardson posted a team-best average coaches grade of 89.8 percent and team-high 105 knockdowns on the season. He was selected 2012 Big 12 Offensive Lineman of Year by league coaches, and also earned various All-America honors and unanimous first-team All-Big 12 selection. As a senior in 2013, Richardson anchored Baylor's offensive line at left guard, in an offense that averaged 76.4 plays in their first five games, piling up 714.4 yards per game. He earned mid-season All-American honors by ESPN as the only selection from an unbeaten Baylor Bears team.
Professional career
At one point, Richardson was considered one of the top prospects for the 2014 NFL Draft. Daniel Jeremiah of NFL.com named him one of college football's “most physical players.”
Buffalo Bills
Richardson was drafted by the Buffalo Bills in the fifth round, 153rd overall. He played in 12 games as a rookie, making four starts. On September 4, 2015, he was released by the Bills. On September 6, 2015, the Bills signed Richardson to their practice squad. On January 4, 2016, Richardson signed a futures contract with the Buffalo Bills. He was released by the Bills on September 2, 2016.
Chicago Bears
On November 15, 2016, Richardson was signed to the Bears' practice squad. He signed a reserve/future contract with the Bears on January 3, 2017. He was waived on September 2, 2017.
In October 2019, Richardson was selected by the Seattle Dragons in the 2020 XFL Draft. He was placed on injured reserve on February 11, 2020. He had his contract terminated when the league suspended operations on April 10, 2020.