Gregory attended Abilene High School, where he developed into one of the greatest athletes in Texas high school history. He earned All-American and All-State honors as a running back in football, while playing a key role in three state championships and Abilene's legendary national record-setting 49 consecutive victories. He was a part of 37 straight wins and never lost a game in high school. As junior he rushed for 1,174 yards. In his final year he posted 1,142 rushing yards, 19 receptions for 553 yards, 82 passing yards, 22 touchdowns, 56 extra points and received the High School Player of the Year Award from the Texas Sportswriters Association. Gregory received All-State honors and was one of the region's brightest pro-baseball prospects as a switch-hitting catcher, while contributing to the school winning two state championships. The Cleveland Indians valued his signing so much, that they offered a contract with a $75,000 bonus, the third-richest ever at the time. He was named Honorable-mention All-state as a shooting guard in basketball. Although he didn't practice track because of time conflicts with the other sports, but was involved in the Abilene Recreation Department program, registering a 100-yard dash in 9.8 seconds. In 1969, he was inducted into the Texas High SchoolFootball Hall of Fame. In 2013, he was inducted into the Big Country Athletic Hall of Fame.
College career
After graduating from high school in 1956, he was a highly touted and sought after college football recruit and ultimately accepted an athletic scholarship from Southern Methodist University. The NCAA would later give a one-year no sanction probation to SMU, for a $300 summer job as an oil scout, given to Gregory by a Dallas-based oil company. In his sophomore year, with Don Meredith as his quarterback, he had 55 rushes for 208 yards and 16 receptions for 189 yards. The next season, he led the Southwest Conference and tied for ninth in the nation with 30 receptions for 369 yards, while registering 50 runs for 269 yards. He played as a halfback, punter, defensive back and was voted the team's player of the year. As a senior, he was limited with a rib injury, recording 28 rushes for 95 yards and 8 receptions for 120 yards in 10 games. He finished his college career, earning three letters each in football and baseball. He also received All-Southwest Conference honors in both sports. In 1984, he was inducted into the SMU Athletics Hall of Fame.
Professional career
Gregory was selected in the ninth round of the 1961 NFL Draft by the Dallas Cowboys and in the thirteenth round of the 1961 AFL Draft by the Dallas Texans. He opted to sign with the Cowboys, turning down multiple offers to play professional baseball. As a rookie, he was used at 5 different positions both on offense and defense. The next year his career was cut short by a knee injury he suffered in training camp, although he was able to play in 6 games. On September 10, 1963, he was released and signed to the taxi squad, where he would finish the year and his contract.