Patterson signed with the Celtics shortly before the start of training camp in 1954. He was waived by the team two days before the start of the regular season. After he was cut, Patterson returned to UConn and earned his bachelor's degree in Physical Education. After graduating, Patterson, who was a member of the Reserve Officer Training Corps in college, went into the United States Army. He spent two years in the Army and was stationed at Fort Sill. In 1957, Patterson returned to professional basketball as a member of the St. Louis Hawks. He was released after spending three months with the team, during which time he played in 4 games, scoring 7 points in 13 minutes of play. After his release, Patterson worked in the Hawks' front office for 4–6 weeks as part of the team's effort to integrate. He spent the 1958–59 and 1959–60 seasons with the Scranton Miners of the Eastern Basketball Association.
Music executive
After retiring from basketball, Patterson worked as a salesman. He spent six years at Technical Tape Corporation, working as a sales trainee, assistant to the Vice President of Sales, assistant to the General Manager, and assistant to the Purchasing Agent. Patterson left Technical Tape Corporation for RCA, where he worked at an RCA Records factory before becoming a salesman in the New England territory. He was later transferred to the company's promotions office in New York City. He then spent five years with Warner Bros. Records. In 1969, he was appointed manager of artist relations east at Dot Records. In 1970, he returned to RCA as head of promotions. Among the acts at RCA at this time were Hall & Oates, Waylon Jennings, and Vickie Sue Robinson. Later that year, he moved to Chess Records, where he was named national promotions director. In 1971, Patterson was promoted to Sales Director. In 1972, Patterson became Eastern district sales manager for Warner/Reprise Records. In 1973, he joined Warner Brothers’ subsidiary Casablanca Records, where he was charge of promotions for the new label. Casablanca's acts included Kiss, Donna Summer, The Village People, and Toni Tennille. He later moved to Motown, where he worked for eight years in sales and promotions. He worked at Bertelsmann Music Group for eight months in 1991 before retiring. He came out of retirement in the mid-1990s to serve as vice president of marketing and promotion for Monad Records.
Personal life
Patterson and his wife Queen have two children, Worthy III and Tanya. Worthy III played basketball at Pepperdine University.