Williams was born in Wrexham and attended Acton Park School before moving to Grove Park Grammar School, who he represented at football. He also played football for the Victoria Youth Club whilst working for an insurance company. He joined Wrexham as an amateur in August 1949, making five league appearances before being signed by West Bromwich Albion in November 1950.
West Bromwich Albion
At West Bromwich, he made his debut as a centre-forward before switching, firstly to wing-half, before settling into the full-back position. In 1954, Williams helped West Bromwich to reach the runners-up position in the Football League and seemed certain to replace the injured Stan Rickaby in the FA Cup Final, but manager Vic Buckingham opted instead for the more experienced Joe Kennedy. Described as having "a first rate temperament, splendid positional sense and a sure kick", Williams later developed a "notable" full-back partnership with Don Howe. Williams remained at West Bromwich for 12 years, making 226 league appearances, scoring six goals.
Southampton
In September 1962, he joined Southampton for a fee of £15,000; the "Saints" manager Ted Bates needed Williams's experience to help guide the club towards the First Division. Williams made his debut on 19 September 1962, in a 2–1 victory over Chelsea, when he took over at right-back from Roy Patrick. He rarely missed a match over the next four years, although in 1965–66, he switched to left-back with the right-back position being filled by several players, including Ken Jones and Tommy Hare, before the signing of David Webb in March 1966. Williams's final match for Southampton came on 22 April 1966, shortly before the end of the season which saw the Saints celebrate promotion to the top flight for the first time.
International career
Williams made his debut for Wales in a friendly against Austria on 9 May 1954. He played for Wales on 43 occasions, including all Wales's group stage matches at the 1958 FIFA World Cup in Sweden where Wales met Brazil in the Quarter-finals, going out 1–0 to a goal from Pelé. In his autobiography "My Life and the Beautiful Game", written in 1977, Pelé says of this match:
After his playing career he held various coaching and management jobs including as a trainer with West Bromwich Albion, Aston Villa, Morton and Southampton and brief spell as manager with Iranian club Paykan in 1970–71, and Norwegian club Viking FK in 1974. Williams settled in Southampton, and after leaving the game he became a tyre salesman, and later a financial controller for a transport company.