Palmer was born in Skegness and began playing non-league football with Skegness Town before being spotted by Nottingham Forest, signing in March 1958. He established himself in Andy Beattie's side as a tough tackling midfielder and made 90 league appearances for Forest, scoring 14 goals and playing in seven different positions. His performances for Forest earned him recognition as a reserve for the England under-23 side. On the lookout for new players as Stoke City suffered five successive defeats early in 1963–64, manager Tony Waddington was so impressed by Palmer's display he sought out Beattie after the match to broker a deal. Palmer cost Stoke £30,000 and was seen as a long-term replacement for the ageing Eddie Clamp. He was blessed with boundless energy and regularly ran back to make a tackle in his own penalty area before charging up field again. He had a decent goal ratio for a midfielder averaging one every five games. In his first season with Stoke he played in the final of the League Cup, as Stoke lost to Leicester City after two legs. He was also known for having a short fuse and at the end of the 1964–65 season with the squad preparing for a tour of the United States, Palmer was involved with a training ground altercation with Maurice Setters. Setters apologised to Waddington but Palmer did not and was made to remain in England. He returned to the side for the 1965–66 season and there was speculation that a call up to the England side would be forthcoming. Alas it never came and Palmer made few friends when he publicly criticised the Stoke crowd for showing a lack of support. He tabled a transfer request in December 1966 and, although it was granted, it took him 12 months to find a new club. In the meantime Palmer starred in a remarkable 4–3 win away at West Ham United in 1967–68, having been informed prior to kick-off that his family had been involved in a car crash. He filled in at centre back for the injured Alan Bloor and at half time Stoke were 3–0 down before coming back to claim an unlikely victory. His legs began to go and in February 1968 he was sold to Sunderland for £70,000 and signed his Stoke career off by scoring the winning goal away at Southampton. His time at Roker Park was marked by friction with the management and despite being just 28 his reputation for confrontation counted against him and no other English club was willing to sign him. So he then moved to South Africa playing for Cape Town City and Hellenic before returning to help Dennis Viollet at Crewe Alexandra in 1971. He then played for non-league Hereford United and made a return to South Africa with Durban United and Berea Park. After a long illness Palmer died on 12 March 2014 at a Brighton hospice, at the age of 73.