The club was formed in memory of Shimshon Rouzi, a former Maccabi Tel Aviv player who died in the 1948 Arab-Israeli War. It was commonly referred to as the club of Kerem HaTeimanim, a neighbourhood of Tel Aviv. The club spent its first few years in the lower leagues, before winning promotion to the top flight in 1960 as Liga Alef champions. In 1962–63 the Shimshon finished 4th, the club's highest position to date. In 1965–66, Shimshon player Moshe Romano was the division's joint top scorer alongside Maccabi Netanya's prolific striker Mordechai Spiegler, with 17 goals. The club also made their first appearance in a State Cup final, going down 2–1 to Hapoel Haifa. Another 4th-place finish was achieved in 1969–70, with Romano again finishing as the league's top scorer. The following season they went one better by finishing second to champions Maccabi Netanya. However, the success was not sustained, and a season after finishing second, the club narrowly avoided relegation, finishing one place above the drop zone. The 1972–73 was even worse, as the club was relegated alongside Hapoel Marmorek. Despite their setback the previous season, Shimshon had a successful season in 1973–74, finishing second in Liga Alef North and took part in the new promotion/relegation play-offs. They finished second in the play-off group, and were promoted back to Liga Leumit. The early 1980s were a period of relative success, as the club finished third in 1979–80 and 1984–85, and another second-place finish achieved in 1982–83. A second cup final was reached in 1986, but they lost 2–1 to Beitar Jerusalem. In 1987–88 the club qualified for the Intertoto Cup. Although they reached another cup final in 1989–90, losing again, 1–0 after extra time to Hapoel Kfar Saba, the club finished bottom of Liga Leumit for the first time in their history, and were relegated to Liga Artzit. Unable to bounce back to the top flight, and overshadowed by more popular and successful teams in Tel Aviv (Hapoel, Maccabi and Bnei Yehuda, the club merged with fellow Tel Aviv strugglers Beitar to form Beitar Shimshon Tel Aviv. In 2014, the club returned to the map of Israeli football after 14 years.
