The club was established in 1958 as Ashford Albion and joined Division Two of the Hounslow & District League. They finished second in their first season, earning promotion to Division One. They finished as runners-up again the following season, and were promoted to the Premier Division, adopting their current name in 1964. However, the club pulled out the Premier Division during the 1964–65 season, resulting in their record being expunged. The following season the first team took over from the reserves in Division Two. The club finished third in Division Two in 1965–66, resulting in promotion to Division One. After finishing as runners-up in Division One the following season, the club joined the Premier Division B of the Surrey Intermediate League for the 1967–68 season. They finished fourth in their first season, earning promotion to Premier Division A. After claiming the runners-up spot in 1972–73, they won the division in 1974–75. Ashford became founder members of the Surrey Premier League in 1982. In 1989–90 they were league runners-up and won the Surrey Premier Cup, beating Farnham Town Reserves 3–0 in the final. In 1990 they were formally renamed Ashford Town to avoid confusion with the Kent club which previously shared the same name, but are now named Ashford United. In the same year they joined the Combined Counties League. They finished as runners-up in 1993–94, and won the league the following season. They went on to win the league in each of the next three seasons, and again in 1999–2000, after which they were promoted to Division Three of the Isthmian League. After finishing third in their first season in Division Three, they were promoted to Division Two. In 2002, they were moved to Division One South as a result of league reorganisation and, in 2004, switched to the Western Division of the Southern League. In 2005–06, they finished second in the division, and were promoted, being placed in the Isthmian League Premier Division. In 2006–07 they won the Isthmian League Cup and, in 2008–09, the Surrey Senior Cup. At the end of the 2009–10 season, the club were relegated and placed in Division One Central of the Southern League; although they were offered a late reprieve from relegation after Merthyr Tydfil were liquidated, the club declined to take it and started the 2010–1 season in Division One Central. In 2011–12 the club won both the Middlesex Senior Charity Cup and the Aldershot Senior Cup. The 2013–14 saw the club finish bottom of Division One Central, resulting in relegation back to the Combined Counties League. After finishing third in the Premier Division in their first season back in the league, they were runners-up in 2015–16 and were promoted back to Division One Central of the Southern League. The following season saw the club win the Middlesex Senior Charity Cup for the third time, beating Wealdstone 2–1 in the final. At the end of the 2017–18 season they were transferred to the SouthCentral Division of the Isthmian League.
Ground
Ashford Town have played home games at Short Lane in Stanwell since 1986. The ground has a capacity of 2,550, of which 250 is seated and covered. The ground was renamed the Robert Parker Stadium in 2010 in honour of Bob Parker, who had served as chairman for 28 years.