Originally from Port Melbourne before moving to Broadmeadows at age 14, Swan was zoned from Broadmeadows to the Carlton Football Club in the Victorian Football League and played under-19s and reserves football there from 1973 until 1975, before walking out on the club. In 1976, he joined Port Melbourne in the VFA, and was part of the club's senior premiership team in his first season. Standing at only 173 cm tall, Swan played a variety of roles, before he cemented a position as the centreman in the strong Port Melbourne team which won three consecutive premierships from 1980 until 1982. He won the J. J. Liston Trophy as the best and fairest player in the VFA First Division in both 1982 and 1983; he originally finished second behind Preston's Geoff Austen for the 1982 award on countback, but was later made a joint-winner of the award when the countback was retrospectively eliminated; he won the 1983 award outright by a margin of three votes. Altogether, in thirteen seasons with Port Melbourne, Swan played 219 games, won four premierships, two club best and fairest awards and served as club captain from 1986 until 1988. He was named as a wingman in the Port Melbourne Team of the Century. In 1989, at age 32, Swan left Port Melbourne after he and the club were unable to come to terms for a new deal, in large part because the club believed that he was too old for the deal he was requesting. As a result, he moved to the Williamstown Football Club, Port's bitter rivals. He played at Williamstown for five years, winning club best and fairests in 1989 and 1991, and won one premiership – the fifth in his VFA career, in 1990. In the 1990 Grand Final against Springvale, Swan kicked the winning goal for Williamstown with fifty seconds remaining, to complete a famous come-from-behind victory by two points after Williamstown had trailed by 34 points early in the final quarter; Swan was not known for his long kicking, and most observers, including Swan, thought that the fifty metre set shot was well beyond his range. In August 1993, Swan played his 301st VFA game, passing Fred Cook to become the all-time VFA games record holder, a title he still holds as of 2019; he played one more match before retiring at the end of the season.
Personal life
Like his father before him, Swan's primary career outside football was on the wharves in Port Melbourne, where he was a union delegate. Swan's son Dane is also a highly decorated Australian rules footballer, who played for in the Australian Football League ; Dane's achievements include an AFL premiership, a Brownlow Medal, a Leigh Matthews Trophy, and, like his father, a premiership with Williamstown.