Celeski signed for Perth Glory for the 2005–06 A-League season from Bullen Zebras playing 18 times and won the club's media award, but was released at the end of the season. Celeski graduated from the Victorian Institute of Sport and Australian Institute of Sport. In March 2007 Celeski was signed by Perth Glory again. A notable performance, with Perth Glory, in the A-League occurred on Saturday, 15 December 2007 – where Celeski scored a hat-trick in Perth Glory's 4–2 win over Sydney FC at the Sydney Football Stadium. Afterwards Celeski signed for Melbourne Victory at the end of the 2007–08 season in a one-year deal, post-mutual termination of his Perth Glory contract, primarily to bolster the squad for the AFC Champions League. Celeski scored his first goal for his new club in their 3–2 win over Adelaide United on 31 October 2008. Celeski signed a new 2-year contract with Melbourne Victory on 23 December 2008. Celeski went on to play a role in the Melbourne team, performing an important role in their title-winning campaign, while also not missing a league match until late 2009. This was to be stopped on 3 September of that year however, when he suffered a season ending anterior cruciate ligament injury, in the opening of a match against Newcastle Jets. On 20 July 2013, Celeski represented the A-League All Stars in the inaugural A-League All Stars Game against Manchester United, a match in which the A-League All Stars were thrashed 5–1, courtesy of goals from Danny Welbeck, Jesse Lingard and Robin van Persie. Just 4 days later, on 24 July 2013, he represented Melbourne Victory against Liverpool, a match in which the Victory lost 2–0, courtesy of goals from Steven Gerrard and an own-goal from Victory captain Adrian Leijer. He was the only Victory player to have played in both of those exhibition matches. On 30 July 2013, just a few days after representing the A-League All Stars against Manchester United and representing Melbourne Victory against Liverpool, Billy Celeski transferred from Melbourne Victory to UAE Pro-League club Al-Shaab. On 24 February 2014, Celeski transferred to Chinese Super League side Liaoning Whowin. On 29 May 2014, he was released along with Artyom Filiposyan. On 6 August 2014, Celeski signed with the Newcastle Jets. He was reportedly sacked by Jets owner Nathan Tinkler, following a player revolt midway through the season. On 3 February 2016, Celeski joined J1 League club Ventforet Kofu.
