Bailly began his senior career at Genk in 2002 but was loaned out to Beringen-Heusden-Zolder for the duration of the 2003–04 season, along with ten other Genk players. Bailly replaced Jan Moons as Genk's first-choice goalkeeper at the beginning of the 2006–07 season; he played 90 minutes in every single Belgian League match for the club that season. He kept 14 clean sheets and received no bookings. His good performance contributed to Genk's surprising second-place finish.
Borussia Mönchengladbach
He joined Borussia Mönchengladbach during the winter of season 2008–09 and immediately adapted to the Bundesliga. He was elected best player of gameday 20 in the league. During the 2011–12 season he was initially loaned out to Swiss side Neuchâtel Xamax, for whom he never played a single match. In the second part of the season, he was loaned back to Genk. After the season, he returned to Mönchengladbach where his contract was not renewed, allowing him to sign as a free agent player for OH Leuven.
Leuven
At OHL, he was rejoined with coach Ronny Van Geneugden, who was his coach during his period as a youth player at Genk. After a series of good performances, Bailly was rewarded on 2 December with a one-year extension, giving him a contract until the end of the 2013–14 season.
Celtic
In July 2015, Bailly moved to Scottish club Celtic for an undisclosed transfer fee. He made his debut on 18 July 2015 in a pre-season friendly against SD Eibar and saved a penalty kick from Dani Nieto during a 4–1 win for Celtic. He began the season as second choice to Craig Gordon, but made his competitive debut in the Scottish League on 22 August 2015 in a 3–1 win away at Dundee United. Bailly's next appearance was in the fourth round of the Scottish Cup on 10 January 2016, deputising for the suspended Craig Gordon. He had little to do as Celtic eased to a comfortable 3–0 win over lower league Stranraer, and his only save came with 12 minutes remaining when he saved a Craig Malcolm header. Bailly did not play for Celtic during the 2016–17 season, as he fell to third choice behind Craig Gordon and Dorus de Vries. Celtic offered to loan Bailly to Raith Rovers in February 2017, but Bailly rejected the move.
International career
Bailly is a member of Belgium national team. He was named in a provisional 30-man squad for the UEFA Under-21 Championship in 2007. He also represented Belgium at the 2008 Olympic Games, where Belgium reached the semi-finals, knocking outItaly en route. Belgium eventually finished in fourth place. He received his first call up to the full international squad in the summer of 2007 for a Euro 2008 qualifying tie against Portugal, although he did not play. He featured in several more squad selections before winning his first cap on 10 October 2009 in a 2010 FIFA World Cup qualifying tie at home against Turkey. He played the full 90 minutes and kept a clean sheet in a 0–0 draw. Over the next year, Bailly made seven more international appearances, his last cap to date being in a 4–4 draw against Austria in October 2010.
