Perbet was born in Le Puy-en-Velay, Haute-Loire. He finished his youth graduation with hometown's Le Puy Foot 43 Auvergne, joining Ligue 2 side Clermont Footin the summer of 2003. On 2 August 2003, he made his professional debut, playing the last ten minutes in a 0–0 home draw against FC Rouen; his first professional goal came on 27 September, netting the game's only in a home success over FC Gueugnon. Perbet finished his first season with 26 league appearances and 4 goals. In the 2004–05 season, Clermont finished 18th, only one point above relegation, and despite the club reaching the quarter-finals in both Coupe de la Ligue and Coupe de France, Perbet's contribution was limited, only finding the net five times in 32 matches, rarely being a starter. In August 2005, after appearing two times for Clermont during the season, Perbet joined Championnat National side AS Moulins. He found a goalscoring touch during his one-season spell at the Auvergne side, scoring 23 goals in only 31 appearances, and finishing the season as a topscorer alongside Jawad El Hajri of US Boulogne. Despite Perbet's contribution, Moulins finished the season 19th, and was subsequently relegated.
Strasbourg and loans
On 30 May 2006, Perbet joined RC Strasbourg in a three-year deal. He made his debut in a pre-season friendly against FC Bayern Munich, netting the last of a 2–0 success. However, he was sparingly used during the campaign, and was assigned to the reserves in the fourth division. On 5 January 2007, Perbet moved to Charleroi on loan until the end of the season. After helping the Belgian side finish 5th by scoring six goals in 13 matches, he returned to his parent club, freshly promoted to Ligue 1. On 14 July, Perbet joined Angers SCO, also in a temporary deal. After only appearing 11 times and without scoring, he returned to Strasbourg. In January 2008, due to Perbet's limited playing time, he was linked with a move to Austrian Bundesliga team SK Rapid Wien; however, nothing came of it.
Belgium
On 8 July 2008, Perbet moved to Belgian Pro League side Tubize. In his first season he scored 13 goals in 31 appearances, but could not prevent his side's relegation; in his second, Perbet maintained his goalscoring touch, netting 11 goals in 16 matches. On 7 January 2010, Perbet moved back to the top level, signing a two and a half-year deal with Lokeren for a rumoured €350,000 fee. He appeared ten times in 2009–10, scoring against Mechelen on 17 April. In late July, Perbet suffered a knee injury which left him sidelined for some of the 2010–11 season. On 5 January 2011, Perbet joined Mons on loan until the end of the season. He was a key offensive unit during the promotion campaign, scoring 14 goals in 14 matches. On 3 August, he signed a three-year permanent deal with Mons, netting a career-best 28 goals during the season.
Villarreal
On 25 January 2013, Perbet moved teams and countries again, joining Villarreal CF on loan until the end of the season. He made his debut on 9 February, starting and scoring twice in a 3–0 home win over SD Ponferradina. He finished the season with 11 goals in 18 matches, as the Amarillos returned to La Liga at first attempt. On 7 July, Villarreal exercised the buyout option and signed Perbet until 2016, for a rumoured €1.4 million fee. He made his debut in the Spanish top flight on 24 August, playing the entire second half in a 2–1 home success over Real Valladolid; his first top flight goal came seven days later, the first of a 3–0 win at CA Osasuna.