A left back, Elder was born in Sydney, Australia and began his youth career with Manly United, before moving to England at the age of 16 to begin a scholarship in the academy at Leicester City. After completing his scholarship at the end of the 2012–13 season, he was awarded his first professional contract. Graduating to the Foxes' U21 team, he progressed to sign a new two-year deal at the end of the 2014–15 season and a further extension a year later, which kept him at the King Power Stadium until June 2019.
On 8 January 2015, Elder was loaned out to League Two club Mansfield Town for one month. He made his professional debut two days later, starting in a 2–1 away loss against Burton Albion. He became a regular starter, which saw his loan extended to the end of the 2014–15 season. Elder made 21 appearances for the Stags and returned to Leicester at the end of the season.
On 29 August 2015, Elder joined League One side Peterborough United on a one-month loan, which was later extended until January 2016. He made 21 appearances and scored the first senior goal of his career, which came in a 4–0 thrashing of Doncaster Rovers on 24 October 2015. A foot injury suffered in a match versus Blackpool on 19 December led to the early termination of Elder's loan.
On 30 July 2016, Elder joined Championship side Brentford on loan for the duration of the 2016–17 season. He was an ever-present in league matches during the opening month of the season, before suffering a knee injury during a 2–0 win over Brighton & Hove Albion on 10 September 2016. Elder's recovery was delayed by a thigh injury and he returned to the King Power Stadium for treatment in mid-December 2016. He returned to training with Brentford on 16 January 2017, but failed to make an appearance before being recalled by Leicester City two weeks later.
On 31 July 2017, Elder joined Wigan Athletic on a season long loan. He made his debut on 5 August at Milton Keynes Dons but was sent off in the 44th minute. Wigan won the match 1–0.
On 2 January 2019, Elder joined Ipswich Town on loan for the remainder of the 2018–19 Championship season.
Hull City
On 8 August 2019, Elder joined Championship side Hull City for an undisclosed fee, signing a three-year contract. The move reunited him with Grant McCann, his manager at Peterborough United. Elder made his debut on 14 September 2019, in a 2–2 draw at home to Wigan Athletic.
