In 1799, he was chosen to represent Washington County at the convention that drafted the second Kentucky Constitution. From 1800 to 1802, he represented Washington County in the Kentucky House of Representatives. He then moved to Nelson County, which he represented in the Kentucky House from 1804 to 1806. On December 10, 1806, he was commissioned an associate justice of the Kentucky Court of Appeals. He was elevated to Chief Justice of the court on April 11, 1807. Later that year, he resigned and moved to Nashville, Tennessee, where he again took up the practice of law. He was elected as a Democratic-Republican to the 12th and 13th Congresses and served from March 4, 1811, until his resignation in July 1814. He then became a member of the Tennessee House of Representatives from 1819 to 1825, and in 1820 was commissioner to settle the boundary line between Tennessee and Kentucky. He was elected as a Jacksonian in 1829 to the United States Senate to fill the vacancy in the term ending March 4, 1833, caused by the resignation of John H. Eaton to join the Cabinet of President Andrew Jackson; reelected in 1832 and served from October 19, 1829, to July 4, 1838, when he resigned to accept a Cabinet position. During this time he served as chairman of the U.S. Senate Committee on Post Offices and Post Roads, U.S. Senate Committee on the Judiciary. He entered the Cabinet when he was appointed Attorney General of the United States by President Martin Van Buren in July 1838. He resigned the post in December 1839, having been elected as a Democrat to the United States Senate on November 19, 1839, to fill the vacancy in the term commencing March 4, 1839, caused by the resignation of Ephraim Foster; the question of his eligibility to election as Senator while holding the office of Attorney General of the United States having been raised, he resigned on December 14, 1839, and was reelected to the Senate the same day, serving from December 14, 1839, until his death in Nashville, a little over a year later. During this stint in the upper house of the U.S. Congress he served as chairman of the U.S. Senate Committee on Revolutionary Claims in the 26th Congress.
Death
His grave can be found at the Nashville City Cemetery in Nashville, Tennessee. After his death, four American counties were named in his honor. The four counties are located in Illinois, Iowa, Missouri and Tennessee. Grundy Center, Iowa, located in Grundy County, Iowa are both also named in his honor. Grundy Center's annual festival, called "Felix Grundy Days", are held each July, marking the start to the annual Grundy County Fair, located in Grundy Center.
Legacy
He was a mentor to future President James K. Polk. Polk purchased Grundy's home called "Grundy Place" and changed the name to "Polk Place". He lived and died there after his presidency. It was demolished in 1901.