After establishing a law practice in Hancock County, Holsey ran for as seat in the Georgia General Assembly. After losing two close races in 1823 and 1824, Holsey was elected in 1825 as one of three State Representatives from Hancock County, in the Georgia House of Representatives. He served one term before eventually moving to Harris County, Georgia. In 1835, he was elected as a Jacksonian Representative from Georgia to the 24th United States Congress to serve the remainder of the term left vacant when James C. Terrell resigned due to poor health. During his first term, he served from October 5, 1835 until March 4, 1837. Holsey switched political parties to win reelection to his seat as a Democrat in the 25th Congress. His entire congressional service spanned from October 5, 1835, until March 3, 1839. After his congressional service, he moved to Athens, Georgia, where he took up farming, and eventually journalism after he purchased the Southern Banner. In his editorials, Holsey "advocated the Missouri Compromise, the annexation of Texas and the strict enforcement of the fugitive slave law." However, in 1850 South Carolina threatened secession, and attempted to embroil Georgia. While Holsey was "unflinchingly opposed to federal encroachments, and strongly favored states rights" he was nonetheless "bitterly opposed secession and the Banner became the leading exponent of the union cause in the Sixth congressional district, if not in the state." Holsey, the political lightning rod, and his newspaper would soon become involved in what became known as the "Tugalo Tragedy", when an insane or disgruntled woman named Jane Young entered the newspaper office looking for Holsey. She didn't find him, but in a case of mistaken identity, shot an employee and fled into the street. Holsey was not injured, but claimed that the woman was after him for his unionist views. Ms.Young was brought before mayor Cincinnatus Peoples, and sentenced to two years in prison for assault with the intent to murder. Soon afterward, Holsey, as an outspoken proponent of unionism, and attempting to capitalize on his recent brush with death, was compelled to run for the 33rd United States Congress as a unionist Democrat, on what was called the "Tugalo Ticket" "against the disunion wing and the Whigs". He was defeated by the populist Democrat William Morton. Within a year after his defeat, Holsey sold the newspaper and began practicing law again, in Butler, Georgia, with his brother-in-law, Col. Daniel W. Miller.
Death
He died at Brightwater, his estate near Butler, and was buried there.
