Kyle was born in Sardis, Mississippi, the son of James M. Kyle and Susannah Kyle. As a boy, he attended the local schools and worked on his family's farm. He attended Bethel College through his junior year, then returned home to work on the farm. He later resumed his education at Cumberland University Law School, from which he received a LL.B. degree in 1874. He then attained admission to the bar and began a law practice in Batesville, Mississippi. After several years in Batesville, Kyle returned to Sardis, where he continued practicing law.
Political career
Kyle served as Democratic mayor of Sardis from 1879 to 1881, an event that was noteworthy for the fact that Kyle had not been a candidate for the office and had not campaigned for it. In 1881, Kyle was elected to the Mississippi State Senate, and he served until 1885. He declined to be a candidate for a second term, and in 1886 the state legislature chose him for a seat on the Mississippi Railroad Commission. Kyle had not sought the position, but still defeated John Marshall Stone in the legislative vote. In 1887, he was elected chairman of the Mississippi Democratic Party's executive committee, and he oversaw the party's efforts in that year's statewide elections.
U.S. Representative
In 1890, Kyle was elected to the United States House of Representatives. He was re-elected in 1892 and 1894, and served in the House from March 4, 1891 to March 3, 1897. He was not a candidate for renomination in 1896 election, declining to run because he opposed the Democratic Party's adoption of the Free silver position in its 1896 platform.
Later life
After leaving the House, Kyle resumed practicing law. He also became involved in several banking and business ventures. In 1900, Kyle was a candidate for the presidency of Mississippi Agricultural & Mechanical College and lost by one vote when the school's board of trustees voted 5 to 4 to offer the position to J. C. Hardy. In 1902, Kyle announced his candidacy in the 1903 gubernatorial election, but withdrew after his business and legal interests prevented him from becoming an active candidate. He retired from business in 1912, and died in Sardis on July 6, 1913, eleven days before his 62nd birthday. He was interred in Rosehill Cemetery.
Family
In 1879, Kyle married Sallie G. Heflin of Sardis. They were the parents of a son, John Curtis Kyle Jr..