Leroy A. Stafford was born on Greenwood Plantation near Cheneyville, south of Alexandria in south Rapides Parish, Louisiana. He was the eldest son of wealthy land owner Leroy Stafford and his second wife, Elizabeth Calliham. His father died when he was thirteen years old. Stafford was educated at Bardstown, Kentucky, and attended college in Nashville, Tennessee. In 1843, at the age of twenty-one, he returned to Louisiana and married Sarah Catherine Wright, the daughter of Dr. Jesse D. Wright and the former Sarah R. Grimball. The Staffords had ten children—George Waters, Sally, Elizabeth Eloise, David Theophilus, Cornelia Knox, Julia Laura, Jesse Wright, Rosalind, Leroy Augustus Jr., and Kate Keary.
Military life
Stafford operated plantations in Rapides Parish for a couple of decades, and served as the sheriff of the parish in 1846–47 before leaving for the Mexican–American War. He enlisted in the "Rapides Volunteers" and later served in Captain Ben McCullough's Texas Rangers. With Louisiana's secession in 1861, Stafford formed the "Stafford Guards", a militia unit that was soon mustered into Confederate service as Company B, 9th Louisiana Infantry with Stafford as its first captain. Stafford rose through the ranks and became the regiment's colonel in early 1862 when Richard Taylor was promoted. With the death of Brig. Gen. William E. Starke early in the Battle of Antietam, Stafford was elevated temporarily to command of the brigade. He suffered a minor wound in his foot during the battle, but soon recovered. In the reorganization of the Army of Northern Virginia in the fall of 1862, Stafford and the 9th Louisiana regiment were reassigned to Harry T. Hays brigade, the Louisiana Tigers. Stafford served at the battles of Chancellorsville and Gettysburg in 1863. In October of that year, he was promoted to brigadier general and assigned command of the 2nd Louisiana Brigade.
The Gen. Leroy A. Stafford Camp #358 of the Sons of Confederate Veterans in Alexandria, Louisiana, is named in honor of the fallen general. The Gen. Leroy Stafford Camp #3 of the United Confederate Veterans in Shreveport, Louisiana, was named for General Stafford as well. It was formed in 1891 and existed until the last member died in 1944. In its 53 years of existence, over 250 Confederate Veterans were members. Stafford's son, David Theophilus Stafford, was later the sheriff of Rapides Parish, having served for four terms from 1888 to 1904. Many of Stafford's descendants were named after him.