When the Civil War broke out, Gregg was commissioned a captain in the 3rd U.S. Cavalry on May 14, 1861. He then joined the volunteer army in June as a captain in the 5th Pennsylvania Reserve Regiment. He was elected colonel of the regiment on June 20, 1861 and resigned the following day to accept a commission in the regular army as a captain in the 6th U.S. Cavalry. On November 14, 1862, Gregg was promoted to colonel of the 16th Pennsylvania Cavalry. He then commanded many different cavalry brigades in the various reorganizations of the Army of the Potomac. He led the Third Brigade at the battles of Chancellorsville and Gettysburg in a division commanded by his cousin David Gregg. In October 1863, he earned another brevet to lieutenant colonel in the regular army for the Battle of Sulphur Springs. He was wounded at the Second Battle of Deep Bottom and won a brevet again on October 7, 1864 to full colonel in the regular army. On December 12, 1864, President Abraham Lincoln nominated Gregg for appointment to the grade of brevet brigadier general of volunteers to rank from August 1, 1864, for gallant and meritorious service in the engagement and defenses of Richmond on the Brock Turnpike and at the Battle of Trevilian Station and the United States Senate confirmed the appointment on February 20, 1865. Gregg briefly was in command of the Cavalry Corps, Army of the Potomac from February 10, 1865 to February 24, 1865. On April 7, 1865, Gregg was slightly wounded at the Battle of Sayler's Creek, captured the next day north of Farmville, Virginia, and was released two days later. Gregg was mustered out of the volunteer service on August 11, 1865. On January 13, 1866, President Andrew Johnson nominated Gregg for appointment to the grade of brevet major general of volunteers to rank from March 13, 1865 and the U.S. Senate confirmed the appointment on March 12, 1866. On July 17, 1866, President Johnson nominated Gregg for appointment to the grade of brevet brigadier general in the regular army to rank from March 13, 1865 and the U.S. Senate confirmed the nomination on July 23, 1866.
Postbellum
After the end of hostilities Gregg remained in the Army. He was named colonel of the 8th U.S. Cavalry on July 28, 1866, a position his cousin David McMurtrie Gregg had desired. He then reported for duty at Camp Whipple in the Arizona Territory. He led a series of expeditions against Indians into the Mojave Desert. He was sent to the New Mexico Territory, where he commanded Fort Union from 1870 to 1873, and led efforts in that region to pursue and subdue the Apache. In 1872, he led a reconnaissance expedition to survey and map the Texas Panhandle. Gregg retired from active service on April 2, 1879. He died in Washington, D.C. on January 26, 1892, and is buried in Arlington National Cemetery in Virginia.