Waller married Ann Winston Carr, first cousin to Patrick Henry, in Spotsylvania County, Virginia, in December 1760, and their children were: Elizabeth Waller, b. 1762; Mary Winston Waller, b. October 26, 1763, Henry County, Virginia, d. July 22, 1828, Martinsville, VA; John Waller, b. 1765, Spotsylvania, VA, d. 1842, Horsepasture, Virginia; Carr Waller, b. 1767; Ann Waller, b. 1770; George Waller, b. 1773; William Waller, b. 1775; Edmund Waller, b. 1779, d. 1817, Waller's Ford, VA. George Waller owned extensive land in the area that is now Fieldale, Virginia, along the Smith River. By 1768 he was established as a tobacco farmer and local politician, also overseeing the ford over the Smith River. The plantation he owned later became known locally as both "Waller's Ford" and the "Village of Waller". One source says: "Early black landowners in the area were descendants of slaves owned by Colonel Waller. Some surnames taken by those descendants included Finney, Morris, Stovall, Napper, Baker, Mitchell, and Waller. Another slave descendant, this one from the Hairston line, was Jordan Hairston who owned land near what is now 10th Street in Fieldale. The Smith River tributary called Jordan Creek runs through the center of the original Village of Waller and was named for Jordan Hairston."
During the Revolutionary War period Waller trained militia on a drill field near the Smith River, as Adjutant and part of "Penn's Regiment" named for Abram Penn. He was made an ensign in the regiment on September 25, 1775, in Pittsylvania County, Virginia. In March 1781 he marched a detachment of three companies drawn from 21 companies of Henry County militia 70 miles south to Hillsborough, North Carolina in support of General Edward Stevens in the Battle of Guilford Court House. He was afterwards promoted and commissioned a colonel of militia. Colonel Waller was with General George Washington at the surrender of Lord Cornwallis at Yorktown, after which he returned to his home in Henry County to live out his life as a gentleman planter. He was also an early tax commissioner for the county, and along with Robert Hairston, one of Henry County's earliest sheriffs.
After the Revolutionary War, he became more interested in local politics. The first court held in Henry County was held in January 1777, and he was one of the first justices, along with Abram Penn, Edmund Lyne and Peter Saunders.
Death and burial
He served his county and state until his death on November 18, 1814. He is buried in the Oakwood Cemetery in Martinsville, Virginia.
