Sweet was born Benjamin Jeffery Sweet on April 24, 1832, in New York City. He later moved to Chilton, Wisconsin. Sweet died on January 1, 1874, and is buried at Rosehill Cemetery in Chicago, Illinois. He had two noteworthy daughters: Ada Celeste Sweet was a social reformer and pioneer for women serving in government office; Winifred Bonfils was a journalist and humanitarian.
Soon after the outbreak of the American Civil War, on July 16, 1861, Sweet joined the 6th Wisconsin Volunteer Infantry Regiment of the Union Army and was given the rank of major. On September 17, 1861, he was promoted to lieutenant colonel. On September 5, 1862, Sweet was promoted to colonel and given command of the 21st Wisconsin Volunteer Infantry Regiment. During the Battle of Perryville, in Kentucky, despite being sick from malaria, he led his men in combat until he was seriously wounded in his right arm. The wound ultimately left Sweet paralyzed in that arm for the rest of his life. Sweet resigned his commission on September 8, 1863. On September 25, 1863, Sweet was appointed a colonel in the Veteran Reserve Corps and assigned to the garrison at the Union Army prisoner-of-war camp for Confederate States Army soldiers at Camp Douglas, Chicago, Illinois. Following recuperation from his injury, Sweet was given command of Camp Douglas after Brigadier General William W. Orme resigned on May 2, 1864. At Camp Douglas, Sweet oversaw what some observers have called the unnecessarily harsh and cruel treatment of Confederate prisoners, but which other observers have seen as no less so than Confederate treatment of Union prisoners. This notwithstanding, Sweet would soon be lauded for "discovering" and "thwarting" a bogus plot by spies for the Confederacy to liberate Confederate prisoners of war and attack Chicago on the eve of the 1864 United States presidential election. Sweet's efforts to prevent the conspirators from achieving their objective earned him the thanks of the United States Department of War. On December 12, 1864, President Lincoln awarded Sweet the rank of brevet brigadier general of volunteers to rank from December 20, 1864, and the U.S. Senate confirmed the award on February 14, 1865. Sweet resigned from the army on September 19, 1865. He died from a sudden illness in 1874.