James Montgomery was born to James and Mary Baldwin Montgomery in Austinburg, Ashtabula County, Ohio, on December 22, 1814. He migrated to Kentucky in 1837 with his parents and eventually taught school there. He married, but his first wife died shortly after the wedding, so he married again to Clarinda Evans. They moved to Pike County, Missouri, in 1852, and then to Jackson County and finally Bates County while awaiting the organization of Kansas for settlement. In 1854 Montgomery purchased land near present-day Mound City, Kansas, where he became a leader of local Free-state men and was a fervent abolitionist. In 1857 he organized and commanded a "Self-Protective Company", using it to order pro-slavery settlers out of the region. Conflict with other pro-slavery elements led territorial governor James W. Denver to dispatch U.S. Army soldiers in to restore order. Montgomery at times cooperated with the abolitionist John Brown and considered a raid to rescue Brown after his capture in Virginia, but snow in Pennsylvania upset his plan.
On July 24, 1861, Montgomery was commissioned as colonel of the 3rd Kansas Infantry of U.S. Senator James H. Lane's Kansas brigade, with Montgomery as second-in-command of the brigade. Discipline was lacking under Montgomery, and both the 3rd, 4th, and 5th Kansas would be consolidated into the 10th Kansas Infantry in April 1862. Lane's Kansas brigade was notorious for its Jayhawker-style raids into Missouri at the start of the war, particularly the Sacking of Osceola. Noted historian Albert Castel describes Montgomery as "a sincere, if unscrupulous, antislavery zealot." Montgomery was authorized to raise a regiment of African-American infantry in January 1863 that would become the 2nd South Carolina. Throughout 1863 and part of 1864, Montgomery practiced his brand of warfare in South Carolina, Georgia, and Florida. In June 1863, Montgomery commanded a brigade, including his own 2nd South Carolina and the 3rd Rhode Island Heavy Artillery, in operations along the coast resembling his earlier Jayhawk raids. The most famous of his operations was the Raid at Combahee Ferry in which 800 slaves were freed with the help of Harriet Tubman. Montgomery led a raid on the coastal town of Darien, Georgia, which he ordered looted and burned even though it was not defended and had not offered any resistance. Colonel Robert Gould Shaw, commander of the all-black 54th Massachusetts Volunteers condemned the action, and in a private letter gave Montgomery's reason for burning the town as "that the Southerners must be made to feel that this was a real war, and that they were to be swept away by the hand of God, like the Jews of old." Montgomery stated to Shaw, "We are outlawed, and therefore not bound by the rules of regular warfare." Montgomery's raids were part of a Union strategy to damage the Confederate states' ability to supply food and materials for their war effort. Some held that this strategy would in the end result in the loss of fewer lives, and a shorter and less protracted struggle, by driving the Confederacy to quickly surrender. Montgomery commanded a brigade in the Battle of Olustee, where Union forces were pushed from Florida. In September 1864 he resigned his commission and returned to Kansas. He ended his military career as colonel of the 6th Kansas State Militia, active in October of that year during Confederate General Sterling Price's raid.