Whittaker was born into slavery on the Chesnut Plantation in Camden, South Carolina. He studied privately with Richard Greener, the first African American to graduate from Harvard college. Whittaker later attended the University of South Carolina, then a freedmen's school. He was appointed to the United StatesMilitary Academy at West Point in 1876 after receiving an appointment from South Carolina Representative Solomon L. Hoge. For most of his time at West Point, he was the only black cadet, and he was ostracized by his white peers. On the morning of April 5, 1880, he was found tied to his bed, unconscious, bleeding, and bruised. His hands and face had been cut by a razor, and burned pages from his Bible were strewn about his room. Whittaker told administrators that he had been attacked by three fellow cadets, but his account of the morning was not believed. West Point administrators said that he had fabricated the attack to win sympathy. Initially, Whittaker was held by a court of inquiry, where he was defended by Martin I. Townsend and his old friend, Richard Greener, and finally granted a court-martial. After more than a year of nationally publicized hearings, Whittaker was found guilty in an 1881 court martial and expelled from West Point. The prosecuting attorney was West Point Judge Advocate Major Asa Bird Gardiner, later a Sachem of Tammany Hall in New York and disgraced New York District Attorney, who blatantly talked of the "inferior" and "superior" races and commented that "Negroes are noted for their ability to sham and feign." His defense was led by Daniel Henry Chamberlain assisted by Greener. Though the verdict was overturned in 1883 by President Chester A. Arthur, West Point reinstated the expulsion on the grounds that Whittaker had failed an exam. In his later life, Whittaker was a teacher, lawyer, high school principal in Oklahoma City, and psychology professor in South Carolina. He died in Orangeburg, South Carolina in 1931. Whittaker was buried at Orangeburg Cemetery. His sons, Johnson Whittaker Jr. and Miller Whittaker, both served as Army officers in World War I. In addition, a grandson joined the all-black Tuskegee Airmen in World War II and a great-grandson served as a first lieutenant in the Vietnam-era Army.
In the 1970s, a book about Whittaker by John Marszalek, a historian at Mississippi State University, drew attention to his case. In 1994, a television movie based on the book aired, which generated momentum for the movement to award Whittaker a posthumous commission. On July 25, 1995, President Bill Clinton awarded the commission to Whittaker's heirs, saying, "We cannot undo history. But today, finally, we can pay tribute to a great American and we can acknowledge a great injustice."
is a 1994 TV movie about the case. Matter of Honor, a stage play by Michael Chepiga retelling Whittaker's story while at West Point, was produced at the Pasadena Playhouse in Pasadena, California, in September 2007.