After moving to New York, he was associated with the dry goods business of Woodward, Baldwin & Co., which did a lot of business in the South, and controlled several of the largest manufacturers in the South. In this role, he served as president of the Louisville and Nashville Railroad for four years when he resigned due to management differences with the other major stockholders. In 1884, he was appointed a member of the New York Aqueduct Commission by then Gov. Grover Cleveland, who was a close friend of his. He succeeded George W. Lane and served for four years, retiring in 1888. After the Commission, where he was instrumental in the construction of the Croton Aqueduct, he returned to railroads and with D. J. Mackey, he invested in the Evansville and Crawfordsville Railroad. He was also a director of the New-York Life Insurance Company and a trustee of the New-York Security and Trust Company. In 1894, then President Cleveland appointed him Naval Officer of the Port of New York, Baldwin replaced Theodore B. Willis, who had been appointed by President Benjamin Harrison. He served in this role until his death in 1897.
Society life
In 1892, his son, daughter, and himself were included in Ward McAllister's "Four Hundred", purported to be an index of New York's best families, published in The New York Times. He owned a cottage in Newport, known as the "Baldwin villa" which was located at the corner of Bellevue Avenue and Narragansett. The cottage was rented to Gouverneur Kortright in 1895. In 1896, he rented Mrs. A. M. King's Newport cottage on Ayrault Street for the season. He hosted Donald M. Dickinson at the home in July 1896. Baldwin was a member of the Manhattan Club, the Metropolitan Club, the Union Club, the Knickerbocker Club, the Democratic Club and Reform Clubs, the Southern Society, the Down Town Association, the South Side Sportsmen's Club, and the American Geographical Society. He was also a governor of the Newport Casino and member of the Westchester Country Club.
Personal life
Baldwin was married to Sarah "Sallie" Roman. She was the daughter, and sole heir, of James Dixon Roman, a U.S. Representative who was president and part owner of Old Hagerstown Bank. Together, Baldwin and Sallie were the parents of four children:
Susie Blow Baldwin
James Dixon Roman Baldwin, a lawyer who did not marry.
Louise Roman Baldwin, who married William Benjamin Bristow, the son of Benjamin Bristow, the 30th U.S. Treasury Secretary and 1st Solicitor General, in 1907.