Educated at Bartlett's Academy near his hometown, Baldwin enrolled in Bethany College (Bethany, WV) in 1848 and graduated from there with a B.A. in 1852. He perused collegiate positions in Missouri in Platte City (1853) and Savannah (1855) and helped found the Missouri State Teachers Association in 1856, becoming its Vice President. The following year he enrolled at Lancaster County Normal School in his native Pennsylvania to receive training specific preparing teachers for the classroom.

Baldwin opened the Indiana Normal in 1858, the state’s first Normal School and the fifth in the United States.

From mid-1863 to early-1864 Captain Baldwin served in the Indiana Infantry during the American Civil War.

In 1867 Baldwin founded the North Missouri Normal School and Commercial College in Kirksville, Missouri (now Truman State University). In 1870 the school was designated as the first public teaching college in Missouri and Baldwin served as its first president.

He resigned the position in 1881 to assume leadership of Sam Houston Normal Institute. With a need for additional space, the Austin College Building was renovated to add a third floor during his tenure and the Main Building became the second structure on campus.

Baldwin received an honorary LL.D. from Bethany College in 1891. That same year he left Huntsville to accept the chair of the Pedagogy Department at the University of Texas at Austin. He retired as Professor Emeritus in 1897 and died two years later.

To honor Baldwin, the Baldwin House was renamed in his honor, as was the 1918 Education Building shortly before it was demolished. A stained glass window was presented by the class of 1899 for placement in the Memorial Hall of Old Main.