Dr. Rufus W. Bailey, a former president and professor at Austin College, had acquired an eight-acre tract on a site now part of Oakwood Cemetery and erected a house which he named "Buena Vista," but which became known as "The Steamboat House" because its unusual design evoked the image of a double-decker steamboat.  The house was a wedding gift for Bailey’s son, Frank, but Huntsville lore says the couple refused to live there.

In 1862, Bailey rented the house to Sam Houston, the-then disposed governor, who died of pneumonia there in July 26, 1863.

In November of 1863, Frank Bailey sold the house and grounds to A. C. McKeen for $4,000.

The house was sold again, this time for $1,500, on December 25, 1866 to Pleasant W. Kittrell, a Huntsville physician and former member of the legislature.  He died in the house on September 29, 1867.  His family continued to live there until January 1874, when his widow traded the property to Thomas J. Goree for a house in Madison County.  Goree, superintendent of the Texas prisons, used convict labor to remodel the front of the building and give it a Victorian style.

Goree owned the house and grounds until April 1891, when they were purchased by I. N. Smith for $2,250.  Smith in turn sold the property in August 1917 to the Lamkin Brothers, a Huntsville hardware firm; in March 1925 the property was bought by the Oakwood Cemetery Association for $3,500.

In 1928 the association sold the house to J. H. Johnson, who moved it one-half mile to North Main Street.

Houston businessman J. E. Josey bought the house for $500 in 1933 and announced plans to restore it to its original appearance.  Efforts for this project were delayed until 1936 when the house, long used for squatters and the homeless, was dismantled and moved to the museum grounds where it was formally presented to the state on March 2, 1936 - Texas Independence Day.

Restoration took less than a year and the refurbished house was opened on March 2, 1937.