Ground was broken for the Men's Gymnasium in November, 1860, and the building was opened with appropriate exercises on Saturday, March 30, 1861. It was built by the "Gymnasium Association." It was located in Tappan Square, northwest of Tappan hall, and was a one-story frame building, about 75 by 25 feet, with a higher central portion and two wings. It was removed in 1867 to provide a location for Society Hall.