John Proctor was an early settler of Ipswich and received a grant of land along what is now Elm Street. Proctor and his wife Elizabeth moved to Salem in 1666. They were both charged and imprisoned in the Salem Witch Trials. Elizabeth was pregnant and her sentence was never executed, but John Proctor was hung. The Boston minister Cotton Mather came to watch and congratulated the crowd of spectators for their diligence.
Samuel S. Baker, active in real estate, bought the lot at 16 Elm Street and built this house in 1835. He sold it to shoemaker William S. Tozer (1804-1860) in 1841. He was married to Adeline Mears. Their daughter Susan Story Tozer died in 1840 at 4 years of age.
Their son John M. Tozer died in Newport News of “Congestive Chills” (pneumonia) while serving in the Civil War, and his name is inscribed on the Civil War Monument at Meetinghouse Green. The other son Joshua shares a tombstone with John at the Old Burying Ground and is assumed to have also died in the Civil War. Thomas Franklin Waters records that William H. Tozer was one of over 300 Ipswich volunteers in the Civil War and at the end of the 19th Century he owned the Daniel Lummus house on High Street. There was once a Tozer’s Grocery on Hammatt Street and the family name continues in Ipswich today.
The house consists of three combined structures. The four bay addition may be an older structure
which was moved t o the site and incorporated into the original Baker house. There is a fine central staircase, and the full-height basement features a chamfered summer beam, suggesting that it may have been re-used from a first period home. This house has the same address, (16 Elm Street) as the Choate house that was dismantled and is now a major display at the Smithsonian. That house sat where the Police parking lot is now. Enhanced 911 renumbering of houses gave this house, formerly #17, that same address.
Elm Street originally had a small hill. The soil was used to fill the marshy land between Washington Street and High Street that was filled in 1870 to create Central Street. Although Elm Street is anchored at the river end by the Old Town Hall and at the County Street end by Benjamin Grant house, Elm Street and most of South Main Street are not part of the surrounding East End, Meeting House Green, or South Green National Historic Districts and are not included in the National Register of Historical Places. It is included in the Ipswich Architectural Preservation District.
- Waters, T.F., Ipswich i n the Massachusetts Bay Colony, Vol . I. , p. 450