Hermitage derives its name from a building that was part of the first Pickering College private school. The Hermitage was used as the school gymnasium. It somehow survived the fire that ravaged Pickering College in 1905. The school rebuilt in Newmarket and the Hermitage, considered a derelict building was then sold to E.L. Ruddy of Toronto. Ruddy restored the Hermitage turning it into a magnificent Greek Revival mansion that was used as the family summer home until 1960. Ironically, after surviving the Pickering College fire years earlier, the Hermitage would later also be destroyed by a fire. When the present day neighbourhood was developed in the late 1980s the Hermitage name was chosen to commemorate its history. Hermitage Community Park contains a replica artesian well that would be very similar to the one the Ruddy’s would have used at their Hermitage property.