When James Pinchot decided to build his summer home - Grey Towers - in 1886 he choose Milford Pennsylvania as the location. This was a natural choice since his grandfather has come to Milford from France in 1816, purchased 400 acres of land and opened a general store and lumber business. James had grown up in the area and it held many fond memories for him With Richard Morris Hunt as his architect he spent $44,000 to build a 41-room vacation home constructed of local products by local craftsmen.