In colonial days, this village was known as Stirling, for the British earl who was the proprietor of Long Island. Its port was called Winter Harbor because it was accessible to ships even when Town Harbor was frozen over. Greenport's main street was laid out in 1827 and Southold's only village was incorporated in 1838. Greenport was famous as a whaling center and for its fishing and ship building industries.

 As the terminus for the Long Island Railroad, Greenport celebrated the arrival of the first train on July 25, 1844. The headquarters of the Stirling Historical Society is in a circa 1831 whaling captain's home, open 1-4, July 4- Labor Day, Wed. Sat., & Sun. There is also a new Maritime Museum located on the dock by the Shelter Island Ferry, right next to the Railroad Museum.