The middle colonies were ruled largely by the British monarchy until William Penn was granted land by the throne and formed Pennsylvania. Voters had to be Christian, as well as anyone wishing to hold office, but the local government did not force settlers to attend church or to pay taxes to support the church, as in other colonies. Estimated Population of American Colonies 1620 to 1780 From the Beginning of the Great Migration Through the Revolutionary War. Series Z-19 U.S. Census The small colonies inside of the Middle Colonies would be the New York, New Jersey, Delaware, and Pennsylvania Colonies. In 1664 the Duke Of York founded the New York Colony because he did it from building a colony on land, which was, eventually, captured by the Dutch. Out of all of the English colonies, the settlers in the middle colonies came from the most varied backgrounds. By 1664, the city of New York best illustrated these varied backgrounds with inhabitants that included Scots, French, Dutch, Swedes, Germans, Norwegians, Irish, Poles, Portuguese, and Italians who were the forerunners of millions to come.