Your castle influences your population and kingdom's growth. It can be upgraded up to forty times, with each time resulting in an increase in your kingdom's population. The layout of your kingdom is not very important. Just keep your castle in the center to make it easier to keep your people happy and productive.