If you want a larger population in your kingdom, you'll need to find lost souls int he forest and give them coins. Also, make sure that you don't destroy camps. It's easy to accidentally do that by cutting down trees that are located along the edges of a camp, so make sure you leave those standing or you'll lose camps and the potential population growth they would have enabled.