You will eventually get the ability to Leash your Followers and have them work wherever needed. Because this costs Belief, use this technique to progress faster only it really needed. Otherwise, simply expand to get people close enough to where you want them to work.