For advanced spell casters, when you are pulling the control stick in the direction you want to pull, wiggle it around that point. You will find that your Carpe Retractum spell is quicker at pulling. The best place to try this is the Prefect's Bathroom.