When the Prince and Farah attempt to cross the drawbridge you have lowered, many sand creatures will appear to battle you. Just run past them on the left and into the open doorway with the switch inside. They will not follow, and you wll not have to fight them.