This can only be done after the events that take place in Ogre Castle late in the game. When you're done entirely with that area, return to the house of Kondo's daughter, which is just one screen west of Omi Village's outside area. Speak with Kondo's daughter to get an unlimited supply of ladders.