To unlock these modes in the UPDATED VERSION you must perform the actions below. They are already unlocked in the original version.

Boss Attack - Clear the Sanctuary in Story Mode.
Curly Story - Defeat Curly in Story Mode.
Nemesis challenge stage - Have the Nemesis.
Sanctuary Time Attack - Clear the Sanctuary in Story Mode.