To unlock Expert Mode you must to earn all 15 medals in Nintendo 64 mode or 3DS mode.