In Challenge Mode, beat each of the Minigames once on any difficulty. Then purchase the Secret Map from the Item Shop for 500 coins. A Warp Pipe will appear to take you to the Toy Stadium Minigame.