Go south in Toad Town until you get to the area with the Li'l Oink Farm. Go left immediately after you enter this area (there is a narrow passageway that's not fenced off). Hit the tree with your hammer and a pipe will appear on the right. You can now enter the playroom, but need either a Silver and/or Gold card to play the games.