Based on how many Toads you have collected via Toad Rally, your castle's size will increase.

Level	How to unlock
Castle Level 02	Play Toad Rally at least once.
Castle Level 03	Attain at least 25 toads in Toad Rally.
Castle Level 04	Attain at least 100 toads in Toad Rally.
Castle Level 05	Attain at least 200 toads in Toad Rally.
Castle Level 06	Attain at least 400 toads in Toad Rally.
Castle Level 07	Attain at least 600 toads in Toad Rally.
Castle Level 08	Attain at least 800 toads in Toad Rally.
Castle Level 09	Attain at least 1,000 toads in Toad Rally.
Castle Level 10	Attain at least 1,200 toads in Toad Rally.
Castle Level 11	Attain at least 1,400 toads in Toad Rally.
Castle Level 12	Attain at least 1,600 toads in Toad Rally.
Castle Level 13	Attain at least 1,800 toads in Toad Rally.
Castle Level 14	Attain at least 2,000 toads in Toad Rally.
Castle Level 15	Attain at least 2,200 toads in Toad Rally.
Castle Level 16	Attain at least 2,400 toads in Toad Rally.
Castle Level 17	Attain at least 2,600 toads in Toad Rally.
Castle Level 18	Attain at least 2,800 toads in Toad Rally.
Castle Level 19	Attain at least 3,000 toads in Toad Rally.
Castle Level 20	Attain at least 4,000 toads in Toad Rally.