Go to the Mabe village library. Use pegasus boots as soon as you walk in to ram into the bookshelf knocking down a book. Read the book and it will give you directions to the color dungeon.