You first need to speak to the guy who gives you the crystal (outside the Goblin Fortress). He looks like he is digging on the outside of a mountain. Be sure the crystal is on the ground first.

Don't pick up the crystal, and instead talk to the goblin who rallies the troops. Tell the goblins to search for treasure. Most of the time, one of them will grab the magic crystal for you. As long as the digging team fetches the crystal for you, it will remain available (if a bit chancy) for you to gain.