The key is hidden in the ruins on the left side of the land, after you have built the stone bridge to cross the river. Across the river is one set of ruins, then to the left and south of that is a large briar patch that needs to be removed. Once that is done, cut down the large dead tree to get across the chasm to the last area. The ruins of the observatory are in this area. After your workers clear that area, the Libra Key will appear.