The different items that you can use to upgrade your field houses will rot away after different periods of time. The following materials can be used to upgrade:

Grass/ Hay: The weakest
Lumber: Average
Stone: Good
Golden Lumber: Excellent