Buildings that generate you resources over time, such as your Quarries, Farms, and Lumbermills, should always be upgrade.