Avoid colliding with other vehicles. While any damage incurred doesn't affect performance, the act of hitting another vehicle or wall diminished your speed.