Shoot hostage: With a gun, press X to get a hostage, then press Square.
Break hostage's neck: While unarmed, press X to get a hostage, then press Square.