You can unlock different theme music for the jukebox by collecting trinkets.

Music	How to unlock
Pipe Dream	Collect twenty trinkets.
Popular Potpourri	Collect eighteen trinkets.
Positive Force	Collect eight trinkets.
Potential For Anything	Collect twelve trinkets.
Predestined Fate	Collect sixteen trinkets.
Presenting VVVVVV	Collect ten trinkets.
Pressure Cooker	Collect fourteen trinkets.
Pushing Onwards	Collect five trinkets.