The directory on your computer for Minecraft saved worlds and settings differs depending on what OS you're running.

On Windows: C:\Users\username\AppData\Roaming\.minecraft
Take note that the AppData directory is hidden, so be sure to show hidden files from within Folder Options.

On Mac OS: /Users/username/Library/Application Support/minecraft
On Linux: /home/username/.minecraft

The "Saves" folder is what you want to back up.