Permalinks are the "permanent URLs" of your web pages. If you're just starting out with a site, you're going to want to make sure the URL structure is user and search engine friendly -- i.e. example.com/this-is-the-permanent-url/ instead of example.com/?p=12345.
Even if the site isn't brand new, you may want to change your permalink settings to a more user-friendly structure. This is more involved, however, and would require adding the necessary 301 redirects to your htaccess file (redirecting the old, ugly URLs to the new ones).