Some of today's websites put us back to 1994 while loading. This is due to some ignorance of the technical behavior of current browser engines and the fact that nearly any of these sites uses social plugins written in Javascript or wrong jQuery usage.
Nowadays it's very easy for everyone to build a website based on a CMS. If you take a look on Wordpress, you'll find a huge variety of full-featured templates. They're free and you don't have to get into code for yourself.
Finally you add some JS-widgets for Facebook-button, twitter-button or other things to your site. Now if a visitor goes to your site and it takes about 10secs before he sees anything. If Facebook API is not available (which is quite often) it even could take longer! And if someone has a smartphone with UMTS he won't wait 1 minute to load your crappy website. But how to figure out what's being wrong and what can you do better? I'll now try to give you a short overview of the answers:
1. Delayed loading of javascripts
First thing are javascripts. They're blocking your website content until they're loaded if not placed right in HTML. Nearly every site today uses the jQuery-Framework, Google Analytics, the Tweet-Button or Facebook plugins. All these scripts are up to 200kb in size and often they're linked in the head of the webseite because you've read that in some docs out there in the web.
But you can easily imagine how a webbrowser works: First the head of the website is loaded completely to get informations about the website and all links will be loaded and interpreted(!) within that step. Until now the real content of the website is not loaded at all.
If I have a head part which contains files that in summary are about 1.5mb and I now try to reduce it to 40kb (which indeed is possible), I'd see that the site which took about 30secs to load before now loads in 2secs! But how is that done without abandon the javascripts? So we can load javascript everywhere in our HTML markup and due to the fact that javascript is used on clientside we can load it after loading the whole content. So put the js links directly after your content but before the closing html-element and you'll have a huge difference in your loading behavior.
2. Reduce the number of linked files
Many people split their css files into several files to keep the survey of codes. Sometimes you have plugins that are loading their own css files additionally. That's not best practice!
In general the rule is to combine as many files as possible in as few files as possible. For small site you now should have one single css file to be loaded initially. And of course you shouldn't load other files via @imports inside the css files as this is very slow. But the best practice is not best if it doesn't fit to your project. So if you need to target different devices it makes sense to load a main-css which targets all media and additionally another file per device. This can easily be done with css media-queries.
The same issue is with images: every single image generates a http-request and this takes time. To reduce that you can use CSS-Sprites if you have a couple of images with rollover or sth. like that to improve the speed of your website.
3. Well formatted and valid HTML-structures
A valid HTML-Code is a good condition for a fast interpretation by the browser. If there are errors here the browser has to fix them first and that takes time. You should take care of unnessecary elements (wrapper/empty-tags). summary: more code = more load-time, more errors = more load-time.
4. Optimize your server / configure your CMS right
I'm sorry but I can't give too specific answers here as they rely on the server environment you use and not all are available on all servers. So if you have a good provider he lets you do cool things to reduce the server load.
Every CMS offers some Caching methods. They should be used in any setup if possible. If you need a uncached call maybe for dynamic content this can be done but should be the exception. MODX CMS by the way offers the correct methods for that by structure. However Wordpress needs plugins for the caching methods. And you should minify your javascipts and CSS files for a production environment.
If you have a cool provider you even can use a PHP accellerator via htaccess like gzip or PHPAccellerator. This increases load-time on server-side considerably.
5. Javascript lazy-loading
The last thing is about Javascript. You can use javascripts that can load other scripts in a specific order and they're doing it asynchronouous so there's a huge difference to commonly used loading behavior of the website. Some scripts that do it well are: control.js, head.js and Apache ANT Builder. Please consider to know how to use them correctly as they could break up speed improvements if not!