Colocation is usually the cheapest but you'll want to find one close to you so you can go in and replace hardware if it fails. That can get pricey. Colocation is where you provide the server and you rent space within a data center.

If you rent a server from someone then they will handle the hardware so you don't need anything close to where you live. This is usually more expensive than colocation. With this you can get managed and unmanaged. Managed is where the people you are renting from will keep software up to date and any other management tasks that need to be done on the server. Unmanaged means you do all of that. Managed is more expensive than unmanaged.

If you don't know anything about running servers and your website absolutely needs it's own box then I'd recommend spending the extra money to get a managed server.

There is also virtual private hosting. You share a box and resources with other people but you have your own server running as a virtual machine. You are guaranteed a certain amount of resources though. This is for those that need more than share hosting provides but don't have a need for a dedicated server. I believe you can get this in managed and unmanaged as well. Cheaper than a dedicated server and sometimes colocation.

__________________
Eric