It's called pagination, and it's relatively easy to do actually. You can find various pagination classes and function snippets littered across the web that all do pretty much the same thing with minor variations in the coding style of how it gets done.

Some resources to check out:
Pagination - what it is and how to do it
php pagination - Google Search
Class: Pagination (pagination, paginate) - PHP Classes
-m