Why you shouldn't rely entirely on an IP

Wildhoney · 11-02-2007, 08:54 PM

Filtering users by their IP address may seem like a good idea when you're beginning PHP, even coders that should know better have been caught giving too much trust to an IP. I remember the first PHP project I did, entitled rMetal - which was a website dedicated to various bands from the metal genre in my younger days, I didn't know about sessions and I didn't care to read up, either. As a consequence my system's login was constructed by if your IP address is the same as the one in the database, you're OK. Boy, was I in for a shock!

Today I ensure that no trust is placed on the user's IP address. It cannot be trusted. For once, where I live, the ISP uses web proxy servers and so everybody in my local area - stretching 5+ fairly large sized towns, all have the same IP address when they visit a website, whereas they have unique IPs for other services. This is partly for security and partly to save the tight ISP some bandwidth - they simply load in pages from the web proxy unless the target page has been modified.

Take the following PHP code into consideration and place it at the forefront of your mind:

PHP Code:


			
var_dump($_SERVER['REMOTE_ADDR']);

I'm sure the majority of you recognise what this code does. It will echo out my IP address with its data type and length. When I run the script it gives me:

PHP Code:


			
string(11) "127.0.0.1"

Note: I'm not really 127.0.0.1, stops any of you little rats getting my real IP, though!

A conservative estimate would be that 5,000 other households have this identical IP address that I have, and thus relying on the IP address is clearly a big no no! "What's the solution?" you might ask. Well...

...The honest truth is there is no solution. We can never be 100% certain that the member visiting our website is truly unique. Sure we have cookies, IP addresses and sessions (related to cookies), but these are not reliable. Perhaps the best way to identify and tag a visitor is by hashing some information together and constructing them a fingerprint identity.

If you remember from this article, every single header sent from the client to the server is optional. Put another way, the client's browser decides whether or not to set it. The HTTP protocol only expects the page request. It can extract the client's IP address from the TCP/IP packet so although this can be spoofed, if the client wants a response, which the request will need to complete TCP's 3-way-handshake, then spoofing it would be a monumental exercise in futility.

The HTTP_USER_AGENT is a HTTP parameter that is set in the HTTP header and extracted by PHP and placed into the $_SERVER predefined array. This parameter contains information on the user's browser and operating system. Mine being:

Quote:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9

Naturally, many people will share the same HTTP_USER_AGENT as me, but we cut the chances of a duplicate if we concatenate the IP address to it and then hash it. Something like the following would give us more possibilities:

PHP Code:


			
$szFingerprint = md5($_SERVER['HTTP_USER_AGENT'] . $_SERVER['REMOTE_ADDR']);

This gives me the following fingerprint identity:

Quote:

a54aa1ff349280792a8ef780697f06ae

See what we've done? We've increased the chances of creating a duplicate fingerprint by many times. Now for the same fingerprint to be generated, the user would have to:

Have the same version of Firefox
Have the same version of Windows
Have the same IP address
Have the same browser language

There is of course a good chance that if me and another user share point 3 then we will also share point 4, but even still, you can see our chances have increased significantly since naively relying on the IP address alone.

Sadly, as previously touched upon, the header parameters are options and so if HTTP_USER_AGENT is empty then we'll be merely hashing our IP address on its own which is a pointless exercise. Unfortunately, there's very little we can do here other than set cookies to rat them out. The good news is that that almost every browser, and certainly every new browser, sets a HTTP_USER_AGENT and so if identifying a user is crucial, preventing users from accessing the website who do not have HTTP_USER_AGENT set is a path you may consider:

PHP Code:


			
if(!isset($_SERVER['HTTP_USER_AGENT']))

{

    die('You must have HTTP_USER_AGENT set.');

}



$szFingerprint = md5($_SERVER['HTTP_USER_AGENT'] . $_SERVER['REMOTE_ADDR']);

As you can see, there are both advantages and disadvantages. I load up the same website and my fingerprint is now different because I am using Internet Explorer as opposed to Mozilla Firefox:

Quote:

Internet Explorer: 36f0679b96c335e5f8694a9b8f957f61
Mozilla Firefox: a54aa1ff349280792a8ef780697f06ae

They are clearly very different from one another, and no matter which browser I use, I will be seen as a different person even though I have the same IP address as I did 5 seconds ago. To conclude, everybody is unique on the Internet if they wish to be, this is where sessions stepped in to introduce some law and order in a lawless and orderless environment. Denying individuals without a HTTP_USER_AGENT would more often than not be a lot less than the individuals with Javascript disabled, which is around 6% according to W3Schools' browser statistics.

bluesaga · 11-07-2007, 10:10 AM

Nice article, what do you recommend then instead? Sessions are not a clear indicator of a unique visitor either, how can we track unique visitors correctly if a large number of them will not have unique IP's?

Karl · 11-07-2007, 12:38 PM

I'm hoping someone can put me wrong on this one, but I'm fairly sure there isn't an alternative. There's only so much data we can get our hands on, unfortunately there just isn't enough data to determine if one user is truly valid.

bluesaga · 11-07-2007, 12:58 PM

I have read some discussion about HTTP_X_FORWARDED_FOR before, so after a little browsing i found this on the php.net website

PHP Code:


			
if ($_SERVER["HTTP_X_FORWARDED_FOR"]) {
   if ($_SERVER["HTTP_CLIENT_IP"]) {
    $proxy = $_SERVER["HTTP_CLIENT_IP"];
  } else {
    $proxy = $_SERVER["REMOTE_ADDR"];
  }
  $ip = $_SERVER["HTTP_X_FORWARDED_FOR"];
} else {
  if ($_SERVER["HTTP_CLIENT_IP"]) {
    $ip = $_SERVER["HTTP_CLIENT_IP"];
  } else {
    $ip = $_SERVER["REMOTE_ADDR"];
  }
}

Seems like that will give a more accurate IP (granted not a definite one)

Edit: By the looks of things HTTP_X_FORWARDED_FOR can contain a comma delimited list (when a user has been through multiple proxies) so it might be worth exploding that aswell.

Wildhoney · 11-07-2007, 01:06 PM

Yea, I covered that in this article but unfortunately it is an optional parameter. There really is no way to detect a unique visitor. As Karl mentioned, we only have limited data, the user agent is a good one but again it's optional.

Nor · 11-07-2007, 01:10 PM

Nice Articles, but I don't think foot prints are the best bet for certain things, say your building a script to see how many time this user visited? If they switch between browsers, pretty inefficient, specially if there banned.

bluesaga · 11-07-2007, 01:12 PM

Yea, you can create a pretty good hash of a user though from all the information combined, but to be honest you should never go by any information that is sent by the user. Including potential headers, ip's etc, majority of things like that can be spoofed.

Its best to create a session, and simply have the user login again if they don't have the cookie.

Nor · 11-07-2007, 01:17 PM

Quote:

Originally Posted by bluesaga

Yea, you can create a pretty good hash of a user though from all the information combined, but to be honest you should never go by any information that is sent by the user. Including potential headers, ip's etc, majority of things like that can be spoofed.

Its best to create a session, and simply have the user login again if they don't have the cookie.

Only way I'd use a fingerprint is to check if this user has changed browsers and ip's more then one then he would have to verify his account on next page view and change his password.(Good idea for a project if anyone likes it)

Karl · 11-07-2007, 01:39 PM

Nor, I think you're misunderstanding the point of the fingerprint in this case. This fingerprint would be useful where you want the try and recognize the same IP, for example, if you were creating a rating system you wouldn't want a user to able to submit a rating, then switch from FF to IE and submit again.

Nor · 11-07-2007, 01:52 PM

Quote:

Originally Posted by Karl

Nor, I think you're misunderstanding the point of the fingerprint in this case. This fingerprint would be useful where you want the try and recognize the same IP, for example, if you were creating a rating system you wouldn't want a user to able to submit a rating, then switch from FF to IE and submit again.

wtf...my point exactly, if you switch you can resubmit again..reread what your saying.

Quote:

Internet Explorer: 36f0679b96c335e5f8694a9b8f957f61
Mozilla Firefox: a54aa1ff349280792a8ef780697f06ae

right if your checking based on a finger print your hash will be either or, if you used firefox, just go to IE or Opera and resubmit pretty pointless..

Karl · 11-07-2007, 02:26 PM

Ah yes, my mistake. I gave the wrong example, the point behind this type of fingerprint is to protect against session hijacking. This method would be especially helpful to stop, for example, one person attepting to hijack another persons session by masking their IP. Assuming they're using different browsers, the fingerprint could detect this type of attack.

And by the way, it doesn't hurt to be polite to people. After all, if I'm going to be hit with a blunt attitude for trying to help, then why should I even bother in the first place?

Salathe · 11-07-2007, 02:31 PM

There is no sure fire way of detecting and tracking a unique visitor, that's just the nature of how things on the web are done. I think we're all in agreement with that. However one particular issue that I could see cropping up here is something like when a certain big ISP who shall remain nameless rotates their IPs sometimes on a page-by-page basis. Obviously an extreme event but in that instance, the hash would be different and the visitor would be deemed "unique". Then again, that example shows quite clearly why you shouldn't rely on an IP. :)

Nor · 11-07-2007, 02:57 PM

Quote:

Then again, that example shows quite clearly why you shouldn't rely on an IP.:)

Or fingerprint :).....

Tanax · 11-07-2007, 03:24 PM

So, how would you be able to get as most accurate info on what IP the user is on? And, could you combine the X_FORWARDED_FOR with the HTTP_USER_AGENT to a "best-we-can-get" -function which would try to get a "fingerprint" of the user?

Wildhoney · 11-07-2007, 04:15 PM

Well, as Salathe rightly pointed out, some ISPs rotate the IP on every page, and as Salathe is not man enough to say it himself (grins), I'll say it myself and face the consequences! As AOL rotates the IP on every page, identifying unique visitors is something that can never be determined.

However, as Bluesaga mentioned, and I have verified this with another source, then the comma-separated X_FORWARDED_FOR would be extremely useful. What I couldn't locate is the ordering of the comma+space separated X_FORWARDED_FOR and so if anybody knows, please let me know. Let's assume that I am correct in assuming that the original IP will be at the beginning of the X_FORWARDED_FOR attribute. The following code would work:

PHP Code:


			
function getUserIP()
{
    if(isset($_SERVER['X_FORWARDED_FOR']))
    {
        if(strpos($_SERVER['X_FORWARDED_FOR'], ',') === false)
        {
            return $_SERVER['X_FORWARDED_FOR'];
        }
        
        return trim(reset(explode(',', $_SERVER['X_FORWARDED_FOR'])));
    }
    
    return $_SERVER['REMOTE_ADDR'];
}

This would check to see if there are any commas in X_FORWARDED_FOR, and if there are, return the first IP. If not then return X_FORWARDED_FOR as-is. If there is no X_FORWARDED_FOR set at all then return the address from REMOTE_ADDR. As pointed out in another article of mine, the two popular web-based proxies do not set the X_FORWARDED_FOR attribute as they are poorly coded and do not understand web-standards.

Let's mimic the X_FORWARDED_FOR and add the following code that calls our getUserIP function:

PHP Code:


			
$_SERVER['X_FORWARDED_FOR'] = '127.0.0.1, 192.168.0.1';

echo 'User IP: ' . getUserIP();

This would rightfully return 127.0.0.1. If, however, my assumption is incorrect and the first proxy is at the end of the X_FORWARDED_FOR attribute, which I highly doubt, then replace the reset function with end. Though knowing the correct order would be much appreciated!

Salathe · 11-07-2007, 04:27 PM

According to Wikipedia (X-Forwarded-For) the accepted order is to have "the left-most being the farthest downstream client, and each successive proxy that passed the request adding the IP address where it received the request from." So your assumption on the IP order is correct.

P.S. I did say that there is no way of identifying unique visitors and I was even man enough to start off my post with that point. But it's a valid enough point to be hammered home by repeating it. :)

Tanax · 11-08-2007, 08:59 PM

Quote:

Originally Posted by Wildhoney

Well, as Salathe rightly pointed out, some ISPs rotate the IP on every page, and as Salathe is not man enough to say it himself (grins), I'll say it myself and face the consequences! As AOL rotates the IP on every page, identifying unique visitors is something that can never be determined.

However, as Bluesaga mentioned, and I have verified this with another source, then the comma-separated X_FORWARDED_FOR would be extremely useful. What I couldn't locate is the ordering of the comma+space separated X_FORWARDED_FOR and so if anybody knows, please let me know. Let's assume that I am correct in assuming that the original IP will be at the beginning of the X_FORWARDED_FOR attribute. The following code would work:

PHP Code:


			
function getUserIP()

{

    if(isset($_SERVER['X_FORWARDED_FOR']))

    {

        if(strpos($_SERVER['X_FORWARDED_FOR'], ',') === false)

        {

            return $_SERVER['X_FORWARDED_FOR'];

        }

        

        return trim(reset(explode(',', $_SERVER['X_FORWARDED_FOR'])));

    }

    

    return $_SERVER['REMOTE_ADDR'];

}

This would check to see if there are any commas in X_FORWARDED_FOR, and if there are, return the first IP. If not then return X_FORWARDED_FOR as-is. If there is no X_FORWARDED_FOR set at all then return the address from REMOTE_ADDR. As pointed out in another article of mine, the two popular web-based proxies do not set the X_FORWARDED_FOR attribute as they are poorly coded and do not understand web-standards.

Let's mimic the X_FORWARDED_FOR and add the following code that calls our getUserIP function:

PHP Code:


			
$_SERVER['X_FORWARDED_FOR'] = '127.0.0.1, 192.168.0.1';



echo 'User IP: ' . getUserIP();

This would rightfully return 127.0.0.1. If, however, my assumption is incorrect and the first proxy is at the end of the X_FORWARDED_FOR attribute, which I highly doubt, then replace the reset function with end. Though knowing the correct order would be much appreciated!

I see, that looks really nice!!

But can't you use for example HTTP_USER_AGENT in the function aswell?
Or wouldn't that work?

Karl · 11-08-2007, 09:22 PM

Could use HTTP_USER_AGENT together with the function call, if you wish:

PHP Code:


			
echo 'User Fingerprint: ' . md5($_SERVER['HTTP_USER_AGENT'] . getUserIP());

Tanax · 11-09-2007, 06:04 PM

Okey, so how's this?

PHP Code:


			
        public function getFingerprint() {
            
            if(isset($_SERVER['HTTP_USER_AGENT'])) {
                
                if(isset($_SERVER['X_FORWARDED_FOR'])) {
                        
                    if(strpos($_SERVER['X_FORWARDED_FOR'], ',') === false) {
                
                        return md5($_SERVER['HTTP_USER_AGENT'] . $_SERVER['X_FORWARDED_FOR']);
            
                    }
            
                    return md5($_SERVER['HTTP_USER_AGENT'] . trim(reset(explode(',', $_SERVER['X_FORWARDED_FOR']))));
        
                }
        
                return md5($_SERVER['HTTP_USER_AGENT'] . $_SERVER['REMOTE_ADDR']);     
                
            }
            
            return false;                        
            
        }

Wildhoney · 11-13-2007, 03:15 PM

Looks good to me!