Fetching facebook-friends

Tanax · 07-27-2011, 05:03 PM

Whoa, long time I was here!
And long time I've actually done any PHP coding.

In any case, I've seen these apps on Facebook that lets you see who defriended you. Unfortunately these apps were removed, I'm guessing Facebook doesn't want you to keep track of who's defriending you since it's something "negative" and Facebook's all about positive.. right?

Anyways.
In order to use the Facebook SDK-PHP API I need an APP ID .. I don't want to register an application for something that I will solely use on my local machine. So how would I go about fetching everyone from my friends-list?

Is there a "guest" APP ID I can use from my local machine? I was thinking about somehow fetching the HTML content of the friends-page and (again) "somehow" strip everything and only save the friends name, userid, etc.

Any suggestions or ideas on how to proceed to be able to create this?

codefreek · 07-27-2011, 10:00 PM

Welcome Back Tanax,

The simplest way that, my brain can come up with at this
moment is use curl or something like that & as you said get
the friend list & then i would take the names and generate
a id for, individual names, save it up & then you can count
the names by id and search etc,

then compare by name id etc and let's say id
32 is not on updated list run check on name for
id 32 and then print out lets say id 32 = Jane doe,
well Jane doe has defriended you.
(because she ain't on the list anymore).

-cf

Tanax · 07-27-2011, 10:23 PM

Quote:

Originally Posted by codefreek

Welcome Back Tanax,

The simplest way that, my brain can come up with at this
moment is use curl or something like that & as you said get
the friend list & then i would take the names and generate
a id for, individual names, save it up & then you can count
the names by id and search etc,

then compare by name id etc and let's say id
32 is not on updated list run check on name for
id 32 and then print out lets say id 32 = Jane doe,
well Jane doe has defriended you.
(because she ain't on the list anymore).

-cf

Hey Codefreek and thanks for the reply!
Yes thank you, I already have a system to store and manage the actual comparing of the friendslist. Regarding the id, I don't have to generate one, they already have a facebook id, for instance mine is: "562191521" -> http://www.facebook.com/profile.php?id=562191521

The thing I need help with is how to actually fetch the friends. How exactly would you suggest using cURL? :)

Tanax · 07-27-2011, 10:26 PM

Or rather, the cURL part wouldn't be that hard. I just googled it and while I'll still have to read up on it, it looks fairly straight-forward.

However, how would I do the actual stripping-part to get only the friends?
I suck at regex :/

Tanax · 07-28-2011, 12:26 AM

Okay, so I've managed to come a bit on the way.

First snag I hit was that when using cURL from localhost it didn't recognize that I was logged in and returned a loginpage. I've managed to get it working and logging in aswell as redirecting to the friendslist(Account->Edit Friends): http://www.facebook.com/friends/?everyone&ref=tn

The problem now is that it doesn't list everyone. It only lists "Recently Interacted". How to I get cURL to send so that it selects "All friends" from the dropdown-list?

The regex is still an issue.
I got this so far:

php Code:

preg_match_all( '@profile\.php\?id\=(.*?)\">(.*?)<@i', $line, $matches );

$line is result from cURL.
However it brings back some weird results. Perhaps someone can doublecheck the regex for me?

Tanax · 07-28-2011, 03:03 PM

Okay, I made some changes and it works GREAT now to fetch the names and only the names of your friends. Prints them out in an easy way. Still have to make the database and that.. but for now it seems like the matching of links and all that fuzz is complete.

HOWEVER!
I still need some help with cURL to get to the page I want.
Like I explained in my previous post, I need to somehow via cURL select an item from a dropdown-list. How would I do that?

And then, after I've selected that, I need to be able to go to the "next" page since the friends-list is paginated. How would I do that?

Here's the script so far:

PHP Code:


			
<?php

include_once('simple_html_dom.php');

$user_id    = '';
$user_email = "";
$user_pass  = "";

$fp = fopen("example.html", "w");
$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, 'https://login.facebook.com/login.php?&next=http://www.facebook.com/friends/?everyone&ref=tn');
curl_setopt($ch, CURLOPT_POSTFIELDS,'email='.urlencode($user_email).'&pass='.urlencode($user_pass).'&login=Login');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_exec($ch);

// So we can view the facebook-page
include('example.html');

$html = file_get_html("example.html");
$count = 0;
$found = 0;

foreach( $html->find('a') as $link )
{
    
    $count++;
    if( strpos( $link, 'profile.php?id=' ) && !strpos( $link, $user_id ) && !strpos( $link->innertext, 'img' ) )
    {
        
        $found++;
        echo 'Link #' . $found . ': ' . $link->innertext;
        echo '<br />';
        
    }
    
}

echo $count . ' links found where ' . $found . ' links matched profile-links';

To test it, remember to:
1) Download simple_html_dom from http://simplehtmldom.sourceforge.net/
2) Create an empty example.html
3) Create an empty cookie.txt

codefreek · 07-28-2011, 04:08 PM

Quote:

And then, after I've selected that, I need to be able to go to the "next" page since the friends-list is paginated. How would I do that?

you need to check how to url changes when you change a page,
and then tell curl the new string for new page.
if you know what i mean

check this

check this also.

Tanax · 07-28-2011, 08:31 PM

Quote:

Originally Posted by codefreek

you need to check how to url changes when you change a page,
and then tell curl the new string for new page.
if you know what i mean

check this

check this also.

Unfortunately it doesn't change at all. I'm suspecting they are either using POST to change the page or JS/AJAX to load another page.

Tried searching for how cURL handles javascript but no luck.

Tanax · 09-02-2011, 11:24 AM

I actually solved this by using the mobile site instead. It uses no(or less) javascript so it's easier to get the friends from there. Script is fully functional and I've loaded all my friends into a local database.