How to get radom proxies from an simple html page and place them in cURL for parsing!

soma56 · 06-18-2010, 02:35 AM

Well, I've taken a crash course on cURL and learned how to successfully parse through proxies. I can strip html as well. Not bad considering I thought PHP was a cough syrup medicine a month ago. Now I'd like to rotate those proxies. Let's assume I have can get a list of proxies on an html page that are separated by a <br /> tag. How would I go about going there and grabbing a random proxy from the list?

I'm assuming I'll have to use either cURL or file_get_contents. Hopefully the example below will give you a better understanding of what I'm trying to do:

Code:

$proxypage = file_get_contents('myproxylist.html');

$proxy = "An ARRAY of the proxylist?";

$rand_proxy = ("randomly grab from ARRAY?", $proxy);


$curl_handle=curl_init();
curl_setopt($curl_handle,CURLOPT_URL, $url);
curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,2);
curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);
curl_setopt($curl_handle,CURLOPT_PROXY, "$rand_proxy");
curl_setopt($curl_handle, CURLOPT_TIMEOUT, 100);
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);

I guess the next question would be, as I'll be using proxies with this program over and over again within a short time - would I have to download, create and array and choose a random proxy every single time or is there a more efficient way?