TalkPHP - View Single Post - Need Help: Getting Meta Description, Keywords and URL

Wildhoney · 06-17-2009, 11:59 PM

Something to be aware of, but officially you're not allowed to trawl Google's results. I have done myself, and you can do, but any system that draws attention to the fact you're crawling Google, and they will ban the IP address.

Bing, on the other hand, offer an API, and so do Yahoo, so if you're not too fussed about the search engine, then you can use either of those two, and avoid Google. Google actually prompt you with a Captcha if you hammer their server.

06-17-2009, 11:59 PM	#4 (permalink)
Wildhoney La Vida es Sueño Join Date: Sep 2007 Location: Oldham Posts: 2,280 Thanks: 90	Something to be aware of, but officially you're not allowed to trawl Google's results. I have done myself, and you can do, but any system that draws attention to the fact you're crawling Google, and they will ban the IP address. Bing, on the other hand, offer an API, and so do Yahoo, so if you're not too fussed about the search engine, then you can use either of those two, and avoid Google. Google actually prompt you with a Captcha if you hammer their server. __________________ The man who comes back through the Door in the Wall will never be quite the same as the man who went out.