Hi dear BHW friends, I have created an important PHP script that runs from my PC (using xampplite + localhost + FF) that scrapes G results. The script outputs hundreds of G results. To overcome the G-IP-Block for consistent automated queries and scraping, I at least need to embed the HideMyIp2009 into the script to change IPs frequently. (I hope this query makes some sense to my dear readers! ) What should I do? Or how other way should I go about it? Many Thanks! P.S.: Can't use this script from my regular website for obvious reasons. Also have searched G for the answer but I was unable to find the right answer to my specifics. Not using any APIs also.
I have dynamic IP and I also use HideMyIP with FF which has helped me overcome the G IP ban many times. I wanted to know whether I will be able to use both FF & hideMYIP with this local script (I want to make this script run through FF and not from WeBuilder, which is giving me output at present through its built-in browser). The script is running using WeBuilder which isn't able to change my dynamic IP and so it's getting the G's IP ban which stays for about 2 hours. P.S.: Can I use some sort of FF plugin to make the local script run from FF and also get HideMYIP to work successfully?
How are you displaying the results. Are you just echoing\printing them to a page? If you are then i would consider running a mySQL database and storing all your results there for easy management and viewing later on. If you go this route you can run your script several ways Call it directly: [xampp path]\php\php.exe [path to script] Use the web browser: http://localhost/[script name].php how are you interfacing with big g to get the results. curl?
Thank you very much! Actually I have client-server experience (Sybase-SQL server-Powerbuilder), but very limited PHP knowledge. I have a project partner, and we just began this PHP program 2 days ago. We got the initial program working 100% successfully although we both are complete PHP noobs. So using very-limited PHP skills: I am using fgets and dumping results on screen/text file, then to excel to manipulate things. I would have given you the entire code but my partner is against giving it as we are building this base to get into lots of other major automated things using iMacros, Excel, Google, PHP etc to build a skeleton working model, then after succeeding in what we are after, wiill get into much bigger things for personal use only. Most of the other tests are completely successful and our program is ready. If we are able to scrape what we need without the IP bans, then we are almost 100% there! All other things like excel/VBA inputs/outputs are successful after we have fed the scraped data which we were able to get successfully thorugh our basic PHP program, but incurring the G IP ban several times in the process. If we are able to get over this problem, we are really through. I wanted to use the database at a later stage after the pilot project was complete. If we are able to get the data through this PHP script without G stoppages, then all other things are easy for us.
There is another module in the project where we are using FF and HideMyIP to get up to 1,000 results effectively at a go without the IP ban, but this module requires a different scraping algorithm and we thought that local PHP would help us in this case, as there aren't any softwares or scripts available for the specific purpose. We have also created the PHP script on our own successfully, but we need the same FF-HideMyIP combo (just to get the HMIP working as we don't know how we can get the different IPs without FF and HMIP combination), and also we cannot put this script on my website which is on a shared plan... don't want others there to suffer from our testing and getting IP bans from G... So, basically I need the PHP local script to run from FF and also use the HMIP as we were able to do it successfully for the other module. But I think, the localhost nature of the script is unable to interact with HMIP... Sorry, I can't explain this part properly to you, but I hope that you'll understand!
Hi! Thanks & Rep given! Your suggestion worked for us in a big way! It was simple, but as a noob I wasn't to know it before someone explained... you did! Thanks again!
