[CHRISTMAS GIFT] Simple Google Scraper

[bimbimo]

kokoloko75, thx bro! for ur usefull tool!

There is one important item Miss! Can u add plz checkbox, for saving "full url" from google. For example, i search "inurl:********.php?pid= site:*.ua" and i wanna get urls like "site.ua/page.php?pid=34&lang=1","site.ua/page.php?pid=1278".

WE(security guards ) rly need this Plz add this

Thx.

 

[kokoloko75]

kokoloko75, thx bro! for ur usefull tool!

There is one important item Miss! Can u add plz checkbox, for saving "full url" from google. For example, i search "inurl:********.php?pid= site:*.ua" and i wanna get urls like "site.ua/page.php?pid=34&lang=1","site.ua/page.php?pid=1278".

WE(security guards ) rly need this Plz add this

Thx.

Understood nothing, sorry

EDIT : What you ask is a basic feature, present since the version... 1.0.

Beny

 

[royserpa]

merry christmas and happy new year 2013!!
(too early, i know) but ftw

 

[bimbimo]

Understood nothing, sorry

EDIT : What you ask is a basic feature, present since the version... 1.0.

Beny

sry for my albanian english.

but... look...
keywords:

inurl:order.php?id= site:*.ua

result:
***.ua/order.php
***.ua/order.php
***.ua/order.php

BUT NEEDED THIS:
***.ua/order.php?id=3
***.ua/order.php?id=1
***.ua/order.php?id=371

Need full link, with parametrs... Make it plz.....

 

[kokoloko75]

sry for my albanian english.

but... look...
keywords:

inurl:order.php?id= site:*.ua

result:
***.ua/order.php
***.ua/order.php
***.ua/order.php

BUT NEEDED THIS:
***.ua/order.php?id=3
***.ua/order.php?id=1
***.ua/order.php?id=371

Need full link, with parametrs... Make it plz.....

Ah, I understood
It's a bug due to Google changes.
Before, the software scraped full links (otherwise it's useless, you're right).

I'll fix that soon.

Beny

 

[bimbimo]

Ah, I understood
It's a bug due to Google changes.
Before, the software scraped full links (otherwise it's useless, you're right).

I'll fix that soon.

Beny

thx bro! we are waiting

 

[kokoloko75]

Update (1.4.1) :
- Bug fix : Google changes

I changed the engine scraping process.
Now, it should be bulletproof for a lot of future Google changes (or anyway easy to update).

Download (unchanged) :

http://bit.ly/SimpleGoogleScraper

Beny

 

[bimbimo]

Ur best!
But now one more thing...

need to exlude noneeded url. regular expression cant help when there are keywords with diff parametrs.

try to scrap with keyword: inurl:"order.php?id=" site:*.ua
u will get urls also with no parametrs and even url "schema d.o.t org s.l.a.s.h WebPage".

seems smthng going wrong
thx.

 

[kokoloko75]

Ur best!
But now one more thing...

need to exlude noneeded url. regular expression cant help when there are keywords with diff parametrs.

try to scrap with keyword: inurl:"order.php?id=" site:*.ua
u will get urls also with no parametrs and even url "schema d.o.t org s.l.a.s.h WebPage".

seems smthng going wrong
thx.

You can use the function "Remove URLs Not Containing" with as filter : order.php?id

Beny

 

[haleluo]

Really thank you, I am newbie,
Can you please tell me how to use it?

 

[templaries]

 

[who_was_phone]

Thanks it is very nice

 

[bimbimo]

You can use the function "Remove URLs Not Containing" with as filter : order.php?id

Beny

try to read FULL text.
i said

need to exlude noneeded url. regular expression cant help when there are keywords with diff parametrs.

for example, i have 1k keywords
part of them:
inurl:List.php?CatID= site:*.ua,
inurl:listcategoriesandproducts.php?idCategory= site:*.ua,
inurl:modline.php?id= site:*.ua,
inurl:myaccount.php?catid= site:*.ua,
inurl:news.php?id= site:*.ua,
inurlrder.php?BookID= site:*.ua,
inurlrder.php?id= site:*.ua,

so how u will use ur "Remove URLs Not Containing"?? ha???
thats why i ask to make clear parse.

 

[kokoloko75]

try to read FULL text.
i said

...

so how u will use ur "Remove URLs Not Containing"?? ha???
thats why i ask to make clear parse.

Haaa ?!
I'll try to improve the parsing engine.

Thanks for your feedback.

Beny

 

[bimbimo]

Sry Beny =)
thought it was other man. didnt read nick
thats why little "agression" :1:

 

[zenglihunter]

It's really good tool！Thks

 

[jpgaga]

You are a genius. Thanks.

 

[tiuz89]

Thanks. This is a really great tool.
But I have one problem. I have 10 working proxies, but it seems as if the program just scrapes without proxies.
When I google myself from the local ip i get the google captcha (IP blocked), when I google with proxy i get no captcha.

I'm using version 1.4.1

I think it would be helpfull if there would be something like a small debug-box where u see what the programm is currently trying (scraping "keyword" with ip 127.0.0.1).

I made a changelog for you. cos i had to search through all posts, maybe this helps other people to know what the software is able to do.

Update (1.4.1) :

- Bug fix : Google changes

Update (Version 1.4) :

- Feature added : Ability to load URLs from a file
- Feature added : Remove entries not containing...
- Minor bugs fixed
- Stability and performance improved...

Update (Version 1.3) :

- Addon : PageRank checker
- Addon : Google Results Number Scraper
- Addon : Regular Expression Extractor
- Feature added : Remove entries containing...
- Feature added : Support all Google locations
- Bug fix : Regex extractor no results
- Bug fix : latest Google update
- Bug fix : Google Results Number Scraper no results
- Minor bugs fixed
- Stability and performance improved...

Update (Version 1.2.1) :

- Win XP Button fix

Update (Version 1.2) :

- Google Results Number Scraper (scrape the number of results of a keyword)
- Regular Expression Extractor (extract any element of a page)
- Several improvements...

Update V1.1 :

- Proxy rotation
- Google period (anytime or past 24h)
- Choose number pages to parse (1-100)
- Improved performances...

Features 1.0:

- Fast!
- Full threaded
- Support connection via a proxy
- Support delay between queries
- Support advanced search terms (inurl, intitle, site, ...)
- Intelligent scraper engine to avoid Google ban
- Portable application (single executable)
- Quick start help file included (and tooltips on fields/buttons)

 

[kokoloko75]

Thank you Tiuz for the changelog

The debug box is a good idea but it's a bit too much work, and for the moment I'm barely busy.
But I keep the idea for future.

What I plan to add in the next version will be :
- Feature added : Updates checker
- Feature added : Google ban detection (warning)
- Addon : Proxy harvester
- Addon : Google proxy checker
- Addon : Links extractor (in/out)
- Bug fix : Remove Selected
- Bug fix : Scraping multiple pages
- Bug fix : Scraping with proxies

Beny

 

[randyya]

Great tool Beny,, it works great for me...and i give my first rep on BHW to you..