1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

URL Scraping Tool

Discussion in 'BlackHat Lounge' started by irfanlone, Dec 4, 2016.

  1. irfanlone

    irfanlone Junior Member

    Joined:
    Aug 8, 2013
    Messages:
    120
    Likes Received:
    19
    Can anyone here recommend a good scraping tool.I used to work with Scrapebox in the past,but their URL harvestor isn't working the way it used to, in the past.I even have GScraper,but heard some bad things about it,so i doubt it might not be working.Any suggestions in this regard would be appreciated.
     
  2. bluecoder

    bluecoder Jr. VIP Jr. VIP

    Joined:
    Dec 26, 2014
    Messages:
    253
    Likes Received:
    55
    Location:
    Skpe: leadomatics
    Home Page:
    As per the market and past, scrapebox is best even am using in my daily work hope most of them too...so what your actual requirement so we can suggest based on that...
     
  3. irfanlone

    irfanlone Junior Member

    Joined:
    Aug 8, 2013
    Messages:
    120
    Likes Received:
    19
    There is some issue with URL harvestor,it only spits errors.Now don't tell me the proxies are bad or something as i have been using the same proxies in the past with Scrapebox,but less productive today.
     
  4. bluecoder

    bluecoder Jr. VIP Jr. VIP

    Joined:
    Dec 26, 2014
    Messages:
    253
    Likes Received:
    55
    Location:
    Skpe: leadomatics
    Home Page:
    Yes Google as u know they are very keen in noticing and making changes frequently...try Bing n Yahoo it will give good results...and try Google search API in engine selection...even for me sometimes I noticed Google will not give much results....
     
  5. smarty84

    smarty84 Regular Member

    Joined:
    Oct 20, 2012
    Messages:
    364
    Likes Received:
    34
    Gender:
    Male
    Hi, I also think that the scrapebox is best. I hope someone here can help you with the issue that you are facing with the scrapebox.
     
  6. irfanlone

    irfanlone Junior Member

    Joined:
    Aug 8, 2013
    Messages:
    120
    Likes Received:
    19
    How many threads should i use with 10 proxies or should i use the proxies provided by Scrapebox?
     
  7. irfanlone

    irfanlone Junior Member

    Joined:
    Aug 8, 2013
    Messages:
    120
    Likes Received:
    19
    I made the changes in the setting but still getting errors even using Yahoo and Bing search engines.However one thing i forgot to mention is that i am using site operator,but i don't think it make such a big difference.
     
  8. Skyebug77

    Skyebug77 Jr. VIP Jr. VIP

    Joined:
    Mar 22, 2012
    Messages:
    1,924
    Likes Received:
    1,353
    Occupation:
    Marketing
    Location:
    Portland,Or
    Not sure what you are trying to actually do, but you might be interested in the tools at the site in my sig. A Couple different scrapers.
     
  9. irfanlone

    irfanlone Junior Member

    Joined:
    Aug 8, 2013
    Messages:
    120
    Likes Received:
    19
    Main purpose is for URL harvesting.
     
  10. Kamrul Ahsan

    Kamrul Ahsan BANNED BANNED

    Joined:
    Nov 29, 2016
    Messages:
    4
    Likes Received:
    0
    Gender:
    Male
    scrapebox is the best mat
     
  11. bartosimpsonio

    bartosimpsonio Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 21, 2013
    Messages:
    12,025
    Likes Received:
    10,816
    Occupation:
    WHEREZ MA
    Location:
    BITCOINS AT?
    Home Page:
    Scrapebox and GScraper both do the job. If there's errors then save the messages and contact their support, they'll tell you what's wrong. Something must be wrong with your footprint or you need to update SB or GS to get the latest search engine profiles.
     
  12. Skyebug77

    Skyebug77 Jr. VIP Jr. VIP

    Joined:
    Mar 22, 2012
    Messages:
    1,924
    Likes Received:
    1,353
    Occupation:
    Marketing
    Location:
    Portland,Or
    Yes there are a few applications there that do url harvesting.
     
  13. irfanlone

    irfanlone Junior Member

    Joined:
    Aug 8, 2013
    Messages:
    120
    Likes Received:
    19
    I already send a ticket,waiting for their reply.How many proxies are you using? and how many urls are you able to scrape?.
     
  14. BackY

    BackY Senior Member

    Joined:
    Dec 28, 2012
    Messages:
    1,151
    Likes Received:
    266
    When the fuck are u going to learn to use punctation????? FFS
     
  15. extremeboy

    extremeboy Jr. VIP Jr. VIP

    Joined:
    Jul 8, 2010
    Messages:
    3,185
    Likes Received:
    667
    Occupation:
    World Best RANK Tracker SERPCloud.com
    Home Page:
    you will need better dedicated proxies and it should be fine with SB or GSc
     
  16. JustUs

    JustUs Power Member

    Joined:
    May 6, 2012
    Messages:
    626
    Likes Received:
    582
    I just checked Gscraper. Bing works, Google does not.
     
  17. irfanlone

    irfanlone Junior Member

    Joined:
    Aug 8, 2013
    Messages:
    120
    Likes Received:
    19
    Good to hear that..
     
  18. irfanlone

    irfanlone Junior Member

    Joined:
    Aug 8, 2013
    Messages:
    120
    Likes Received:
    19
    Could anyone tell me how to filter keywords, by the number of words it has.Scrapebox generates keywords that has well over 10 words,which i don't need.Kindly tell me how to do that.
     
  19. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    15,600
    Likes Received:
    9,559
    Occupation:
    PROVIDING PROXIES FOR GSA SCRAPING.
    Location:
    BHW
    Home Page:
    the only problem with gscraper is there is 0 support
    no update since march - so if it works use it, if not dont
    bother there support they reply to no1
     
  20. irfanlone

    irfanlone Junior Member

    Joined:
    Aug 8, 2013
    Messages:
    120
    Likes Received:
    19
    It is kinda funny.I deleted my ScrapeBox copy and download it again, and after updating, it seems to work fine for atleast Bing and other search engines.There must have been some corrupted file in my ScrapeBox folder.