1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

This can be modded for ScrapeBox Alternative?

Discussion in 'BlackHat Lounge' started by Capo Dei Capi, Oct 17, 2016.

  1. Capo Dei Capi

    Capo Dei Capi BANNED BANNED

    Joined:
    Oct 23, 2014
    Messages:
    754
    Likes Received:
    1,732
    If I learn python, I would be able to modify the script to use it as a alternative to scrapebox that is as good or better? Or would one have to totally build the scrapebox alternative from scratch.

    https://github.com/NikolaiT/GoogleScraper
     
    Last edited: Oct 17, 2016
  2. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,724
    Likes Received:
    1,993
    Gender:
    Male
    Home Page:
    I mean ... why?

    I use a lot of python scripts and its not nearly as good as delphi for this sort of thing. Scrapebox has 6 years experience at being awesome in this specific niche, its $67 and they spent over a year rewriting over 1.1 million lines of code to make it what it is.

    They answer support 365 days a year and they work on Scrapebox 365 days a year. For $67 why on earth would you even contemplate writing your own thing to do whats already been done? I mean you could go work at McDonalds flipping burgers and earn the $67 faster then you could get a python script to even hobble along and get 1/10th of the production of even scraping google.

    I have a video of Scrapebox scraping over 1 million urls per Minute from google. Thats the raw power of scrapebox.

    If you need a spoon to eat lunch, are you going to go dig a mine in your back yard, get some raw material, smelt it and pour it into a mold for a spoon, wait for it to cool, all so you can eat lunch? Or would you just go to the store and buy a spoon? Thats about the equivalent of what your asking.

    So again I ask why?
     
    • Thanks Thanks x 4
  3. Lunaaa

    Lunaaa Jr. VIP Jr. VIP

    Joined:
    Jan 18, 2016
    Messages:
    2,415
    Likes Received:
    1,244
    Gender:
    Male
    Location:
    Canada
    You should definitely learn python. In trying to right now. And maybe do that just for the experience. But Scrapebox is amazing. There's no point in remaking it. Better to make other programs for your own use
     
  4. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,724
    Likes Received:
    1,993
    Gender:
    Male
    Home Page:
    Yes I agree with you here, learn python. I learned it and I have a ton of scripts running augmenting what scrapebox does. But I only build a script to fill a gap between what data scrapebox outputs and how I need that data in some other program or part of my process etc.. So in other words, I use it to augment things scrapebox can't do, not reinvent the wheel.
     
    • Thanks Thanks x 1
  5. bartosimpsonio

    bartosimpsonio Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 21, 2013
    Messages:
    12,027
    Likes Received:
    10,817
    Occupation:
    WHEREZ MA
    Location:
    BITCOINS AT?
    Home Page:
    It's well documented, but I haven't looked at the code.

    Of course you can change the whole thing to suit your needs. You can rewrite Scrapebox in Python if you want, but the advantage isn't really recreating something that already exists, but to create your own custom stuff with it.

    Python can talk to any database or spreadsheet program. It gets fun when you start interlinking your tools.
     
  6. Capo Dei Capi

    Capo Dei Capi BANNED BANNED

    Joined:
    Oct 23, 2014
    Messages:
    754
    Likes Received:
    1,732
    There's such a freaking cult on here when it comes to Scrapebox. Maybe I just want something that runs on linux rather than a windows based program.

    In that video you did, you just said dedicated server You never mentioned the specs of it, so it does look like you are hyping up scrapebox.
     
  7. Joseph Lich

    Joseph Lich BANNED BANNED

    Joined:
    Nov 25, 2015
    Messages:
    402
    Likes Received:
    79
    It's a popularization and improvement relationship. As software publisher the former is more important. Obivous it can't suit for very very advanced user's need, so also the other softwares. I want to learn Python too but haven't time. Goolge likes Python, hackers like Python.
     
  8. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,724
    Likes Received:
    1,993
    Gender:
    Male
    Home Page:
    Scrapebox already works is all Im saying. Im a business man. Why would I waste my time and money to rebuild something that runs on linux when I could just get a windows machine and then dump the results in dropbox and do whatever else I wanted on linux.

    If you feel the need to reinvent scrapebox no one is stopping you, but you are the one who came to ask, and so we are giving you the feedback. If you don't like it, thats fine.

    The server is a $90 dedicated server from solidseovps. Its fairly low in specs, their BX200. When scrapebox rewrote V2 from scratch probably half the effort went into optimizing the harvester. It could have run on an even slower machine, the machine isn't the limiting factor for scraping google.

    Googles pages are already optimized and while clunky code can eat resources, scrapebox is ultra optimized in the harvester department, the proxies are the key in that video. Having that sort of supply of proxies that are not google banned is what lets it go so fast. So sure there is hype I guess, but I prefer to call it excitement, I mean I think getting a million urls per minute is pretty fantastic. :)

    The proxy package is no longer sold, but cost $750. At any rate I still harvest a lot from google and bing today, but to be honest I have a dozen servers, and I still couldn't even deal with a million urls per minute. I mean if it harvested 60 million urls per hour 1.4 billion urls per day. I can't process 1.4 billion urls per day so the video was to show capability, not to say that someone would accutually sustain it. I mean my scraping is fully automated. I load in weeks or months worth of keywords and walk away and the scraping runs 24/7 so you just don't need that much in function. But none the less Ive not seen anything as powerful as scrapebox to date, and when I say powerful I don't just mean pure power I mean power as in the ability to function efficiently and diversify in a way that allows me to make money.
     
  9. Peter Ngo

    Peter Ngo Jr. VIP Jr. VIP

    Joined:
    Apr 23, 2013
    Messages:
    2,037
    Likes Received:
    1,636
    Occupation:
    I browse BHW for a living
    Location:
    The Internet
    Is there any specific function you are looking for that scrapebox can't do?

    Gscraper used to be a big competitor of scrapebox in term of scraping until they give fuck all about their customers.

    Scrapebox 2.0 scraping speed is pretty crazy also, it all comes down to quality proxies, if you have good proxies, it is easy for scrapebox to hit 1mil URLs per minute as mentioned by @loopline
     
    Last edited: Oct 19, 2016
  10. PHPInjected

    PHPInjected Elite Member

    Joined:
    Apr 25, 2014
    Messages:
    2,144
    Likes Received:
    1,873
    Occupation:
    100% Unique Content Writer
    Location:
    Overriding Methods
    Home Page:
    First, you're not gonna make something 'better' than scrapebox for its purposes. It does what it's supposed to do and has so many addons, that coding your own is essentially pointless. What's not pointless is learning python or any other programming language. You can make your own software based on scrapebox if you want, it'll help you learn pyhton/c#/java, but whatever you use it wouldn't be as stable as scrapebox.
     
  11. Google Prince

    Google Prince Jr. VIP Jr. VIP

    Joined:
    Dec 24, 2015
    Messages:
    158
    Likes Received:
    89
    Location:
    Google's Search Engine
    Yes learn python as it is a great introduction to coding in general and has a pretty simple syntax that will help you understand other languages easily once you learn enough.

    But for the sake of god reserve the millions of lines you will be writing to re-create such a powerful program as Scrapebox & use it for another program that can be beneficial like a paypal monkey generator....

    Now I would pay you way more then a one time fee of $67 for that!!!!!
     
  12. Capo Dei Capi

    Capo Dei Capi BANNED BANNED

    Joined:
    Oct 23, 2014
    Messages:
    754
    Likes Received:
    1,732
    Scrapebox is such a freaking cult on here. Any competitor or open source alternative is deemed as heresy.

    I'm not asking about price or if it will take a long time, I'm asking if what I posted can be modded to work as good or better than scrapebox, obviously since it is open source people can make contributions compared to scapebox which is closed source.
     
  13. Joseph Lich

    Joseph Lich BANNED BANNED

    Joined:
    Nov 25, 2015
    Messages:
    402
    Likes Received:
    79
    How many contributions will satisfy you, is this considered? Without enough quantity, the project will abort.
     
  14. towelfox

    towelfox Junior Member

    Joined:
    Jan 11, 2012
    Messages:
    173
    Likes Received:
    55
    I think something that works on Linux would be nice. Like others have said you won't dislodge scrapebox in the near future but if you can produce a workable tool and learn along the way then it's a win-win.

    If you're serious about doing it I'd look at building a daemon with a local web interface. I haven't looked at the posted source so I'm just talking about something that I think would be useful. I'm a paid up scrapebox user btw but I'd still consider alternatives.