1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scrape any website

Discussion in 'Black Hat SEO Tools' started by WebScraping, Mar 20, 2013.

  1. WebScraping

    WebScraping Newbie

    Joined:
    Jun 19, 2012
    Messages:
    31
    Likes Received:
    14
    Hi,

    I got some free time and I'm very quick in building spider for scraping website so I'm offering for free for a short period of time this service. Pm me here or add on skype: skypetomislav
     
    • Thanks Thanks x 2
  2. Scritty

    Scritty Elite Member Premium Member

    Joined:
    May 1, 2010
    Messages:
    2,807
    Likes Received:
    4,496
    Occupation:
    Affiliate Marketer
    Location:
    UK
    Home Page:
    Are we talking about a bot here!
    I'm looking for a keyword based content scraping tool that grabs from any number of sources for content curation and such like. AS many sources and scrapes as possible.

    Let me know if this is what you are on about.

    Scritty
     
  3. WebScraping

    WebScraping Newbie

    Joined:
    Jun 19, 2012
    Messages:
    31
    Likes Received:
    14
    Sure I can write such a bot too, pm me for details or add on skype: skypetomislav
     
  4. edgematch

    edgematch Elite Member

    Joined:
    May 24, 2010
    Messages:
    2,538
    Likes Received:
    1,949
    Occupation:
    You can never guess!
    Location:
    :noitacoL
    I hope you can distribute what you have developed as a gift to this community.
     
  5. ultra.marine

    ultra.marine Registered Member

    Joined:
    Oct 5, 2012
    Messages:
    80
    Likes Received:
    101
    Location:
    Macedonia
    even though I don't need your services, thanks :D
     
  6. moonlighsunligh

    moonlighsunligh Jr. VIP Jr. VIP Premium Member

    Joined:
    May 1, 2010
    Messages:
    1,623
    Likes Received:
    218
    I do not need a tool, but if you are an experienced scrapper and use Python I would like to ask a few questions.
     
    Last edited: Mar 21, 2013
  7. newman21

    newman21 Newbie

    Joined:
    Mar 21, 2013
    Messages:
    22
    Likes Received:
    3
    You are added to skype, thanks
     
  8. WebScraping

    WebScraping Newbie

    Joined:
    Jun 19, 2012
    Messages:
    31
    Likes Received:
    14
    Most of the time I use python , well how experienced am I relay don't know, but for now I didn't see a website which can't be scraped :)
     
  9. saber210

    saber210 Supreme Member

    Joined:
    Sep 1, 2011
    Messages:
    1,358
    Likes Received:
    500
    Location:
    -
    Is it free?

    Can I have a pinterest image scrapper? :)
     
  10. moonlighsunligh

    moonlighsunligh Jr. VIP Jr. VIP Premium Member

    Joined:
    May 1, 2010
    Messages:
    1,623
    Likes Received:
    218
    Cool. There isn't a site that cannot be scraped (except probably javascript ones, where a click is needed), but it is the trick to scrape in the most efficient way.

    Also I would like to make python look pretty much the same as VB.net so I could add easy GUI.

    So here are few questions (answer to at third one):
    1) What IDE do you use (if any)
    2) What Gui do you use?
    3) What library for scraping?
     
    Last edited: Mar 22, 2013
  11. trader3920

    trader3920 Newbie

    Joined:
    Mar 22, 2013
    Messages:
    15
    Likes Received:
    0
    Occupation:
    Trader
    Location:
    Boston, MA
    Home Page:
    I could use this
     
  12. Nuker

    Nuker Regular Member

    Joined:
    Jan 21, 2011
    Messages:
    352
    Likes Received:
    32
    i'm also into scrapping, but using PHP
     
  13. WebScraping

    WebScraping Newbie

    Joined:
    Jun 19, 2012
    Messages:
    31
    Likes Received:
    14
    There are many tricks that work with javascript so scraping this kind of sites is not relay a problem any more. If I need something to be build fast and to deliver data in short time ( few hours) I use scrapy framework for python. As by defaullt this framework doesn't support javascript with a few modifications it can be setup to support.

    For all of my programming I use eclipse,GUI-CLI. Ah library , some basic library's You will need to get You started would be the parsers library(lxml,beautiful soup,HtmlXpathSelector...), something that implements the browser request(mechanize,selenium webdriver..) and ofc librarys for storing the data to csv, xml, excel.

    Hope it helps, I have posted a short tutorial in prev post, but there are many online tutorials that explain it 100 times better then I did. But again You can allways add me on skype and ask any question You like. I relay like a good challenge when it comes to scraping :)
     
    • Thanks Thanks x 1