1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Google is testing random element classes in their serp?

Discussion in 'Black Hat SEO' started by Knappe, Aug 23, 2016.

  1. Knappe

    Knappe Newbie

    Joined:
    Jan 27, 2015
    Messages:
    23
    Likes Received:
    2
    Since maybe one year or so i've noticed that its gotten a lot harder to use scraping tools for the Google serp in General. Google is apparently cracking down on scrapers that "abuse" their services, the latest comment from them being the new adwords launch will have restricted data access for non- and low paying users.

    I've been doing a bit of detailed scraping in the Serps (detailed being also registering the pos. no and other data in this case), and i've noticed that a lot of classes for the divs that i target in order to pinpoint the content that i actually want to scrape gets substituted after a unknown time period.

    Is it a test? Can we excpect more shit from the assholes on Big G soon? Time is due for a Anti trust?
     
  2. Wilvent

    Wilvent BANNED BANNED

    Joined:
    Aug 15, 2016
    Messages:
    15
    Likes Received:
    2
    Gender:
    Male
    Try using a syntax that doesn't target the source of websites. Ex: inrul:"something" , intext:"your text" , etc.
    You are right that it is getting harder to scrape google, especially if you can't break their captcha automatically.
     
  3. Knappe

    Knappe Newbie

    Joined:
    Jan 27, 2015
    Messages:
    23
    Likes Received:
    2
    To visit the relevant serp page i use the get data:

    https://www.google.ch/?gws_rd=ssl#q=casino+bonus&start=10
    https://www.google.ch/?gws_rd=ssl#q=Einzahlungsbonus&start=100

    So, I don't fill the formula and klick buttons, the search is already done when i visit the page. Apart from that i use random waits, random url queries to visit, proxies, clear cookies and diffirent user agents. So it should be fine.

    My problem is i Can't seem to get a method scraping lots of results (several rows in the datasheet) from the serp at the same time using Xpath du to google screwing up the classes that i target. I have not found a xpath target wich relatively can select several targets yet.