Neural networks and scrapers

Discussion in 'General Programming Chat' started by Nick1, Dec 19, 2011.

  1. Nick1

    Nick1 Junior Member

    Joined:
    Oct 16, 2009
    Messages:
    196
    Likes Received:
    45
    Does anyone have experience with using neural networks to code scrapers and such?

    Are they worth the trouble learning for such a use? It seems like it is an interesting subject, I just want to know if it has a real practical advantage in time saved in the long run.
     
  2. Chris22

    Chris22 Regular Member

    Joined:
    Sep 29, 2010
    Messages:
    400
    Likes Received:
    1,063
    I fucked around using a neural network to help solve captchas on certain sites and read text from images, and had varying degrees of success.

    If you're using a ready made open source one to do something, ie OCR, then it's not too much of a hassle to implement.

    The real ballache comes from when you need to customize it to do something more custom and out of the ordinary.
     
    • Thanks Thanks x 1
  3. Nick1

    Nick1 Junior Member

    Joined:
    Oct 16, 2009
    Messages:
    196
    Likes Received:
    45
    Yeah, I know OCR is a pretty common use for neural networks.

    Jeff Heaton has an interesting blurb on it in his introductionary neural network book.

    "The neural network-based bot detailed below is provided with the name of a famous person. It uses this information to perform a Yahoo search and obtain information on
    the person. The bot ?reads? all of the information found on the person and attempts to determine the individual?s correct birth year."

    I just got the book, so I was wondering about the possibilities. If I could automate simple tasks, like scraping a list of suppliers and e-mailing them a request for price quote, it would be very cool.
     
  4. Chris22

    Chris22 Regular Member

    Joined:
    Sep 29, 2010
    Messages:
    400
    Likes Received:
    1,063
    That sounds amazing, I don't think I have the understanding of those types of neural networks yet to be able to make em do something useful like that though :(