1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

What programming language can be used to access other websites and gather information?

Discussion in 'General Programming Chat' started by drake45, Oct 10, 2013.

  1. drake45

    drake45 Newbie

    Joined:
    Nov 24, 2012
    Messages:
    21
    Likes Received:
    1
    There are certain web apps that can access external websites and gather information.

    One example is a serp checker:


    User submits URL + Keywords to a website > The website uses this information to search google serp's and return the keywords positions > the website tells the user his rank position.


    Can such actions be accomplished with Javascript/php? If not then what language can develop such web apps which allow a website to gather and return information from external websites?
     
  2. Diplomat

    Diplomat Jr. VIP Jr. VIP Premium Member

    Joined:
    Oct 25, 2011
    Messages:
    872
    Likes Received:
    410
    Occupation:
    CEO
    PHP will work, Javascript not :)
     
  3. change

    change BANNED BANNED

    Joined:
    Sep 27, 2013
    Messages:
    54
    Likes Received:
    23
    php cUrl will do that
     
  4. neutralhatter

    neutralhatter Jr. VIP Jr. VIP Premium Member

    Joined:
    Jun 23, 2010
    Messages:
    430
    Likes Received:
    330
    This is wrong. Javascript will work fine. Many languages will, but the real question is what is the fastest way...
     
  5. DarkPixel

    DarkPixel Jr. VIP Jr. VIP Premium Member

    Joined:
    Oct 4, 2011
    Messages:
    1,328
    Likes Received:
    1,239
    Location:
    ↓↓↓↓
    Home Page:
    Python, Java, PHP, Javascript, Ajax, etc can all be used.

    In other words, almost every language will do, you just have to choose if you want the data received on the server or client side.
    I would suggest you start with PHP, since it's pretty straightforward (google PHP curl), and the online PHP community is huge.
     
  6. Gophering

    Gophering Junior Member Premium Member

    Joined:
    Mar 21, 2013
    Messages:
    115
    Likes Received:
    279
    Occupation:
    Unemployed
    Location:
    EU
    Any Turing complete language will work here.
     
    • Thanks Thanks x 1
  7. sm754

    sm754 Registered Member

    Joined:
    Mar 21, 2012
    Messages:
    93
    Likes Received:
    38
    Occupation:
    Farmer
    Location:
    Azerbaijan
    Whatever languages have good HTML parsing libraries (regex doesn't count).
    C/C++, C#, Java, PHP seem to all have decent options.
     
  8. jazzc

    jazzc Moderator Staff Member Moderator Jr. VIP

    Joined:
    Jan 27, 2009
    Messages:
    2,468
    Likes Received:
    10,147
    Scraping in Postscript sounds fun :eek:
     
    • Thanks Thanks x 1
  9. Gophering

    Gophering Junior Member Premium Member

    Joined:
    Mar 21, 2013
    Messages:
    115
    Likes Received:
    279
    Occupation:
    Unemployed
    Location:
    EU
    Pfff, imagine scraping in brainfuck.
     
    • Thanks Thanks x 1
  10. IceHD

    IceHD Jr. VIP Jr. VIP Premium Member

    Joined:
    May 8, 2013
    Messages:
    486
    Likes Received:
    198
    Occupation:
    SEO, SEM, HTML5&CSS3 Coder
    Location:
    Not Romania
    I'm using SimplePie, it's PHP library that read xml feeds and uses cache/mysql to show the data. I also have a Pyton script that work greate, but I don't have the skills to enhace it.
    I suggest using Python or C#, because of the seed.
     
  11. apoorv

    apoorv Regular Member

    Joined:
    Aug 31, 2011
    Messages:
    301
    Likes Received:
    62
    Haha!

    --

    To the OP, for that specific example, pretty much anything would work: Ruby or Python using Mechanize, PHP using cURL, or CasperJS if you are going to use JS (though the speeds are slow).