1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

PHP Programmer

Discussion in 'Introductions' started by namkcits, Sep 3, 2008.

Tags:
  1. namkcits

    namkcits Newbie

    Joined:
    Sep 18, 2008
    Messages:
    2
    Likes Received:
    0
    Hello BHW,

    I'm just a humble PHP programmer looking to make some dough. I specialize in scrapers including storing data in an efficient relational data model using PHP and MySQL.

    I also program attractive, intuitive systems to interact with the data. None of that tabled ugly layout crap [the kind of stuff your competitors use ;)].

    I'll try to interact with BHW as much as possible, however I'm in my final year at college trying to earn by Bachelors so you know how that works...
     
  2. BozoClown

    BozoClown Junior Member

    Joined:
    Jan 4, 2009
    Messages:
    150
    Likes Received:
    106
    You are welcome gotekian. Looks like you have a lot to offer. There is a lot of information here for you too.
     
  3. PauloPT

    PauloPT Jr. VIP Jr. VIP Premium Member

    Joined:
    May 22, 2008
    Messages:
    344
    Likes Received:
    1,124
    Occupation:
    Rainbow hatter
    Location:
    Portugal
    It's always good to have programmers here, welcome.
     
  4. r-webb-k

    r-webb-k BANNED BANNED

    Joined:
    Dec 19, 2006
    Messages:
    373
    Likes Received:
    407
    yes it is paulopt i think i already have a project for him lol
     
  5. namkcits

    namkcits Newbie

    Joined:
    Sep 18, 2008
    Messages:
    2
    Likes Received:
    0
    Here's some more info on what I do...

    I specialize in scraping content from single websites. How it usually works is you'd show me a site, explain what content you want, how you would want it used, and I provide the database within a few days (one site had over 3.5 million records and took a week to finish collecting/parsing). I can also custom build an application to interface with the data or put the data in a format to interface with an existing application.

    A lot of content would be inefficient (slow and bulky) to store in a single table, so I go through and store the information in multiple tables (relation-izing) where applicable. For example, in the db dump below, I have stripped the states from the colleges as having the word "Alaska" repeated hundreds of times would be inefficient.

    A client of mine needed a list of all schools for a website in which students register. I found a site with information on thousands of schools and the following is a truncated version of the db dump.

    Code:
    CREATE TABLE `school` (
      `id` int(11) NOT NULL auto_increment,
      `state` int(11) NOT NULL,
      `name` varchar(255) collate latin1_general_ci NOT NULL,
      PRIMARY KEY  (`id`),
      KEY `state` (`state`)
    ) ENGINE=MyISAM  DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci AUTO_INCREMENT=7263 ;
    
    INSERT INTO `school` (`id`, `state`, `name`) VALUES
    (1, 1, 'Alabama Agricultural and Mechanical University'),
    (2, 1, 'Alabama Southern Community College'),
    (3, 1, 'Alabama State College of Barber Styling'),
    (4, 1, 'Alabama State University'),
    (5, 1, 'American College of Computer and Information Sciences'),
    (6, 1, 'American Sentinel University'),
    (7, 1, 'Andrew Jackson University'),
    (8, 1, 'Athens State University'),
    (9, 1, 'Auburn University'),
    (10, 1, 'Auburn University Main'),
    (11, 1, 'Auburn University at Montgomery'),
    (12, 1, 'Bessemer State Technical College'),
    (13, 1, 'Bevill State Community College'),
    (14, 1, 'Birmingham-Southern College'),
    (15, 1, 'Bishop State Community College'),
    (16, 1, 'Blue Cliff Career College'),
    (17, 1, 'Calhoun Community College'),
    (18, 1, 'Capps College'),
    (19, 1, 'Capps College: Foley'),
    (20, 1, 'Central Alabama Community College'),
    (21, 1, 'Chattahoochee Valley Community College'),
    ...
    (84, 1, 'University of Alabama System Office'),
    (85, 1, 'University of Alabama at Birmingham'),
    (86, 1, 'University of Alabama in Huntsville'),
    (87, 1, 'University of Mobile'),
    (88, 1, 'University of Montevallo'),
    (89, 1, 'University of North Alabama'),
    (90, 1, 'University of South Alabama'),
    (91, 1, 'University of West Alabama'),
    (92, 1, 'Virginia College'),
    (93, 1, 'Virginia College Technical'),
    (94, 1, 'Virginia College at Huntsville'),
    (95, 1, 'Virginia College at Mobile'),
    (96, 1, 'Virginia College: Huntsville'),
    (97, 1, 'Wallace State Community College at Hanceville'),
    (98, 2, 'Alaska Bible College'),
    (99, 2, 'Alaska Pacific University'),
    (100, 2, 'Alaska Vocational Technical Center');
    
    
    CREATE TABLE `state` (
      `id` smallint(5) unsigned NOT NULL auto_increment,
      `name` varchar(32) collate latin1_general_ci NOT NULL,
      `short` varchar(8) collate latin1_general_ci default NULL,
      PRIMARY KEY  (`id`)
    ) ENGINE=MyISAM  DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci AUTO_INCREMENT=52 ;
    
    INSERT INTO `state` (`id`, `name`, `short`) VALUES
    (1, 'Alabama', 'AL'),
    (2, 'Alaska', 'AK'),
    (3, 'Arizona', 'AZ'),
    (4, 'Arkansas', 'AR'),
    (5, 'California', 'CA'),
    (6, 'Colorado', 'CO'),
    (7, 'Connecticut', 'CT'),
    ...
    (48, 'Washington', 'WA'),
    (49, 'West Virginia', 'WV'),
    (50, 'Wisconsin', 'WI'),
    (51, 'Wyoming', 'WY');
    
    With the above example, all schools from a particular state can be easily obtained (SELECT * FROM school WHERE state_id = 12) and getting state information from a school can also be easily obtained (SELECT school.name AS school_name, state.* FROM school, state WHERE school.id = state.school_id). [I typed these from my head and they might contain typo's :p]

    I do have a couple databases in my repitour already. I do a lot with PHP and MySQL development, scraping is just one of them. Need a programmer? Just ask ;)

    My aim and yahoo is 'gotekian'.
     
  6. weblaffin

    weblaffin Registered Member

    Joined:
    Feb 20, 2008
    Messages:
    53
    Likes Received:
    1
    Hello gotekian,

    do you know if is posible to create a script for yahoo! news scraping content ?

    I mean full news article!

    Thank you!
     
  7. Nixte

    Nixte Regular Member

    Joined:
    Jan 29, 2008
    Messages:
    264
    Likes Received:
    114
    That would be definitely a nice feature! And then output the whole as a rss feed so my autoblogger can post it ;)

     
    Last edited: Sep 3, 2008