1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Free data mining (to test my data mining service)

Discussion in 'BlackHat Lounge' started by jimBlackHat, Jan 28, 2015.

  1. jimBlackHat

    jimBlackHat Newbie

    Joined:
    Jan 14, 2015
    Messages:
    3
    Likes Received:
    1
    Hi,

    if you need web data collected, let me know. I am developing a web data mining service which needs to be tested with real projects.

    I will offer this for free.

    If you are interested, please send me the following details:
    - What website you want to crawl.
    - What data you need to extract.
    - What format do you need: XLS, CSV, XML, JSON

    Please keep in mind that I cannot accept every request due to hardware and crawler limitations.
    But I will do my best and hey it's free.

    EDIT:
    My web crawling service has a web GUI. But in the first phase of testing I will configure the crawler. I just need to know what you guys want to get crawled to get a basic idea of real world projects.
    In a later phase I need people that actually use my service on their own.

    What I have successfully crawled in the past:
    - yellowpages
    - amazon
    - web shops

    What is not (yet) working:
    - websites where you have to log in to see the content
     
    • Thanks Thanks x 1
    Last edited: Jan 29, 2015
  2. r3dr00m

    r3dr00m Newbie

    Joined:
    Jan 26, 2015
    Messages:
    8
    Likes Received:
    3
    I have few ideas, but wait, neither one of us can PM each other;)
     
  3. divok

    divok Senior Member

    Joined:
    Jul 21, 2010
    Messages:
    1,056
    Likes Received:
    644
    Location:
    .IN
    can you scrape scores from websites like soccerway?
    is it gui based , or we tell you what to scrape? more information on your service will be good .!
     
    • Thanks Thanks x 1
  4. masterbeta

    masterbeta Newbie

    Joined:
    Jan 29, 2015
    Messages:
    1
    Likes Received:
    0
    Oh no... perhaps we shall answer to each other on this threat 15 times so we can get the pm feature? lol
     
  5. r3dr00m

    r3dr00m Newbie

    Joined:
    Jan 26, 2015
    Messages:
    8
    Likes Received:
    3
    I'm pretty sure it's against BHW TOS, but getting more information about your service wouldn't be a bad thing. What exactly can you scrape? I know you want to test it, but so far what are your limitations?
     
  6. jimBlackHat

    jimBlackHat Newbie

    Joined:
    Jan 14, 2015
    Messages:
    3
    Likes Received:
    1
    I have edited my first post. You can post your requirements in this thread or you can send me an e-mail to my gmail address. name is sirjimbojones123

    divok: What exact data do you need from soccerway? I can try to scrape it.
     
  7. r3dr00m

    r3dr00m Newbie

    Joined:
    Jan 26, 2015
    Messages:
    8
    Likes Received:
    3
    Like I said. I've few ideas. Will send you an e-mail.
     
  8. jimBlackHat

    jimBlackHat Newbie

    Joined:
    Jan 14, 2015
    Messages:
    3
    Likes Received:
    1
    Sounds good. Waiting for your requirements.
     
  9. jimBlackHat

    jimBlackHat Newbie

    Joined:
    Jan 14, 2015
    Messages:
    3
    Likes Received:
    1
    To the people who sent me PM's. I cannot answer PM's yet because of my low posting count.
    Please mail me to sirjimbojones123 at gmail