Need help with...Spider/Crawling web

X

xmonkeys

Newbie

Jan 17, 2011

22

1

• Mar 19, 2013

• #1

Hello,

I am looking for someone or someway to crawl free adult sites, and post their videos on my sites. I have access to their csv/dump, but I want to create an automated way of getting them without having to continually manage them.

Does anyone know of a good spider, or crawler that can get the necessary info?


thanks.

 

S

seriousjoker

Registered Member

Feb 12, 2013

75

33

• Mar 19, 2013

• #2

I can make one for you, if you are intrested.

 

innozemec

Elite Member

Aug 19, 2011

5,404

1,815

• Mar 19, 2013

• #3

i think you will need to hire a coder to create the bot specifically to your needs

 

jazzc

Elite Member

Jan 27, 2009

2,964

14,717

• Mar 20, 2013

• #4

Wget, curl, aria2c, teleport pro

 

I

It Should Be Easy

Newbie

Mar 18, 2013

7

0

• Mar 22, 2013

• #5

Since you said "someway" I am not sure if you are a developer yourself. But the Scrapy framework in Python is great for almost any kind of crawling tasks.