Ok so I am learning PHP and mySql at the moment, and I had this idea how to put it to use.

http://what.online-casinotest.com/blackhatworld.php

My idea was based on the site wn.com, but its smaller scale.
Basically what it does:
Someone enters a keyword->
It checks in the DB if the keyword has been entered before->
If so, user gets redirected to the ready site.
If not, The script pulls:

Wikipedia article
ezine Article
3 Youtube videos
3 Yahoo! Answers Q&As
And a flickr image

related to the Keyword entered
after that all that is packed up into a new file and the user gets send there.
In the sidebar of the pages there are 3 other random sites pulled from the DB and interlinked.
If you enter a new keyword on the site you landed on, its the same again.
No since those pages are actual .php files they can be indexed by the BIG G.
I know its all duplicate, but i think if it gets big enough (lets say 100.000 pages) there will be plenty of random hits to it.
This is just a beta stage on a slow server so be patient. it might take 10 seconds if you enter a KW that has not been entered before...
But if you enter it again, shit is instant:P

ToDo:
Better Design
Adsense
Sitemap (Not to hard, just have it loop through the DB and echo all the files)
index.php
some tweaking on the script (image fails 1 out of 10 times, wiki fails 5 out of 10 times)


http://what.online-casinotest.com/blackhatworld.php


Please Guys!
Let me know what you think, and what i should change.
I hope you like it

Server down?

nope
just got a domain + hosting
so took it off of the subdomain sucker

http://whattoknowabout.net/indexx.php
Few notes:
Index.php is not done yet, so you gotta use the indexx for now:P
i chose the domain name since i figured i would get a ton of long tail hits once it gets going (what to know about indians, what to know about faceremoval) and so on. 100.000 keywords or so.
The design of the inner pages is almost final, but might undergo some more changes.
sitemap is located at: http://whattoknowabout.net/sitemap/sitemap.php and it automatically pulls all current pages ready to crawl for google.
Please dont click my ads to death as there are no privacy policy and stuff in place
tell me what you think:P

Not a bad idea. Somewhat slow and some errors visible. That template is awful and you need to work on your ad placement.

I would try and include some more text sources too, rather than just the usual suspects.

Are you multithreading your fetches in the background?

Thanks:P
I know its slow, but once you enter the keyword again (or someone else did before) its instant. I kind of liked the simple template and i figured the placement wasn't 100% (or even 10%:P)
Im working on getting more sources (forums, twitter, news, amazon/shopping.com)
No multithreading as im new to php and have no clue on how to do so...
The image scraping is the weak spot right now as it fails way to much...
working on the google API but the pictures turn out really small...

Have a look at the curl_multi family of functions.

flickr is always a bitch to work with and is notoriously unreliable.

The basic structure of the template is OK but the implementation doesn't look very professional to my eye, it looks like spam. You need to get your ads front and centre and above the fold more. If you read up about adsense hotspots you will see that putting your vertical block on the left will probably get your a better CTR.

I would put your image links to other pages at the bottom where they are less likely to be clicked.

Congrats on getting it up and running though!

hey i like it, it looks like a good idea, just spend some time building it up,tweaking tweaking and once again, tweaking...

mh im not using curl at all at the moment but i will have a look at it.
Damn there is a lot to learn i was so happy i had it figured out as far as i am but i know that its WAY to slow to be good right now.
Im really planning on kicking out flickr. the pictures are really unrelated to. you dont even get a straight shot of any product or so...
Aight ill put them down, and try to pump the add-placement.
looking at the template now... i dont like it anymore
but yea it was kind of more a test of my not-existing skills but then i liked it too much to not put it online.

Wow
Thats nice
Workin my ass off learning cUrl to multithread the request ( i needed multithread inside multithread too )
i even made a 2 page construckt where the first does something, saves it into a .txt file and posts the name of the txt file to the next which reads it and continues the work.
everything is nice, smooth and kind of fast (alot faster than before)
And now...
999 error from yahoo due to to many requests
That blows!

Good work, I agree the template/design could use some work. However the script works great aside from a few errors from time to time. I would love to have a copy of the script.

Originally Posted by wkrappen91

999 error from yahoo due to to many requests
That blows!

Welcome to the wonderful world of scraping. Unfortunately that's just a limitation of this kind of site design. Imagine what it's going to be like when you've got dozens / hundreds of surfers on the site at the same time...

UPDATE:

whattoknowabout.net
Double Multi Threading is now in place
index.php posts keyword to a.php
a.php performs first set of 4 requests, saves all the shit to .txt file, calls
b.php
which grabs .txt file, performs second set of 4 requests and puts out the result.

Obviously BETA right now
no database, no file that is created, just the pure output in b.php.
check it out, (dont enter querys with space just yet:P still need to figure out which urls need "+" and which need "%20"

Great work man

Thanks for your nice script, I`d like a wish list with your further vision.
1.Passable to save flickr image within our own domain site folder.
2.Scrape image from google image search and filter the scale with owner decide.
ex.

Code:

http://unmth.com/Google-Images/index.php

Regard.
G.

I just got a 404 message thats not good.I want to see.PM me when it is up.

Originally Posted by galus

Thanks for your nice script, I`d like a wish list with your further vision.
1.Passable to save flickr image within our own domain site folder.
2.Scrape image from google image search and filter the scale with owner decide.
ex.

Code:

http://unmth.com/Google-Images/index.php

Regard.
G.

What do you mean thanks for the nice script?
I hope you werent able to leech it or something?
Why would i waste space on my hosting for flickr images? it makes no sense.
Less space used, less traffic used, and their server is pry faster than mine:P

Originally Posted by dizzspaceteam

I just got a 404 message thats not good.I want to see.PM me when it is up.

just go to the http://whattoknowabout.net/ and enter a (one word) keyword.
the indexx and all the inner pages are deleted due to change to multithreading.
The generated pages are not static as of now (like they used to) but they are just a preview and still need to be processed and saved to .php/entered into the mysql database and so on.
Also i need a design but i think i have an idea. not sure if i can pull it off... more a coder than a drawer...
hope you like it

Originally Posted by wkrappen91

Why would i waste space on my hosting for flickr images? it makes no sense.
Less space used, less traffic used, and their server is pry faster than mine:P

No no no!

Download the images, use imagemagick to slightly resize them, optimize them and remove the metadata - this is to make them unique in googlebot's eyes.

Give them keyword filenames eg. your-keyword-here.jpg.

Link your thumbs to the big pic, or put the big pic on its own page and surround it with niche keywords (and ads of course).

You will start getting google image search traffic = profit.

aight good idea
will get to that.
what do you think about the multithreading?
speed better that way?

Originally Posted by Autumn

No no no!

Download the images, use imagemagick to slightly resize them, optimize them and remove the metadata - this is to make them unique in googlebot's eyes.

Give them keyword filenames eg. your-keyword-here.jpg.

Link your thumbs to the big pic, or put the big pic on its own page and surround it with niche keywords (and ads of course).

You will start getting google image search traffic = profit.

Yes, u r right.
I saw some good traffic also from image search.
To op, I just admire your effort, not for tricks or leech propose.
Good day.

Ah i was kind of scared for a moment there
But thanks in that case
I will get to the image-traffic thing as soon as the rest stands on its feet.

Looks pretty sweet man!

Now take what you've got, add proxy support and adapt it for use as a stand alone scraper and site generator for your own targeted niche keywords. Start cranking out hundreds of niche sites a day. $$$

Unless you have multiple IPs and you want to rotate your external IP address with squid etc, you're going to start running up against resource limits - ezine, wiki, flickr all will block you if you start scraping them en masse.

Update: Fuck Flickr
New Bing images scraper in place.
Way better images for the KW plus the speeds are almost half as much as they used to be.

Actually im planning on letting it run for a while, generating a LOT of content and picking up TONS of longtail KWs (things NO ONE searches for and that only exist on my page that way (eg. combination of yahoo question plus a word from the ezine article).
Having 100.000 pages will get me enough of those trash KW hits i think
After it generates a single penny ill sell it:P

Take a look at the new image scraping

Edit: BTW. You can now enter more than one word per request ("online games", "black hat world")...


Update 2: Images are now safed to the server and resized. also there is a lightbox when you click on the image and it comes up larger

Edit 3: For some reason GoDaddy is letting me down real hard Site is slow as **** right now...

pretty cool idea. not so sure that auto blogging is a good idea though.
but whatever..

Nice project!
cUrl is powerful!

I am planning to make my own tools too, but at this point I have to make my specifications clear, in order to choose between a c# and cUrl/php approach...then I'll get my hands dirty...

check out the new template

What to you think?
http://whattoknowabout.net/template/index2.html

looks nic, would be a great wordpress ms "WPMU" script, when a search is done, it creates a WPMS page on subdomain. And it does a few post. Then its a automated micro nisch script.

Originally Posted by wkrappen91

check out the new template

What to you think?
http://whattoknowabout.net/template/index2.html

Much better. I would make the tabs slightly longer around the submit form, because on my browser (Firefox on OS X) the submit button gets pushed past the end of the tab, or even better make the submit button a nice image that matches the theme and looks all web 2.0-ish.

I wanna see a huge adsense block sitting right under that submit form!

I'm impressed with your productivity, you're making me feel quite lazy.

OK so now if i want to post on a blog how will that happen?
Dizz

Originally Posted by Autumn

Much better. I would make the tabs slightly longer around the submit form, because on my browser (Firefox on OS X) the submit button gets pushed past the end of the tab, or even better make the submit button a nice image that matches the theme and looks all web 2.0-ish.

I wanna see a huge adsense block sitting right under that submit form!

I'm impressed with your productivity, you're making me feel quite lazy.

Ok Good thing i know that now.
I dont quite now how to use images as buttons but ill figure it out Thanks for the input.
I know, this project cracks me up im lazy usually too..

Edit: Check Template now.
What u think?

Originally Posted by dizzspaceteam

OK so now if i want to post on a blog how will that happen?
Dizz

What to you mean if you want to post on a blog?
Its not supposed to post to any blogs.
its a standalone script that generates pages on the fly as the user puts in a keyword.
or
i did not quite understand the question:P

I would keep the input field and the submit button on the same tab, it's not entirely clear that the image is actually the submit button otherwise.

Maybe you could look in to putting a little bit of javascript onSubmit() to pop an alert box if the surfer tries to click submit without entering anything in the input box, eg. "You must enter a search term."

Originally Posted by Autumn

I would keep the input field and the submit button on the same tab, it's not entirely clear that the image is actually the submit button otherwise.

Maybe you could look in to putting a little bit of javascript onSubmit() to pop an alert box if the surfer tries to click submit without entering anything in the input box, eg. "You must enter a search term."

Maybe ill connect them with an arrow:P ill think of something cause i like it that way
And yes, the empty search has to be caught.
just as if someone enters "index" (since it would overwrite my index.php in the final script...)

Its almost done
All i need now is the index and some minor tweaks in the code...
Check it out: http://whattoknowabout.net/

*Bump*
Its pretty much final now.
I would be happy if everyone could enter a few KWs so the thing gets rolling

Code:

http://whattoknowabout.net

Update:
Breaking news!
Automatic rewriting is now in place.
the content that gets shown on the final pages is (close to) 100% unique and i think pretty readable..
Tell me what you guys think.
And.
Does any one know how to get rid of this site now?
It was just a test of programming / proof of concept, and im bored by it now and dont have the time to do anything with it...
Should i list it on flippa? (but there isnt any revenue or anything... just the domain, the concept, the script and the design...)

How much are you looking to sell it for?

Not much
just trying to get my investment + time put in back (1 year of hosting+domain+25-30 hours of work)
so around €150-€200...
Do you think thats to much / not enough?

*Bump*
Anyone want this script?
Domain+Hosting+Script+Support when needed.
$300 is my price
Ask me to go lower and i will
Just kindof in need of some float cash right now...