Help me - How to generate automatically 100K webpages?

Basically I have a DB which contains huge number of names and I would like to create a website where all those names will be made into pages and the page title should be the DB value or etc.

Basically a main website and from there You can view a list or etc and see all those names. And all pages have the same content basically but with some variables (links). What I need is that those pages are static and seam to be useful for google etc.

I hope u understand what I need? Also when I have 100K different HTML files or maybe Wordpress posts, then wont my host f**k me in the a**?

Thanks!

huh sound funny you have to code your php codes to interact with db and make some posts in word press by reading db tabls one by one

Sorry, I cannot edit my post - anyways I took a look into wordpress db but I did not come up with a good idea how to convert these into posts.

The page content should not be nothing more than just Thanks for Visiting and Click for more info...

The thing dont have to be done with WP but WP has Categories and it is built for g luvin...(sitemaps,breadtrails and etc)

blackmagicmaster - do you have some clue how do to it? Im sorry im not exactly "smart" with mysql and php, I can understand some, but not much...

there is program to generate php code from your sql data. or you can make xml feed to your blog post.

hmm could you explain a bit more? i have no clue how to generate 100k posts with my database field names which would contain the same text only the more info link would be renamed...

I was thinking that maybe somehow... oooh I have no idea on how do to it...

You don't really need to "generate" any pages. You can have a single PHP page that produces them dynamically for each visitor. The pages never really exist. You can have a single page do all this.

Combining mod_rewrite and a simple PHP page that makes an SQL query would do the trick. You probably don't want to have a single index page with that many outgoing links.

EDIT: mpruben beat me to it.

but in this case google would not cache them am I right? It is important that google would see them ( and all other se-s)

Strange problem Iv got, never have thought about this :/

eestisiin said:

but in this case google would not cache them am I right? It is important that google would see them ( and all other se-s)

Strange problem Iv got, never have thought about this :/

Click to expand...

Google would absolutely cache them as long as it could reach them. Whether or not they would be indexed has a lot more to do with how you'd promote the site and the quality of the information you'd be posting.

One way to do it would be to create an index page with, for example, a table containing every letter in the alphabet (A-Z). When the user clicked on one of the letters, it would take them to a dynamically generated page that performed an SQL query to search for the first 20-50 records whose index value begins with that letter. Use pagination to link to all of the other possible records.

So, if you had a list of all first names, your index page would have a link like:

Using mod_rewrite, you'd forward any request for http://somesite.com followed by any character to a paginated search page for the character 's.' On that page, you'd generate links using the value of whatever kind of records you're using. To continue the first names example:

Once again, you use mod_rewrite to forward all requests for http://somesite.com/[A-Z]/[*] to do an SQL query for whatever value of [*]. This would allow you to pull whatever other information is in that specific record and display it however you please.

WordPress and almost every other popular CMS use schema that is at least somewhat similar, so you can rest assured that if the content is quality and you create a decent link bed, your site will get indexed.

Hope that helps you begin to move in the right direction.

yep, what you need is a php coder

I think, theres also a programme floating around here, that does the same thing (takes a dbase and makes a site) - is it xsitepro2?

Ok, now I understand the procedure at least and I feel kind a dumb to ask more questions because u laid it all here correctly but the sad part is that I dont actually understand how to generate dynamic pages...

Ooh well, ill try to search for google for help.

And if anyone has any more bones to throw here about the mod_rewrite thingy or has built a similar website, then please help

Thanks a lot!

Here are a couple of tutorials:
mod_rewrite: http://www.workingwith.me.uk/articles/scripting/mod_rewrite
PHP/MySQL: http://www.designplace.org/scripts.php?page=1&c_id=25

Neither of those are very elegant solutions nor are they exceptionally well-written, but they do provide code and examples.

For a more general overview, check out the following:
http://www.w3schools.com/PHP/php_mysql_intro.asp
http://www.sitepoint.com/article/apache-mod_rewrite-examples/

eestisiin, if I have the time, I'll put together an example for you in the next day or so.

It doesn't look as if you're trying to fine tune something but sounds more like you're trying an approach that requires skills beyond your skill-set. Unless you feel like it's an incredible technique, I'd try something different or maybe look to outsource it.

@oldenstylehat - it would be marvelous, just a sample page would do the magic, im trying to read atm but I have never used it like this, i can check php scripts and etc and get their logic, but I cant put one up by myself - you are a true helper

@bobfrapples - yeah I got an idea and thought that I could give it a try, but I just did not have any clue how difficult it can be for me. Outsourcing takes time and I believe that it can be done by me also

I every once in a while curse myself that i have no clue about programming in any language.. and im not able to learn it also - cant teach new tricks to an old dog

Again - I cant edit my post...

Anyways - I had some success, I found out that wordpress has a plugin which enables to import posts with CSV format!!

Now I have just two questions :

1) How could I make an PHP script which would select the "Title" value from "Project" database table named "Stuff" and then make an csv file that would look like this..I have used in a past a php script ( i think i still have it) it generated fake filenames from some db and made exes, but I dont know how to output it into text file?

wp_title|wp_post_date|wp_category|wp_content|field1|field2|field3
My Post Title|2007-12-31 23:59:59|fun-stuff|Long post text|value|value|valaue
Another Post|2008-01-31 23:59:59|fun-stuff|Long post text|value|value|valaue

2) And is there a way to add something like title field into the content area?? The content area is in html format this means that it should be possible to add something like this [For more information about "Value from titletag" click here <-link]

My post is laid out so bad, im not following myself on this post - have to get some sleep..

Is this the same thing as, say you type in something on a search engine like whatever.mp3 and a site pulls it up like they have exactly what your looking for but there is no such file to download? As in its going to pull up whatever you search for in a big database?

Actually not, but yeah how are those websites made? They usually respond to the exact keyphrase I have used or smth.

Is it working like so that the income search queries make up a page and each time a new search phrase comes in it will save a new page??

I still need help with generating the csv file to be built from db...

so that the csv file would be like this

title from db|some date|category entered by me (always the same)|text which would title from db and for more info click here|and other values are also writen, so those must not change

Help is needed, cause I cant figure out how to make a php script on my local PC to build a csv file which would be built like I said

I just posted some example code and a video on our blog that will hopefully give you folks some ideas about how this type of dynamic page generation works. Check it out here.

Like I say in the post, don't use this code exactly as it is, because it definitely isn't very secure or efficient, but hopefully it will help.