|
|
|
 |

02-04-2009, 04:39 PM
|
 |
Regular Member
|
|
Join Date: Oct 2007
Posts: 233
Thanks: 29
Thanked 97 Times in 19 Posts
Reputation: 16
|
|
Help me - How to generate automatically 100K webpages?
Basically I have a DB which contains huge number of names and I would like to create a website where all those names will be made into pages and the page title should be the DB value or etc.
Basically a main website and from there You can view a list or etc and see all those names. And all pages have the same content basically but with some variables (links). What I need is that those pages are static and seam to be useful for google etc.
I hope u understand what I need?  Also when I have 100K different HTML files or maybe Wordpress posts, then wont my host f**k me in the a**?
Thanks!
|

02-04-2009, 05:21 PM
|
|
BANNED
|
|
Join Date: Dec 2008
Location: www.blackhatworld.com
Posts: 589
Thanks: 223
Thanked 928 Times in 104 Posts
Reputation: 20
|
|
Re: Help me - How to generate automatically 100K webpages?
huh sound funny you have to code your php codes to interact with db and make some posts in word press by reading db tabls one by one
|

02-04-2009, 05:23 PM
|
 |
Regular Member
|
|
Join Date: Oct 2007
Posts: 233
Thanks: 29
Thanked 97 Times in 19 Posts
Reputation: 16
|
|
Re: Help me - How to generate automatically 100K webpages?
Sorry, I cannot edit my post - anyways I took a look into wordpress db but I did not come up with a good idea how to convert these into posts.
The page content should not be nothing more than just Thanks for Visiting and Click for more info...
The thing dont have to be done with WP but WP has Categories and it is built for g luvin...(sitemaps,breadtrails and etc)
blackmagicmaster - do you have some clue how do to it? Im sorry im not exactly "smart" with mysql and php, I can understand some, but not much...
Last edited by eestisiin; 02-04-2009 at 05:25 PM.
|

02-04-2009, 06:42 PM
|
 |
Junior Member
|
|
Join Date: Aug 2008
Posts: 184
Thanks: 79
Thanked 41 Times in 26 Posts
Reputation: 12
|
|
Re: Help me - How to generate automatically 100K webpages?
there is program to generate php code from your sql data. or you can make xml feed to your blog post.
|

02-04-2009, 06:47 PM
|
 |
Regular Member
|
|
Join Date: Oct 2007
Posts: 233
Thanks: 29
Thanked 97 Times in 19 Posts
Reputation: 16
|
|
Re: Help me - How to generate automatically 100K webpages?
hmm could you explain a bit more?  i have no clue how to generate 100k posts with my database field names which would contain the same text only the more info link would be renamed...
I was thinking that maybe somehow... oooh I have no idea on how do to it...
|

02-04-2009, 06:59 PM
|
|
Registered Member
|
|
Join Date: Jan 2008
Posts: 52
Thanks: 20
Thanked 19 Times in 14 Posts
Reputation: 11
|
|
Re: Help me - How to generate automatically 100K webpages?
You don't really need to "generate" any pages. You can have a single PHP page that produces them dynamically for each visitor. The pages never really exist. You can have a single page do all this.
|

02-04-2009, 07:02 PM
|
 |
trophaeum's bitch
|
|
Join Date: Apr 2008
Posts: 1,895
Thanks: 630
Thanked 1,171 Times in 436 Posts
Reputation: 60
|
|
Re: Help me - How to generate automatically 100K webpages?
Combining mod_rewrite and a simple PHP page that makes an SQL query would do the trick. You probably don't want to have a single index page with that many outgoing links.
EDIT: mpruben beat me to it.
|

02-04-2009, 07:04 PM
|
 |
Regular Member
|
|
Join Date: Oct 2007
Posts: 233
Thanks: 29
Thanked 97 Times in 19 Posts
Reputation: 16
|
|
Re: Help me - How to generate automatically 100K webpages?
but in this case google would not cache them am I right?  It is important that google would see them ( and all other se-s)
Strange problem Iv got, never have thought about this :/
|

02-04-2009, 07:12 PM
|
 |
trophaeum's bitch
|
|
Join Date: Apr 2008
Posts: 1,895
Thanks: 630
Thanked 1,171 Times in 436 Posts
Reputation: 60
|
|
Re: Help me - How to generate automatically 100K webpages?
Quote:
Originally Posted by eestisiin
but in this case google would not cache them am I right?  It is important that google would see them ( and all other se-s)
Strange problem Iv got, never have thought about this :/
|
Google would absolutely cache them as long as it could reach them. Whether or not they would be indexed has a lot more to do with how you'd promote the site and the quality of the information you'd be posting.
One way to do it would be to create an index page with, for example, a table containing every letter in the alphabet (A-Z). When the user clicked on one of the letters, it would take them to a dynamically generated page that performed an SQL query to search for the first 20-50 records whose index value begins with that letter. Use pagination to link to all of the other possible records.
So, if you had a list of all first names, your index page would have a link like:
Code:
http://somesite.com/s/
Using mod_rewrite, you'd forward any request for http://somesite.com followed by any character to a paginated search page for the character 's.' On that page, you'd generate links using the value of whatever kind of records you're using. To continue the first names example:
Code:
<a href="http://somesite.com/s/steve/">Steve</a>
Once again, you use mod_rewrite to forward all requests for http://somesite.com/[A-Z]/[*] to do an SQL query for whatever value of[*]. This would allow you to pull whatever other information is in that specific record and display it however you please.
WordPress and almost every other popular CMS use schema that is at least somewhat similar, so you can rest assured that if the content is quality and you create a decent link bed, your site will get indexed.
Hope that helps you begin to move in the right direction.
|
|
The Following User Says Thank You to oldenstylehats For This Useful Post:
|
|

02-04-2009, 07:19 PM
|
|
Registered Member
|
|
Join Date: Dec 2006
Posts: 64
Thanks: 0
Thanked 0 Times in 0 Posts
Reputation: 10
|
|
Re: Help me - How to generate automatically 100K webpages?
yep, what you need is a php coder
I think, theres also a programme floating around here, that does the same thing (takes a dbase and makes a site) - is it xsitepro2?
|

02-04-2009, 07:20 PM
|
 |
Regular Member
|
|
Join Date: Oct 2007
Posts: 233
Thanks: 29
Thanked 97 Times in 19 Posts
Reputation: 16
|
|
Re: Help me - How to generate automatically 100K webpages?
Ok, now I understand the procedure at least and I feel kind a dumb to ask more questions because u laid it all here correctly but the sad part is that I dont actually understand how to generate dynamic pages...
Ooh well, ill try to search for google for help.
And if anyone has any more bones to throw here about the mod_rewrite thingy or has built a similar website, then please help
Thanks a lot!
|

02-04-2009, 08:15 PM
|
 |
trophaeum's bitch
|
|
Join Date: Apr 2008
Posts: 1,895
Thanks: 630
Thanked 1,171 Times in 436 Posts
Reputation: 60
|
|
Re: Help me - How to generate automatically 100K webpages?
eestisiin, if I have the time, I'll put together an example for you in the next day or so.
|

02-04-2009, 08:41 PM
|
|
Jr. VIP
|
|
Join Date: May 2008
Posts: 360
Thanks: 79
Thanked 208 Times in 111 Posts
Reputation: 15
|
|
Re: Help me - How to generate automatically 100K webpages?
It doesn't look as if you're trying to fine tune something but sounds more like you're trying an approach that requires skills beyond your skill-set. Unless you feel like it's an incredible technique, I'd try something different or maybe look to outsource it.
|

02-04-2009, 09:29 PM
|
 |
Regular Member
|
|
Join Date: Oct 2007
Posts: 233
Thanks: 29
Thanked 97 Times in 19 Posts
Reputation: 16
|
|
Re: Help me - How to generate automatically 100K webpages?
@oldenstylehat - it would be marvelous, just a sample page would do the magic, im trying to read atm but I have never used it like this, i can check php scripts and etc and get their logic, but I cant put one up by myself - you are a true helper
@bobfrapples - yeah I got an idea and thought that I could give it a try, but I just did not have any clue how difficult it can be for me. Outsourcing takes time and I believe that it can be done by me also
I every once in a while curse myself that i have no clue about programming in any language.. and im not able to learn it also - cant teach new tricks to an old dog :P
|

02-04-2009, 10:43 PM
|
 |
Regular Member
|
|
Join Date: Oct 2007
Posts: 233
Thanks: 29
Thanked 97 Times in 19 Posts
Reputation: 16
|
|
Re: Help me - How to generate automatically 100K webpages?
Again - I cant edit my post...
Anyways - I had some success, I found out that wordpress has a plugin which enables to import posts with CSV format!!
Now I have just two questions :
1) How could I make an PHP script which would select the "Title" value from "Project" database table named "Stuff" and then make an csv file that would look like this..I have used in a past a php script ( i think i still have it) it generated fake filenames from some db and made exes, but I dont know how to output it into text file?
wp_title|wp_post_date|wp_category|wp_content|field 1|field2|field3
My Post Title|2007-12-31 23:59:59|fun-stuff|Long post text|value|value|valaue
Another Post|2008-01-31 23:59:59|fun-stuff|Long post text|value|value|valaue
2) And is there a way to add something like title field into the content area?? The content area is in html format this means that it should be possible to add something like this [For more information about "Value from titletag" click here <-link]
My post is laid out so bad, im not following myself on this post - have to get some sleep..
|

02-05-2009, 02:15 AM
|
 |
Registered Member
|
|
Join Date: Jul 2008
Posts: 67
Thanks: 99
Thanked 22 Times in 6 Posts
Reputation: 10
|
|
Re: Help me - How to generate automatically 100K webpages?
Is this the same thing as, say you type in something on a search engine like whatever.mp3 and a site pulls it up like they have exactly what your looking for but there is no such file to download? As in its going to pull up whatever you search for in a big database?
|

02-05-2009, 08:23 AM
|
 |
Regular Member
|
|
Join Date: Oct 2007
Posts: 233
Thanks: 29
Thanked 97 Times in 19 Posts
Reputation: 16
|
|
Re: Help me - How to generate automatically 100K webpages?
Actually not, but yeah how are those websites made? They usually respond to the exact keyphrase I have used or smth.
Is it working like so that the income search queries make up a page and each time a new search phrase comes in it will save a new page??
|

02-05-2009, 07:45 PM
|
 |
Regular Member
|
|
Join Date: Oct 2007
Posts: 233
Thanks: 29
Thanked 97 Times in 19 Posts
Reputation: 16
|
|
Re: Help me - How to generate automatically 100K webpages?
I still need help with generating the csv file to be built from db...
so that the csv file would be like this
title from db|some date|category entered by me (always the same)|text which would title from db and for more info click here|and other values are also writen, so those must not change
Help is needed, cause I cant figure out how to make a php script on my local PC to build a csv file which would be built like I said
|

02-06-2009, 04:28 AM
|
 |
trophaeum's bitch
|
|
Join Date: Apr 2008
Posts: 1,895
Thanks: 630
Thanked 1,171 Times in 436 Posts
Reputation: 60
|
|
Re: Help me - How to generate automatically 100K webpages?
I just posted some example code and a video on our blog that will hopefully give you folks some ideas about how this type of dynamic page generation works. Check it out here.
Like I say in the post, don't use this code exactly as it is, because it definitely isn't very secure or efficient, but hopefully it will help.
|
|
The Following User Says Thank You to oldenstylehats For This Useful Post:
|
|
| Thread Tools |
Search this Thread |
|
|
|
| Display Modes |
Linear Mode
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
|
|
|
|