My new development.
- Thread starter blackonator
- Start date
blackonator
Junior Member
- Dec 27, 2012
- 132
- 16
The testing has already begun, the good thing is, I've seen my domain count increese 10 fold with the submit a link into our database.
Which is brillient, thank you everyone for having some sort of faith. YES OK I know the results are taking a little longer then expected, but bare with me while I sort the optimization of the back end
Which is brillient, thank you everyone for having some sort of faith. YES OK I know the results are taking a little longer then expected, but bare with me while I sort the optimization of the back end
blackonator
Junior Member
- Dec 27, 2012
- 132
- 16
No, it'll be put in que and should be indexed within 24hours.
But like I said, they're is the point of being beta, I wanted to release this project to show people WHAT I can do. (Mostly locals) of coruce.
I want it to be a success, however it probally won't happen over night and I will be monitoring the website every day to make improvements
On the card next is;
webmasters area for the SEO implention.
then I'll start building more SEO extras with the algo, but the idea's I got, google have not done them yet (if they have then.. well yeah)
But I'm getting a lot of possative feedback due to the fact of indexing websites on it's own, and not using the google engine, yahoo etc etc (that's all that dogpile do)
But like I said, they're is the point of being beta, I wanted to release this project to show people WHAT I can do. (Mostly locals) of coruce.
I want it to be a success, however it probally won't happen over night and I will be monitoring the website every day to make improvements
On the card next is;
webmasters area for the SEO implention.
then I'll start building more SEO extras with the algo, but the idea's I got, google have not done them yet (if they have then.. well yeah)
But I'm getting a lot of possative feedback due to the fact of indexing websites on it's own, and not using the google engine, yahoo etc etc (that's all that dogpile do)
Last edited:
blackonator
Junior Member
- Dec 27, 2012
- 132
- 16
The unfortunate fact is that I am building a DB server now.. to colo in UK.
Bare with me! I have not got that much money!
Bare with me! I have not got that much money!
bananaman5000
Regular Member
- Mar 5, 2010
- 201
- 130
i second this, you have no way of financing massive datacenters necessary to compete with google/yahoo. however, if you made a really accurate travel search, deal search, etc., you could get a lot of traffic, not to mention turning it into an app
blackonator
Junior Member
- Dec 27, 2012
- 132
- 16
Can't post links at the mo, but it's /submitWebsite.php
Thanks.
Thanks.
blackonator
Junior Member
- Dec 27, 2012
- 132
- 16
Think the problem with it is the DB is taking so long to load results which, averagely. the results are coming around 7-10seconds.. I will be improving this by Monday next week.
blackonator
Junior Member
- Dec 27, 2012
- 132
- 16
Okay, we've just deployed another dedicated server so the results should be faster soonish.
blackonator
Junior Member
- Dec 27, 2012
- 132
- 16
looks like we're getting results from around 8-9seconds now.. that's with 45 bots running... with only 15 bots running it gets to around 7 seconds... which is quite good reallly.. getting better.
- Apr 25, 2011
- 19,849
- 28,315
Don't know if you can divulge, but interested to know what hardware you are running(ie how much computing power it takes to produce those results), and what sort of power you need to achieve <1second results??
blackonator
Junior Member
- Dec 27, 2012
- 132
- 16
<1 I am currently building an Intel quad core processor x2, 32GB ram, 1TB hard drive hopefully will produce the result I want.
Now on to the current server, it's not hanging too bad really. but currently running 2GB ram, 4 core's 100mbit connection...
It's the optimizing of the DB that's really helping push it though. (but as you can see, the current server is rubbish)
Now on to the current server, it's not hanging too bad really. but currently running 2GB ram, 4 core's 100mbit connection...
It's the optimizing of the DB that's really helping push it though. (but as you can see, the current server is rubbish)
Last edited:
blackonator
Junior Member
- Dec 27, 2012
- 132
- 16
Now got the current server down to 4 seconds result time..
cash202
Elite Member
- Mar 12, 2011
- 1,801
- 2,828
Are you using RDBMS? If yes, drop it now and go for NoSQL.
blackonator
Junior Member
- Dec 27, 2012
- 132
- 16
No.. I'm not.