hey loopline how many keywords do you use for 11 million urls? hehehe
also, 504 it happens often even without proxies.
Permanently Closed Marketplace Sales Thread
- Thread starter Sweetfunny
- Start date
- Status
hey loopline how many keywords do you use for 11 million urls? hehehe
also, 504 it happens often even without proxies.
- Oct 5, 2009
- 2,363
- 1,207
Don't run sbox without proxies. I am guessing you were trying to troubleshoot/diagnose the problem, and I get that, just please...please...for your own sake...don't. I've seen other people just add to their problems by running naked and it just adds to their woes.
This is meant as free advice, and constructive help - not a sermon, just a thought.
okay, pep talk is over...now, let's see. An HTTP 504 is a gateway timeout, which is probably why loops thought it might be your proxies. Good educated guess on his part.
Are you running through any sort of tunneling or obfuscation? A VPN or the like? A service like hide my IP or hide my ass? Anything like that which could cause issues with upstream replies?
Another possibility is that your router or your OS has invoked some form of flood protection. I know that in the interests of security WinXP introduced some sort of TCP/IP fuckery that in SP2 that can wreak havoc on large amounts of data such as sbox creates. Can't remember off the top of my head if that clusterf lives on in Vista and above, but also some routers will pitch a fit over what they perceive as data floods. Google it a bit or do an advanced or this thread, I know I and at least a few others have posted info and links about it before. Applying the mods to alter the flood protections will improve your overall internet experience any way and if your system is properly secured flood protection is pointless IMHO.
Those are about the next two theories I can think of. Let us know what you find if you want. Happy Boxing!
peace - gregs.
onlinewealth
Junior Member
- Mar 13, 2007
- 159
- 102
Hey Frankie,
Is there a way to install this on a domain I have? The reason I ask is because I've had it installed on 2 different Vista machines and both of them have crashed and I had to completely reformat (recover) the hard drives. I'm not saying SB was the cause but that's the only software program both machines used.
Any thoughts on this and my orig question?
OW
Is there a way to install this on a domain I have? The reason I ask is because I've had it installed on 2 different Vista machines and both of them have crashed and I had to completely reformat (recover) the hard drives. I'm not saying SB was the cause but that's the only software program both machines used.
Any thoughts on this and my orig question?
OW
Sevenhelmets
Registered Member
- May 9, 2010
- 52
- 7
I just came here to post that I hit the ceiling with my 1,000,000 scrape, only to see your post a few posts up from mine. Dang!
How do you get 10 million scraped? I left mine running overnight, and when I just checked it now, scrapebox had saved the file to the folder, and stopped the run dead on 1mil scraped.
namewithnogaps
Junior Member
- Mar 28, 2010
- 199
- 74
hi,
can you tell me which option in comment poster should i use to post comments to drupal b2e or expression engine? is it the manual poster?
also. What blogs besides wordpress can fast poster do? thanks
thanks
can you tell me which option in comment poster should i use to post comments to drupal b2e or expression engine? is it the manual poster?
also. What blogs besides wordpress can fast poster do? thanks
thanks
mataff
Junior Member
- Sep 21, 2008
- 139
- 55
Been running three instances of SC on a crappy XP machine here at home. Try running it with XP SP3 and you shouldn't have any problems.
I wouldn't mind trying this on a windows server OS to see if it runs smoothly.
- Jan 25, 2009
- 6,378
- 3,774
Yeah really the only limitations on scraping is that each file is going to be maxed at 1 Mil and your keywords will limit you on how many results you can get. If you loaded up, say 1 million keywords I would imagine you could pretty much use the results to start your own search engine. lolIt saves off any urls harvested that is over 1Mil into a folder called harvester_sessions in your scrapebox directory for that instance. Each file is saved off as 1million urls.
Its odd that it would stop it right at 1Mil, but I suppose it could. I would just double check your most recent harvester session.
- Jan 25, 2009
- 6,378
- 3,774
All work on slow poster. B2E will work on fast poster. All will work on manual as well.
http://www.youtube.com/watch?v=p_vYOMxY4pY
- Jan 25, 2009
- 6,378
- 3,774
It will work fine on a Windows server, but not on linux style.
I was running it my vista laptops with no probs as far as that goes, but it wasn't fast enough or had enough ram to use it intesively. If you want to hammer it you need a better machine (maybe, dunno what you have).
I have a core i5 with 8GB RAM. It runs super smooth, no issues. I run 10+ instances at a time. Harvest millions of urls, addons with large files to proccess, check links for 100s of thousands of blogs. I mean I can still max out my pc, depending on what I am doing, but its 1000 times better then when I ran it on my laptop. I am running win7, but again its going to work fine on Vista/XP NP.
- Jan 25, 2009
- 6,378
- 3,774
I think it was like 250K or something. Can't remember. I only load in like maybe 80K something keywords then Merge it with my footprint file. Depending on what I am doing that can drastically inflate it. Like I have a footprint file for fast poster. Where it will take each keyword and pull it for wordpress, moveabletype and B2evo. Which obviously triples the keywords input.
- Oct 5, 2009
- 2,363
- 1,207
Well, how much bandwidth it uses all depends on how you use it of course. I haven't had any issues with my ISP, but I'm on an unmetered connection and my ISP is...well...pretty cool with things. Can't make any promises for how yours or anyone else's would react.
As far as running it on a server, yes that can be done, the app's resource requirements as far as CPU/RAM and so forth are actually pretty light. Sweetfunny has tried to make it as non-invasive as possible. I'd go with a fairly basic VPS, or if you already have a server try it, either way just check if they have any bandwidth limitations. That's the main area where resource usage can be a concern. As far as system resources, I've seen and heard of the app running on some really low-end machines.
Another advantage of running it on a separate machine is you can do stuff like scrap 18 kajillion search engine results, run slow poster mode to your heart's content and so forth.
onkelmicha
Registered Member
- Mar 15, 2010
- 74
- 11
WHy Do I keep getting error 503 when I scrape and how come that the software completely freezes when I press stop harvesting?
and how can I find out how long it has gone on my keyword list, so I dont have to re scrape all off the urls again if I want to make sure I get them all?
Last edited:
swillbrandt
Newbie
- Sep 1, 2010
- 6
- 0
is there a demo?
- Oct 5, 2009
- 2,363
- 1,207
First thing I'd do if I were you would be try turning down the # of connections you're running.
onkelmicha
Registered Member
- Mar 15, 2010
- 74
- 11
I have 75 m/bit download and 8 m/bit up...
shoouldnt it be able to handle 150 X 2 threads and 80 X 2 threads,,,
I was running more threads the first days then it started to happen and IM getting it now even when I lower the settings.
- Oct 5, 2009
- 2,363
- 1,207
Are those the up/down speeds your ISP tells you that you'll get, or are they tested via something like speedtest?
- Status