Permanently Closed Marketplace Sales Thread

[loopline]

hello,
I've really became fan of automator lately.The flexibility it provides is simply enormous.But lot of times some task just gets stuck at last few connections for long time.It would be better if one could automatically stop task after X min or if threads are lower than 2 for X min then stop and move to next task.
Any workaround for this?

Thanks

What task? In V 2.0.0.50 all the components have a connection timeout set when things get below X percent which should solve this. But I guess it depends on what your doing as its not in all the addons for example.

 

[wacsuppot]

damn the 2.0 is so awesome. Trying the proxy scraper and it works like a beast. In 2-3 hrs almost 8000+ google pass. Even if 10% work for a day then this is so good to scrape so many things.

Regards,

 

[Evostance]

Quick question.

Running Scrapebox inside a virtualbox VM, the proxies don't seem to work. They either get 404, timeout or randomly do respond but quite slowly.

I expect this is an issue with Virtual Box, however was wondering if anyone had a similar issue, and how they got around it?

 

[Aaman]

Day before yesterday i bought this product. Great Post

 

[Evostance]

OK, got this working now.

I do have a question though.

Is this a CPU intensive program, or is it restricted by the number of proxies you have and their speed? I have a VPS with 2 Xeon E5's and it's only utilising 1-3% of the CPU, harvesting on average 30 URLs per second.

Is there anything I can do to speed things up, or do I need more/faster proxies, and can I downgrade to 1 CPU?

 

[loopline]

damn the 2.0 is so awesome. Trying the proxy scraper and it works like a beast. In 2-3 hrs almost 8000+ google pass. Even if 10% work for a day then this is so good to scrape so many things.

Regards,

Its pretty awesome, 2.0 is more optimized in everything then 1 was. 1 was never built to be what it turned into, with V2 they started with the end in mind and built it from the ground up to be what it is and laid the ground work for it to grow as well.

OK, got this working now.

I do have a question though.

Is this a CPU intensive program, or is it restricted by the number of proxies you have and their speed? I have a VPS with 2 Xeon E5's and it's only utilising 1-3% of the CPU, harvesting on average 30 URLs per second.

Is there anything I can do to speed things up, or do I need more/faster proxies, and can I downgrade to 1 CPU?

Its not really a CPU intensive program for most things. Harvesting is very optmized and uses low CPU for me as well. Posting and Link checker and other elements like the page scanner that require an entire web page to be loaded and processed, those things use more CPU.

The number of working proxies will be your bottle neck for harvesting, not CPU. As for whether or not you can downgrade to 1 CPU, I don't think there is a right or wrong answer, it just depends on what you need and how fast you want to go. If you are only harvesting then sure. If you plan to post/link check or use lots of other functions I would do some test runs with what you plan to use and see how it fares before you change.

Also if changing means you have to move to a new VPS then that will count as a transfer and you can only do that 1 time per month, so if you move and in 3 days decide to move again you will need to purchase an additional transfer for $10. Transfers reset on the 1st of each month.

 

[zoewarrior]

Hi Matt

Which search engines in the harvester use Google API? I try to find this information using Google Search but no luck!

 

[cottonwolf]

Can scrapebox be compiled for debian 8?

 

[loopline]

Hi Matt Which search engines in the harvester use Google API? I try to find this information using Google Search but no luck!

Well Deeperweb is the one I use the most, as its fast, has its own ip bans etc... however I believe that charter.net verizon.net google api all use google. I think there are perhaps a couple others, but I can't remember.

Can scrapebox be compiled for debian 8?

Sweetfunny would ultimatley have to answer this, but I believe the answer is no. I mean in theory maybe it could, I don't even know, but for example with some of the support I see on here, people can't even use it with windows. So it doesn't make sense to invite people to try and use it on Debian and create more support. However regardless of that I would say there is a bigger mac user base then debian (of people that would use Sbox I mean) and I know delphi can compile to mac, so I would assume if they went any direction they would go that direction, but I have no idea if thats something they will ever do. Its hard to say. So could it, possibly, does it make sense from a stand point of will it make more money then it costs in time and support, probably it would be a loss for them so it seems unlikely to happen.

 

[rogerke]

Hi Loop or Sweetfunny,

Is there a way to extend the Page Scanner to CSS as well? I need to check Youtube pages for comments with an attribute similar to this:

<span class="load-comments can-reply" data-cid="z134ufiimke5fp0t4220jrijryavcjuih04" data-tab="" data-vid="4dKuCqQ4G7g">
<a href="#" onclick="return false;" class="show-more">

Obviously it doesn't show up in the HTML source code and Googlebot doesn't index it either, so directly scraping Google won't work either.

 

[andyf91]

Just signed up! Excited to get this going

 

[loopline]

Hi Loop or Sweetfunny,

Is there a way to extend the Page Scanner to CSS as well? I need to check Youtube pages for comments with an attribute similar to this:

<span class="load-comments can-reply" data-cid="z134ufiimke5fp0t4220jrijryavcjuih04" data-tab="" data-vid="4dKuCqQ4G7g">
<a href="#" onclick="return false;" class="show-more">

Obviously it doesn't show up in the HTML source code and Googlebot doesn't index it either, so directly scraping Google won't work either.

Can you pm me an example page? Ive not tried it, but let me ask, are all such pages formatted the same, I mean do they coincide with a given theme or cms etc... ?

Also the less obvious but important question, are there other common markers that may be indexed by a search engine? As that would obviously be an easier route.

 

[rogerke]

Can you pm me an example page? Ive not tried it, but let me ask, are all such pages formatted the same, I mean do they coincide with a given theme or cms etc... ?

Also the less obvious but important question, are there other common markers that may be indexed by a search engine? As that would obviously be an easier route.

Hi Loop,

I can just share the URL publically. First, go got to this awesome video (https://www.youtube.com/watch?v=LyCLfL_ffqQ) with Chrome. Then go to George Geoker's comment, right click the "view all 3 replies" and open the Chrome Dev Tool to inspect the element. Now you'll see the attribute I mentioned before, but it's not in the HTML source code.

Google does index some common marker pages ("view all * replies"), but it's only 1% or so of the pages with that marker. Not enough pages for my purpose unfortunately. Besides, I also need it to check for pages were I dropped my own comment earlier and which aren't yet updated with the correct "view all * replies" in Google.

 

[gromhold]

anyone have issue with count prev/recent positions in RT ?

 

[gromhold]

today checking result

it should be |208|233|+##|

 

[delta_squadron]

anyone have issue with count prev/recent positions in RT ?

Yep. They're both blank (-) for me as well, since yesterday.

 

[loopline]

Hi Loop,

I can just share the URL publically. First, go got to this awesome video (https://www.youtube.com/watch?v=LyCLfL_ffqQ) with Chrome. Then go to George Geoker's comment, right click the "view all 3 replies" and open the Chrome Dev Tool to inspect the element. Now you'll see the attribute I mentioned before, but it's not in the HTML source code.

Google does index some common marker pages ("view all * replies"), but it's only 1% or so of the pages with that marker. Not enough pages for my purpose unfortunately. Besides, I also need it to check for pages were I dropped my own comment earlier and which aren't yet updated with the correct "view all * replies" in Google.

My thinking was that if you could generate urls that would call the CSS of pages you could scan them, but in your example its not going to work with the page scanner. What is your end goal though, are you looking for pages where you can comment, or where there are X number of replies or somethign else all together? Im just trying to think if there is another way to do it.

anyone have issue with count prev/recent positions in RT ?

No its working fine for me. If I hit check and then stop it then it clears out the data and then I check again and I get all blanks or old data. However if I let it complete a check and then run another check and let it complete then it shows the the previous and current data just fine.


~~~~~~~~~~~~~~~~~~~~~~~~~

Youtube Video Grabber Video

​

 

[andyf91]

Having trouble, getting failed connection timeout with my proxies. I emailed your support trying to get this going ASAP thank you

 

[rogerke]

My thinking was that if you could generate urls that would call the CSS of pages you could scan them, but in your example its not going to work with the page scanner. What is your end goal though, are you looking for pages where you can comment, or where there are X number of replies or somethign else all together? Im just trying to think if there is another way to do it.

Yes, I'm looking for pages with a certain amount of replies. Without going too much in detail after 500 comments it locks down so your comment will basically be on top forever (or at least until YT changes its top comment algorithm yet again, lol).

I saw some scripts on Github to extract CSS data and check for footprints but I need it customized and am terrible at coding. Also not sure wether these codes work at this scale. Might have to find a custom solution on Freelancer or in the hire a freelancer section here on BHW.

Thanks for looking into it anyway

Edit: the YT scraper in your video is absolutely awesome. When was this added?​

 

[loopline]

Having trouble, getting failed connection timeout with my proxies. I emailed your support trying to get this going ASAP thank you

Failed connection timeout would indicate of course that a connection is timing out. Where are you getting this error? Like in the proxy tester or poster or an addon or?

Make sure Scrapebox is white listed in all security software and make sure of course you have valid proxies loaded.

Yes, I'm looking for pages with a certain amount of replies. Without going too much in detail after 500 comments it locks down so your comment will basically be on top forever (or at least until YT changes its top comment algorithm yet again, lol).

I saw some scripts on Github to extract CSS data and check for footprints but I need it customized and am terrible at coding. Also not sure wether these codes work at this scale. Might have to find a custom solution on Freelancer or in the hire a freelancer section here on BHW.

Thanks for looking into it anyway

Edit: the YT scraper in your video is absolutely awesome. When was this added?
​

Hmmm well Im not sure about that one. The YT Scraper is awesome and it was added maybe 10 days ago? Maybe 2 weeks? Been slow about getting a video up. So what if the YT grabber could include the number of comments a video has? So you could see that there is less then 500, would that do it?

Don't know if it could be added, but we could ask Sweetfunny if its a possibility for the next update.