Permanently Closed Marketplace Sales Thread

Status
Not open for further replies.
Is there a way I can exclude certain words when harvesting? For example when scraping site:facebook.com for profiles exclude the word 'post'.
 
How can I change "Grab comments from harvested URL list" threads count? It seems it is hardcodded to be 30 and cant be changed anywhere ?

It is hard coded and can not be changed.

Is there a way I can exclude certain words when harvesting? For example when scraping site:facebook.com for profiles exclude the word 'post'.

site:facebook.com -"post"

you can minus stuff out, assuming you are using google.
 
The Rank Tracker plugin is not bad but how can I see my previous rankings. Can you please make some changes and make it somethink like this http://prntscr.com/9l8774 or at least add hover that actually shows the ranking of that day.
 
The Rank Tracker plugin is not bad but how can I see my previous rankings. Can you please make some changes and make it somethink like this http://prntscr.com/9l8774 or at least add hover that actually shows the ranking of that day.


When you click a keyword you can see the ranking graph in the segment of the window below that, at the bottom of the window. Its already there an you can already see todays rank just by looking at the rank tracker. Not really sure what your asking as those things are already there.
 
http://prntscr.com/9lje2g
By looking at the picture can you tell the exact spot where I was ranked on 23/12/2015?

I'm just saying the you need to improve the graph so we can actually see our exact previous rankings.
 
I've noticed that the harvester doesn't seem to filter out domains when you use User's Blacklist locally on Remove/Filter -> Use User's Local Blacklist. I've got a bunch of honeypot and social media sites where I can't post with SER, and other junk like 12345678.r.msn.com that google/other SEs has been returning lately.

I guess I can only filter out if I put http:// and https:// before these.

I'm not really sure what should be the proper formatting for domains on the User's Black List.
 
http://prntscr.com/9lje2g
By looking at the picture can you tell the exact spot where I was ranked on 23/12/2015?

I'm just saying the you need to improve the graph so we can actually see our exact previous rankings.

I can tell exactly as accurately as I can in the screen that you posted in your prior post:
http://prntscr.com/9l8774

Your prior screen that you were saying is good doesn't list the number either. In any case I can tell you are consistent in rankings, and your far enough from spot 1 that its largely mute. (IMHO).

I don't work for Scrapebox but Ive asked about this before. They nor likley the people in your screenshot build the graph from scratch. They both use a 3rd party component that they buy from somewhere that builds the graph and then rank tracker for example supplies the data to the graph etc...

For the current delphi compiler (which is what scrapebox is written in) there are limited graphs available, and because of the nature of rank tracking the graph must be inverted and used in the opposite manner as what most graphs are used in because 1 is on the top and not the bottom. Which futher complicates it.

So they have spent a lot of time testing to get the most optimal option. For a $20 plugin it does a great job of basic rank tracking. If I want to go pay 2 or 3 times that much per month, or more, to someone else, then I can possibly get a more custom solution that has a grid with numbers that let you cross reference to see your rank at a glance. But rank tracking to me is more along the lines of

Im so far away I don't care and I just need to improve

or

Im at or near number 1 so if I keep doing well Im in good shape.

Im not saying your doing it wrong to want details but I think its very important to remember that rank tracking is a very broad general idea not a specific metric. You can take 20 people with different browsers and get 20 different sets of SERPs. Also you toss in different ip addresses and it further shuffles things. In some sets of results your page might not even appear at all, and its placing can vary from high page one to page 3 for the same term based on ip and browser. I think most people assume that what they see is what everyone else sees, but the truth is that where your page can land for the same term is a very broad spread. Hence rank tracking is a general guideline of movement in a positive or negative direction but should never be assumed that that is what your customer/user will see.

I've noticed that the harvester doesn't seem to filter out domains when you use User's Blacklist locally on Remove/Filter -> Use User's Local Blacklist. I've got a bunch of honeypot and social media sites where I can't post with SER, and other junk like 12345678.r.msn.com that google/other SEs has been returning lately.

I guess I can only filter out if I put http:// and https:// before these.

I'm not really sure what should be the proper formatting for domains on the User's Black List.

the blacklist is actual domains that contain http:// etc... so it needs to be whatever you want filtered, its url match.

If you put

http://www.domain.com

and what shows up from google is

http://domain.com

its not going to match.

If you want partial matching just put all your blacklist in a text file and do remove/filter >> remove urls containing entries from - and select your text file and you will get it filtered with no need for http: etc... just put

domain.com

or even stuff like

12345678

in the text file.
 
Thanks Matt.

the blacklist is actual domains that contain http:// etc... so it needs to be whatever you want filtered, its url match.

If you put

http://www.domain.com

and what shows up from google is

http://domain.com

its not going to match.

If you want partial matching just put all your blacklist in a text file and do remove/filter >> remove urls containing entries from - and select your text file and you will get it filtered with no need for http: etc... just put

domain.com

or even stuff like

12345678

in the text file.

I've also noticed the Automator and Malware and Phishing addon aren't correctly working together. The phishing addon fails to automatically load the file I've specified in the automator dialog. I've tried it both with a 1000,000 URLs large file and a 4KB sized file. The automator launches the malware filter addon, then the addon doesn't load the file.

The files I specify surely exist.
 
When will the Expired Domain Finder be available?

Thanks Matt.

Unfortunately there's no ETA on this one yet.

I've also noticed the Automator and Malware and Phishing addon aren't correctly working together. The phishing addon fails to automatically load the file I've specified in the automator dialog. I've tried it both with a 1000,000 URLs large file and a 4KB sized file. The automator launches the malware filter addon, then the addon doesn't load the file.

The files I specify surely exist.

This was due to the major update of the Malware and Phishing filter addon, the addon and the Automator have both just been updated so if you download the latest versions of Automator and the Addon everything should be working.
 
I still have a few automator jobs running and I've noticed sometimes the Link Extractor and Proxy Harvester both get stuck on a thread. I've got my proxy harvester template from Matt's youtube videos and modified it for my file paths and then it's got a loop step to continue looping the proxy testing steps.

The proxy harvester however did/does not have an option to cancel/stop that one thread, so that it can continue looping again. the link extractor can/could be at least stop/stopped.

The link extractor job is simple. Get the file and extract outbound links from the URLs in the file, and then save it as %date%-%time%-done.txt in a folder. I've reduced the time outs via automator to 30 for both the "reader(?)"(iirc) and the parser.

The issue might be with my proxies(23 private dedicated, 20 shared), some of them are crap in Asia(about 3).

This is just a notice before updating. I don't know if these will go away after an update.

Thanks for your work on scrapebox @sweetfunny and @softtouch.

This was due to the major update of the Malware and Phishing filter addon, the addon and the Automator have both just been updated so if you download the latest versions of Automator and the Addon everything should be working.

edit2:

Also, would it be possible to add the "Custom Title/Name" you can give to your Scrapebox instances to child processes(I mean addons and plugins)?

I run multiple link extractors and all of them have the same long name, so my only way to know which to shut down(if it gets stuck) is to look at the resource usage in task manager. And task manager sucks on windows srv 2012.

See the image for what I mean:

20160104scrapebox.jpg
 
Last edited:
I still have a few automator jobs running and I've noticed sometimes the Link Extractor and Proxy Harvester both get stuck on a thread. I've got my proxy harvester template from Matt's youtube videos and modified it for my file paths and then it's got a loop step to continue looping the proxy testing steps.

The proxy harvester however did/does not have an option to cancel/stop that one thread, so that it can continue looping again. the link extractor can/could be at least stop/stopped.

The link extractor job is simple. Get the file and extract outbound links from the URLs in the file, and then save it as %date%-%time%-done.txt in a folder. I've reduced the time outs via automator to 30 for both the "reader(?)"(iirc) and the parser.

The issue might be with my proxies(23 private dedicated, 20 shared), some of them are crap in Asia(about 3).

This is just a notice before updating. I don't know if these will go away after an update.

Thanks for your work on scrapebox @sweetfunny and @softtouch.



edit2:

Also, would it be possible to add the "Custom Title/Name" you can give to your Scrapebox instances to child processes(I mean addons and plugins)?

I run multiple link extractors and all of them have the same long name, so my only way to know which to shut down(if it gets stuck) is to look at the resource usage in task manager. And task manager sucks on windows srv 2012.

See the image for what I mean:

Working backwads if you right click the process you can look at its file path. Thats what I do to determine things. I number things so I have folders for Scrapebox 1, Scrapebox 2 etc.. .and the addons and plugins will fall inside those paths so as long as you know which number you launched it from your good. Or you could name the folder Job X and then launch the exe from it and then in turn the addon and then you would know. Just thinking out loud.


As for the proxy harvester and the link extractor if you look in the configuration options/settings you will the option to kill the threads if it goes below X. Just set that to a good number like 2 or 3. I have this on the link extractor in a loop and it kills when its below 3 and its been running for literally weeks without a hitch. I think it had a crash at 1 point do to something else entirely but at any rate it works like a charm.
 
Domains found by Expired Domain Finder need some additional check,mainly to check if a seeming good/authoritive
domain had been touched by many notorious spammers. This can done via ahrefs etc.
To bulk checking, paid api calls can eliminate this limits. I mention this here, so please don't call out at me if the purchased stuff is not so strong as it expected.
Easy method is checking domains on pr6 and above pages.
Build several sites just using Expired Domain Finder is not a problem.
With large data collected, premium domains will show up.
If knows where has more good domains than other places, then
time spend explorer would reduced.
 
Just gifted myself with the product bundle :). A great start in 2016!
 
the automator gives empty result when link extractor addon is scheduled
 
Am i able to get hidden data and attributes using this ? I am looking for a tool to help me get variants for products that are not released yet.
 
Just gifted myself with the product bundle :). A great start in 2016!

Gifted yourself, nice. :D

the automator gives empty result when link extractor addon is scheduled

Can you give more details and specifics? Perhaps you can give some screenshots?

Am i able to get hidden data and attributes using this ? I am looking for a tool to help me get variants for products that are not released yet.

I guess that depends on what you mean. The custom data grabber can grab html components that are on the page. However if you are looking to scrape search engines for html components then no as search engines do not allow that.
 
Status
Not open for further replies.
Back
Top