Can I match scraped results (URLs) with the respective keywords using Scrapebox?

SergeyShok

Registered Member
Joined
Aug 23, 2018
Messages
62
Reaction score
18
Hello to everyone.

While scraping Google SERP with Scrapebox I came across a problem. When the process of scraping is done I receive only the list of scraped URLs. But how to match them with the keywords these URLs came from?
I got around 40k keywords so I received 4,000,000 URLs. Do I now have to process them manually to match the urls with the keywords?
 
Hello to everyone.

While scraping Google SERP with Scrapebox I came across a problem. When the process of scraping is done I receive only the list of scraped URLs. But how to match them with the keywords these URLs came from?
I got around 40k keywords so I received 4,000,000 URLs. Do I now have to process them manually to match the urls with the keywords?

I have good news and not quite as good of news.

If you go to settings >> connections timeouts and other settings >> more harvester settings >> save additionally keywords with urls. Place a check mark in this box.

Then when you harvest you will have 2 lists automatically saved in the harvester_sessions folder (thats inside your main scrapebox folder). 1 file with just the list of urls. The other file with the urls with the keywords attached. Thats the good news.

The not quite as good of news is that if you didn't already have this box checked, then currently you have only a list of url. So you would need to rescrape that list of keywords to have them saved with the urls. But moving forward you would always have the urls and the keywords.
 
I have good news and not quite as good of news.

If you go to settings >> connections timeouts and other settings >> more harvester settings >> save additionally keywords with urls. Place a check mark in this box.

Then when you harvest you will have 2 lists automatically saved in the harvester_sessions folder (thats inside your main scrapebox folder). 1 file with just the list of urls. The other file with the urls with the keywords attached. Thats the good news.

The not quite as good of news is that if you didn't already have this box checked, then currently you have only a list of url. So you would need to rescrape that list of keywords to have them saved with the urls. But moving forward you would always have the urls and the keywords.

Thanks. It worked out for me. Also as a user, I'd like to tell you that there should be an option to see the Google's position. So the perfect .txt file should be as follows:
Keyword 1 | URL 1 | Position 1
Keyword 1 | URL 2 | Position 2

This is implemented by other SERP scrapers and the absence of this feature in your tool turned out to be a huge disappointment for me.
Just take it into consideration for the following upgrade :)
 
Thanks. It worked out for me. Also as a user, I'd like to tell you that there should be an option to see the Google's position. So the perfect .txt file should be as follows:
Keyword 1 | URL 1 | Position 1
Keyword 1 | URL 2 | Position 2

This is implemented by other SERP scrapers and the absence of this feature in your tool turned out to be a huge disappointment for me.
Just take it into consideration for the following upgrade :)
I don't work for Scrapebox.

But I can tell you this has been looked at in the past, and for various reasons, it has not been implemented.

Its largely subjective anyway, as that SERP position can vary wildly all over the map and sometimes not even show up for all sorts of reasons. I actually talk about it here in detail:
http://scrapeboxfaq.com/why-does-ra...ank-then-what-i-see-in-a-browser-or-elsewhere

Sufficient to say it would cause exponentially more problems then it solves and the whole ranking position is highly subjective anyway.
 
I don't work for Scrapebox.

But I can tell you this has been looked at in the past, and for various reasons, it has not been implemented.

Its largely subjective anyway, as that SERP position can vary wildly all over the map and sometimes not even show up for all sorts of reasons. I actually talk about it here in detail:
http://scrapeboxfaq.com/why-does-ra...ank-then-what-i-see-in-a-browser-or-elsewhere

Sufficient to say it would cause exponentially more problems then it solves and the whole ranking position is highly subjective anyway.
i fort you did own scrapebox deep down.

shocked , your posted all over the .net with scrapebox.
 
But you're definitely the reason a lot of us use it.

Glad to be of service. :)

i fort you did own scrapebox deep down.

shocked , your posted all over the .net with scrapebox.

No I don't back nearly a decade ago I was doing grunt work to pay bills. By that I mean when I went on vacation at AT&T I would work full time on my business and I would work on my lunch hour and weekends and after work. I was just trying to get the bills paid at that point.

I was doing a bunch of manual work and I found scrapebox. I had a conversation with SweetFunny and i even asked them if they were legitimate or just selling stolen software. They still laugh at me to this day about that LOL.

At any rate it was of course legitimate and so I bought and it saved me tons of time. That let me make more money then I figured out how to actually leverage it to make even more money. But SEO tools died pretty quick in those days (maybe still today I guess too) and so I wanted it to keep going so I tried to donate a percentage of what I made.

Scrapebox would have none of it. So I looked for some other way to help. They had only a handful of videos and I watched day in and day out where they spent their time answering the same questions over and over like a broken record.

Well thats pointless and that was development time they didn't have. Mind you scrapebox was new and as it was they were pushing out 1-3 updates a day sometimes. I mean I would email at breakfast and ask them "hey any chance you could add this feature?" and at lunch time there was an update with the feature. Scrapebox is more complex now and its not that simple to just add features that fast. Plus typically the majority of small features are already added and its larger features now then it was then when it first started.

So I knew answering repetitive questions was a problem for them and for me and every one as that meant less development.

So I started doing videos. They loved it and it helped people and so I kept going. I kept making more money, they had to answer less questions and was able to develop more and the community was able to get answers faster. It just all scaled well.

Well they put my videos on their site, and then after a while I started to be seen as an expert in Scrapebox. Well then people started coming to me for stuff and I made money selling lists and I was offered deals and JV type stuff.

I learned a valuable lesson from it all. I started just to give back to the community and help me and Scrapebox and everyone and I kept going. I still do it to give back to the community, because thats just who I am, but a side effect is I still get people that come to me. In fact I had coffee with a guy just today here in Phoenix that we are going to work together and we have some great opportunities. He even made reference today how he had watched so many of my videos over the years.

So what I do because I want to help, it pays me back in ways I couldn't possibly have foreseen or sought out. So everyone wins and I like doing it. Its kind of nice to have people message me and tell me how much the videos have helped them. :)

So I like doing it, Scrapebox wins, the community wins and I win. 9 years later, over a million views and 56,000 hours worth of videos watched. Can't go wrong there.
 
Last edited:
What an absolutely beautiful reply.

You have been the cornerstone of that community for years now. And the power of community never ceases to amaze me.

As a matter fact I heard of your lists before I heard of Scrapebox. And from there on it was an endeavour towards understanding what your lists meant and why people were all over it.

I then saw a bunch of your videos, bought my first Scrapebox license (Had a client pay for it, because I was having Paypal issues back then) and everything went to his email.

Used it hard for a few years, until the client went MIA. Then I forgot all about it. I then bought my own license a few years ago and I use it every other day.

I've said this before - I attribute most of my success with SB to you and your intricate understanding of the software and its use cases. Had it not been for those videos, I would've been clueless about it and that wouldn't have been so good.

So once again, thank you. I know you get this all the time, but this marketer had a changed perspective of marketing and traditional SEO - thanks to your taking the time and effort to make those videos. I still refer to them, every now and then.
 
I don't work for Scrapebox.

Oops :) I just figured you are the owner. You know, I've been using this tool for a week or so. I came across your videos and thought - this perfect youtube channel belongs to the owner for sure.
Thank you for the answer to my questions. Great job!
 
What an absolutely beautiful reply.

You have been the cornerstone of that community for years now. And the power of community never ceases to amaze me.

As a matter fact I heard of your lists before I heard of Scrapebox. And from there on it was an endeavour towards understanding what your lists meant and why people were all over it.

I then saw a bunch of your videos, bought my first Scrapebox license (Had a client pay for it, because I was having Paypal issues back then) and everything went to his email.

Used it hard for a few years, until the client went MIA. Then I forgot all about it. I then bought my own license a few years ago and I use it every other day.

I've said this before - I attribute most of my success with SB to you and your intricate understanding of the software and its use cases. Had it not been for those videos, I would've been clueless about it and that wouldn't have been so good.

So once again, thank you. I know you get this all the time, but this marketer had a changed perspective of marketing and traditional SEO - thanks to your taking the time and effort to make those videos. I still refer to them, every now and then.

Thanks for the kind words. :)

Im glad that the videos have been a help to you. I hope to make many more in 2019 and perhaps they will continue to be of good value. I wish you the best of luck!

Oops :) I just figured you are the owner. You know, I've been using this tool for a week or so. I came across your videos and thought - this perfect youtube channel belongs to the owner for sure.
Thank you for the answer to my questions. Great job!

Thanks!
 
Back
Top