Web Scraping Advice
- Thread starter LukeN153
- Start date
TheToolbox
Elite Member
- Jul 25, 2021
- 1,568
- 643
- Apr 27, 2016
- 6,342
- 2,537
You can use Python.
- Jan 25, 2009
- 6,378
- 3,774
I need clarification to be able to more accurately help you.
You want to scrape the web looking for sites that have certain code?
Or you have a list of websites and you want to scan them and see if the contain certain code ?
or
- Aug 9, 2017
- 5,569
- 6,858
To search for a certain piece of code rather than keywords you can try https://publicwww.com/ but their database is a bit limited.
So for a particular WP theme I would search for "/wp-content/themes/themenamehere"
So for a particular WP theme I would search for "/wp-content/themes/themenamehere"
- Mar 10, 2010
- 42,697
- 107,138
Somebody found something that is going to make them money once they find the process to gather the information.
I hope this does turn into revenue for you as you are setting up a nice base for that.
LukeN153
Newbie
- Aug 3, 2020
- 17
- 5
You've never heard of anyone advertising this type of software? hmmm, interesting...
- Mar 10, 2010
- 42,697
- 107,138
I know about advertising using software such as you are looking for. I've done that in the past myself but my comment was on you finding an opportunity that could turn into money for you should what you are offering to those buy your product or service.
It reminds me of when I would sell in real life. I'd not target the businesses on the main roads but rather the ones on the off streets as I had little to no competition because almost everyone else would only target the businesses on the main roads.
LukeN153
Newbie
- Aug 3, 2020
- 17
- 5
I've created an Escrow payment plugin for WooCommerce and I'm trying to scrape some leads.
You are exactly right, trying to target some smaller businesses in some niche industries!
Zazdrel
Junior Member
- Jul 8, 2019
- 110
- 58
It depends on what kind of information you need to get.
I handle this kind of task with xpath queries and screaming frog