- Jan 17, 2015
- 5,263
- 4,956
I've been looking for a scraping tool to do a very simple task in multiple pages, but just be configured on the go.
For example, lets say I have a list of items and some pagination, and I need to scrape name of the item and the URL, and go into the page, 2, 3, 4 and so on.
But considering that each site will not have more than 50 pages, I won't need to do this like regularly, but once every day at most, and I could leave a simple Chromium browser just on the background scraping everything for a couple of minutes and come just to see the scraped content in CSV or Google Sheets or whatever
I was thinking on maybe a Chrome extension, or a service or something. The simplest the better. I don't want to do a very complex thing with Puppeteer or anything like that. I would prefer to do this with a Chrome Extension, because generally there is some login involved, so I would like to set up the browser with the credentials, and leave it there. Nothing too fancy, some Xpaths to locate the item names and the a href for the items urls, and not much else.
I've seen some Chrome extensions that actually can scrap 1 Xpath, but no automation pagination so far (which would be interesting to minimize the manual work)
Any good solution you could recommend me?
For example, lets say I have a list of items and some pagination, and I need to scrape name of the item and the URL, and go into the page, 2, 3, 4 and so on.
But considering that each site will not have more than 50 pages, I won't need to do this like regularly, but once every day at most, and I could leave a simple Chromium browser just on the background scraping everything for a couple of minutes and come just to see the scraped content in CSV or Google Sheets or whatever
I was thinking on maybe a Chrome extension, or a service or something. The simplest the better. I don't want to do a very complex thing with Puppeteer or anything like that. I would prefer to do this with a Chrome Extension, because generally there is some login involved, so I would like to set up the browser with the credentials, and leave it there. Nothing too fancy, some Xpaths to locate the item names and the a href for the items urls, and not much else.
I've seen some Chrome extensions that actually can scrap 1 Xpath, but no automation pagination so far (which would be interesting to minimize the manual work)
Any good solution you could recommend me?
