Best tool(s) to Scrape & translate articles + upload on WordPress?

Robzkie

Regular Member
Joined
Apr 27, 2017
Messages
347
Reaction score
83
The title says it.

I have a list of url. They're all articles that I want to translate in another language using Deepl. After that i need to upload all these articles on. WP.

What you think is the best solution? I'd like to minimize the expenses on VAs.

One idea I have would be to scrape the urls with Scrapebox, but there's no API integration with Deepl for what I know.

Otherwise I can scrape the articles and upload them in bulk on WP (any plugin for this?), And use Translatepress, which is not cheap, though.

Or I can build a custom automation for this, but have no idea of how much it could cost me.

Any idea?
 

BtcTradeNG

Junior Member
Joined
Feb 22, 2017
Messages
164
Reaction score
146
i have a working approach for this.
I created php scraper that takes the HTML of post content for any site, still has to be adjusted for the url you wanna usse it for though.

I can code a scraper for you if you want, you'd have to run it via google cloud console (Free and very fast)

The scraper takes page conteent in html and puts it into a CSV file, title, content , featured image, tags, etc.

Once it's in that format, i get it into any wordpres site using WP ALL IMPORT plugin (They have a templating system for importing post, alot of crazy stuff can be done).
1632715505510.png


If you need to translate, make sure you've already uuploaded to wordpress, then find a translation service that integrates with wordpresss... that way you solve the last issue.
Any who, DM me and i'll helpout when i can. it's a 3-4 step stuff, no need for VA.
 

hkhkhkhkhk123

Regular Member
Joined
May 26, 2021
Messages
234
Reaction score
76
does this auto site make money? Do you want to at least have some pictures attached to your post?
 

loopline

Jr. VIP
Jr. VIP
Joined
Jan 25, 2009
Messages
5,841
Reaction score
3,297
Website
contactformmarketing.com
The title says it.

I have a list of url. They're all articles that I want to translate in another language using Deepl. After that i need to upload all these articles on. WP.

What you think is the best solution? I'd like to minimize the expenses on VAs.

One idea I have would be to scrape the urls with Scrapebox, but there's no API integration with Deepl for what I know.

Otherwise I can scrape the articles and upload them in bulk on WP (any plugin for this?), And use Translatepress, which is not cheap, though.

Or I can build a custom automation for this, but have no idea of how much it could cost me.

Any idea?
Scrapebox does not have integration for that api, but there are other spinner apis in the article scraper plugin. Also the article scraper plugin can bulk post to wordpress.
 

loopline

Jr. VIP
Jr. VIP
Joined
Jan 25, 2009
Messages
5,841
Reaction score
3,297
Website
contactformmarketing.com
Ah! I didn't know that. That could be one point for Scrapebox.

@loopline, do you think I'll need proxies if I want to scrape and post let's say 50-100 articles in a day?
Well your posting to your own sites. So as long as your own server doesn't ban your IP, then you should be fine on the posting side. If its wordpress.com sites then you may want to slow it down or use private proxies, but 100 a day is pretty low, even if you do it every 5 mins, thats decently slow and totally doable in a day.

As for scraping, it depends on what sites the articles are coming from, like if its a directory like ezine articles you will need proxies, but some directories will not ban you for 50 to 100 articles if you go slow and are not hammering all at once. Else you can try some private proxies, bu I would suspect handful is enough.
 

Robzkie

Regular Member
Joined
Apr 27, 2017
Messages
347
Reaction score
83
Sounded great until I read that the plugin is only for Windows...
 

Salamouna

Jr. VIP
Jr. VIP
Joined
Aug 28, 2014
Messages
5,770
Reaction score
2,820
Website
www.blackhatworld.com
i have a working approach for this.
I created php scraper that takes the HTML of post content for any site, still has to be adjusted for the url you wanna usse it for though.

I can code a scraper for you if you want, you'd have to run it via google cloud console (Free and very fast)

The scraper takes page conteent in html and puts it into a CSV file, title, content , featured image, tags, etc.

Once it's in that format, i get it into any wordpres site using WP ALL IMPORT plugin (They have a templating system for importing post, alot of crazy stuff can be done).
View attachment 186721


If you need to translate, make sure you've already uuploaded to wordpress, then find a translation service that integrates with wordpresss... that way you solve the last issue.
Any who, DM me and i'll helpout when i can. it's a 3-4 step stuff, no need for VA.
For free?
 

Robzkie

Regular Member
Joined
Apr 27, 2017
Messages
347
Reaction score
83
does this auto site make money? Do you want to at least have some pictures attached to your post?

It will ame some money eventually, but for now it's a test on translated content.

Images, well I guess the best course of action would be to get some new ones, but that's of secondary importance to me now
 

mihag28

Newbie
Joined
Apr 22, 2013
Messages
5
Reaction score
0
The title says it.

I have a list of url. They're all articles that I want to translate in another language using Deepl. After that i need to upload all these articles on. WP.

What you think is the best solution? I'd like to minimize the expenses on VAs.

One idea I have would be to scrape the urls with Scrapebox, but there's no API integration with Deepl for what I know.

Otherwise I can scrape the articles and upload them in bulk on WP (any plugin for this?), And use Translatepress, which is not cheap, though.

Or I can build a custom automation for this, but have no idea of how much it could cost me.

Any idea?
Contact me for the deal! Thank you!
 

loopline

Jr. VIP
Jr. VIP
Joined
Jan 25, 2009
Messages
5,841
Reaction score
3,297
Website
contactformmarketing.com
Sounded great until I read that the plugin is only for Windows...
Mac lacks some of the components needed to work.

Basically neither windows nor mac was designed to work with scrapebox and such powerful software with massive connection capability and overall seo/internet marketing capability. that said, mac is even less friendly in design for this then windows.
 

Robzkie

Regular Member
Joined
Apr 27, 2017
Messages
347
Reaction score
83
WP Automatic Plugin does this.
scrapes and posts

@C4rnage569, Looks like it does!

I tested the demo version, but I can't figure out a few things:

1. Is the Deepl integrations working properly? I ask because they also have an integration with Sheets which is not great
2. How do you set it up to scrape content from a given list of URLs? Looks like you can scrape as many pages as you want, but from one domain at the time
 

Impaler

Newbie
Joined
Apr 1, 2021
Messages
3
Reaction score
0
@C4rnage569, Looks like it does!

I tested the demo version, but I can't figure out a few things:

1. Is the Deepl integrations working properly? I ask because they also have an integration with Sheets which is not great
2. How do you set it up to scrape content from a given list of URLs? Looks like you can scrape as many pages as you want, but from one domain at the time
Did you manage to figure out if DeepL integration works properly?
 

itz_styx

Jr. VIP
Jr. VIP
Joined
May 8, 2012
Messages
2,267
Reaction score
1,642
Website
argo-content.com
i used yandex API for my software to automatically translate scraped articles, worked well, but also depends on the language of the original article.
there used to be a free version of the API, but now there is only the paid version.
 

Royal96

Jr. VIP
Jr. VIP
Joined
Mar 16, 2019
Messages
208
Reaction score
118
Sounded great until I read that the plugin is only for Windows...

You can run it on a VPS. Take a look at Hyonix, they also have a sales thread here. For $12/m you can get a VPS with 4GB RAM which is more than enough to run these kind of software unless you're scraping a lot or running multiple software / threads at once. Then you can also let it run in the background, over night, etc. I run all of the scraping on a VPS, I think my laptop appreciates that as well :D
 

getivan

Junior Member
Joined
Sep 1, 2017
Messages
136
Reaction score
146
One of the Scrapebox premium plugins has a translation feature.
No idea how good or bad it might be, though.

Once you have your content, you could also create a small macro with something like "pulover's macro creator".
You could use that to copy/paste into your translator sessions, perhaps.
 

Robzkie

Regular Member
Joined
Apr 27, 2017
Messages
347
Reaction score
83
Hey guys, so far the easiest solution was the WordPress Automatic Plugin.

It does exactly what I asked here. I'm testing it with the Deepl API, and the translations with the free version of Deepl are crap. Not sure if the Pro version works any better.
 
Top