Best tool(s) to Scrape & translate articles + upload on WordPress?

R

Robzkie

Senior Member
Joined
Apr 27, 2017
Messages
948
Reaction score
329
The title says it.

I have a list of url. They're all articles that I want to translate in another language using Deepl. After that i need to upload all these articles on. WP.

What you think is the best solution? I'd like to minimize the expenses on VAs.

One idea I have would be to scrape the urls with Scrapebox, but there's no API integration with Deepl for what I know.

Otherwise I can scrape the articles and upload them in bulk on WP (any plugin for this?), And use Translatepress, which is not cheap, though.

Or I can build a custom automation for this, but have no idea of how much it could cost me.

Any idea?
 
Advertise on BHW
i have a working approach for this.
I created php scraper that takes the HTML of post content for any site, still has to be adjusted for the url you wanna usse it for though.

I can code a scraper for you if you want, you'd have to run it via google cloud console (Free and very fast)

The scraper takes page conteent in html and puts it into a CSV file, title, content , featured image, tags, etc.

Once it's in that format, i get it into any wordpres site using WP ALL IMPORT plugin (They have a templating system for importing post, alot of crazy stuff can be done).
1632715505510.png


If you need to translate, make sure you've already uuploaded to wordpress, then find a translation service that integrates with wordpresss... that way you solve the last issue.
Any who, DM me and i'll helpout when i can. it's a 3-4 step stuff, no need for VA.
 
Ha! This is already a nice speed up!
 
does this auto site make money? Do you want to at least have some pictures attached to your post?
 
Robzkie said:
The title says it.

I have a list of url. They're all articles that I want to translate in another language using Deepl. After that i need to upload all these articles on. WP.

What you think is the best solution? I'd like to minimize the expenses on VAs.

One idea I have would be to scrape the urls with Scrapebox, but there's no API integration with Deepl for what I know.

Otherwise I can scrape the articles and upload them in bulk on WP (any plugin for this?), And use Translatepress, which is not cheap, though.

Or I can build a custom automation for this, but have no idea of how much it could cost me.

Any idea?
Click to expand...
Scrapebox does not have integration for that api, but there are other spinner apis in the article scraper plugin. Also the article scraper plugin can bulk post to wordpress.
 
loopline said:
Also the article scraper plugin can bulk post to
Click to expand...

Ah! I didn't know that. That could be one point for Scrapebox.

@loopline, do you think I'll need proxies if I want to scrape and post let's say 50-100 articles in a day?
 
Robzkie said:
Ah! I didn't know that. That could be one point for Scrapebox.

@loopline, do you think I'll need proxies if I want to scrape and post let's say 50-100 articles in a day?
Click to expand...
Well your posting to your own sites. So as long as your own server doesn't ban your IP, then you should be fine on the posting side. If its wordpress.com sites then you may want to slow it down or use private proxies, but 100 a day is pretty low, even if you do it every 5 mins, thats decently slow and totally doable in a day.

As for scraping, it depends on what sites the articles are coming from, like if its a directory like ezine articles you will need proxies, but some directories will not ban you for 50 to 100 articles if you go slow and are not hammering all at once. Else you can try some private proxies, bu I would suspect handful is enough.
 
Sounded great until I read that the plugin is only for Windows...
 
BtcTradeNG said:
i have a working approach for this.
I created php scraper that takes the HTML of post content for any site, still has to be adjusted for the url you wanna usse it for though.

I can code a scraper for you if you want, you'd have to run it via google cloud console (Free and very fast)

The scraper takes page conteent in html and puts it into a CSV file, title, content , featured image, tags, etc.

Once it's in that format, i get it into any wordpres site using WP ALL IMPORT plugin (They have a templating system for importing post, alot of crazy stuff can be done).
View attachment 186721


If you need to translate, make sure you've already uuploaded to wordpress, then find a translation service that integrates with wordpresss... that way you solve the last issue.
Any who, DM me and i'll helpout when i can. it's a 3-4 step stuff, no need for VA.
Click to expand...
For free?
 
hkhkhkhkhk123 said:
does this auto site make money? Do you want to at least have some pictures attached to your post?
Click to expand...

It will ame some money eventually, but for now it's a test on translated content.

Images, well I guess the best course of action would be to get some new ones, but that's of secondary importance to me now
 
Robzkie said:
The title says it.

I have a list of url. They're all articles that I want to translate in another language using Deepl. After that i need to upload all these articles on. WP.

What you think is the best solution? I'd like to minimize the expenses on VAs.

One idea I have would be to scrape the urls with Scrapebox, but there's no API integration with Deepl for what I know.

Otherwise I can scrape the articles and upload them in bulk on WP (any plugin for this?), And use Translatepress, which is not cheap, though.

Or I can build a custom automation for this, but have no idea of how much it could cost me.

Any idea?
Click to expand...
Contact me for the deal! Thank you!
 
Robzkie said:
Sounded great until I read that the plugin is only for Windows...
Click to expand...
Mac lacks some of the components needed to work.

Basically neither windows nor mac was designed to work with scrapebox and such powerful software with massive connection capability and overall seo/internet marketing capability. that said, mac is even less friendly in design for this then windows.
 
C4rnage569 said:
WP Automatic Plugin does this.
scrapes and posts
Click to expand...

@C4rnage569, Looks like it does!

I tested the demo version, but I can't figure out a few things:

1. Is the Deepl integrations working properly? I ask because they also have an integration with Sheets which is not great
2. How do you set it up to scrape content from a given list of URLs? Looks like you can scrape as many pages as you want, but from one domain at the time
 
Robzkie said:
@C4rnage569, Looks like it does!

I tested the demo version, but I can't figure out a few things:

1. Is the Deepl integrations working properly? I ask because they also have an integration with Sheets which is not great
2. How do you set it up to scrape content from a given list of URLs? Looks like you can scrape as many pages as you want, but from one domain at the time
Click to expand...
Did you manage to figure out if DeepL integration works properly?
 
i used yandex API for my software to automatically translate scraped articles, worked well, but also depends on the language of the original article.
there used to be a free version of the API, but now there is only the paid version.
 
Robzkie said:
Sounded great until I read that the plugin is only for Windows...
Click to expand...

You can run it on a VPS. Take a look at Hyonix, they also have a sales thread here. For $12/m you can get a VPS with 4GB RAM which is more than enough to run these kind of software unless you're scraping a lot or running multiple software / threads at once. Then you can also let it run in the background, over night, etc. I run all of the scraping on a VPS, I think my laptop appreciates that as well :D
 
One of the Scrapebox premium plugins has a translation feature.
No idea how good or bad it might be, though.

Once you have your content, you could also create a small macro with something like "pulover's macro creator".
You could use that to copy/paste into your translator sessions, perhaps.
 
Hey guys, so far the easiest solution was the WordPress Automatic Plugin.

It does exactly what I asked here. I'm testing it with the Deepl API, and the translations with the free version of Deepl are crap. Not sure if the Pro version works any better.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top