FabioSelau
Senior Member
- Oct 19, 2018
- 812
- 451
A while back I created a project where I built a Scrapy post bot that automatically published to my blog with an expired domain, and I got millions of hits over that year and a half. (https://www.blackhatworld.com/seo/journey-domain-expired-scrapy-ia-my-new-blog.1613608/)
Now I'm reworking my code to release it as open-source. The idea is that you configure your WordPress site, add target sites, and based on the configured cron (every hour, every 6 hours...) it retrieves posts from the target WordPress sites (with proxy if configured), rewrites them with AI (if enabled), uploads the image (if enabled), adds your site's watermark (if enabled), publishes to your site, pings IndexNow (if enabled), and finally calls your webhook if configured.
The entire architecture uses a Redis queue with several configurable queue types, such as sequential, parallel... The architecture was designed to be scalable and easy to implement features, meaning that if instead of retrieving the original image, we need to create one using some AI service, we can easily implement it.
Below are some features:
AI Content Engine
* Smart Rewriting: Uses LLMs (GPT-4o, DeepSeek) to rewrite titles and content, avoiding plagiarism.
* Context Awareness: Preserves the original meaning while improving SEO and readability.
* Internal Linking: Automatically builds internal links based on your site's history (SEO Link Building).
Orchestration & Performance
* Queue System: Powered by Redis & BullMQ. Handles massive workloads without timeouts.
* Concurrency Control: Configurable execution modes (Sequential or Staggered) to prevent server overload.
* Robust Scheduling: Built-in CRON scheduler to fetch new content automatically.
️ Media Management
* Auto-Upload: Extracts images from source, downloads, and uploads them to the WordPress Media Library.
* Watermarking: Automatically applies watermarks to images before uploading.
* Featured Image: Sets the featured image automatically.
️ Network & Security
* Proxy Support: Native support for Scrape.do (API Mode - freemium) or Standard Tunnel Proxies.
* Stealth Mode: Mimics real browser headers to bypass basic anti-bot protections.
Modern Dashboard
* React + Vite: A blazing fast interface to manage sites, view logs, and monitor queues.
* Real-time Stats: Monitor success rates, errors, and queue status instantly.
The idea is to improve the AI prompts and have it recreate the article using all SEO best practices, as well as integrate with WordPress SEO plugins.
The project will be open-source with Docker and PM2 already configured; just do a `git clone` and a `docker compose` and everything will be ready.
Do you have any suggestions for features?
Now I'm reworking my code to release it as open-source. The idea is that you configure your WordPress site, add target sites, and based on the configured cron (every hour, every 6 hours...) it retrieves posts from the target WordPress sites (with proxy if configured), rewrites them with AI (if enabled), uploads the image (if enabled), adds your site's watermark (if enabled), publishes to your site, pings IndexNow (if enabled), and finally calls your webhook if configured.
The entire architecture uses a Redis queue with several configurable queue types, such as sequential, parallel... The architecture was designed to be scalable and easy to implement features, meaning that if instead of retrieving the original image, we need to create one using some AI service, we can easily implement it.
Below are some features:
AI Content Engine
* Smart Rewriting: Uses LLMs (GPT-4o, DeepSeek) to rewrite titles and content, avoiding plagiarism.
* Context Awareness: Preserves the original meaning while improving SEO and readability.
* Internal Linking: Automatically builds internal links based on your site's history (SEO Link Building).
* Queue System: Powered by Redis & BullMQ. Handles massive workloads without timeouts.
* Concurrency Control: Configurable execution modes (Sequential or Staggered) to prevent server overload.
* Robust Scheduling: Built-in CRON scheduler to fetch new content automatically.
️ Media Management
* Auto-Upload: Extracts images from source, downloads, and uploads them to the WordPress Media Library.
* Watermarking: Automatically applies watermarks to images before uploading.
* Featured Image: Sets the featured image automatically.
️ Network & Security
* Proxy Support: Native support for Scrape.do (API Mode - freemium) or Standard Tunnel Proxies.
* Stealth Mode: Mimics real browser headers to bypass basic anti-bot protections.
Modern Dashboard
* React + Vite: A blazing fast interface to manage sites, view logs, and monitor queues.
* Real-time Stats: Monitor success rates, errors, and queue status instantly.
The idea is to improve the AI prompts and have it recreate the article using all SEO best practices, as well as integrate with WordPress SEO plugins.
The project will be open-source with Docker and PM2 already configured; just do a `git clone` and a `docker compose` and everything will be ready.
Do you have any suggestions for features?