bandititf9
Regular Member
- Jun 20, 2012
- 423
- 196
Good luck man.
Your tool is pretty decent from what you showed, looking promising!
Your tool is pretty decent from what you showed, looking promising!
I only published 3 articles on my hobby website, the automated websites will be up when I finish all the features. Don't want to post the articles until they're polished. Probably will take me a week or two to finish this.How many articles have you done so far, as in published?
Very Awesome journey, Thank you for the information you're sharing here, it will be fun to follow it, till the end.
My questions :
1- What Python Libraries you used to build the script?
2- Any guide or schema for creating a similar tool? (because I have some basics in python and don't know where to start from)
I experimented with this and even GPT3 produces worse results than my non-abstract methods. The difficult element is identifying which parts of articles you need to extract.There are some free GPT3 alternatives (https://nlpcloud.io/gpt-3-open-source-alternatives-gpt-j-gpt-neo.html). Can these be used to create new content instead of just scrapping old content. I think combining bits and pieces of content from multiple places will make it loose the contextual flow and will make it look more spammy even if it is unique. I think AI can do much better if it understands the trillions of content training data and churns out something meaningful and maybe something new and creative.
- Selenium and Helium if I need to automate a browser
- Nider for writing on images
- Pillow for image manipulation
- sentence_transformers, torch for ML
- spacy for NLP
- python-unsplash
- For uploading to WP REST API I'm just using requests/json/base64
Python 3.6 works the best with these, above there's gonna be dependency hell.
Just divide the project into small steps. For example:
I will set up a simple script for you guys later this week so you got some place to start.
- Take a keyword.
- Scrape top 10 results from Google
- Get something from these sites. Combine into an article. I used to extract contents using Newspaper.
- Paraphrase. Start with this: https://huggingface.co/tuner007/pegasus_paraphrase
What does ML stands for?
- Selenium and Helium if I need to automate a browser
- Nider for writing on images
- Pillow for image manipulation
- sentence_transformers, torch for ML
- spacy for NLP
- python-unsplash
- For uploading to WP REST API I'm just using requests/json/base64
Python 3.6 works the best with these, above there's gonna be dependency hell.
Just divide the project into small steps. For example:
I will set up a simple script for you guys later this week so you got some place to start.
- Take a keyword.
- Scrape top 10 results from Google
- Get something from these sites. Combine into an article. I used to extract contents using Newspaper.
- Paraphrase. Start with this: https://huggingface.co/tuner007/pegasus_paraphrase
Machine LearningWhat does ML stands for?
My WordPress sites aren't like most WordPress sites. I'm using headless with REST API, ACF, custom posts, pages load like static sites because they are fully cached at all times. I'm using it for the plugin ecosystem. It speeds up the process of replicating sites for me. Unless I'd be looking into making hundreds of sites which isn't necessary.If I was you, I would get rid of Wordpress cause it does nothing here.
Think outside of the box. What kind of article are you writing? What kind of article would be easy to construct this way without repetitions? An easy example is a list of items. But there are several other types you can do. There's a lot of stuff you can look at: meta-descriptions, headings, latent semantic keywords...Cool journey, any tips on knowing which parts of articles to use. So it keeps being a coherent article?
My solution is much faster and more accurate. I tried several GPT-3 tools. I also got access to the GPT-3 API itself a year ago.Yet another interesting journey
Have you considered using GPT-3 based tools like jarvis for your hobby site?
Good luck!