A new AI journey! (content automation)

Advertise on BHW
How much money have you spent on the project? Ballpark.
 
How do you do keyword research? Do you scrape google autocomplete?
 
Currently I'm working on REST API and a control gate if the URL already exist in the sitemap.

For questions keywords: I'm doing something similar to what Sartre is doing, PAA scraper google auto complete, weed out what makes no sense with embeddings/cosine and cluster them in silos.

for everything but questions: I was using yive ranker, but the credit system pissed me so much I coded something with the google-ads API and it feeds the csv based on what I'm looking for in term of volume/difficulty/KGR/etc
Then I get title ideas from the GPT3.5 API.
domkis30 said:
How do you do keyword research? Do you scrape google autocomplete?
Click to expand...
 
This is really inspiring but I'm thinking of starting a similar journey.
So can I be in touch with you?
 
bomnaox said:
This is really inspiring but I'm thinking of starting a similar journey.
So can I be in touch with you?
Click to expand...
Sure, don't hesitate. You can pm me or post here
 
Talyn said:
Hey all,

After making good money with fiction books on KDP for years, it's time for me to embark on a new journey. I got tired of dealing with ghostwriters and the fact that the life cycle of non-fiction books is getting shorter and shorter.

I have no proper CS education, even if I have a few agile certs, but learned "on the field" for years, I'm not even halfway decent in coding, but I'm still learning every day.

At the beginning of this year, I started playing with Chat GPT, which was getting a lot of hype. Since I can code a bit in Python, I started playing around with the unofficial API at that time and then moved on to the official Davinci and recently the GPT3.5 API.

As of today, I have created three sites:

  1. The first website:
    I used the official Davinci-003 API and unofficial/wrapper ChatGPT (which was a pain in the ass to deal with because of incomplete answers). The algorithm was really basic, just a list of keywords and ideas to use for each keyword in a CSV file. The niche is kind of related to finance and it's not getting a lot of traffic (and the search volume in google keywords is non existent).

  2. The second website focuses on as few uncommon niches:
    It's a pure PAA website, with 700-800 words articles. I switched to the GPT-3.5 turbo and I introduced new few functionalities, like pushing articles to WordPress automatically, basic article outlining process (pictures, videos, etc.)
    So far, the website has 5k pages indexed and is getting some decent traffic.
    This website also pushed me to work on a solution for indexing pages faster. I implemented the functionality to send the URL to GSC with the Google API, and so far, it works well.

  3. The third site is a PAA/education site in a not too narrow niche.
    This time I focused on my PAA scrapper, which feeds a an input CSV and then creates engaging but basic outlines before prompting/generating articles. It was a great introduction to Re3 and how to deal with numerous prompts to generate a consistent article (without getting millions of introductions/conclusions in the same article). This website is getting decent traffic with 600 articles ranging from 1500 to 12000 words long. For now, I don't plan to generate more articles as I hit a wall with content generation and am struggling to get a good algorithm for Re3. I'm still not happy with my outputs, and I'd like to have GPT3.5 generate more "storytelling" articles. I'm actively working on it.
    The whole site got generate within 72h and got indexed within 48h so my way to push the articles to google seems to be working
  4. The fourth website is in a highly competitive niche, but with readers who have a tendency to binge shop.
    In order to rank better, I'm focusing on keyword volume and interlinking. Here's what I'm working on:
  • How to validate keywords (cosine analysis)
  • Clustering keywords (maybe K-Means? DBSCAN? Hierarchical clustering? with or without using embeddings?)
  • Interlinking (cosine and embeddings?)
  • How to write titles on thumbnails + quality control (pillow)
  • Code architecture, even if I commented it, it's a mess, and as I added more functionalities, I don't want to turn it into something impossible to maintain.
What I'm working on in the long haul:
  • Improving the content quality with Re3. I'm considering hiring a tutor/teacher to learn faster.
  • Improving quality control before pushing the articles.
  • Streamlining to be able to feed a few keypoints/data and let it run alone with some gate controls.
  • SQL: several CSVs with several thousand lines is a pain to deal with
  • Performance analysis. So far, I track the time it took for each article and the word count.
  • UI, command line is cool, but not fun, I'd like to have a nice dashboard.
  • Multithreading with concurrent.futures / ThreadPoolExecutor as the bottleneck is the API and not the processing power if I don't use any NLP/LLM in local like alpa.
Click to expand...
Congratulations dear..
Best of luck
 
Today I'm working on a keyword cleaner using spaCy, so far it needs some work, as it is getting weird errors.

I'm also looking for a better host, the one I'm using is really bad, their default PHP version is 5.6 and when I set it up to 8.0 or another one I get mysqli PHP extension not found when I try to install WP.
already 4 days my ticket is open and they can't fix it, everytime they answer me it's fixed, it's not.
 
That sounds like you have got already some experience with NLP and AI. Your time will be better spent on that rather than building UI.
Talyn said:
  • UI, command line is cool, but not fun, I'd like to have a nice dashboard.
  • Multithreading with concurrent.futures / ThreadPoolExecutor as the bottleneck is the API and not the processing power if I don't use any NLP/LLM in local like alpa.
Click to expand...
I wouldn't combine these things because of the risk you will have big codebase for a simple project that does 2 or 3 things.

I coded a dashboard and threw it away later on cause I had to implement it for every single feature. It made me spend 5 times as much time before the screen. I had my head busy with UI instead of automation. I couldn't see the money pouring in - the work was moving too slow for a fast moving market like this.

If you can afford and want to turn it into something like a SaaS, hiring someone for frontend / backend work is a smart idea. First, you must make decent money.

Most ballers on this forum skip UI for projects that aren't SaaS. And I would advise doing the same since I just learned it the hard way. Coding backend, frontend, automation, AI all combined together is expensive as hell.

I worked at a simple email marketing automation app one time and the budget of $100k was sufficient for a year of work. It wasn't successful as money went for development instead of marketing.
 
reaaski said:
That sounds like you have got already some experience with NLP and AI. Your time will be better spent on that rather than building UI.

I wouldn't combine these things because of the risk you will have big codebase for a simple project that does 2 or 3 things.

I coded a dashboard and threw it away later on cause I had to implement it for every single feature. It made me spend 5 times as much time before the screen. I had my head busy with UI instead of automation. I couldn't see the money pouring in - the work was moving too slow for a fast moving market like this.

If you can afford and want to turn it into something like a SaaS, hiring someone for frontend / backend work is a smart idea. First, you must make decent money.

Most ballers on this forum skip UI for projects that aren't SaaS. And I would advise doing the same since I just learned it the hard way. Coding backend, frontend, automation, AI all combined together is expensive as hell.

I worked at a simple email marketing automation app one time and the budget of $100k was sufficient for a year of work. It wasn't successful as money went for development instead of marketing.
Click to expand...

Thank you for your input!
I think I will limit the UI with a basic: instance is running / instance crashed, or something similar, as I would like to be able to get an overview without circling between different instances in command line/bash

I don't plan to take the SaaS road, the support would be totally insane, I already have to deal with insane clients in my normal stable business, and dealing with twice the teaching/educating clients is a big no for me.
If I'd take that road, it would be only a SaaS article generator: generating ideas on a keyword, the outline, the article with basic Re3 and an option for import/export, and payment based on a credit system indexed on token consumption.
But if I'm at that stage it means my AI system is up, operational and bringing some serious money to fund this SaaS service.

small update: I finished the keyword cleaner with Spacy, it takes ~10min to clean a csv file with 400 lines, 6 columns header.
I'm now stress testing it with my giant unclean PAA keywords list (~15k lines)
 
Last edited:
BroTherm said:
Are you using Spacy's similarity?
Click to expand...
Yes with their en_core_web_lg model. Basically, I take pair of keywords and use Spacy's similarity. It's not super elaborated and could be done in a better way, but as an MVP it's quite ok.
I got the inspiration from Blog Pro's journey.

I'm working now on a social signal bot, it will work independently from the content generator as I don't want to pin 6000 articles in 2 days.
The MVP will get the list of articles from the sitemap and use AI to write a short engaging description, use pillow to write an eye catching title on the picture and post it.
Then I will work on a system to post in priority articles with title keyword gets the most volume
 
Super interesting journey, you seem like a bright person asking yourself the right questions, sincerely wishing you the best of luck!
Would be interested in hearing more the NLP side of stuff and your simplified Re3 implementation. Did you manage to solve the lack of 'flow' between the different sections of the blog post for example?
 
Sesh7 said:
Super interesting journey, you seem like a bright person asking yourself the right questions, sincerely wishing you the best of luck!
Would be interested in hearing more the NLP side of stuff and your simplified Re3 implementation. Did you manage to solve the lack of 'flow' between the different sections of the blog post for example?
Click to expand...

Thanks!
You can improve the flow without going to deep in the NLP side; think you're making a collage with openai outputs, you paste your main pieces, then you make the space in between more continuous by asking the openai API to fill in the spaces between your main parts, just by doing that you can reach average copywriter quality with the GPT-3.5 model

To go above and beyond, you'd need to train/fine-tune a model to your needs.

Today I worked on:
- the keyword cleaner, a brute approach with spaCy, even with pipelines, won't work or would need serious power when you reach keywords lists with a few k inside, as I was working on pairs, you can easily go into 100-150m similarity checks, which is insane.

- The core algorithm for article generation, I picked a tight niche, the website has a 800-1000 articles potential for the core, and some keywords with good KD&KGR have decent volume, so I take that opportunity to take it a step further (flow and readability)
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top