100% Automated Review Website From Scratch. My Journey Learning Python.

[beDreamful]

Hey there folks. I haven't been involved in anything here for a long time due to personal reasons. To keep it short, I'm making the career switch from campaign manager in digital marketing to software development. I learned Java a while ago making apps & I needed a project to get my hand deep into a project to learn python. This thread is to help me with accountability, help others achieve similar goals & maybe learn from people with a lot more experienced.

My goals are:
- Create a bot that will create a complete, SEO-optimized, review article using only an amazon URL.
- Every interaction with a website needs to look human.
- Articles need to have an infographic kinda vibe. The rest will be generated or spun somehow.
- The bot needs to optimize everything for voice search & google snippets.
- The bot needs to build the page on the WordPress website & do everything needed (which I'm not sure what it is...)

I could keep on writing more but I'll add them as I go.

The tools I'm using right now are just an old laptop with Linux and VS Code.

I'll try to update this thread as much as I can. It gives me a break from coding, swearing and slamming my head on the keyboard. I'll try to give everyone a look into my way-over-detailed log or the output of the bot frequently.

The first step is web scraping. I need a module that will handle 100% of the data gathering online. This is what I am working on right now. I need lots of data to write a proper review page from an amazon page.

Ask any questions, I don't mind them.

PS: I'll be referring the bot as MIA. The project was meant to be called Intelligent Marketing Assistant which is such a bad name. The acronym is IMA. As I was writing the first few lines of code, one of the Fast & Furious movies was on TV in the background. One of the girl's name is MIA and she's kinda hot. The MIA Bot was born.

 

[AlekhyaDas]

All the very best for your venture! Looks really interesting, I'll be following towards your journey.

 

[sunilbarnaba]

My goals are:
- Create a bot that will create a complete, SEO-optimized, review article using only an amazon URL.
- Every interaction with a website needs to look human.
- Articles need to have an infographic kinda vibe. The rest will be generated or spun somehow.
- The bot needs to optimize everything for voice search & google snippets.
- The bot needs to build the page on the WordPress website & do everything needed (which I'm not sure what it is...)

Your Goals are very interesting, one of my favorite subjects. All the best and I am following you.

 

[beDreamful]

All the very best for your venture! Looks really interesting, I'll be following towards your journey.

Thanks! I hope it will be interesting and full of knowledge.

Your Goals are very interesting, one of my favorite subjects. All the best and I am following you.

Thank you. Stay tuned buddy.

 

[Sandie2018]

That’s an absolutely fascinating idea what you are doing. I will follow your journey and hopefully be a future customer and buy the bot

 

[soheb123]

Just started learning python myself! Good luck!

 

[beDreamful]

That’s an absolutely fascinating idea what you are doing. I will follow your journey and hopefully be a future customer and buy the bot

Thanks mate. I might have a BST someday. Who knows. Gotta make stuff work first lol.

Just started learning python myself! Good luck!

Thanks! How do you like it? I prefer it over Java.

 

[SKF]

Interesting idea, gonna save a lot of time

 

[AntiHacker]

Good luck!

 

[beDreamful]

Interesting idea, gonna save a lot of time

You bet. It will take lots of time to make tho.

Edit: Typo

 

[beDreamful]

Here's what's going on so far. MIA is using her friend SCRAPER to gather data online. SCRAPER uses Selenium and Mozilla Firefox Webdriver to access Amazon pages. I tried sending requests directly but it would send me an error message and a link to their API.

Here's what happens when I feed MIA an Amazon link.

Link (Nutella Snack Packs): https://www.amazon.ca/dp/B07CB87Z15...uPWNsaWNrUmVkaXJlY3QmZG9Ob3RMb2dDbGljaz10cnVl

MIA Log. - The log is over-detailed for debugging purposes. You can kinda get what's going on step-by-step.

Mia Log # Sending URL to Scraper
Scraper Log # Amazon Page Data Init
Scraper Log # Gathering Data
Scraper Log # Cleaning Listing Name
Log # (,) Spacer Found
['Nutella and Go Snack Packs', ' Chocolate Hazelnut Spread with Breadsticks', ' Perfect Bulk Snacks for Kids', ' 52 Grams', ' Pack of 10']
Log # Nutella and Go Snack Packs
Scraper Log # Cleaning Brand Name
Log # (Brand Found
['Brand', ' Nutella']
Log # Stripping Whitespace
Log # Nutella
Scraper Log # Splitting About Section
Log # Snippet One: Pantry-perfect box holds 10 single packs, ideal for your Nutella-loving family and individually packaged for kids
Log # Snippet Two: The perfect mid-morning or after school and office snack: delicious Nutella hazelnut spread plus crunchy breadsticks
Log # Snippet Three: Dip, crunch, smile, repeat — a one-of-a-kind snacking experience
Scraper Log # Done
Mia Log # Data Received. Extracting..
Mia Log # Creating Article
Writer Log # Generating Subtitle
Log # Open Subtitle Database
Log # Randomize Number
Log # N = 3
Log # Return Subtitle From Database
Log # Does it stand out of the rest?
Mia Log # Writting Article
Mia Log # Article Finished

MIA Article. - Here's the current output. Really Basic Stuff.

NEW ARTICLE ------------------------------------------------------

Nutella and Go Snack Packs Review
Does it stand out of the rest?

-img-carrousel--------------------

Is it good? Here's a few perk from Nutella :

1.) The perfect mid-morning or after school and office snack: delicious Nutella hazelnut spread plus crunchy breadsticks
2.) Pantry-perfect box holds 10 single packs, ideal for your Nutella-loving family and individually packaged for kids
3.) Dip, crunch, smile, repeat — a one-of-a-kind snacking experience


Nutella and Go Snack Packs Overview

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna
aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia
deserunt mollit anim id est laborum.


Is Nutella and Go Snack Packs worth it?

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna
aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia
deserunt mollit anim id est laborum.


End of generation -----------------------------------------------

I going back in. I need to refine the way I extract the product name from the listing name. Since it's now 'working', I'll keep building the SCRAPER module.

 

[soheb123]

Thanks mate. I might have a BST someday. Who knows. Gotta make stuff work first lol.

Thanks! How do you like it? I prefer it over Java.

I got my initial training in JS so Java was always my annoyance. I think for me it’s a tie between JS and python so far. Both are easy to get the hang of

 

[xantan]

Best of luck. Gonna be watching this journey

 

[DDi]

i have something similiar in mind doing. just the content creation will be a bit different. did you figure out how to create posts in wordpress just using the database?
every post in wordpress is basically just one database entry, but also connected to some other databases. but creating that post should just be an simple sql create in the database. but still not 100% sure how to do it, iam also super new to python. so nice journey wish you good luck and work hard

 

[beDreamful]

I got my initial training in JS so Java was always my annoyance. I think for me it’s a tie between JS and python so far. Both are easy to get the hang of

I agree completely. Thanks for dropping in.

Best of luck. Gonna be watching this journey

Thanks mate

i have something similiar in mind doing. just the content creation will be a bit different. did you figure out how to create posts in wordpress just using the database?
every post in wordpress is basically just one database entry, but also connected to some other databases. but creating that post should just be an simple sql create in the database. but still not 100% sure how to do it, iam also super new to python. so nice journey wish you good luck and work hard

I'm not sure yet. One thing I know is that the article will be generated in pieces like this:
title.txt
subheading.txt
img1.jpeg
snippet_title.txt
affiliate_link.txt

Then I will probably get the bot to load the right pieces in a pre-made template. I don't know anything about SQL so far. I could simply fetch the text from a simple desktop file.

 

[Shaybib]

Most of this can be done with wp automatic plugin. Maybe you can built an extension on it

 

[beDreamful]

Most of this can be done with wp automatic plugin. Maybe you can built an extension on it

Thanks for the input! I'd rather start from scratch since it will be more customizable toward my goal. I will also learn a lot more along the way. I might be wrong but I think every piece of code can leave a footprint (patterns, etc.) . I'd rather not use any tool that can have a known footprint that can hurt me in the long run.

I will definitely look at how it works.

 

[blackcatseo20]

Hey OP, congrats and good luck learning python, you're off to a great start!

One question, are you planning to copy the Amazon listing description onto your site, and if so won't Google not index it since it's just a copy of another site?

 

[beDreamful]

Hey OP, congrats and good luck learning python, you're off to a great start!

One question, are you planning to copy the Amazon listing description onto your site, and if so won't Google not index it since it's just a copy of another site?

Thanks!

Nope. I'm gathering all the info from the amazon page but everything will be spun/generated. Here an example pulled from my "Nutella Log" from earlier:

Log # (,) Spacer Found
['Nutella and Go Snack Packs', ' Chocolate Hazelnut Spread with Breadsticks', ' Perfect Bulk Snacks for Kids', ' 52 Grams', ' Pack of 10']
Log # Nutella and Go Snack Packs

You can see that the listing name is a bunch of keywords delimited by some spacer. I try to scrape the name the best I can out of it to then use it:

Nutella and Go Snack Packs Review
Does it stand out of the rest?

I hope it makes sense!

 

[vyactor]

Learning some programming language was one of my choices in late 2018. I literally lost interest in coding in 2019. Good luck with achieving those goals, OP.