[FREEBIE] Web scraping/Automation CLI tool

[aandryyy]

Hello,

As someone here suggested to me before offering my developer services it's better to validate my skill set and profile.

So I'd love to take 1/2 litlle-sized web automation/scraping project completely free. The catch is that I will be relating it here for all the other member.

I'd prefer not to do any GUI but if it will be used by more than 2/3 people I'll gladly add one.

I'll come here to this post every 1/2 days after 18:30 (after work) in order to update you about the freebies.

What I'll use for this bot: Python (requests, beautifulSoup, selenium, undetected-chrome, ecc.).

 

[paydette]

Nice offer. Good luck!

 

[imanseptian]

Thanks for this offer. I hope you can make automatic envato element download.

 

[Jeff Vh]

I'm interested in your web scraping skills! How do we proceed from here?
Would love to work with you.

 

[Chielou June]

Thanks for the giveaway. I Don't have any request atm, but please don't vanish after taking user ideas.
I have seen many similar kinds of stuff here in this section.

Thanks, n GL

 

[thetoothfairy]

Can you automate transfering money from the bank to my account

 

[polecat]

Sounds cool wish you good luck with this.

 

[Starblazer]

Please work on a project that scrapes articles based on a keyword and mixes 5-10 articles to create fresh content. The content need not be unique but should not have a single source.

We will manually spin & optimize it for SEO. This is needed by almost everyone in CPA niches.

 

[tidelow]

Please work on a project that scrapes articles based on a keyword and mixes 5-10 articles to create fresh content. The content need not be unique but should not have a single source.

We will manually spin & optimize it for SEO. This is needed by almost everyone in CPA niches.

Here here. I would 2nd this idea.

 

[aandryyy]

Hey, I'm reading now all your requests but I'll choose with a first come first serve order.

Thanks for this offer. I hope you can make automatic envato element download.

I have done your envato automatic downloader. I don't have a premium envato account so I just created an account and make it download from the free section.
For now not many checks are implemented (if you already have the file, etc.) but of course everything can be added.

Like a medium-sized project with envato I can think of could be:

• Have a premium account
• Launch *NOT MY* (for legal purpose) automatization that:
  • Downloads all the envato file
  • Save the envato product info in a database
  • Automatically create a page for download in your envato not-so-legal website
  • Upload the file to an upload service
  • add the uploaded file with everything you want on the download page (redirection, publicity, adfly, etc.)
• ?? donate all the revenue to some no-profit??

Here a video of the program working:
ok, no video, I uploaded it to vimeo but the moderators keep rejecting it. How should I upload it?


You call it via python3 envato.py username password, and it downloads every file in the free section.
If you want to add some features or that I do it with the premium files just pm me and we can start working on it.

Also pm me for the bot, I'll send it to you

 

[aandryyy]

I'm interested in your web scraping skills! How do we proceed from here?
Would love to work with you.

Just write me a dm

 

[Jeff Vh]

Just write me a dm

I believe you’ve locked this in settings? I cannot send you one.

 

[aandryyy]

Please work on a project that scrapes articles based on a keyword and mixes 5-10 articles to create fresh content. The content need not be unique but should not have a single source.

We will manually spin & optimize it for SEO. This is needed by almost everyone in CPA niches.

Let me understand it better.

Which type of article are we talking about?
From where they need to be scraped?

When you say mixing 5-10 articles creating fresh content, the content needs to have a logical sense? Or the article just need to have multiple phrases from the different article?
Or the newly generated content needs to be "human-readable" and seems authentic by a person?

I can do this, but it's more a big-sized project so I think I can't do it as a freebie. But if someone wants to invest some money just let me know!

 

[tidelow]

Let me understand it better.

Which type of article are we talking about?
From where they need to be scraped?

When you say mixing 5-10 articles creating fresh content, the content needs to have a logical sense? Or the article just need to have multiple phrases from the different article?
Or the newly generated content needs to be "human-readable" and seems authentic by a person?

I can do this, but it's more a big-sized project so I think I can't do it as a freebie. But if someone wants to invest some money just let me know!

Well, I don't want to hijack the other dudes idea but I believe he was referring to some of the functionality that the paid autoblogger softwares come with such as:

https://www.rssmasher.com/

One way I imagine it working with multiple sources:

1. Opens Google and searches target keyword
2. Captures top 10 search results (page title, meta description and url)
3. Scraper opens each url, starting from the top, regex finds first paragraph that includes the target keyword, parses out/saves locally.
4. Moves on to next url, ect, continues to scrape the first paragraph that contains the target keyword from each url and saves.
5. Combines all scraped paragraphs into one .txt

Cons: The search results for keywords will be a mix of style of sites/pages, some homepage, some authority news, ect so it will come with false positives and will be troubling to run a simple scan for a common html/css element across them all.

---

Alternatively, if you do the same steps above, except instead of scraping front end html, give the scraper 10 RSS feed sources to pull from.

 

[aandryyy]

Well, I don't want to hijack the other dudes idea but I believe he was referring to some of the functionality that the paid autoblogger softwares come with such as:

https://www.rssmasher.com/

One way I imagine it working with multiple sources:

1. Opens Google and searches target keyword
2. Captures top 10 search results (page title, meta description and url)
3. Scraper opens each url, starting from the top, regex finds first paragraph that includes the target keyword, parses out/saves locally.
4. Moves on to next url, ect, continues to scrape the first paragraph that contains the target keyword from each url and saves.
5. Combines all scraped paragraphs into one .txt

Cons: The search results for keywords will be a mix of style of sites/pages, some homepage, some authority news, ect so it will come with false positives and will be troubling to run a simple scan for a common html/css element across them all.

---

Alternatively, if you do the same steps above, except instead of scraping front end html, give the scraper 10 RSS feed sources to pull from.

Okay, I've understood.
Well, yes it's cool but it's too big for a freebie on my part. Surely doable but more like a medium-sized project for me and my resources right now. But if you want I can do a script that does something simpler with rss feed!

Just tell me. I've already done the envato script, I'll do the last one and then open a marketplace thread.

 

[Starblazer]

Which type of article are we talking about?
From where they need to be scraped?

When you say mixing 5-10 articles creating fresh content, the content needs to have a logical sense? Or the article just need to have multiple phrases from the different article?
Or the newly generated content needs to be "human-readable" and seems authentic by a person?

The article need not be human-readable. When we spin an article from a website, Google detects the original using positions of verbs & paragraphs. So it's necessary to scrape content from multiple URLs to rank on Google. The article need not make sense or look authentic. Most of the users won't even read the content. They just open the article and click on the CTA button.

Here is a scraper posted in BHW but expired. The developer promised to give an update but is currently not active. It has several additional features but they are not necessary.
https://www.blackhatworld.com/seo/get-tor-universal-scraper-free-for-all-bhw-members.962455/

 

[enterpreneur]

Can you work on some things to mass create pinterest pins?