Michael Goode
Newbie
- Aug 7, 2016
- 23
- 2
I'm fairly proficient in python and creating web interfaces. I'm looking for some project ideas to work on publicly which more be of use to someone.
Any ideas?
Any ideas?
Python Project Suggestions
- Thread starter Michael Goode
- Start date
cole9595
Newbie
- May 15, 2018
- 9
- 1
I'm just learning python and have hit a roadblock I can't seem to get past. If you have a free minute would you be able to PM me (or post here if against the rules) and help me? I just need helping figuring out the right xpath/css selector to use for webdriver to select a specific field. The page has a bunch of nested DIVs and nothing I try seems to work. If not, no worries. Take care.
Michael Goode
Newbie
- Aug 7, 2016
- 23
- 2
I'm unable to PM until I reach a higher level of posts
cole9595
Newbie
- May 15, 2018
- 9
- 1
Yeah, I've downloaded 3-4 different chrome extensions and did find one that allows me to select the box and gives me the relative xpath but python is still unable to locate it. I suspect it has to do with the way the page is created (or just my lack of understanding of xpath/webdriver, LOL). I spent 2 hours last night on just getting this one box highlighted and type something into it. fun times! lol
Michael Goode
Newbie
- Aug 7, 2016
- 23
- 2
Marcel Freiberg
Junior Member
- Aug 6, 2017
- 171
- 71
@OP if you are bored you could simple build a little application based on python flask bootstrap celery (+rabbitmq) and beautifulsoup/selenium.
Setup e.g. celery worker threads behind tor-network (depending on scraping goal) on a few vps. Main application a microservice with flask and as Frontend a nice bootstrap application (input + dashboard) consuming the services.
Useful for all different kind of stuff
Setup e.g. celery worker threads behind tor-network (depending on scraping goal) on a few vps. Main application a microservice with flask and as Frontend a nice bootstrap application (input + dashboard) consuming the services.
Useful for all different kind of stuff
Michael Goode
Newbie
- Aug 7, 2016
- 23
- 2
@OP if you are bored you could simple build a little application based on python flask bootstrap celery (+rabbitmq) and beautifulsoup/selenium.
Setup e.g. celery worker threads behind tor-network (depending on scraping goal) on a few vps. Main application a microservice with flask and as Frontend a nice bootstrap application (input + dashboard) consuming the services.
Useful for all different kind of stuff
I have just built a fully functioning flask setup as a trading interface which trades on the Binance Exchange. Unfortunately due to lack of successful trading algorithms to implement I have lost interest in the project. It does fully function and trade automatically though
Michael Goode
Newbie
- Aug 7, 2016
- 23
- 2
Please elaborate
- Apr 26, 2017
- 428
- 156
1)you'd find a big site with community features. A commenting, profile and following system for example. Then you make scripts that create accounts, follow other accounts, comment links or have links in the profile. You just have to do research to find sites like this that don't have sophisticated anti-spam, but are big enough that your links aren't an easy to find footprint
2) parasite making bots for sites like jeuxvideo, change.org. Where you spin articles with a link and google ranks them high
Before making a full bot its worth trying to do it all manually to see how hard it is to make $1. Then a bot system will scale it to much more.
Likely it is the same with Facebook as with Google, they always change class names dynamically so on the next page load might all look different.
Reload few times to see if that is the case here
cole9595
Newbie
- May 15, 2018
- 9
- 1
Excellent suggestion. I just loaded the page on Microsoft Edge which is something I haven't touched in years (just to ensure no cache). It appears that the class names remain the same when reloaded.
Mh better to have the actual page, but how about
//div[contains(@Class, '_1mf _1mj')]
And from there on no clue what you are trying to get...
cole9595
Newbie
- May 15, 2018
- 9
- 1
Thanks, I've tried it before and receive the error "Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[contains(@Class, '_1mf _1mj')]"}"
I'm trying to get the text box that would let me type in text to be able to post to my page. It seems facebook has code that rearranges a few things to remove the "Write something..." placeholder text and allows you to enter what you want. I just can't seem to figure out exactly what I need to do to get selenium to mimic this.
Yep, it's basically what I mentioned with class, could be ID, placeholder etc. Point being Google does that for a bunch of their products, when you log in next time all things are different so makes it very hard to make a bot (with simple class/id etc.) Keep looking!
cole9595
Newbie
- May 15, 2018
- 9
- 1
lol, I've been trying just about every ID, class name, text, etc that I could find. I've spent 4+ hours with no luck. I think I'm going to take a different approach. I'm going to use a 3rd party site (like h@@tsuite) to handle the posting to the page and I'll just use python/selenium to post to that 3rd party site. That way facebook won't see the script, they'll just see the authorized 3rd party posting on my behalf. The 3rd party site may be easier to figure out too. lol
Appreciate your time.
More than likely yes. Most other sites don't change classes etc on each login so you should definitely find it easier
Michael Goode
Newbie
- Aug 7, 2016
- 23
- 2
I'm thinking of maybe some sort of social media bot combined with a web server interface for the user