[Python] - How would you parse sponsored post from Facebook Newsfeed

CPAHaus

CPAHaus

Junior Member
Joined
Oct 26, 2016
Messages
115
Reaction score
21
Hey all o/

So I am playing around with python + selenium and for now I am trying to pull out sponsored post.

BUT... I can't figure out how to parse that out.
I was thinking about searching for the "Sponsored" string in the html source but it seems thats just not best way to do it.

Anyone has a better (working) method?

Happy New 2017 :)
 
Advertise on BHW
Selenium has selectors.
  • find_element_by_id
  • find_element_by_name
  • find_element_by_xpath
  • find_element_by_link_text
  • find_element_by_partial_link_text
  • find_element_by_tag_name
  • find_element_by_class_name
  • find_element_by_css_selector
To find multiple elements (these methods will return a list):

  • find_elements_by_name
  • find_elements_by_xpath
  • find_elements_by_link_text
  • find_elements_by_partial_link_text
  • find_elements_by_tag_name
  • find_elements_by_class_name
  • find_elements_by_css_selector
http://selenium-python.readthedocs.io/locating-elements.html

Try with Xpath. http://www.w3schools.com/xml/xpath_syntax.asp. Don't forget your element might not be available instantly after page load. Check this for Wait For operations http://selenium-python.readthedocs.io/waits.html. This coupled with a selector should work for you.
 
pasdoy said:
Selenium has selectors.
  • find_element_by_id
  • find_element_by_name
  • find_element_by_xpath
  • find_element_by_link_text
  • find_element_by_partial_link_text
  • find_element_by_tag_name
  • find_element_by_class_name
  • find_element_by_css_selector
To find multiple elements (these methods will return a list):

  • find_elements_by_name
  • find_elements_by_xpath
  • find_elements_by_link_text
  • find_elements_by_partial_link_text
  • find_elements_by_tag_name
  • find_elements_by_class_name
  • find_elements_by_css_selector
http://selenium-python.readthedocs.io/locating-elements.html

Try with Xpath. http://www.w3schools.com/xml/xpath_syntax.asp. Don't forget your element might not be available instantly after page load. Check this for Wait For operations http://selenium-python.readthedocs.io/waits.html. This coupled with a selector should work for you.
Click to expand...

Thanks for the reply @pasdoy

I've already written some small script with selenium where I can scroll down few times so the FB newsfeed can load more posts (and sponsored posts), and I've managed to pull out ahref links with BeautifulSoup but currently It's beyond me how to detect if there is some "sponsored post" or not :)

I guess I will keep trying :D
 
Nice that you found your way around it. With xpath you can get a list of posts, and filter each post if the xpath to SPONSORED is not there. I wonder if that would work. To work within an element object you can do: obj.xpath('./div') where ./ says to search from this element in the tree.
 
I will try it out today and let you know how it went :)
If I manage to get out what I wanted, I will light a money candle in your name bro ;)
 
You just need to know how to identify these sponsored posts. Look up what they have in common.
If you can't fetch them with BeautifulSoup methods, you could try using regular expressions. They will definetly be able to fetch the post, if you have the right expression of course.
 
try not to parse html with regex. maybe javascript if you really have to.
 
It's not that it can't be done or must not, it's just if it can be avoided why not.
 
Anyone interested into 5minutes of picking his brain about this python parsing for FB Sponsored ads?
Thanks :)
 
Any updates? Were you successfully able to identify the meaningful tags for the sponsored posts?
 
See if some texts inside it has some particular id and go for xpath to get the whole box, if you find some text inside it that is only in "Sponsored Post" box, you can get it with xpath by using "acenstor" keyword
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top