[Code] Scrape Google PAA Using Python + Requests

import requests
ModuleNotFoundError: No module named 'requests'
pip install requests

There are many libs that you need to install separately from python core, so there is a lib which install libs, called pip. If you have not installed pip you can use:

python -m pip install requests
 
import requests from lxml.html import fromstring import lxml.html header = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36', 'Accept-Language': 'en-US,en;q=0.9', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' } query = 'weight loss tips' response = requests.get(f'https://www.google.com/search?q={query}&start=0', headers=header).text tree = lxml.html.fromstring(response) node = tree.xpath('//@data-q') print(node)
I ran this code with help of a friend.
The queries returns only 4 results. Is that the limitation of this or do I have to modify something in order to get more questions scraped per query?
 
I ran this code with help of a friend.
The queries returns only 4 results. Is that the limitation of this or do I have to modify something in order to get more questions scraped per query?
There is no limitations in the code as I can see, it was the result for this query:


query = 'weight loss tips'
 
import requests
ModuleNotFoundError: No module named 'requests'
You should install "Requests" Module open just type "python pip install requests" , you can watch a video how to do that in youtube , i'm building a tool right now for this matter
 
I'm not tech savvy but this one looks compelling
Saved up
 
It's working nice. Very basic but it's a good starting point for something better. Thanks :)
 
That a nice code but your ip will be blocked after too many requests even when you are rotating user agents you need to use proxies at least , i can design user friendly interface for it with the nessecarry adjustement (for free ofcourse) if you guys interested
Please Do for me
 
Please Do for me
I already did it , it took me 2h straight to complete whole GUI project , i didn't change anything iam_ironman script (credit to him) i just build GUI around it
you can watch how it works :
img404.jpg


and you can download the app here : https://mega.nz/file/U5YSwIDa#AcihlHrQlnTKADE9e-wWXPbe8r8UsHbBx8YLEZHyIH0

if you have any suggestions i'm happy to help, it is a simple basic app , but it can be adjust it for advance requierements
 
Back
Top