webscrapping when acces gets denied

C45HC0W · Mar 14, 2019

I want to read out this list of odds of different betting websites using python and request and beautifulsoup however my access gets denied
this is my code:

Code:

import requests
from bs4 import BeautifulSoup as bs

url = 'oddsportal.com/soccer/europe/europa-league/dynamo-kyiv-chelsea-lIjhcPn4/'

r = requests.get(url)
soup = bs(r.text,'html.parser')
txt ='bt1'
results = soup.find_all(id= "odds-data-table")
for result in results:
    info = result.text
    print(info)

TasDePixels · Mar 14, 2019

what do you exactly mean by access gets denied ?

you have to check what's the output of this line :

r = requests.get(url)

alain12 · Mar 14, 2019

You need to assign a correct URL first. It should start with http:// or https://

That may be your problem.

edit: just tried the snippet your provided. The request goes through if you add 'http://', but the URL returns a 404 anyway, so get a working URL first.

C45HC0W · Mar 14, 2019

no thats not it. I removed it because everytime i write ht tp blackhat markes it as spam.

C45HC0W · Mar 14, 2019

so when I run the code and print out the results between the html text shows up this:
The page you requested is not available.
Page not found
This page not exist on OddsPortal.com!

is this maybe cause because I didnt put in useragent or something?

bartosimpsonio · Mar 14, 2019

Well you either have the url mispelled when you send it to requests or it's blocking the requests user agent. Lots of sites block python and wget.

C45HC0W · Mar 14, 2019

yes it was the useragent. case closed

webscrapping when acces gets denied

C45HC0W

Regular Member

TasDePixels

BANNED

alain12

Registered Member

C45HC0W

Regular Member

C45HC0W

Regular Member

bartosimpsonio

Elite Member

C45HC0W

Regular Member

Main Menu

Marketplace

Making Money

BlackHatWorld

Other