Trying to Create Website Scraper

S

slow312

Newbie

Jan 13, 2016

21

1

• Aug 21, 2017

• #1

Hello all,

I am trying to create a website scrapper for job posting sites like Indeed to return results for me via python and beautifulsoup. However, I cannot seem to get it to work for certain websites that has a unicode thing? The output is just random numbers.

Should I just pay on fiverr for a gig for someone to handle this issue?

 

cryptons

Supreme Member

Jun 12, 2013

1,408

494

• Aug 21, 2017

• #2

use python with scrappy can be easily done!

 

M

Marcel Freiberg

Junior Member

Aug 6, 2017

171

71

• Aug 21, 2017

• #3

Hi,
I would suggest to pay someone on fivvr to google the answer for you

Google beautyfulsoup unicode and Look at the stackoverflow Thread "Python correct encoding Websites" from 2016
It will explain how to Encode the response and to get the right Output.

Good luck
Mf

 

S

slow312

Newbie

Jan 13, 2016

21

1

• Aug 22, 2017
• Thread Starter Thread Starter

• #4

Marcel Freiberg said:

Hi,
I would suggest to pay someone on fivvr to google the answer for you

Google beautyfulsoup unicode and Look at the stackoverflow Thread "Python correct encoding Websites" from 2016
It will explain how to Encode the response and to get the right Output.

Good luck
Mf

Click to expand...

haha thank you.

 

M

Marcel Freiberg

Junior Member

Aug 6, 2017

171

71

• Aug 22, 2017

• #5

Glad to help, if you have further questions do not hesitate to ask. Sometimes it is useful to get a hind for the right search keywords.

Best wishes with your scraper
mf