Anyone good with selenium (python)?
- Thread starter bk071
- Start date
Brian Alexander
Regular Member
- Aug 12, 2016
- 454
- 317
I've been using Selenium Webdriver with .NET for years.
PM me the issue - I can probably help you out.
PM me the issue - I can probably help you out.
bk071
Elite Member
- Nov 24, 2010
- 3,302
- 8,845
Figured it out myself.
Thanks anyway, @Brian Alexander!
Thanks anyway, @Brian Alexander!
bk071
Elite Member
- Nov 24, 2010
- 3,302
- 8,845
What do you need help with?
Is there a way we can talk in particular on this website?
Iamhere123
Newbie
- Jul 28, 2017
- 13
- 0
Speaking of helping a brother out, anyone know how to scrape into an excel sheet. I've learnt a lot about selenium except for the only thing I wanted which was how to scrape into excel lmao xD
- Mar 21, 2013
- 24,760
- 31,551
Scrape it into CSV and then import into excel.
Iamhere123
Newbie
- Jul 28, 2017
- 13
- 0
Yeah haha except when I do it it just gives me a blank spreadsheet xD. Do you have an example script that works? I'd kill for that..... lmao.
GoDesain
Regular Member
- Feb 26, 2011
- 491
- 224
if you able to scrap using scrapy is much better,
this sample tutorial using scrapy : https://www.blackhatworld.com/seo/create-simple-python-scraper-bot-with-scrapy.946490/
Iamhere123
Newbie
- Jul 28, 2017
- 13
- 0
Scrapy is good but it can't replace an actual browser. Selenium gets the job done where you want to emulate a browser and have no fingerprinting. I thing apache is not working on my pc for some reason . Maybe that's why I keep getting blank excels. Who knows *shrugs*. Time for 5 hours of bug fixing... Scrapy is good if you have lots of pages but selenium is great where you want to maintain a good relationship with the webpage and act as a user instead of just an api.
luckystomp
Newbie
- Nov 1, 2012
- 15
- 0
@Iamhere123
selenium is great if you need to emulate a human
as for excel - check stackoverflow there are lots of examples
and most of them working good
you can use csv.writer or DictWriter
depends on your needs
selenium is great if you need to emulate a human
as for excel - check stackoverflow there are lots of examples
and most of them working good
you can use csv.writer or DictWriter
depends on your needs
- Mar 21, 2013
- 24,760
- 31,551
There's probably a loose comma somewhere breaking the CSV. Don't kill for this, it's a very trivial script
Iamhere123
Newbie
- Jul 28, 2017
- 13
- 0
Hahaha yeah probably a bit overkill huh? I can scrape the data well enough but any excel actions don't seem to work. I have installed apache a few times :S. I wonder if there's a way to verifiy the install through windows :S. I'll be looking at lots of stackoverflow q's to see if I can't copy someone else's code and hope that works or just try and do a fresh install of apache. I'm sure it's probably something simple af I've overlooked though xD
Iamhere123
Newbie
- Jul 28, 2017
- 13
- 0
Turns out I was missing a few apache files so it wasn't working properly. Now I'm going to have to dig through some stackoverflow q's and see if I can't write to excel with the extracted data .
.
bk071
Elite Member
- Nov 24, 2010
- 3,302
- 8,845
Turns out I was missing a few apache files so it wasn't working properly. Now I'm going to have to dig through some stackoverflow q's and see if I can't write to excel with the extracted data
Sounds simple. Write to a text file and use semicolon (
as the delimiter. Then import into excel by going to Data tab and clicking the "From Text" option.
It will ask you about the delimiter, choose semicolon. All done.
Edit: Of course you can use the CSV library. But the above approach is what I've tried and it worked perfectly. Didn't feel the need to use the csv lib.
Shoot me a PM if you want me to take a look.
rafark
Elite Member
- Jan 15, 2013
- 3,249
- 1,979
I'm no python programmer but I recently parsed a large spreadsheet fule. What I can say is that don't read it at once, it's better to stream it line by line. An even more optimized way is to convert it to csv and work it from that as Bart Simpson suggested.
akashasnani
Regular Member
- Mar 30, 2011
- 311
- 122
using csv library u can do it very easily.
Iamhere123
Newbie
- Jul 28, 2017
- 13
- 0
So getting extracted data and putting it straight into excel is slow where there is a a large quantity of data?
Iamhere123
Newbie
- Jul 28, 2017
- 13
- 0
Sounds interesting . So the delimiter allows you to use the text file and keep the data separate in the text file. Any good examples on this? II'll see if stack overflow has some examples and try it out.
. So the delimiter allows you to use the text file and keep the data separate in the text file. Any good examples on this? II'll see if stack overflow has some examples and try it out.