How many Ko I use when I scrape 1 webpage's source code ?

Arold79

Newbie
Joined
Feb 28, 2020
Messages
20
Reaction score
2
Hello,
my question is perhaps basic but I need to know the answer for my project.
I would like to know how many Ko I download when I scrape one webpage source code.
For exemple, with Firefox, if I show this webpage source code : view-source:https://www.thekitchn.com/how-to-make-chocolate-chip-cookies-cooking-lessons-from-the-kitchn-109117
How many Ko do I use to show this webpage source code ?
If I copy & past the source code in NotePad++ and save it as a .txt file. The space of the file is 251Ko.
Does I use 251Ko of data to show this source code ? Are is that more complicated ?
Thank you very much :)
 
To fully render a page Browser could load other files like .css & java script.
It is rather simple, if page size is 251KB, than you have to use 251KB to fully load a page.
But it could be far more complex if you are counting, DNS requests, SSL certificate & HTML headers data. Also source code could be sent over internet gzipped.
 
Hello VirtualSystems :) Thank you for your answer.
Yes I would like to be sure if I scrape only the source code with my mobile with 4G Internet 10 times a day, how many Ko data do I need.
So I'm speaking about only the source code, not the fully loaded page with css, font etc.
When you write : "Also source code could be sent over internet gzipped"
Could you please tell me more about that ? Is that possible to scrape a source code already gzipped ? Or do I will need to gzipped myself (so the weight of the source code downloaded would be the same, isn't it ?) ?
 
If the website you are scraping, has gzip compression enabled, then you will receive the source code compressed, if you request for it. I don't know how you scrape, but cURL, has this option. If you only receive 251 Kb, 10 times means 2510 Kb. If the page you are scraping changes frequently, then take in consideration +- 10 Kb.
 
Back
Top