Wave_Tronic
Newbie
- May 19, 2024
- 19
- 8
Hello BHW,
Just finished another spider using Scrapy. Last one I made pulled info on lego mini figurines. This time I went with car data. End result is a SQLite DB of 266 rows with Name, Price and Rating. With every website's structure being different, I ran into some snags with my crawler following bogus links. With the help from Chat GPT I was able to get this sorted out in no time. I am no 1337 code, but with this AI assistance I can make things work and get a good explanation as to why they worked as well. Feels like cheating sometimes sure, but I prefer to walk away with a working bot. I'm trying to get this new skill monetized.
Anyone else feel like this tech has boosted their skills? It has also had me chasing red herrings at times too, so it's not perfect. I'm happy with the end result. The next step will be to add the used cars to this DB. Anyone know if this will be something I can sell or does it need to be much larger? I'm also curious if anyone has rented cloud compute to host a database to charge for access.
Happy coding to all!
Just finished another spider using Scrapy. Last one I made pulled info on lego mini figurines. This time I went with car data. End result is a SQLite DB of 266 rows with Name, Price and Rating. With every website's structure being different, I ran into some snags with my crawler following bogus links. With the help from Chat GPT I was able to get this sorted out in no time. I am no 1337 code, but with this AI assistance I can make things work and get a good explanation as to why they worked as well. Feels like cheating sometimes sure, but I prefer to walk away with a working bot. I'm trying to get this new skill monetized.
Anyone else feel like this tech has boosted their skills? It has also had me chasing red herrings at times too, so it's not perfect. I'm happy with the end result. The next step will be to add the used cars to this DB. Anyone know if this will be something I can sell or does it need to be much larger? I'm also curious if anyone has rented cloud compute to host a database to charge for access.
Happy coding to all!
