punkinhead
Regular Member
- Feb 19, 2015
- 450
- 37
I'm moving over to much bigger vps, so that should help, but I could swear FL is just inefficient in how it deals with lists as they grow. It basically slows to a dead stop as the blacklist, etc grows. Dev has told me to delete scraped lists and so on to speed up. Real system killer, though, appears to be all the scraping it does per account hundreds of times a day.
So... the last bit, at least, I could address if I scrape everything ahead of time. If, for instance, I scraped all 3M followers of a user ahead of time, then just fed that list without scraping for my bot accts to follow, it should be much quicker, no?
I'm generally aware of issues that larger lists should be broken up, etc... but need specifics. If you are scraping your lists ahead of time, what program are you using, and are you getting the full list or only top few thousand? If full list, what's the right way at size to chunk it before feeding it to FL?
So... the last bit, at least, I could address if I scrape everything ahead of time. If, for instance, I scraped all 3M followers of a user ahead of time, then just fed that list without scraping for my bot accts to follow, it should be much quicker, no?
I'm generally aware of issues that larger lists should be broken up, etc... but need specifics. If you are scraping your lists ahead of time, what program are you using, and are you getting the full list or only top few thousand? If full list, what's the right way at size to chunk it before feeding it to FL?