LakeForest
Supreme Member
- Nov 11, 2009
- 1,268
- 1,816
...makes me want to cry.
1 step forward, 2 steps back... This might be the first version of Scrapebox I roll back to a previous version.
Grievances
1. Can't add sources to find proxies from clipboard to Harvester (F***)
2. Harvester is almost a second thought as it's the third option in the "Import Proxies" menu
3. In the Harvester, you need to highlight and select remove to delete. Checking them and pressing remove doesn't work.
4. Speed has changed from being reported in ms to "Fast, Medium, Slow". Fast = -1500ms. Medium = 1501 - 3000ms. Slow = +3000ms. (talk about speed being relative...)
5. Doesn't tell time of last operation. If you want to compare, get a stopwatch.
6. "Save Proxies" by default saves the entire list you put in. All 9000 were saved to file or put into Scrapebox. Can't save only Anon or only Google Passed, making the filtering much less useful.
6(2). I don't know if it's good or not that it tests Anon and then Google Passed simultaneously because the amount that pass both tests are relatively consistent now as opposed to 1000 GPassed and 300 IP passed in 1.15.38. I'm not sure what bearing this has on overall testing speed, though.
7. I dare you to put in 300,000+ proxies into the new harvester. I dare you.
8. Filtering the list when testing is completed with "Keep Anonymous Proxies" reduces the amount of anonymous proxies in the list after the test. Perhaps this is a duplicate removal? Filtering by Google Passed doesn't reduce the number of Google Passed, though.
9. I hope the new tester filters duplicates automatically, because I can't find a button to remove dups anywhere in the Proxy Manager.
Postives:(I guess)
1. Shows proxy's country
2. Does Anon check (but no idea on the level)
3. More Descriptive as to why the proxy failed the check (old harvester used to do this before 1.15.38)
4. Saw unusual port numbers passing tests and working in scrapes. Fun!!
5. 200 connections is fast.
6. Can mark proxy sources in harvester as a SOCK source.
7. Scraper is responding...so there's that...or did I speak too soon?
Test
I ran 4 instances testing with a list of 9000 proxies harvested in 1.15.38 because I CAN COPY/PASTE THE SOURCES. 3 instances were version 1.15.39 and had connections set at 25, 100, and 200 (Default is 25. Does Maximum Connections setting of Proxy Harvester not count anymore?) and 1 instance of 1.15.38 running at 100 Requests.
25 connections was painfully slow. 200 connections didn't even finish testing the list all the time. There were ~100 Proxies stuck on "Testing Proxy..."
Results:
(100 Proxy Harvester Max. Connections and 30s Timeout)
25 connections: 2172 Anonymous(filtered to 1244) and 1119 Google Passed
100 connections: 2036 Anonymous(filtered to 1088) and 1065 Google Passed
200 connections: 2205 Anonymous(filtered to 1369) and 1127 Google Passed (wat)
200 connections with Proxy Harvester Max. Connection reduced to 50 from 100: 2124 Anonymous(filtered to 1050) and 1040 Google Passed
100 Requests/Maximum Connections (1.15.38): 483 Google Passed and 356 that also IP Passed.
- This is the first proxy checker I've used that didn't substantially increase results with reduced connections.
- The number of Google Passed results were reduced after Anonymous filtering.
Scraping
I scraped "franklin" (heyyy it's franklin) with the 356 proxies from the old harvester and got 98 results in 1.15.38 and 398 in 1.15.39.
Scraping "franklin" with 1127 proxies from new harvester(200 connections) placed into in 1.15.38 made the scrape fail almost immediately with 0 results. I got 398 scraped urls in 1.15.39. Testing the 1127 proxies in 1.15.38 I got 170 that GPassed and IP Passed and WAS able to scrape, but got 98 results again.
Lately, in all google scrapers, I've been noticing a heavy emphasis to focus on finding the keyword in the domain. This is without a footprint. If anyone has suggestions as to why this is or how to overcome it, do share, please.
Update: wtf...I just put "franklin" in quotes in 1.15.38 with proxies harvested from 1.15.38 and got 566 results. I just scraped in 1.15.39 with franklin in and out of quotes using 1000+ proxies tested and "working" in 1.15.39's tester and got 0 results in two instances and 99 in the other instance.
Proxy Test Consistency
In 1.15.39 retesting has been getting pretty consistent results in terms of what the proxy manager says are good proxies. 1.15.38 is notorious for having results get thrown around the board until you test 3+ times.
Further Tests
I'll be mixing up speeds and working proxies from 1.15.38 and 39 as well as rolling back the version to test some more.
I'm going to sleep. Scrapebox's Proxy issue is at the very least being worked on very diligently and the desire to improve is clearly intended. Fix what's broken, but please don't touch what works.
1 step forward, 2 steps back... This might be the first version of Scrapebox I roll back to a previous version.
Grievances
1. Can't add sources to find proxies from clipboard to Harvester (F***)
2. Harvester is almost a second thought as it's the third option in the "Import Proxies" menu
3. In the Harvester, you need to highlight and select remove to delete. Checking them and pressing remove doesn't work.
4. Speed has changed from being reported in ms to "Fast, Medium, Slow". Fast = -1500ms. Medium = 1501 - 3000ms. Slow = +3000ms. (talk about speed being relative...)
5. Doesn't tell time of last operation. If you want to compare, get a stopwatch.
6. "Save Proxies" by default saves the entire list you put in. All 9000 were saved to file or put into Scrapebox. Can't save only Anon or only Google Passed, making the filtering much less useful.
6(2). I don't know if it's good or not that it tests Anon and then Google Passed simultaneously because the amount that pass both tests are relatively consistent now as opposed to 1000 GPassed and 300 IP passed in 1.15.38. I'm not sure what bearing this has on overall testing speed, though.
7. I dare you to put in 300,000+ proxies into the new harvester. I dare you.
8. Filtering the list when testing is completed with "Keep Anonymous Proxies" reduces the amount of anonymous proxies in the list after the test. Perhaps this is a duplicate removal? Filtering by Google Passed doesn't reduce the number of Google Passed, though.
9. I hope the new tester filters duplicates automatically, because I can't find a button to remove dups anywhere in the Proxy Manager.
Postives:(I guess)
1. Shows proxy's country
2. Does Anon check (but no idea on the level)
3. More Descriptive as to why the proxy failed the check (old harvester used to do this before 1.15.38)
4. Saw unusual port numbers passing tests and working in scrapes. Fun!!
5. 200 connections is fast.
6. Can mark proxy sources in harvester as a SOCK source.
7. Scraper is responding...so there's that...or did I speak too soon?
Test
I ran 4 instances testing with a list of 9000 proxies harvested in 1.15.38 because I CAN COPY/PASTE THE SOURCES. 3 instances were version 1.15.39 and had connections set at 25, 100, and 200 (Default is 25. Does Maximum Connections setting of Proxy Harvester not count anymore?) and 1 instance of 1.15.38 running at 100 Requests.
25 connections was painfully slow. 200 connections didn't even finish testing the list all the time. There were ~100 Proxies stuck on "Testing Proxy..."
Results:
(100 Proxy Harvester Max. Connections and 30s Timeout)
25 connections: 2172 Anonymous(filtered to 1244) and 1119 Google Passed
100 connections: 2036 Anonymous(filtered to 1088) and 1065 Google Passed
200 connections: 2205 Anonymous(filtered to 1369) and 1127 Google Passed (wat)
200 connections with Proxy Harvester Max. Connection reduced to 50 from 100: 2124 Anonymous(filtered to 1050) and 1040 Google Passed
100 Requests/Maximum Connections (1.15.38): 483 Google Passed and 356 that also IP Passed.
- This is the first proxy checker I've used that didn't substantially increase results with reduced connections.
- The number of Google Passed results were reduced after Anonymous filtering.
Scraping
I scraped "franklin" (heyyy it's franklin) with the 356 proxies from the old harvester and got 98 results in 1.15.38 and 398 in 1.15.39.
Scraping "franklin" with 1127 proxies from new harvester(200 connections) placed into in 1.15.38 made the scrape fail almost immediately with 0 results. I got 398 scraped urls in 1.15.39. Testing the 1127 proxies in 1.15.38 I got 170 that GPassed and IP Passed and WAS able to scrape, but got 98 results again.
Lately, in all google scrapers, I've been noticing a heavy emphasis to focus on finding the keyword in the domain. This is without a footprint. If anyone has suggestions as to why this is or how to overcome it, do share, please.
Update: wtf...I just put "franklin" in quotes in 1.15.38 with proxies harvested from 1.15.38 and got 566 results. I just scraped in 1.15.39 with franklin in and out of quotes using 1000+ proxies tested and "working" in 1.15.39's tester and got 0 results in two instances and 99 in the other instance.
Proxy Test Consistency
In 1.15.39 retesting has been getting pretty consistent results in terms of what the proxy manager says are good proxies. 1.15.38 is notorious for having results get thrown around the board until you test 3+ times.
Further Tests
I'll be mixing up speeds and working proxies from 1.15.38 and 39 as well as rolling back the version to test some more.
I'm going to sleep. Scrapebox's Proxy issue is at the very least being worked on very diligently and the desire to improve is clearly intended. Fix what's broken, but please don't touch what works.
