GoldenGlovez
Senior Member
- Joined
- Mar 23, 2011
- Messages
- 924
- Reaction score
- 2,226
Optimize your network for Scrapebox
a GoldenGlovez production
******************************
I read a lot of misguided information everyday about configurations for Scrapebox. The most common being related to Timeout/Maximum Connection settings. I wrote this guide to break some common misconceptions about Scrapebox and help you optimize your network to achieve the greatest potential. I have done my best to write this guide with as little technical jargon as possible. However, this is not a guide to be used lightly. Changes you make here can severely affect the performance of your operating system. It is highly recommended you revert back to your default settings after using Scrapebox.
By default Windows 7 does a pretty good job at optimizing your network card and TCP/IP settings for normal everyday use. Unfortunately, Scrapebox is not an application that falls into that category. Scrapebox hammers your network connection, router and network card with multiple simultaneous connections at a very fast rate. By default your hardware cannot keep up with the demand necessary to run Scrapebox to it's highest potential. In this guide you will make the necessary changes to prepare your network for the high demand usage of Scrapebox.
What this guide will do:
First lets talk a little about what Scrapebox does and how it affects your connection.
When Scrapebox begins posting your new Auto Approve list, it instantly opens up multiple connections from your network to the internet. The amount of connections it opens can range from 1 to 500, defined in the SB Maximum Connections settings. Incidentally, if you were to set your SB installation to attempt 500 connections on an unoptimized network you would almost immediately crush your own network. Default settings on home user level hardware cannot withstand 500 outgoing/incoming connections at a constant rate for very long. Most network hardware is designed to immediately block all connections at the first sign of this. It is a built in feature to help protect home users from malicious activity.
On top of the hardware level, you have limitations in your own Operating System. By default Windows Vista, 7 and Server 2008 disable some resources that we can use to help improve how our network connection handles HTTP Requests, Packet Processing, Memory and CPU management, compression and various other goodies. It is also designed by default to operate optimally for normal network applications. Luckily, Scrapebox is NOT a normal network application. We need to make the necessary changes to accommodate to the needs of this application.
I will jump a little more into the technical side later on. Let's answer some questions first,
I have XX private proxies, how many connections should I use?
This question shows up very frequently here on BHW. And the usual community response is to set your maximum connections to the same number of private proxies you are using. In a default environment this is a great way to insure a reliable posting session. However, the length of time it takes to complete such a session can run past the 24hr mark!
That's fine if you don't mind your connection running for 24 hours to complete that 100,000 URL Blast. Personally, I like to get things done right AND fast. Take a look at the image below.
What you are looking at is not an Auto Approve list, but a fresh URL scrape ran at 100 connections on 10 private proxies. It only took 3 Hours and 30 Minutes to finish a list of 91,000 at a success rate of 33%! The rest is mostly Captcha Blogs or 404 Errors.
Here is a test run using a week old Auto Approve list:
Just over 80% posting success on a 1k Unique Domain AA List. This time just under the 6 minute mark, using 100 connections on 10 private proxies! The second pass posted to the remaining list just under a minute.
These are just two examples of what you can achieve by properly configuring your system for Scrapebox.
What should my Timeout setting be?
I'm going to state right here and now: 90 Seconds. Anything less and you are just going to hurt your success percentage and bring your network to it's knees. When dealing with large amount of connections, it causes a higher level of network congestion that is not seen in normal use. Setting your timeout to 90 seconds, creates the necessary buffer to wait for a response packet back from the website SB is trying to post on.
You should go ahead and change your timeout settings now. This change can be made under Settings > Adjust Timeout Settings > Then move the slider for Faster Commenter all the way to the right.
Ok, I will answer more questions later. Let's move on to making some changes.
Tweaking the Operating System
******************************
You should now be looking at the following window:
The settings we are targeting will be 'Transmit Buffers' and 'Receive Buffers'. Both of these settings sets the number of buffers used by the adapter when copying data to memory. Increasing these values can enhance both receive and transmit performance, but consumes additional system memory.
Receive Buffers: Typically, the default setting is '512' and can be increased in increments of '64' up to a maximum of '2048'. While this can vary from card to card, you're going to want to set this to the maximum number allowed. In my case '2048'.
Transmit Buffers: The next setting is also generally set to '512' by default. Once again it can be increased by increments of '64' up to a maximum of '16384' on most commonly Intel cards. Since, it's unlikely you'll need this many buffers, lets set this to match your maximum receive buffer setting. In my case '2048'.
Two additional settings to look out for (one is mentioned in my original post) are 'Receive Side Scaling' and 'Receive Side Queues'.
Receive Side Scaling (RSS) enables processing for a TCP connection across multiple processors or processor cores. If your adapter does not support RSS, or if your operating system does not support it, the RSS setting is not displayed. If you have the option to enable 'Receive Side Scaling' you should do so (this may also require RSS is enabled in your registry settings as noted and explained in the original post).
Receive Side Scaling Queues allocates queue space to buffer transactions between the network adapter and processors. Configuration of queues is not supported on all adapters that support RSS. The number of queues supported varies depending on the adapter and operating system. Only settings allowed for your adapter and operating system are displayed.
This final setting may provide varying performance gains. It's typically helpful in very high network environments, but may show negligible results with Scrapebox performance. You can test this on your own accord.
Once you've finished with the changes, click OK! Don't panic if you lose connection to your VPS or Dedicated Server, it will be restored usually within a minute. I'd also go ahead and reboot the server after the changes anyways.
Enjoy!
UPDATE #2: Settings for Windows XP/Server 2003
TCP Half Open Connections
(HIGHLY RECOMMENDED)
If you are running Windows XP SP2 or later (up until Vista SP2/Server 2008 R2/Windows 7), there is a limitation that MUST be removed before you can properly use Scrapebox. Windows developers included a ?fix? that limits concurrent outbound connections to only 10 per second. The idea behind this was to help prevent the extremly fast spread of computer worms. However, it creates an unnecessary bottleneck for applications such as Scrapebox.
For more information (and downloadable fix) on removing this limitation please visit: http://www.speedguide.net/articles/windows-xp-sp2-tcpipsys-connection-limit-patch-1497
TCP Receive Window
In Windows Server 2003, Windows XP, and Windows 2000, the Transmission Control Protocol/Internet Protocol (TCP/IP) stack tunes itself in most environments, using larger default window sizes than in earlier versions of Windows. Instead of using a hard-coded default receive window size, TCP as implemented in Windows Server 2003 adjusts the window size to even increments of the maximum segment size (MSS) negotiated during connection setup. Matching the receive window to even increments of the MSS increases the percentage of full-sized TCP segments used during bulk data transmission and thus helps minimize the number of segments sent when large sets of data are transmitted.
Recommended Settings:
1Mbps Connection = 37960
1.5Mbps Connection = 56940
2-3Mbps Connection = 128480
3.5-10Mbps Connection = 256960
11-20Mbps Connection = 513920
21-40Mbps Connection = 1027840
41-80Mbps Connection = 2055680
80+ Mbps Connection = 4111360
MTU Discovery
When enabled, TCP attempts to discover MTU automatically over the path to a remote host. When disabled Windows defaults the MTU setting to 576 which decreases overall performance on high speed connections.
Recommended Setting: Enabled (YES)
Black Hole Detect
When enabled, causes TCP to try to detect ?Black Hole? routers while doing Path MTU Discovery. A ?Black Hole? router does not return ICMP Destination Unreachable messages when it needs to fragment an IP datagram with the Don?t Fragment bit set. TCP depends on receiving these messages to perform Path MTU Discovery. With this feature enabled, TCP will try to send segments without the Don?t Fragment bit set if several retransmissions of a segment go unacknowledged. If the segment is acknowledged as a result, the MSS will be decreased and the Don?t Fragment bit will be set in future packets on the connection. Enabling black hole detection increases the maximum number of retransmissions performed for a given segment.
Recommended Setting: Disabled (NO)
Selective ACKs & Max Duplicate ACKs
TCP may experience poor performance when multiple packets are lost from one window of data. With the limited information available from cumulative acknowledgments, a TCP sender can only learn about a single lost packet per round trip time. An aggressive sender could choose to retransmit packets early, but such retransmitted segments may have already been successfully received.
A Selective Acknowledgment (SACK) mechanism, combined with a selective repeat retransmission policy, can help to overcome these limitations. The receiving TCP sends back SACK packets to the sender informing the sender of data that has been received. The sender can then retransmit only the missing data segments.
Recommended Settings:
Selective ACKs: Enabled (YES)
Max Duplicate ACKs: 2
Time to Live (TTL)
TTL determines the time in seconds and the number of hops a packet lives (every hop reduces it by at least 1). While it does not directly affect speed, a larger value increases the amount of time it takes for a packet to be considered lost, discarded and retransmitted. A value that?s too small can cause packets to be unable to reach distant servers at all.
Recommended Setting: 90
Checksum Offloading
This setting allows for reducing CPU load by offloading some tasks required to maintain the TCP/IP stack to the network card. Theoretically, Windows should automatically detect capable network hardware.
Recommended Setting: Enabled
Window Scaling
The TCP window scale option is needed for efficient transfer of data when the bandwidth-delay product is greater than 64K. For instance, if a T1 transmission line of 1.5Mbits/second was used over a satellite link with a 513 millisecond Round Trip Time (RTT), the bandwidth-delay product is (1500000*.513) = 769,500 bits or 96,188 bytes. Using a maximum buffer size of 64K only allows the buffer to be filled to 68% of the theoretical maximum speed of 1.5Mbits/second, or 1.02 Mbit/s.
By using the window scale option, files can be transferred at nearly 1.5Mbit/second utilizing nearly all of the available bandwidth.
Recommended Setting: Enabled
Timestamps
TCP is a symmetric protocol, allowing data to be sent at any time in either direction, and therefore time stamp echoing may occur in either direction. For simplicity and symmetry, we specify that timestamps always be sent and echoed in both directions. For efficiency, we combine the time stamp and time stamp reply fields into a single TCP Timestamps Option.
In the case of Scrapebox usage this setting should be disabled. It creates an addition 12 byte overhead for each packet being set and further congests the network with unnecessary information.
Recommended Setting: Disabled
Advanced Settings Tab
Internet Explorer Optimization
This setting has no affect on Scrapebox performance and should just be left as default. You can set each field accordingly to improve IE performance.
MaxConnectionsPerServer Setting: 8
MaxConnectionsPer1_0Server Setting: 8
Host Resolution Priority
This section improves DNS and host name resolution in general. It helps web pages load faster, and has negligible effect on downloads. This will help increase the speed at which Scrapebox can load a website for commenting.
Local Priority Setting: 4
Host Priority Setting: 5
DNS Priority Setting: 6
Netbt Priority Setting: 7
LAN Tweaks
These settings will not affect Scrapebox performance. However, they will increase local network data transfer performance significantly.
Recommended Settings:
LAN Browsing Speedup: Optimized
LAN Request Buffer Size: 16384
Dynamic Port Allocation
Scrapebox utilizes a massive amount of network ports during posting operations. Configuring these settings will allow your operating system to better handle the large amount of opening and closing of ports.
MaxUserPort Setting: 65535
TCP TimedWaitDelay Setting: 30
Quality of Service
QOS is used to allocate network resources to applications of higher priority. Disable these settings to insure Scrapebox is not affected by other network applications.
QoS: NonBestEffortLimit Setting: Empty
ToS: DisableUserTOSSetting: Empty
ToS: DefaultTOSValue: Empty
DNS Error Caching
Windows has built-in DNS (Domain Name System) caching, which caches resolved host names for faster access and fewer DNS lookups. This is generally a great feature, with the only downside that failed DNS lookups get cached by default as well. When a DNS lookup fails (due to temporary DNS problems), Windows still caches the unsuccessful DNS query, and in turn fails to connect to a host regardless of the fact that the DNS server might be able to handle your lookup seconds later. One can manually flush failed DNS lookups by typing ipconfig /flushdns in Command prompt. Or you can simply set the 3 values in the Optimizer to ?0″, and it will set the relevant Registry settings.
This is a very important change for Scrapebox. When you are posting comments, Windows by default keeps a record of all domains that are accessed by your network card. Even failed connections that would work during your next run might be passed over as failed again even though the connection was alive.
NegativeCacheTime Setting: 0
NetFailureCacheTime Setting: 0
NegativeSOACacheTime Setting: 0
Gaming Tweak ? Disable Nagle?s algorithm
These settings are related to Nagle?s algorithm. Best to leave these as default as they will not affect our performance with Scrapebox positively.
Setting: Default
Happy Scrapeboxing!
a GoldenGlovez production
******************************
I read a lot of misguided information everyday about configurations for Scrapebox. The most common being related to Timeout/Maximum Connection settings. I wrote this guide to break some common misconceptions about Scrapebox and help you optimize your network to achieve the greatest potential. I have done my best to write this guide with as little technical jargon as possible. However, this is not a guide to be used lightly. Changes you make here can severely affect the performance of your operating system. It is highly recommended you revert back to your default settings after using Scrapebox.
By default Windows 7 does a pretty good job at optimizing your network card and TCP/IP settings for normal everyday use. Unfortunately, Scrapebox is not an application that falls into that category. Scrapebox hammers your network connection, router and network card with multiple simultaneous connections at a very fast rate. By default your hardware cannot keep up with the demand necessary to run Scrapebox to it's highest potential. In this guide you will make the necessary changes to prepare your network for the high demand usage of Scrapebox.
What this guide will do:
[*=left] Increase Network Stability and Reliability
[*=left] Reduce Network Overhead and Improve Scrapebox Performance
[*=left] Optimize your network connection for mass HTTP requests
[*=left] Reduce Comment Poster Times while maintaining Success Rate
[*=left]Stop your network from throttling your connections
First lets talk a little about what Scrapebox does and how it affects your connection.
When Scrapebox begins posting your new Auto Approve list, it instantly opens up multiple connections from your network to the internet. The amount of connections it opens can range from 1 to 500, defined in the SB Maximum Connections settings. Incidentally, if you were to set your SB installation to attempt 500 connections on an unoptimized network you would almost immediately crush your own network. Default settings on home user level hardware cannot withstand 500 outgoing/incoming connections at a constant rate for very long. Most network hardware is designed to immediately block all connections at the first sign of this. It is a built in feature to help protect home users from malicious activity.
On top of the hardware level, you have limitations in your own Operating System. By default Windows Vista, 7 and Server 2008 disable some resources that we can use to help improve how our network connection handles HTTP Requests, Packet Processing, Memory and CPU management, compression and various other goodies. It is also designed by default to operate optimally for normal network applications. Luckily, Scrapebox is NOT a normal network application. We need to make the necessary changes to accommodate to the needs of this application.
I will jump a little more into the technical side later on. Let's answer some questions first,
I have XX private proxies, how many connections should I use?
This question shows up very frequently here on BHW. And the usual community response is to set your maximum connections to the same number of private proxies you are using. In a default environment this is a great way to insure a reliable posting session. However, the length of time it takes to complete such a session can run past the 24hr mark!
That's fine if you don't mind your connection running for 24 hours to complete that 100,000 URL Blast. Personally, I like to get things done right AND fast. Take a look at the image below.
What you are looking at is not an Auto Approve list, but a fresh URL scrape ran at 100 connections on 10 private proxies. It only took 3 Hours and 30 Minutes to finish a list of 91,000 at a success rate of 33%! The rest is mostly Captcha Blogs or 404 Errors.
Here is a test run using a week old Auto Approve list:
Just over 80% posting success on a 1k Unique Domain AA List. This time just under the 6 minute mark, using 100 connections on 10 private proxies! The second pass posted to the remaining list just under a minute.
These are just two examples of what you can achieve by properly configuring your system for Scrapebox.
What should my Timeout setting be?
I'm going to state right here and now: 90 Seconds. Anything less and you are just going to hurt your success percentage and bring your network to it's knees. When dealing with large amount of connections, it causes a higher level of network congestion that is not seen in normal use. Setting your timeout to 90 seconds, creates the necessary buffer to wait for a response packet back from the website SB is trying to post on.
You should go ahead and change your timeout settings now. This change can be made under Settings > Adjust Timeout Settings > Then move the slider for Faster Commenter all the way to the right.
Ok, I will answer more questions later. Let's move on to making some changes.
******************************
Warning! This guide is written for Windows 7/Server 2008 OS only. Many of the changes made here are for Windows 7 environments only. USE THIS GUIDE AT YOUR OWN RISK. I will not be held responsible for any problems you cause by following this guide. You are to proceed at your own risk. Make sure to backup when I tell you to do so, in case you need to restore original settings. Warning!
All of the changes to be made can be done by using the registry editor. To speed up the process we are going to use an application designed specifically for tweaking TCP settings in Windows 7 and Server 2008. Please read about each option before making any changes. Some options require additional changes or hardware support.
To start you will need to download the latest version of SG TCP Optimizer @
http://www.speedguide.net/downloads.php
After you have downloaded SG TCP Optimizer, go ahead and launch the application. There is no installation necessary and will load immediately.
Let's take a look at what you will see:
By default, you will see a lot of options set as default or blank. We will now make the changes needed to optimize your system for heavy Scrapebox usage.
Your first step is creating a BACKUP of your current settings. This can be done by going to FILE > BACKUP CURRENT SETTINGS. Do this now before you make any changes!
After you have completed backing up your settings. Move on to selecting the proper Network Adapter from the drop down box. Make sure to select the connection you use when Scrapebox is in use.
Once you have that done, go ahead and select the CUSTOM option on the lower right hand corner of the application. You will now need to read through the following options and set the accordingly. I have labeled my RECOMMENDED options in red color under each setting. Some options may require Router, Network Adapater and/or CPU support! It is your responsibility to read this information before changing any options.
General Settings
TCP Window Auto-Tuning
TCP Auto-Tuning enables TCP Window scaling by default and automatically tunes the TCP receive window size for each individual connection based on the bandwidth delay product (BDP) and the rate at which theapplication reads data from the connection, and no longer need to manually change TcpWindowSize registry key value which applies to all connections.
Setting: Normal
Congestion Control Provider
By default, Vista and Windows 7 have CTCP turned off, it is only on by default under Server 2008. Turning this option on can significantly increase throughput and packet loss recovery.
Setting: CTCP
TCP Chimney Offload
TCP chimney offload enables Windows to offload all TCP processing for a connection to a network adapter. Offloads are initiated on a per-connection basis. Compared to task offload, TCP chimney offload further reduces networking-related CPU overhead, enabling better overall system performance by freeing up CPU time for other tasks.
Setting: Enabled
Receive-Side Scaling State
The receive-side scaling setting enables parallel processing of received packets on multiple processors, while avoiding packet reordering. It avoids packet reordering and separating packets into "flows", and using a single processor for processing all the packets for a given flow. Packets are separated into flows by computing a hash value based on specific fields in each packet, and the resulting hash values are used to select a processor for processing the flow. This approach ensures that all packets belonging to a given TCP connection will be queued to the same processor, in the same order that they were received by the network adapter.
Setting: Enable (if you have 2 more processor cores, otherwise set as default)
Note: For this setting to work your network adapter needs to support RSS. This can be enabled by going into your network adapter's driver configuration. Under the Advanced tab if there is an option for Receive Side Scaling you need to set this to Enabled.
Here you can see the setting:
Direct Cache Access (DCA)
Windows 7 and 2008 Server (but not Vista) add NETDMA 2.0 Direct cache access support. Direct Cache Access (DCA) allows a capable I/O device, such as a network controller, to deliver data directly into a CPU cache. The objective of DCA is to reduce memory latency and the memory bandwidth requirement in high bandwidth (Gigabit) environments. DCA requires support from the I/O device, system chipset, and CPUs.
Setting: Enabled
Note: You will likely only see performance increase if using a Gigabit router with Cat-6 cables and Gigabit Capable Network Card. This setting requires support from your Network Card, System Chipset and CPUs.
NetDMA (TCPA)
NetDMA enables support for advanced direct memory access. In essence, it provides the ability to more efficiently move network data by minimizing CPU usage. NetDMA frees the CPU from handling memory data transfers between network card data buffers and application buffers by using a DMA engine.
Setting: Enabled
Note: Not available under Windows Vista
Time-to-Live (TTL)
Time-to-live (TTL) is a value in an Internet Protocol (IP) packet that tells a network router whether or not the packet has been in the network too long and should be discarded. For a number of reasons, packets may not get delivered to their destination in a reasonable length of time.
This setting needs to match the timeout setting provided in Scrapebox. We will change the timeout setting for Fast Commenter to 90 Seconds later.
Setting: 90
ECN Capability
ECN is only effective in combination with AQM (Active Queue Management) router policy. It has more noticeable effect on performance with interactive connections and HTTP requests, in the presense of router congestion/packet loss. Its effect on bulk throughput with large TCP Window are less clear.
Setting: Enabled
Note: For this setting to work you router must support ECN. To check if your router supports ECN you can run a test using Internet Explorer at: http://www.microsoft.com/windows/usi...d/default.mspx
After the test completes, it will contain information about your internet connection and if ECN is supported by your router.
Windows Scaling Heuristics
Windows Vista/7 has the ability to automatically change its own TCP Window auto-tuning behavior to a more conservative state regardless of any user settings. It is possible for Windows to override the auto-tuning level even after an user sets their custom TCP auto-tuning level.
Setting: Disabled
Checksum Offloading
This setting allows for reducing CPU load by offloading some tasks required to maintain the TCP/IP stack to the network card. Theoretically, Windows should automatically detect capable network hardware.
Setting: Enabled
Window Scaling
The TCP window scale option is needed for efficient transfer of data when the bandwidth-delay product is greater than 64K. For instance, if a T1 transmission line of 1.5Mbits/second was used over a satellite link with a 513 millisecond Round Trip Time (RTT), the bandwidth-delay product is (1500000*.513) = 769,500 bits or 96,188 bytes. Using a maximum buffer size of 64K only allows the buffer to be filled to 68% of the theoretical maximum speed of 1.5Mbits/second, or 1.02 Mbit/s.
By using the window scale option, files can be transferred at nearly 1.5Mbit/second utilizing nearly all of the available bandwidth.
Setting: Enabled
Timestamps
TCP is a symmetric protocol, allowing data to be sent at any time in either direction, and therefore time stamp echoing may occur in either direction. For simplicity and symmetry, we specify that timestamps always be sent and echoed in both directions. For efficiency, we combine the time stamp and time stamp reply fields into a single TCP Timestamps Option.
In the case of Scrapebox usage this setting should be disabled. It creates an addition 12 byte overhead for each packet being set and further congests the network with unnecessary information.
Setting: Disabled
Advanced Settings Tab
Internet Explorer Optimization
This setting has no affect on Scrapebox performance and should just be left as default. You can set each field accordingly to improve IE performance.
MaxConnectionsPerServer Setting: 8
MaxConnectionsPer1_0Server Setting: 8
Host Resolution Priority
This section improves DNS and host name resolution in general. It helps web pages load faster, and has negligible effect on downloads. This will help increase the speed at which Scrapebox can load a website for commenting.
Local Priority Setting: 4
Host Priority Setting: 5
DNS Priority Setting: 6
Netbt Priority Setting: 7
SynAttackProtect
This is a fairly undocumented feature in included in recent versions of Windows. This setting should be set to default to reduce any unnecessary overhead while running Scrapebox.
Setting: Disabled
Network Memory Allocation
LargeSystemCache Setting: Enabled
Size Setting: Optimized
Quality of Service
QoS: NonBestEffortLimit Setting: 0
DNS Error Caching
Windows has built-in DNS (Domain Name System) caching, which caches resolved host names for faster access and fewer DNS lookups. This is generally a great feature, with the only downside that failed DNS lookups get cached by default as well. When a DNS lookup fails (due to temporary DNS problems), Windows still caches the unsuccessful DNS query, and in turn fails to connect to a host regardless of the fact that the DNS server might be able to handle your lookup seconds later. One can manually flush failed DNS lookups by typing ipconfig /flushdns in Command prompt. Or you can simply set the 3 values in the Optimizer to "0", and it will set the relevant Registry settings.
This is a very important change for Scrapebox. When you are posting comments, Windows by default keeps a record of all domains that are accessed by your network card. Even failed connections that would work during your next run might be passed over as failed again even though the connection was alive.
NegativeCacheTime Setting: 0
NetFailureCacheTime Setting: 0
NegativeSOACacheTime Setting: 0
Gaming Tweak - Disable Nagle's algorithm
These settings are related to Nagle's algorithm. Best to leave these as default as they will not affect our performance with Scrapebox positively.
Setting: Default
Dynamic Port Allocation
MaxUserPort Setting: 65535
TCP TimedWaitDelay Setting: 30
Finished! Go ahead and apply your settings and restart your machine when prompted.
UPDATE #1: Additional network adapter tweaks
Please read the above HowTo before continuing with this information (both sections are pieces of the puzzle). As stated earlier, make sure to proceed with caution when making any changes to your operating system configuration. The information provided in both posts will not only improve Scrapebox performance, but other network intensive applications such as Xrumer7 or SenukeX.
Here's a screen capture of a fresh harvest being posted after changes from these setting changes:
Let's get to it.
The changes we're going to make are directly to the network adapter configuration. While it's not guaranteed these settings will be available on your home level network card, they are typically found on server level network hardware.
We'll first need to open the advanced configuration settings for our network adapter. This can be accomplished by doing the following:
All of the changes to be made can be done by using the registry editor. To speed up the process we are going to use an application designed specifically for tweaking TCP settings in Windows 7 and Server 2008. Please read about each option before making any changes. Some options require additional changes or hardware support.
To start you will need to download the latest version of SG TCP Optimizer @
http://www.speedguide.net/downloads.php
After you have downloaded SG TCP Optimizer, go ahead and launch the application. There is no installation necessary and will load immediately.
Let's take a look at what you will see:
By default, you will see a lot of options set as default or blank. We will now make the changes needed to optimize your system for heavy Scrapebox usage.
Your first step is creating a BACKUP of your current settings. This can be done by going to FILE > BACKUP CURRENT SETTINGS. Do this now before you make any changes!
After you have completed backing up your settings. Move on to selecting the proper Network Adapter from the drop down box. Make sure to select the connection you use when Scrapebox is in use.
Once you have that done, go ahead and select the CUSTOM option on the lower right hand corner of the application. You will now need to read through the following options and set the accordingly. I have labeled my RECOMMENDED options in red color under each setting. Some options may require Router, Network Adapater and/or CPU support! It is your responsibility to read this information before changing any options.
General Settings
TCP Window Auto-Tuning
TCP Auto-Tuning enables TCP Window scaling by default and automatically tunes the TCP receive window size for each individual connection based on the bandwidth delay product (BDP) and the rate at which theapplication reads data from the connection, and no longer need to manually change TcpWindowSize registry key value which applies to all connections.
Setting: Normal
Congestion Control Provider
By default, Vista and Windows 7 have CTCP turned off, it is only on by default under Server 2008. Turning this option on can significantly increase throughput and packet loss recovery.
Setting: CTCP
TCP Chimney Offload
TCP chimney offload enables Windows to offload all TCP processing for a connection to a network adapter. Offloads are initiated on a per-connection basis. Compared to task offload, TCP chimney offload further reduces networking-related CPU overhead, enabling better overall system performance by freeing up CPU time for other tasks.
Setting: Enabled
Receive-Side Scaling State
The receive-side scaling setting enables parallel processing of received packets on multiple processors, while avoiding packet reordering. It avoids packet reordering and separating packets into "flows", and using a single processor for processing all the packets for a given flow. Packets are separated into flows by computing a hash value based on specific fields in each packet, and the resulting hash values are used to select a processor for processing the flow. This approach ensures that all packets belonging to a given TCP connection will be queued to the same processor, in the same order that they were received by the network adapter.
Setting: Enable (if you have 2 more processor cores, otherwise set as default)
Note: For this setting to work your network adapter needs to support RSS. This can be enabled by going into your network adapter's driver configuration. Under the Advanced tab if there is an option for Receive Side Scaling you need to set this to Enabled.
Here you can see the setting:
Direct Cache Access (DCA)
Windows 7 and 2008 Server (but not Vista) add NETDMA 2.0 Direct cache access support. Direct Cache Access (DCA) allows a capable I/O device, such as a network controller, to deliver data directly into a CPU cache. The objective of DCA is to reduce memory latency and the memory bandwidth requirement in high bandwidth (Gigabit) environments. DCA requires support from the I/O device, system chipset, and CPUs.
Setting: Enabled
Note: You will likely only see performance increase if using a Gigabit router with Cat-6 cables and Gigabit Capable Network Card. This setting requires support from your Network Card, System Chipset and CPUs.
NetDMA (TCPA)
NetDMA enables support for advanced direct memory access. In essence, it provides the ability to more efficiently move network data by minimizing CPU usage. NetDMA frees the CPU from handling memory data transfers between network card data buffers and application buffers by using a DMA engine.
Setting: Enabled
Note: Not available under Windows Vista
Time-to-Live (TTL)
Time-to-live (TTL) is a value in an Internet Protocol (IP) packet that tells a network router whether or not the packet has been in the network too long and should be discarded. For a number of reasons, packets may not get delivered to their destination in a reasonable length of time.
This setting needs to match the timeout setting provided in Scrapebox. We will change the timeout setting for Fast Commenter to 90 Seconds later.
Setting: 90
ECN Capability
ECN is only effective in combination with AQM (Active Queue Management) router policy. It has more noticeable effect on performance with interactive connections and HTTP requests, in the presense of router congestion/packet loss. Its effect on bulk throughput with large TCP Window are less clear.
Setting: Enabled
Note: For this setting to work you router must support ECN. To check if your router supports ECN you can run a test using Internet Explorer at: http://www.microsoft.com/windows/usi...d/default.mspx
After the test completes, it will contain information about your internet connection and if ECN is supported by your router.
Windows Scaling Heuristics
Windows Vista/7 has the ability to automatically change its own TCP Window auto-tuning behavior to a more conservative state regardless of any user settings. It is possible for Windows to override the auto-tuning level even after an user sets their custom TCP auto-tuning level.
Setting: Disabled
Checksum Offloading
This setting allows for reducing CPU load by offloading some tasks required to maintain the TCP/IP stack to the network card. Theoretically, Windows should automatically detect capable network hardware.
Setting: Enabled
Window Scaling
The TCP window scale option is needed for efficient transfer of data when the bandwidth-delay product is greater than 64K. For instance, if a T1 transmission line of 1.5Mbits/second was used over a satellite link with a 513 millisecond Round Trip Time (RTT), the bandwidth-delay product is (1500000*.513) = 769,500 bits or 96,188 bytes. Using a maximum buffer size of 64K only allows the buffer to be filled to 68% of the theoretical maximum speed of 1.5Mbits/second, or 1.02 Mbit/s.
By using the window scale option, files can be transferred at nearly 1.5Mbit/second utilizing nearly all of the available bandwidth.
Setting: Enabled
Timestamps
TCP is a symmetric protocol, allowing data to be sent at any time in either direction, and therefore time stamp echoing may occur in either direction. For simplicity and symmetry, we specify that timestamps always be sent and echoed in both directions. For efficiency, we combine the time stamp and time stamp reply fields into a single TCP Timestamps Option.
In the case of Scrapebox usage this setting should be disabled. It creates an addition 12 byte overhead for each packet being set and further congests the network with unnecessary information.
Setting: Disabled
Advanced Settings Tab
Internet Explorer Optimization
This setting has no affect on Scrapebox performance and should just be left as default. You can set each field accordingly to improve IE performance.
MaxConnectionsPerServer Setting: 8
MaxConnectionsPer1_0Server Setting: 8
Host Resolution Priority
This section improves DNS and host name resolution in general. It helps web pages load faster, and has negligible effect on downloads. This will help increase the speed at which Scrapebox can load a website for commenting.
Local Priority Setting: 4
Host Priority Setting: 5
DNS Priority Setting: 6
Netbt Priority Setting: 7
SynAttackProtect
This is a fairly undocumented feature in included in recent versions of Windows. This setting should be set to default to reduce any unnecessary overhead while running Scrapebox.
Setting: Disabled
Network Memory Allocation
LargeSystemCache Setting: Enabled
Size Setting: Optimized
Quality of Service
QoS: NonBestEffortLimit Setting: 0
DNS Error Caching
Windows has built-in DNS (Domain Name System) caching, which caches resolved host names for faster access and fewer DNS lookups. This is generally a great feature, with the only downside that failed DNS lookups get cached by default as well. When a DNS lookup fails (due to temporary DNS problems), Windows still caches the unsuccessful DNS query, and in turn fails to connect to a host regardless of the fact that the DNS server might be able to handle your lookup seconds later. One can manually flush failed DNS lookups by typing ipconfig /flushdns in Command prompt. Or you can simply set the 3 values in the Optimizer to "0", and it will set the relevant Registry settings.
This is a very important change for Scrapebox. When you are posting comments, Windows by default keeps a record of all domains that are accessed by your network card. Even failed connections that would work during your next run might be passed over as failed again even though the connection was alive.
NegativeCacheTime Setting: 0
NetFailureCacheTime Setting: 0
NegativeSOACacheTime Setting: 0
Gaming Tweak - Disable Nagle's algorithm
These settings are related to Nagle's algorithm. Best to leave these as default as they will not affect our performance with Scrapebox positively.
Setting: Default
Dynamic Port Allocation
MaxUserPort Setting: 65535
TCP TimedWaitDelay Setting: 30
Finished! Go ahead and apply your settings and restart your machine when prompted.
UPDATE #1: Additional network adapter tweaks
Please read the above HowTo before continuing with this information (both sections are pieces of the puzzle). As stated earlier, make sure to proceed with caution when making any changes to your operating system configuration. The information provided in both posts will not only improve Scrapebox performance, but other network intensive applications such as Xrumer7 or SenukeX.
Here's a screen capture of a fresh harvest being posted after changes from these setting changes:
Let's get to it.
The changes we're going to make are directly to the network adapter configuration. While it's not guaranteed these settings will be available on your home level network card, they are typically found on server level network hardware.
We'll first need to open the advanced configuration settings for our network adapter. This can be accomplished by doing the following:
[*=left]Open the Start Menu
[*=left]Select Run...
[*=left]Type "ncpa.cpl" without quotes and press enter
[*=left]Right click your Public Network Adapter
[*=left]Select Properties
[*=left]Select Configure
[*=left]Finally, select the 'Advanced Tab'
You should now be looking at the following window:
The settings we are targeting will be 'Transmit Buffers' and 'Receive Buffers'. Both of these settings sets the number of buffers used by the adapter when copying data to memory. Increasing these values can enhance both receive and transmit performance, but consumes additional system memory.
Receive Buffers: Typically, the default setting is '512' and can be increased in increments of '64' up to a maximum of '2048'. While this can vary from card to card, you're going to want to set this to the maximum number allowed. In my case '2048'.
Transmit Buffers: The next setting is also generally set to '512' by default. Once again it can be increased by increments of '64' up to a maximum of '16384' on most commonly Intel cards. Since, it's unlikely you'll need this many buffers, lets set this to match your maximum receive buffer setting. In my case '2048'.
Two additional settings to look out for (one is mentioned in my original post) are 'Receive Side Scaling' and 'Receive Side Queues'.
Receive Side Scaling (RSS) enables processing for a TCP connection across multiple processors or processor cores. If your adapter does not support RSS, or if your operating system does not support it, the RSS setting is not displayed. If you have the option to enable 'Receive Side Scaling' you should do so (this may also require RSS is enabled in your registry settings as noted and explained in the original post).
Receive Side Scaling Queues allocates queue space to buffer transactions between the network adapter and processors. Configuration of queues is not supported on all adapters that support RSS. The number of queues supported varies depending on the adapter and operating system. Only settings allowed for your adapter and operating system are displayed.
This final setting may provide varying performance gains. It's typically helpful in very high network environments, but may show negligible results with Scrapebox performance. You can test this on your own accord.
Once you've finished with the changes, click OK! Don't panic if you lose connection to your VPS or Dedicated Server, it will be restored usually within a minute. I'd also go ahead and reboot the server after the changes anyways.
Enjoy!
UPDATE #2: Settings for Windows XP/Server 2003
TCP Half Open Connections
(HIGHLY RECOMMENDED)
If you are running Windows XP SP2 or later (up until Vista SP2/Server 2008 R2/Windows 7), there is a limitation that MUST be removed before you can properly use Scrapebox. Windows developers included a ?fix? that limits concurrent outbound connections to only 10 per second. The idea behind this was to help prevent the extremly fast spread of computer worms. However, it creates an unnecessary bottleneck for applications such as Scrapebox.
For more information (and downloadable fix) on removing this limitation please visit: http://www.speedguide.net/articles/windows-xp-sp2-tcpipsys-connection-limit-patch-1497
TCP Receive Window
In Windows Server 2003, Windows XP, and Windows 2000, the Transmission Control Protocol/Internet Protocol (TCP/IP) stack tunes itself in most environments, using larger default window sizes than in earlier versions of Windows. Instead of using a hard-coded default receive window size, TCP as implemented in Windows Server 2003 adjusts the window size to even increments of the maximum segment size (MSS) negotiated during connection setup. Matching the receive window to even increments of the MSS increases the percentage of full-sized TCP segments used during bulk data transmission and thus helps minimize the number of segments sent when large sets of data are transmitted.
Recommended Settings:
1Mbps Connection = 37960
1.5Mbps Connection = 56940
2-3Mbps Connection = 128480
3.5-10Mbps Connection = 256960
11-20Mbps Connection = 513920
21-40Mbps Connection = 1027840
41-80Mbps Connection = 2055680
80+ Mbps Connection = 4111360
MTU Discovery
When enabled, TCP attempts to discover MTU automatically over the path to a remote host. When disabled Windows defaults the MTU setting to 576 which decreases overall performance on high speed connections.
Recommended Setting: Enabled (YES)
Black Hole Detect
When enabled, causes TCP to try to detect ?Black Hole? routers while doing Path MTU Discovery. A ?Black Hole? router does not return ICMP Destination Unreachable messages when it needs to fragment an IP datagram with the Don?t Fragment bit set. TCP depends on receiving these messages to perform Path MTU Discovery. With this feature enabled, TCP will try to send segments without the Don?t Fragment bit set if several retransmissions of a segment go unacknowledged. If the segment is acknowledged as a result, the MSS will be decreased and the Don?t Fragment bit will be set in future packets on the connection. Enabling black hole detection increases the maximum number of retransmissions performed for a given segment.
Recommended Setting: Disabled (NO)
Selective ACKs & Max Duplicate ACKs
TCP may experience poor performance when multiple packets are lost from one window of data. With the limited information available from cumulative acknowledgments, a TCP sender can only learn about a single lost packet per round trip time. An aggressive sender could choose to retransmit packets early, but such retransmitted segments may have already been successfully received.
A Selective Acknowledgment (SACK) mechanism, combined with a selective repeat retransmission policy, can help to overcome these limitations. The receiving TCP sends back SACK packets to the sender informing the sender of data that has been received. The sender can then retransmit only the missing data segments.
Recommended Settings:
Selective ACKs: Enabled (YES)
Max Duplicate ACKs: 2
Time to Live (TTL)
TTL determines the time in seconds and the number of hops a packet lives (every hop reduces it by at least 1). While it does not directly affect speed, a larger value increases the amount of time it takes for a packet to be considered lost, discarded and retransmitted. A value that?s too small can cause packets to be unable to reach distant servers at all.
Recommended Setting: 90
Checksum Offloading
This setting allows for reducing CPU load by offloading some tasks required to maintain the TCP/IP stack to the network card. Theoretically, Windows should automatically detect capable network hardware.
Recommended Setting: Enabled
Window Scaling
The TCP window scale option is needed for efficient transfer of data when the bandwidth-delay product is greater than 64K. For instance, if a T1 transmission line of 1.5Mbits/second was used over a satellite link with a 513 millisecond Round Trip Time (RTT), the bandwidth-delay product is (1500000*.513) = 769,500 bits or 96,188 bytes. Using a maximum buffer size of 64K only allows the buffer to be filled to 68% of the theoretical maximum speed of 1.5Mbits/second, or 1.02 Mbit/s.
By using the window scale option, files can be transferred at nearly 1.5Mbit/second utilizing nearly all of the available bandwidth.
Recommended Setting: Enabled
Timestamps
TCP is a symmetric protocol, allowing data to be sent at any time in either direction, and therefore time stamp echoing may occur in either direction. For simplicity and symmetry, we specify that timestamps always be sent and echoed in both directions. For efficiency, we combine the time stamp and time stamp reply fields into a single TCP Timestamps Option.
In the case of Scrapebox usage this setting should be disabled. It creates an addition 12 byte overhead for each packet being set and further congests the network with unnecessary information.
Recommended Setting: Disabled
Advanced Settings Tab
Internet Explorer Optimization
This setting has no affect on Scrapebox performance and should just be left as default. You can set each field accordingly to improve IE performance.
MaxConnectionsPerServer Setting: 8
MaxConnectionsPer1_0Server Setting: 8
Host Resolution Priority
This section improves DNS and host name resolution in general. It helps web pages load faster, and has negligible effect on downloads. This will help increase the speed at which Scrapebox can load a website for commenting.
Local Priority Setting: 4
Host Priority Setting: 5
DNS Priority Setting: 6
Netbt Priority Setting: 7
LAN Tweaks
These settings will not affect Scrapebox performance. However, they will increase local network data transfer performance significantly.
Recommended Settings:
LAN Browsing Speedup: Optimized
LAN Request Buffer Size: 16384
Dynamic Port Allocation
Scrapebox utilizes a massive amount of network ports during posting operations. Configuring these settings will allow your operating system to better handle the large amount of opening and closing of ports.
MaxUserPort Setting: 65535
TCP TimedWaitDelay Setting: 30
Quality of Service
QOS is used to allocate network resources to applications of higher priority. Disable these settings to insure Scrapebox is not affected by other network applications.
QoS: NonBestEffortLimit Setting: Empty
ToS: DisableUserTOSSetting: Empty
ToS: DefaultTOSValue: Empty
DNS Error Caching
Windows has built-in DNS (Domain Name System) caching, which caches resolved host names for faster access and fewer DNS lookups. This is generally a great feature, with the only downside that failed DNS lookups get cached by default as well. When a DNS lookup fails (due to temporary DNS problems), Windows still caches the unsuccessful DNS query, and in turn fails to connect to a host regardless of the fact that the DNS server might be able to handle your lookup seconds later. One can manually flush failed DNS lookups by typing ipconfig /flushdns in Command prompt. Or you can simply set the 3 values in the Optimizer to ?0″, and it will set the relevant Registry settings.
This is a very important change for Scrapebox. When you are posting comments, Windows by default keeps a record of all domains that are accessed by your network card. Even failed connections that would work during your next run might be passed over as failed again even though the connection was alive.
NegativeCacheTime Setting: 0
NetFailureCacheTime Setting: 0
NegativeSOACacheTime Setting: 0
Gaming Tweak ? Disable Nagle?s algorithm
These settings are related to Nagle?s algorithm. Best to leave these as default as they will not affect our performance with Scrapebox positively.
Setting: Default
Happy Scrapeboxing!
Last edited:
)
