J. Spencer Love wrote an explanation on what limits Web and FTP download speed for well-connected users.

Note: Pair is a hosting provider located in Pittsburgh, US.

Subject: Re: Maximum download speed from Pair servers
Date: Mon, 10 Jun 2002 16:59:11 -0400
From: jsl@lovesong.com
Organization: Love Song Productions
Newsgroups: pair.users

...I think your problem is round-trip time, and window size. If there were some other kind of bottleneck, then the simultaneous transfers could not each run at full speed.

Your FTP client establishes a TCP/IP connection to transfer a file. This connection offers a "window" in each direction to the other end, which is the maximum number of bytes that can be transmitted by the sender without being acknowledged as having been successfully received by the other end. This means the receiver can safely expect that no more than window-size bytes of data need to be buffered for that connection, and that any additional data beyond that in sequence order can legitimately be discarded (because it is so defined by the TCP protocol).

When you fetch a file from pair, the FTP server process at pair is the transmitter, and your computer is the receiver, so your computer determines the relevant window size. The program which is receiving, in this case your FTP client, should have some control over the window size, but this depends on the actual program you are using and the operating system whether it allows you to adjust this setting.

The basic problem is this: if the FTP program offers 8 kilobytes (64 kilobits) of window, then the server can only send 8 KB of data, and then must wait until the acknowledgment comes back that those 8 KB have been received before it can send any more. This is a simplification of how TCP/IP does it for explanatory purposes.

Suppose it takes one second for a round trip. Then you can only get 8 KB/sec. If you connect to another server that is closer, you have a shorter round trip, and therefore a higher transfer rate.

TCP/IP version 4 only provides a 16-bit (unsigned) field for the window, so you can only increase the window size to 65535 bytes, which allows a maximum of 65535 data bytes to be unacknowledged in transit at any time. With a one-second round-trip, that's a peak rate of 65 KB/sec, although that is 524,280 bits per second.

Your practical maximum transfer rate can be much lower than this, due to two factors:

1. if the receiving TCP/IP does not offer maximum window with every acknowledgment packet, i.e., if the TCP/IP does not release the buffer until the destination process has emptied it, which may have an additional delay, and doesn't have extra buffering available, then the round trip time for window buffer allocation may be greater than the round-trip time for packet reception and acknowledgment. Depending on your operating system, the time to checksum and reply may be very close to the time measured by the ping command, while the time to open the window may be significantly greater. A good implementation for high throughput should make these times (ping and window) much closer together.
2. If any of the packets do not get through, the sender has to figure that out and send them again. This is going to take at least two round-trips and generally longer. So network congestion that results in packet loss, or an unreliable link, can drastically reduce your average transfer rate - sometimes fast, other times very slow.

The recent change from 60 K to 80 K was probably due to a reduction in round-trip time over the haul from Pennsylvania to Scandinavia. There may not be much you can do to improve that further. The ultimate limit is speed of light, but since light can go around the planet 7 times in a second, a round trip-time of less than 140 milliseconds should be possible (ping times were actually around 100 ms -VM). With maximum window, that would allow up to 458,874 bytes per second. Smaller offered windows would choke your transfer rate sooner.

If you must use FTP or HTTP for your transfer, both of which are tied to TCP, and the ping time doesn't preclude a higher rate, you should see what you can do to increase your offered TCP window. If this is already at the maximum, then unless you can change operating systems, you probably won't see improvement, and even that probably won't help much. At least window size is something that can be changed at the client.

Converting to IP v6 may eventually carry with it a TCP with a bigger window field in its header; I'm not sure because I haven't been doing TCP/IP development in a long time (15 years), but I doubt this aspect of the current protocol can have changed, since so many things would break.

If you can use a streaming server for this, you may be able to get better results, because your description suggests that the bandwidth is available. However, I don't know that pair offers any streaming protocol support, especially on shared servers. Perhaps someone else who knows more about pair's services could comment on that. You might need to find another provider for this purpose.

There was a service that pair was considering which allows content to be delivered from a closer server, reducing round trip time, along with a list of cities where such servers were to be located. I haven't seen anything about that service in some time, and it's not on the pair.com front web page that I can see; does that still exist and might it be an option here?

-- Spencer

Vadim's note: I published the video. The largest (1500 kbps) files that required 200KB/s were placed on a server in Norway, and all other files were hosted on a Pair account in Pittsburgh, US.

How to change the default TCP window size in your network

June 28, 1999
By Eric A. Hall

These instructions provide a starting point for setting the systemwide default for the TCP receive window. For more information on specific platforms, refer to the vendor documentation.

Windows NT 4.0 (I think it is the same for all windows boxes - RM)
Using REGEDIT, navigate to the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Paramaters key and add a new DWORD value called TcpWindowSize if one does not already exist. The system default is 8192 rounded up to a multiple of the MSS (maximum segment size)--this is 8760 for Ethernet--, or four times the MSS, whichever is larger. To set your own value, enter the results from your calculations (be sure to click the "decimal" radio button first) and reboot the system.

Solaris 7
System parameters are tweaked using NDD. To view the current TCP window size, execute the "ndd /dev/tcp tcp_recv_hiwat" command, which will display the value currently assigned with the tcp_recv_hiwat variable. To set the window size, execute the "ndd -set /dev/tcp tcp_recv_hiwat" command. Note that Solaris has a minimum value of six times the MSS, so you cannot set the window size to a multiple of four.

Digital Unix 4.0d
To view the current TCP window size, execute the "sysconfig -q inet tcp_recvspace" command. The default setting is 32 kilobytes. To temporarily change the default value, use the "sysconfig -r inet tcp_recvspace=XXX" command, where "XXX" is the value you want to use. This will affect all new processes, but will not be permanent across reboots. If you want to change this setting on a permanent basis, you will need to rebuild the system kernel.

Linux 2.0 kernel
TCP parameters can be set a number of different ways with Linux, though the most common method is to tweak the system's routing table. The system default is 32 kilobytes, so hard-core users will want to recompile the kernel to use a more appropriate value. To view the current setting, execute the "netstat -r" command, which will display the current routing table, including the MTU (maximum transmission unit) and window sizes associated with each destination. To set the value, modify the system startup scripts so that the default route (and/or internal routes) have more appropriate levels.

NetWare
The TCP window size cannot be modified in NetWare.

(Copied from Network Computing site)

-- Rob Marcinko (robert.marcinko@ubs.com), October 29, 2003

Oldie, but goodie

Your page about througput vs. window size/round trip time was very helpful. I did want to note that there is a TCP extension that allows bigger window sizes by using scaling features. My understanding is that it is fairly well supported now. Wikipedia has more info.

— Todd Blackmon, December 7, 2010

• DrTCP — a utility that helps adjust related registrty settings in newer Windowses.
• SpeedGuide TCP Optimizer — a similar utility.