FTP Upload Killing Down Speeds

[specmike]

I am in the process of uploading several pretty large files (multiple gigabyte sized files, RARed into 14.4mb chunks) to a printer via FTP. I have tried 3 different FTP clients and gotten the same results. They all upload fine and at acceptable speeds for my connection (Bellsouth Extreme DSL 6.0 up/512 down). My issue is that these uploads are totally killing my down speed and I am not able to work online efficiently while these uploads are going. I tested my down speed with an upload going and it is less than 800k down, it is typically around 5000k down.

Any idea what may be killing my down speed during an upload?

Setup is:
Bellsouth Modem.
IBM/ISS Proventia M-10 Security Appliance
Netgear 8 port gigabit switch
Then, my machines are all hardwired proper cabling to the switch.

It's all normally really fast except when using these clients. Also, in the past, I have used 2 of these same FTP clients and not had this issue.

I have tried Leech FTP, FTP Commander Pro, and Smart FTP.

[shaihulud]

You will lose a little bit of bandwidth because of ACK’s received. However, it should not be in an excess to the amount that you are only receiving at 800K. I wonder if you are retransmitting an exuberant amount of packets, or is it that you security gateway device is causing an issue with packets. It could be another layer security you have installed, such as a software firewall that you may have not mentioned? I have never seen an application layer increased or decrease the throughput that a connection can achieve. Although, of course, I have seen the connection(s) effect the application.

Are you using large packet sizes? My advice, you should not use jumbo packets ever. Really, the only compensation you get is if the host system transmits an enormous amount of packets per second. This setting can decreases utilization of the FSB and is usually offloaded by the networking processor via large send offloads, if supported by the OS. It will not increase your bandwidth utilization between two hosts. Matter of fact it can cause issues if ICMP cannot inform the host to decrease the packet size. Since you are, most likely, using PPPoE you have to have a smaller size due to the PPPoE header. If a device on your network cannot receive the ICMP message you may be fragmenting your packets and therefore effecting your utilization.

Just a couple of thoughts, I really do not know your topology and implementation.

[specmike]

Well, I have a residential package for my DSL. So, I am wondering if there is some sort of bandwidth cap on my line.

I went into the advanced settings for the FTP client and put a cap on my up and down speeds for the client at 100k up and down. For now, that seems to be a work around.

My guess is that, once I exceed a predetermined point on my up speed, then some sort of cap hits my down speed. I guess they don't want me hosting a server from home without paying them for a business class package. I may experiement with the cap limits on my FTP client to get it as fast as possible without tripping this cap again. This is all assuming my theory is correct.

Thanks for the tips.

[specmike]

Well, I called my ISP and after I got past the level 1 tech, my problem seems to have disappeared if I am not jumping to conclusions too quickly. My ISP (Bellsouth) was recently bought by AT&T and I am wondering if there was some sort of bandwidth cap in place that they removed when I complained.

Funny tech support item of the day, the level 1 tech thought that it was totally logical and acceptable that a 100kb/s upload (100k speed is capped by my FTP client) would bog my download speed from 6.3MB down to 273k down. Unbeleivable, especially considering that this is a recent problem and considering what I pay these people.

[SupDawg]

Originally Posted by specmike

I am in the process of uploading several pretty large files (multiple gigabyte sized files, RARed into 14.4mb chunks) to a printer via FTP. I have tried 3 different FTP clients and gotten the same results. They all upload fine and at acceptable speeds for my connection (Bellsouth Extreme DSL 6.0 up/512 down). My issue is that these uploads are totally killing my down speed and I am not able to work online efficiently while these uploads are going. I tested my down speed with an upload going and it is less than 800k down, it is typically around 5000k down.

Any idea what may be killing my down speed during an upload?

Setup is:
Bellsouth Modem.
IBM/ISS Proventia M-10 Security Appliance
Netgear 8 port gigabit switch
Then, my machines are all hardwired proper cabling to the switch.

It's all normally really fast except when using these clients. Also, in the past, I have used 2 of these same FTP clients and not had this issue.

I have tried Leech FTP, FTP Commander Pro, and Smart FTP.

Without reading the whole thread, there are many factors that can cause this.

The most prevalent I've seen is with cable modems. Some cheaper/inferior modems have this problems.

The second thing to remember is that any downstream bandwidth requires some upstream bandwidth. IF you are using all of your upstream, it will slow down your downstream because the acks from the downloads are delayed.

Or at least that is the way I understand it. But it shouldn't have that much of an effect because your line isn't that fast. It seems to be more prevalent the faster you go.

[izibo]

Originally Posted by SupDawg

The second thing to remember is that any downstream bandwidth requires some upstream bandwidth. IF you are using all of your upstream, it will slow down your downstream because the acks from the downloads are delayed.

Agreed. Remember, your upload is maxed out (or capped) at 64kBytes/sec or 512kbits/sec (remember, 8 bits in a byte!). If you have you have your uploads set to 100 kB/sec, then you are absolutely saturating your uploads and that is likely to be why your downloads are slowing.

To prove the point, set it to 40 kB/sec upload speed and see how it goes.

[specmike]

Originally Posted by izibo

Agreed. Remember, your upload is maxed out (or capped) at 64kBytes/sec or 512kbits/sec (remember, 8 bits in a byte!). If you have you have your uploads set to 100 kB/sec, then you are absolutely saturating your uploads and that is likely to be why your downloads are slowing.

To prove the point, set it to 40 kB/sec upload speed and see how it goes.

It seems it is multiple factors but I have a temporary work around. I figured out the the FTP server is actually capping me at about 20kb/s so I capped my client at that also. Even at that cap speed, my down speed drops from 6.3MB down to 2.8MB down or less. So much for "Extreme" DSL and their "best effort".

Does it sound reasonable that a 20kb/s upload will cut down speed by more than 50%? Nevermind the physics of it, that is not what I pay them for. A few lines of code on their end would likely resolve this.

The single biggest bottleneck internally was my security device. It has FTP AV scan enabled by default and I was becoming CPU bound on it. I temporarily disabled this scanning. With that scanning on my up speed was stable but, down speed was less than 200kb/s. With it disabled, my down speed goes back up to a blazing 2.8MB down.

Thanks for the help and suggestions.

[shaihulud]

Does it sound reasonable that a 20kb/s upload will cut down speed by more than 50%?

My feeling, from my previous post, was your main issue to be your frontend scanner. But always difficult when presented with limited information, or not present to help.

There are many things that can affect a network and its performance. Here is an RFC that describes the very nature of asymmetry (ftp://ftp.isi.edu/in-notes/rfc3449.txt). I do feel though that your ACK’s should not be limiting your downstream bandwidth as much as you report, and you have proven. You should be able to, with minimal latency connections, have a bit more bandwidth for the downstream total.

You could be having a high amount of retransmits at the lower or higher levels of the layers. At least capture a session, use netstat –s (If using Windows), or/and look into the modem diagnostics for line quality and encapsulated packet transmission statistics.

Are you using non-standard Ethernet packet size? You could be sending fragmented packets to your front end, as I previously mentioned. This can increase packet processing due to the fact that if the ICMP message is not sent/received packet fragmentation will occur. If fragmentation happens, packet processing grows at an exponential rate. This is the function of MTU Discovery, RFC 1191 (http://www.faqs.org/rfcs/rfc1191.html). However, RFC 1191 uses ICMP to change the MTU size when mismatched. If the ICMP message is incapable of informing the host, then all your packets, when of large size, will be fragmented. Even if host is one hop away on your network.

[specmike]

AFAIK, packet size and any other network and transmission related settings are all at the defaults of the involved devices from the NIC on up. I know next to nothing about that so I don't mess with it.

Oddly enough, my wife does this type of FTP transfer about 3-4 times per year. And we are done for now and won't do this again until later summer or early fall. I'll keep this thread in mind if I have to revisit this issue.

Thanks again guys.