On Wed, 14 Jul 2010, Robert Nesius wrote: > On Tue, Jul 13, 2010 at 5:56 PM, Mike Miller <mbmiller+l at gmail.com> wrote: > >> On Tue, 13 Jul 2010, Robert Nesius wrote: >>> >>> I think this is likely a case of bus-contention. Especially if the >>> reads and writes were being sent through the same bus/controller. >>> I've had similar issues when doing things with USB devices. >> >> Maybe I would have better luck if I used a different pair of USB ports. >> I kinda doubt it because it seems like the big problem is with writes. >> Combining reading from one with writing to the other is definitely >> worse, but the major impact on system performance is coming from the >> writes. Maybe slowness of file transfers is an interaction of the two. > > I was thinking about this some more last night and I wasn't completely > happy with my hypothesis. I have actually noticed the "slowing down to > a crawl" behavior when copying from an internal hard-drive to a USB2 > drive myself. A system reboot and repeat attempt sailed through with no > slowdown. I've always wondered about that - seems like our experiences > point to the microcontroller or driver. What is the brand of your > external drive? One is Western Digital and the other is Samsung. I was having the same problem with either one. Check this out: $ uptime 12:54:24 up 224 days, 23:59, 23 users, load average: 0.10, 0.12, 0.09 I really want to reboot this machine, but I'm always using it for some big job. Probably within the next month or two I'll shut it down, upgrade some components, install Ubuntu 10.04 and bring it back up. Then I can test the file transfer speed again. >> That sounds like part of the problem. Is there a better way to copy a >> collection of files and directories from one external USB HDD to >> another? I don't know how to do that with dd -- isn't that just for >> cloning? > > Assuming your block-sizes are the same you can clone a smaller drive to > a bigger drive with dd and use partition-managing software to grow the > partition. I've never done that, but I'm pretty sure that's a way to do > it. I'm also pretty sure someone else on the list can confirm or deny > that. :) Thanks for the tip. I guess I won't be doing that, at least not now. > Lastly, when copying filesystems or very large directories, the key > thing to remember for speeding things up is that you're better off doing > intermediary transmissions as one huge bit-stream. So when I'm sending > things over a network I tar things up into a compressed tarball - send > the tarball, then extract the tarball on the other end - that can be > faster than a straight rsync or network copy. I wouldn't think that > necessary though with everything connected to the machine. Right. In the situation you describe, you have CPUs at both ends to deal with the overhead. Mike