Latency and Performance Q

[pgregg]

Hi,

I just downloaded PowerFolder this evening and installed it on 2 computers as a test setup for a WorldWide file sharing solution (approx 12 sites). PF looks interesting because it appears to be based on P2P tech (which should make it quick)...

However - a shared folder (automatic sync) with a few large files dropped in at each end appears to only use a single TCP connection. I couldn't find any setting, even in Advanced, that would let me boost the number of parallel connections.

This is important because in relatively high latency networks (e.g. transatlantic US-Europe) a TCP stream will max out at around 2mbit/sec even if you have say 20mbit on each end.

The other solution we are looking at is RocketStream who attack this by massively parallelising the TCP streams (or using UDP), but I think I'd be quite happy with a P2P type solution that could do, e.g., 32 parallel connections (on the same file) as we plan on moving some multi-GB files around.

Am I missing something or is this a current limitation in PF?

Thanks,
PG

PS. minor niggle - I changed the default port from 1337 to 33333 (instead of autoassign a port) but it seems that clicking on the Online button to connect to another PF app doesn't let you specify the destination port number.

[B.A.]

Dear pgregg

PowerFolder is able to download from multiple sources but those source need to have the complete file. Since we transfer file changes we need to have a complete basis version of a file before we can start to upload this file. So on a scenario with 4 clients it would work like that: 1.computer got all files and is transferring them to other computers as soon as the next computer has all files this computer will start to upload them also, and the other computers will get them with double speed....and so on.

No we have no limitations in the number of connections.

Are all computers you want to have the files shared are friends (mutual)?
Are all computers members of the folder which contains the Files?
Can you see those other computers as online and connected on your friendlist?

I don't think that the port is a problem but I will ask one of the developers about that.
A faster way to get answers always is to send an email or visit our live-support
http://www.powerfolder.com/documentation.html

best regards
Bernhard Rutkowsky

[Hannibal]

PS. minor niggle - I changed the default port from 1337 to 33333 (instead of autoassign a port) but it seems that clicking on the Online button to connect to another PF app doesn't let you specify the destination port number.

To enter the port please use this syntax:
<hostname>:<port>
example.mydomain.com:33333
should do the trick.

About the parallel TCP connections: PowerFolder currently maintains ONE connection to each connected computer only. But it can download a single from multiple computers.

You could give our UDT/UDP protocol a try, which is optimized for high bandwidth connections.
To force PowerFolder to UDT/UDP it you need to:
* block TCP port 33333 (So that no TCP connections can be established)
* Enable UDT/NAT Traversal under Preferences/Network.

Best regards,
Christian

[pgregg]

rutkowsky and sprajc, thank you both for your replies.

Another thing I just noticed is that when I change the port from 1337 to 33333 that only the TCP listener changes.... the UDP remains listening on 1337... bug?

UDP might be a viable solution because it isn't limited by latency problems that are inherent in TCP - however, in our production WAN I can't do acls (firewalls) to block the TCP... I could install windows firewall or linux iptables, but I'd prefer not to... It would be great if it could do this at the application level. Note that we don't use NAT anywhere in our network - does the "Use NAT traversal" option somehow kick it into using UDP?

To clarify my intentions for Bernhard. Essentially I want a mesh of 12 sites each with a point-point powerfolder. there can only ever be 1 server supplying the whole file and there will only ever be 1 consumer. We currently use direct FTP, but this in unacceptable because performance maxes out due to latency and you also have the interrupted / manual resume delay problems. However, a single TCP stream from one to the other is just too slow. This would appear to be the limit in Powerfolder ?

If I was using e.g. Bittorrent type software, the source file would be broken up into chunks of e.g. 4Mb and I would be transferring 10-20 of these chunks simultaneously. So although the latency will impact each stream, parallelising the transfers means I can max out the available bandwidth and transfer the file much faster than a single FTP stream. In tests moving a single 8GB Vmware image from US to Europe - a 10 stream bittorrent was able to move the entire file across our 8Mbit lines in 30% of the time it took for a single FTP stream.
Unfortunately regular P2P apps are not very business friendly - and not a "install and forget" type service that PowerFolder claims to be (this being my goal).

Looking forward to your reply.

Many thanks,
PG

[B.A.]

About the UDP i cannot give some useful advice on that Christian will do ASP he will be able to give you better answers.

About blocking connections:
http://wiki.powerfolder.com/wiki/Setup_a_network_with_connections_to_only_selected_ clients
but that is (currently) only working with fixed ips.

Sorry that i cannot help you instantly but my knowledge about networking is too limited, to give you answers which would be a reliable base for decisions
best regards

Bernhard Rutkowsky