This project is read-only.

Performance of the PipeStream

Nov 18, 2014 at 6:12 PM
Hello,

im using this library for some time now. Till now the performance never was important to me.
But now i have to execute a command which has a huge amount of result data which simply need to be written to a local file.
The first problem i encountered was the handling. To simply read all received data from the stream and write it to a local file, i ended up with something like that:
SshCommand cmd = client.CreateCommand("cd /mnt; tar -cj ./*");
using(FileStream stream = new FileStream("D:\\backup.tar.bz2", FileMode.OpenOrCreate, FileAccess.Write))
{
    IAsyncResult r = cmd.BeginExecute((IAsyncResult aresult) =>
    {
        (cmd.OutputStream as PipeStream).BlockLastReadBuffer = false;
    });

    (cmd.OutputStream as PipeStream).BlockLastReadBuffer = true;
    stream.SetLength(0);
    byte[] data = new byte[1024 * 1024];
    while(!r.IsCompleted)
    {
        count = cmd.OutputStream.Read(data, 0, data.Length);
        if(count == 0)
            continue;
        stream.Write(data, 0, count);
    }

    string result = cmd.EndExecute(r);
}
The second problem i had was the performance. The thoughput is really low with a high cpu load.

I solved both problems by writing a new Pipe class as a replacement.
With my Implementation i can transfer 1GB data though the pipe in under 2 seconds. With the original implementation the needed time is over 97 seconds, so it is about 47 times fater.
I made made this performance tests in a virtual machine with Windows 7.
This new implentation also supports async. reads / writes and timeouts.

Doing the same like i mentioned above with my implementation would be:
using(FileStream stream = new FileStream("D:\\backup.tar.bz2", FileMode.OpenOrCreate, FileAccess.Write))
{
    PipeStream s = cmd.OutputStream as PipeStream;
    byte[] data = new byte[1024 * 1024];
    while((count = s.Read(readData, 0, readData.Length)) > 0)
    {
        stream.Write(data, 0, count);
    }
}
If you are interrested to use it please send me a message, then i will perform some more tests and add the documentation comments.
I offer this because it is already a great library and like to give something back - not just using it.
Nov 18, 2014 at 8:22 PM
I'm definitely interested as I also noticed some issues with the current implementation.
I can't yet promise we'll integrate your implementation, but I surely want to have a look.
You can postpone adding tests and docs until I've verified it is (or can be) useful for SSH.NET.

Thanks!