OperationTimeout and ConnectionInfo.Timeout don't terminate the process

Apr 23, 2012 at 7:58 AM

i'm using the ssh.net for a while to send files to our clients (by sftp).

recently, larger files were sent (1-2MB) and for only one the sftp clients the process never ends, even though i've set both  ConnectionInfo.Timeout and OperationTimeout

                    sftp.BufferSize = 1024 * 32 - 50;
                    sftp.ConnectionInfo.Timeout = TimeSpan.FromSeconds(30);
                    sftp.OperationTimeout = TimeSpan.FromSeconds(1800);
                    sftp.Connect();

for this client i had to reduce the buffer size, after getting payload exception.
trying to run in debug and step into the processes, it seems that while doing a request to the server, no value was returned, and the method keeps on waiting, i waited for more than half an hour before aborting.

 

how can i configure the sftp class on my side to terminate the process after sometime?

 

is there something with the configuration of the client's computer that i could advice him to change? (i cannot share the credentials of the computer, sorry).

 

thanks!

Apr 27, 2012 at 4:07 PM

Hello maba.

First ensure you are using the latest release, if this doesnt work, try the latest commit, and if that doesn't work...

SSHNet sends a request to the remote server, it receives a response, but it doesn't understand that it's empty. Is this correct?

So, could you figure out:

  1. What kind of request is sent.*
  2. What kind of response is received.*
  3. Call stack(s)
  4. Sample code to reproduce the problem.
  5. As much server information you can get your hands on.

For 1 and 2, you can check the output window for 'ReceiveMessage from server:' and 'SendMessage to server', but requires you use a debug build.

Apr 30, 2012 at 8:22 AM

Thanks, Kenneth.

The problem is that I don't get any response from the remote server.

I checked it with the latest version of ssh.net, in a debug mode, stepping into the code of the ssh.net.

there were two methods in which I've seen the process stopping: this._messageListenerCompleted.WaitOne() & this.SendRequest(request)

I waited in both cases for about 5 minutes before ending the process.

 

I am using the synchrony mode to upload the file:

using (MemoryStream stream = new MemoryStream(byteArray))
{
   sftp.UploadFile(stream, "LoremIpsum.txt");
}

My question, is there a way to set the timeout, that if it's longer than the OperationTimeout I've set, the ssh.net will stop waiting for the response from the server, and dispose + throw exception?

Apr 30, 2012 at 8:34 AM

Hello

What kind of SSH server are you connecting to?

Is it only UploadFile that fails like this, or does it affect DownloadFile and others?

Apr 30, 2012 at 8:49 AM

I'm using the sshnet only to upload files right now, so i cannot tell regarding DownloadFile. However, beside that specific server, the UploadFile works fine.

The server that it happens with is SSH-2.0-OpenSSH_5.6

Apr 30, 2012 at 11:14 AM

Could you provide a call-stack?

Apr 30, 2012 at 11:37 AM

there is no exception thrown, so no stacktrace.

what happen actually is that the method calls the server and waits for a response, that never comes.

The process ends only when I stopped the debug ('kill' the process).

Apr 30, 2012 at 12:04 PM

When it starts waiting, then press pause and you can get a call stack.

Apr 30, 2012 at 12:24 PM

thanks!

here's the call stack:

     Renci.SshNet.dll!Renci.SshNet.Session.SendMessage(Renci.SshNet.Messages.Message message) Line 689    C#
     Renci.SshNet.dll!Renci.SshNet.Sftp.SubsystemSession.SendData(Renci.SshNet.Messages.Connection.ChannelDataMessage message) Line 105    C#
     Renci.SshNet.dll!Renci.SshNet.Sftp.SftpSession.SendRequest(Renci.SshNet.Sftp.Requests.SftpRequest request) Line 254    C#
     Renci.SshNet.dll!Renci.SshNet.Sftp.SftpSession.RequestWrite(byte[] handle, ulong offset, byte[] data, System.Threading.EventWaitHandle wait) Line 395    C#
     Renci.SshNet.dll!Renci.SshNet.SftpClient.InternalUploadFile(System.IO.Stream input, string path, Renci.SshNet.Sftp.SftpUploadAsyncResult asynchResult, Renci.SshNet.Sftp.Flags flags) Line 1274    C#
     Renci.SshNet.dll!Renci.SshNet.SftpClient.UploadFile(System.IO.Stream input, string path, bool canOverride) Line 517    C#
     Renci.SshNet.dll!Renci.SshNet.SftpClient.UploadFile(System.IO.Stream input, string path) Line 491    C#
>    SshNetTest.exe!SshNetTest.Program.Main(string[] args) Line 123 + 0x14 bytes    C#

Apr 30, 2012 at 4:05 PM

Hi again

You are using an old Renci.SshNet.dll or old source code. I can tell by the second line where it says "SendData".

Go ahead to the Download page and download it again, delete the DLL (or rename it..) then add a reference to the newly downloaded dll.

Apr 30, 2012 at 4:47 PM

Hi and thanks for the help!

I've replaced the dll, and did two attempts:

first with the SshNet 4.0 Binary you have released: there I got immediately SshException of "Connection was lost".

second was with the recent source code, where I could step into. There was no exception. It stuck with a different call-stack:

     Renci.SshNet.dll!Renci.SshNet.Session.Dispose(bool disposing) Line 1970 + 0xe bytes    C#
     Renci.SshNet.dll!Renci.SshNet.Session.Dispose() Line 1944    C#
     Renci.SshNet.dll!Renci.SshNet.BaseClient.Dispose(bool disposing) Line 237    C#
     Renci.SshNet.dll!Renci.SshNet.SftpClient.Dispose(bool disposing) Line 1337    C#
     Renci.SshNet.dll!Renci.SshNet.BaseClient.Dispose() Line 214    C#
     SshNetTest.exe!SshNetTest.Program.Main(string[] args) Line 123 + 0x6a bytes    C#

and again, stopped responding.

 

it's interesting to say that using the winscp as a client i am capable of uploading the file to this particular server. I prefer, of course, to use solely the ssh.net as the solution for sftp files upload.

Apr 30, 2012 at 5:15 PM

Hi again

Could you paste your code here? Remove username/password/host first.

May 1, 2012 at 7:38 AM

thanks.

the code here includes examination of the fingerprint, which works fine (tested). the BufferSize was set because of the error "Payload cannot be more then 32768 bytes" received (again, only from this specific client's sftp server).

here's the code i'm using:

 

                byte[] byteArray = Encoding.ASCII.GetBytes(TestFile.ToString());
                
                using (SftpClient sftp = new SftpClient("url", 22, "username", "password"))
                {
                    sftp.HostKeyReceived += delegate(object sender, HostKeyEventArgs e)
                    {
                        var sb = new StringBuilder();
                        foreach (var b in e.FingerPrint)
                        {
                            sb.Append(string.Format("{0:x}:", b).PadLeft(3, '0'));
                        }
                        string fingerPrint = sb.ToString();

                        if (fingerPrint.Substring(0, fingerPrint.Length - 1) == expectedFingerprint)
                            e.CanTrust = true;
                        else
                            e.CanTrust = false;
                    };

                    sftp.BufferSize = 1024 * 32 - 50;
                    sftp.ConnectionInfo.Timeout = TimeSpan.FromMinutes(3);
                    sftp.OperationTimeout = TimeSpan.FromMinutes(10);
                    sftp.Connect();

                    using (MemoryStream stream = new MemoryStream(byteArray))
                    {
                        sftp.UploadFile(stream, "LoremIpsum.txt");
                    }
                }

 

 

May 1, 2012 at 8:20 AM

Could you try this? I just added a .Disconnect() call after the file upload.

byte[] byteArray = Encoding.ASCII.GetBytes(TestFile.ToString());

using (SftpClient sftp = new SftpClient("url", 22, "username", "password"))
{
	sftp.HostKeyReceived += delegate(object sender, HostKeyEventArgs e)
	{
		var sb = new StringBuilder();
		foreach (var b in e.FingerPrint)
		{
			sb.Append(string.Format("{0:x}:", b).PadLeft(3, '0'));
		}
		string fingerPrint = sb.ToString();

		if (fingerPrint.Substring(0, fingerPrint.Length - 1) == expectedFingerprint)
			e.CanTrust = true;
		else
			e.CanTrust = false;
	};

	sftp.BufferSize = 1024 * 32 - 50;
	sftp.ConnectionInfo.Timeout = TimeSpan.FromMinutes(3);
	sftp.OperationTimeout = TimeSpan.FromMinutes(10);
	sftp.Connect();

	using (MemoryStream stream = new MemoryStream(byteArray))
	{
		sftp.UploadFile(stream, "LoremIpsum.txt");
	}
	
	sftp.Disconnect();
}

Also, use the source code for SSHNet and check the Output window in Visual Studio, you should get some SSHNet related output there regarding messages received/sent, could you paste them for this sample?

May 1, 2012 at 9:29 AM

the sftp.Disconnect(); doesn't have affect, since it's either stuck on the UploadFile or throw an exception there.

it never got to the Disconnect method.

when stuck, it was on the method (when pausing), in session.cs:

var hashData = new byte[4 + packetData.Length];

trying to copy the output, with step into it took ages and ended with an exception. this is the data in the output window:

Step into: Stepping over non-user code 'SshNetTest.Program..'
The thread '<No Name>' (0x624) has exited with code 0 (0x0).
'SshNetTest.vshost.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'SshNetTest.vshost.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Dynamic\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Dynamic.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'SshNetTest.vshost.exe' (Managed (v4.0.30319)): Loaded 'Anonymously Hosted DynamicMethods Assembly'
Step into: Stepping over non-user code 'System.Threading.Monitor.Enter'
Step into: Stepping over non-user code 'Renci.SshNet.Sftp.Requests.SftpRequest.RequestId.get'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.List'
Step into: Stepping over property 'Renci.SshNet.Sftp.Requests.SftpWriteRequest.SftpMessageType.get'.  To step into properties, go to Tools->Options->Debugging and uncheck 'Step over properties and operators (Managed only)'.
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.Add'
Step into: Stepping over non-user code 'Renci.SshNet.Sftp.Requests.SftpRequest.RequestId.get'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.AddRange'
Step into: Stepping over non-user code 'Renci.SshNet.Sftp.Requests.SftpWriteRequest.Handle.get'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.AddRange'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.AddRange'
Step into: Stepping over non-user code 'Renci.SshNet.Sftp.Requests.SftpWriteRequest.Offset.get'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.AddRange'
Step into: Stepping over non-user code 'Renci.SshNet.Sftp.Requests.SftpWriteRequest.Data.get'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.AddRange'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.AddRange'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.ToArray'
Step into: Stepping over non-user code 'System.Array.CopyTo'
Step into: Stepping over non-user code 'System.Array.CopyTo'
Step into: Stepping over non-user code 'Renci.SshNet.Channels.Channel.RemoteChannelNumber.get'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.Connection.ChannelMessage.ChannelMessage'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.Message.Message'
Step into: Stepping over non-user code 'Renci.SshNet.Common.SshData.SshData'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.Message.Message'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.Connection.ChannelMessage.ChannelMessage'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.Connection.ChannelMessage.LocalChannelNumber.set'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.Connection.ChannelDataMessage.Data.set'
Step into: Stepping over non-user code 'Renci.SshNet.Channels.Channel.IsOpen.get'
Step into: Stepping over non-user code 'Renci.SshNet.Channels.Channel.ServerWindowSize.get'
Step into: Stepping over non-user code 'Renci.SshNet.Channels.Channel.ServerWindowSize.get'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.Connection.ChannelDataMessage.Data.get'
Step into: Stepping over non-user code 'Renci.SshNet.Channels.Channel.ServerWindowSize.set'
Step into: Stepping over non-user code 'System.Net.Sockets.Socket.Connected.get'
Step into: Stepping over non-user code 'System.RuntimeType.Name.get'
Step into: Stepping over non-user code 'System.Threading.Thread.CurrentCulture.get'
Step into: Stepping over non-user code 'System.RuntimeType.GetCustomAttributes'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.MessageAttribute.Name.set'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.MessageAttribute.Number.set'
Step into: Stepping over non-user code 'System.Reflection.CustomAttribute.CreateCaObject'
Step into: Stepping over non-user code 'System.Reflection.CustomAttribute.GetCustomAttributes'
Step into: Stepping over non-user code 'System.Reflection.CustomAttribute.GetCustomAttributes'
Step into: Stepping over non-user code 'System.Linq.Enumerable.SingleOrDefault<object>'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.MessageAttribute.Name.get'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.Connection.ChannelMessage.LocalChannelNumber.get'
Step into: Stepping over non-user code 'string.Format'
Step into: Stepping over non-user code 'System.Diagnostics.TraceSource.TraceEvent'
Step into: Stepping over property 'Renci.SshNet.Security.Cryptography.BlockCipher.BlockSize.get'.  To step into properties, go to Tools->Options->Debugging and uncheck 'Step over properties and operators (Managed only)'.
Step into: Stepping over non-user code 'System.RuntimeType.GetCustomAttributes'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.MessageAttribute.Name.set'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.MessageAttribute.Number.set'
Step into: Stepping over non-user code 'System.Reflection.CustomAttribute.CreateCaObject'
Step into: Stepping over non-user code 'System.Reflection.CustomAttribute.GetCustomAttributes'
Step into: Stepping over non-user code 'System.Reflection.CustomAttribute.GetCustomAttributes'
Step into: Stepping over non-user code 'System.Linq.Enumerable.SingleOrDefault<object>'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.List'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.Connection.ChannelMessage.LocalChannelNumber.get'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.AddRange'
The thread '<No Name>' (0x11d4) has exited with code 0 (0x0).
The thread '<No Name>' (0x124c) has exited with code 0 (0x0).
The thread '<No Name>' (0x17c8) has exited with code 0 (0x0).
The thread '<No Name>' (0x142c) has exited with code 0 (0x0).
Step into: Stepping over non-user code 'Renci.SshNet.Messages.Connection.ChannelDataMessage.Data.get'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.AddRange'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.AddRange'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.ToArray'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.List'
Step into: Stepping over non-user code 'Renci.SshNet.Messages.MessageAttribute.Number.get'
Step into: Stepping over non-user code 'System.Collections.Generic.List<byte>.ToArray'
Step into: Stepping over non-user code 'System.Array.CopyTo'
Step into: Stepping over non-user code 'System.Array.CopyTo'
Step into: Stepping over non-user code 'System.Security.Cryptography.RNGCryptoServiceProvider.GetBytes'
Step into: Stepping over non-user code 'System.Array.CopyTo'
Step into: Stepping over non-user code 'System.Threading.Monitor.Enter'
Step into: Stepping over non-user code 'System.Array.CopyTo'
Step into: Stepping over non-user code 'System.Array.CopyTo'
Step into: Stepping over property 'Renci.SshNet.Security.Cryptography.BlockCipher.BlockSize.get'.  To step into properties, go to Tools->Options->Debugging and uncheck 'Step over properties and operators (Managed only)'.
The thread '<No Name>' (0x1620) has exited with code 0 (0x0).
Step into: Stepping over property 'Renci.SshNet.Security.Cryptography.BlockCipher.BlockSize.get'.  To step into properties, go to Tools->Options->Debugging and uncheck 'Step over properties and operators (Managed only)'.
A first chance exception of type 'System.Net.Sockets.SocketException' occurred in System.dll
A first chance exception of type 'System.Net.Sockets.SocketException' occurred in Renci.SshNet.dll
The thread '<No Name>' (0x1580) has exited with code 0 (0x0).
Step into: Stepping over property 'Renci.SshNet.Security.Cryptography.BlockCipher.BlockSize.get'.  To step into properties, go to Tools->Options->Debugging and uncheck 'Step over properties and operators (Managed only)'.
The thread '<No Name>' (0x117c) has exited with code 0 (0x0).
Step into: Stepping over property 'Renci.SshNet.Security.Cryptography.BlockCipher.BlockSize.get'.  To step into properties, go to Tools->Options->Debugging and uncheck 'Step over properties and operators (Managed only)'.
A first chance exception of type 'System.Net.Sockets.SocketException' occurred in System.dll
A first chance exception of type 'System.Net.Sockets.SocketException' occurred in Renci.SshNet.dll
A first chance exception of type 'System.Net.Sockets.SocketException' occurred in SshNetTest.exe

 

hope it gives a clearer understanding what is happenning here.

May 1, 2012 at 9:43 AM

Could you add this XML to the app.exe.config file, this should produce a log file called SshNetTrace.log somewhere.. :p

<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
    <system.diagnostics>
        <trace autoflush="true" />
            <sources>
                <source name="System.Net.Sockets" maxdatasize="1024">
                    <listeners>
                        <add name="MyTraceFile"/>
                    </listeners>
                </source>
                <source name="System.Net" maxdatasize="1024">
                    <listeners>
                        <add name="MyTraceFile"/>
                    </listeners>
                </source>
            </sources>

            <sharedListeners>
                <add
                  name="MyTraceFile"
                  type="System.Diagnostics.TextWriterTraceListener"
                  initializeData="SshNetTrace.log"
                />
            </sharedListeners>

            <switches>
                <add name="System.Net" value="Verbose" />
                <add name="System.Net.Sockets" value="Info" />
            </switches>

    </system.diagnostics>
</configuration>
May 1, 2012 at 12:01 PM

i have noticed that the process is stuck only on larger than (about) 35kb files. on the shorter files i get an exception.

the logs when stuck are short (as i 'kill' the process several minutes afterwards, without it ending) and the logs are just:

System.Net Information: 0 : [3948] Current OS installation type is 'Client'.
System.Net.Sockets Information: 0 : [3948] Socket#27806816 - Created connection from *my IP* to *the sftp IP*.

 

again this time the call-stack was not the dispose, but:

     Renci.SshNet.dll!Renci.SshNet.Session.SendMessage(Renci.SshNet.Messages.Message message) Line 699    C#
     Renci.SshNet.dll!Renci.SshNet.Channels.Channel.SendMessage(Renci.SshNet.Messages.Connection.ChannelDataMessage message) Line 446    C#
     Renci.SshNet.dll!Renci.SshNet.Channels.Channel.SendData(byte[] buffer) Line 211    C#
     Renci.SshNet.dll!Renci.SshNet.Sftp.SubsystemSession.SendData(byte[] data) Line 105    C#
     Renci.SshNet.dll!Renci.SshNet.Sftp.SftpSession.SendMessage(Renci.SshNet.Sftp.SftpMessage sftpMessage) Line 81    C#
     Renci.SshNet.dll!Renci.SshNet.Sftp.SftpSession.SendRequest(Renci.SshNet.Sftp.Requests.SftpRequest request) Line 267    C#
     Renci.SshNet.dll!Renci.SshNet.Sftp.SftpSession.RequestWrite(byte[] handle, ulong offset, byte[] data, System.Threading.EventWaitHandle wait) Line 408    C#
     Renci.SshNet.dll!Renci.SshNet.SftpClient.InternalUploadFile(System.IO.Stream input, string path, Renci.SshNet.Sftp.SftpUploadAsyncResult asynchResult, Renci.SshNet.Sftp.Flags flags) Line 1281    C#
     Renci.SshNet.dll!Renci.SshNet.SftpClient.UploadFile(System.IO.Stream input, string path, bool canOverride) Line 524    C#
     Renci.SshNet.dll!Renci.SshNet.SftpClient.UploadFile(System.IO.Stream input, string path) Line 498    C#
     SshNetTest.exe!SshNetTest.Program.Main(string[] args) Line 123 + 0x14 bytes    C#

 

on the shorter files that end with exception the logs are:

System.Net Information: 0 : [5892] Current OS installation type is 'Client'.
System.Net.Sockets Information: 0 : [5892] Socket#27806816 - Created connection from *my IP* to *the sftp IP*.
System.Net.Sockets Error: 0 : [5548] Exception in the Socket#27806816::Disconnect - A blocking

operation was interrupted by a call to WSACancelBlockingCall
System.Net Error: 0 : [5892] Exception in the
#65934242::UnhandledExceptionHandler - Connection was lost
System.Net Error: 0 : [5892]    at SshNetTest.Program.Main(String[] args) in D:\TestProjects\SshNetTest\Program.cs:line 150
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

Coordinator
May 1, 2012 at 1:37 PM

Hi,

 

Soryy for getting into this discussion so late, very busy with other project.

 

What seems to happen, based on that: this._messageListenerCompleted.WaitOne(); it either received disconnect message or session was disposed somehow.

Basiclly it stuck while waiting for another task, which listens for incoming messages to stop and it never happens.

 

I'll see if I can quickly add timeout for all those WaitOne methods, cause I think I should never have WaitOne() but always with some timeout, which should throw an execption, in case of timeout, the only exception would be dispose methods, since if I recall correctly, no exceptions shoud be thrown from the dispose methods.

 

I will try to make this change quickly and commit it today so you could take a look.

 

Thanks,

Oleg

May 1, 2012 at 2:04 PM

Thank you, that would be really helpful.

looking forward for this update.

Coordinator
May 1, 2012 at 2:25 PM

It was done, so please taker a look at latest version and let me know provides you with solution.

 

Thanks,

Oleg

May 1, 2012 at 3:08 PM

Thank you so much! it does not stuck in all the tests i've done, and identifies the error, when occurs without waiting.

still, for some reason i cannot deliver files to this specific sftp server using the ssh.net, which i prefer (only when using winscp, it works).

the first time it gets to WaitHandle, the var index = EventWaitHandle.WaitAny(waitHandles, operationTimeout); is 0, and then it throws the exception.

 

for a large file, i get the following log:

System.Net Information: 0 : [2636] Current OS installation type is 'Client'.
System.Net.Sockets Information: 0 : [2636] Socket#27806816 - Created connection from *my IP* to *the sftp IP*.
System.Net.Sockets Error: 0 : [2636] Exception in the Socket#27806816::Send - A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied
System.Net Error: 0 : [2636] Exception in the
#12549444::UnhandledExceptionHandler - A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied
System.Net Error: 0 : [2636]    at SshNetTest.Program.Main(String[] args) in D:\TestProjects\SshNetTest\Program.cs:line 150
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

 

and for smaller file i get immediately:

System.Net Information: 0 : [6060] Current OS installation type is 'Client'.
System.Net.Sockets Information: 0 : [6060] Socket#31038138 - Created connection from *my IP* to *the sftp IP*.
System.Net.Sockets Error: 0 : [7000] Exception in the Socket#31038138::Disconnect - A blocking operation was interrupted by a call to WSACancelBlockingCall
System.Net Error: 0 : [6060] Exception in the
#12547953::UnhandledExceptionHandler - Connection was lost
System.Net Error: 0 : [6060]    at SshNetTest.Program.Main(String[] args) in D:\TestProjects\SshNetTest\Program.cs:line 150
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

 

i would really appreciate your assistance, or ways to explain it to the client, how to properly configure his sftp server, so the error won't reoccur.

Thanks!

Coordinator
May 1, 2012 at 3:27 PM

Can you try to run your code from different machine or even better from different network.

Based on the socket exception, I want to exclude possibility of something on your network interrupts the connection.

I guess the best way for me to figure it out is to see and debug it myself.

If you think its possible, can you contact me privately with account where to connect to and possible file which causes you a problem and I will try to investigate it.

But the earliest I could do that is on Friday, since I have something to finish before meeting I have on Thursday.

 

Thanks,

Oleg

May 1, 2012 at 3:45 PM

Thank you, Oleg. it is really appreciated.

I've tried it on a different machine, different network, different environment (and actually a different country), however i still got error.

on a large file:

System.Net.Sockets Error: 0 : [3084] Exception in the Socket#60068066::Send - A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied

and on a smaller file:

System.Net Error: 0 : [3268] Exception in the #43332040::UnhandledExceptionHandler - Connection was lost

 

I would be happy to use your assistance, I won't be available before the beginning of next week, so I'll contact you then, if it's okay.

thanks!

Sep 24, 2012 at 12:39 PM

Upload File never ending:

Have met same issue with sshnet source submit #19813.
Obviously, for this server, the problem occurs when sent file exceed a certain size.

In a way, I could bypass the issue using async upload, then looping until timeout is reached.
Then, the trick is to avoid calling EndUploadFile when "IsCompleted" is still false because the infinite wait() come from there.
However, this does not solve the real issue: How will I finally transfer my files ?

using (var file = File.OpenRead(CurrentOutputDir + ".zip"))
{
UploadResult = sftp.BeginUploadFile(file, CurrentOutputDir + ".zip", null, null) as SftpUploadAsyncResult;
//  Wait for upload to finish                        
long upLoadTimeOut = 0;
bool uploadCompleted = false;
while (!uploadCompleted && upLoadTimeOut < 180)  
{
Thread.Sleep(1000);
upLoadTimeOut += 1; // add 1 second
uploadCompleted = true;//  Assume upload completed
if (!UploadResult.IsCompleted) uploadCompleted = false;
}

// Trick to avoid getting stuck: do not call "EndUploadFile" unless upload flagged as complete

if (UploadResult.IsCompleted)
sftp.EndUploadFile(UploadResult);
else if (upLoadTimeOut>=120 && !UploadResult.IsCompleted)
logger.Warn("SFTP upload timed out after 3 minutes. File may not be fully transferred.");
else
logger.Error("SFTP upload fatal error.");

...

...

 

 

Nov 24, 2012 at 2:35 AM

I've also encountered this problem in version 2012.3.9 under the .Net 3.5 runtime.  The "Infinite" timeout doesn't happen although it takes the full connection timeout before things finally fail.

From what I've been able to decipher I've run across two main problems that contributing to the problem (at least for my case)

The problem originates when the library attempts to send a packet size that exceeds the window size indicated by the server when the connection was established.  The server responds with a disconnect message with the info "data exceeds window size".

The library contains the following code when dealing with the disconnection message

    private void HandleMessage(DisconnectMessage message)
    {
        this.OnDisconnectReceived(message);

        //  Shutdown and disconnect from the socket
        if (this._socket != null)
        {
            lock (this._socket)
            {
                if (this._socket != null)
                {
                    this.SocketDisconnect();

                    //  When socket is disconnected wait for listener to finish
                    if (this._messageListenerCompleted != null)
                    {
                        //  Wait for listener task to finish
                        this.WaitHandle(this._messageListenerCompleted);
                        this._messageListenerCompleted.Dispose();
                        this._messageListenerCompleted = null;
                    }

                    this._socket.Dispose();
                    this._socket = null;
                }
            }
        }
    }


The code will first lock on the socket object and then waits on "this._messageListenerCompleted".  
Unfortunately it will wait forever (or untill the timeout on the wait is triggered) since the this._messageListenerCompleted.Set() doesn't occur until after the message is finished processing and existed the read loop.
    this.ExecuteThread(() =>
    {
        try
        {
            this.MessageListener();
        }
        finally
        {
            this._messageListenerCompleted.Set();
        }
    });


While stuck here it also has a lock on this._socket and will prevent the main thread from proceeding with the attempts to send the other buffer chunks.

The above describes the deadlock situation.  But the library would never send the file successfully anyways without following protocol.
The code seems to have code dealing with the window size but a closer look reveals problems
        protected void SendMessage(ChannelDataMessage message)
        {
            //  Send channel messages only while channel is open
            if (!this.IsOpen)
                return;

            if (this.ServerWindowSize < 1)
            {
                //  Wait for window to be adjust
                this._session.WaitHandle(this._channelWindowAdjustWaitHandle);
            }

            this.ServerWindowSize -= (uint)message.Data.Length;
            this._session.SendMessage(message);
        }


In the above the line:
    this.ServerWindowSize -= (uint)message.Data.Length;
will wrap to a very high number when the data length exceeds the ServerWindowSize.
In the end though, the ServerWindowSize is never used to limit the individual packet sizes.
The default is:
    this.BufferSize = 1024 * 32 - 52;
and will use anything up to MAXIMUM_PAYLOAD_SIZE = 1024 * 32
both of which are too big for my particular server which enforces a window size of 16k.

I can complete avoid the deadlock condition and successfully send a file only when I've explicitly reduced the buffer size to a value less than 16k.

Jan 22, 2013 at 10:49 PM

Oleg,

Have you by any chance had a look at mhesler's analysis above? I'm also getting random hangs on SftpClient. Note I've been running SSH.NET code from October last year, will update to latest commit.

Coordinator
Jan 23, 2013 at 2:00 PM

Hi,

Thanks for reminding me about this thread.

I did not use this anayzes yet to fix that particular problem but what I did already is reduced a default buffer size to 16KB since it was causing too many problems in other cases and after analyzing putty source code I realized they also use 16KB as a default.

Did you try to use smaller buffer size? Did it fixed your problem?

 

I will take a look at deda locking issue that was described in earlier post since it should not be this way.

 

Sorry for delay in answering this post.

Thanks,

Oleg

Jan 23, 2013 at 2:42 PM

I've updated my production code with the latest version of SSH.NET and will report back if timeout happens again. Unfortunately I am unable to reproduce this in tests, it only happens randomly on production server after multiple days of running fine. I will report back here if it happens again or in a month if it is still running fine. 

Beyers

Coordinator
Jan 23, 2013 at 2:48 PM

ok, no problem.

But I still will have to look at this dead lock issue.

 

Thanks,

Oleg

Coordinator
Jan 25, 2013 at 1:33 AM

mhesler,

 

Did you happen to use latest version of the library?

After checking my code I completely rewrote that section so there should be no more locking in this section.

Can you check it out if possible and let me know if you still experience problems there?

 

Thanks,

Oleg

Feb 14, 2013 at 8:13 AM
thanks reawakening the thread and for the updates.
i was testing the last commit to see if OperationTimeout works as i predicted.

i set: sftp.OperationTimeout = TimeSpan.FromSeconds(1);
and used asynchronous upload, usinf this code:
                        sftp.ConnectionInfo.Timeout = TimeSpan.FromSeconds(1);
                        sftp.OperationTimeout = TimeSpan.FromSeconds(1);


                        sftp.Connect();



                        using (stream = new MemoryStream(byteArray))
                        {
                            IAsyncResult sftpASynch = sftp.BeginUploadFile(stream, fullPath, null, null);

                            if (!sftpASynch.AsyncWaitHandle.WaitOne(100))
                            {

                                sftp.EndUploadFile(sftpASynch);

                                Console.WriteLine("send file by sftp process was terminated after: " + sftpOperationTimeout + " seconds");

                            }
                        }
                        Console.WriteLine("Send by SFTP finished. Path:" + fullPath);
after some seconds (although OperationTimeout was set to 1 second) the file was uploaded to the server and was not terminated as supposed to.
Coordinator
Feb 14, 2013 at 2:43 PM
Hi,

I think there is a little misunderstanding about timeout parameter.
What it basically means is if there is no response from the server for lets say 30 sec, then throw an exception.
But if upload goes correctly and it keep uploading and server did not disconnect for example or no other unexpected error occurred on the server then timeout will not be hit.

Hope it explains a little more.

Thanks,
Oleg