SFTP Parallel Downloads again

Jan 25, 2013 at 8:03 PM
Edited Jan 25, 2013 at 8:52 PM

Hi Oleg,

I have to bother you again.

Found a new problem: after your fix in the isConnected method everything worked fine. But now I encountered a new problem. This is how to reproduce the problem: Starting 4 or more parallel downloads over one connection with <BeginDownload> and after everything is finished calling <EndDownload> on each (in the asyncCallback Method). If some of the files are not present on server side I get the "no such file" error. That's OK so far. But very often I get an error which is called "the safehandle has been closed" (Object disposed Exception) after <EndDownload>. The Problem is, that this exception occurs on downloads which are actually existent on the server.

So If you have 8 parallel Downloads with 4 existing and 4 non existing files you will get the following exceptions: 4 times "no such file", and 4 times Download OK. That’s how it should be. But sometimes you will get 6 times "no such file" PLUS 2 times "safe handle has been closed" and 2 times Download OK. So what happened to the 2 other files which were actually there? If the operation timeout is being set it even throws some “timeout” errors, even when the timeout is set to a ridiculous high amount (e.g. 60 seconds). The whole operations didn’t take 60 Seconds at all, so why it is throwing this exception?

Tried different servers again, it’s not related to them. Also tried latest Sourcecode.

Hope you understand what I'm trying to explain. A Stacktrace for better understanding:

 

   Renci.SshNet.Common.SshException: Safe handle has been closed. ---> System.ObjectDisposedException: Safe handle has been closed.     
   bei Renci.SshNet.Sftp.SubsystemSession.WaitHandle(WaitHandle waitHandle, TimeSpan operationTimeout) in C:\FTPDownloader\Renci.SshNet\SubsystemSession.cs:Zeile 165.     
   bei Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError) in C:\FTPDownloader\Renci.SshNet\Sftp\SftpSession.cs:Zeile 298.     
   bei Renci.SshNet.SftpClient.InternalDownloadFile(String path, Stream output, Action`1 downloadCallback) in C:\FTPDownloader\Renci.SshNet\SftpClient.cs:Zeile 1343.     
   bei Renci.SshNet.SftpClient.BeginDownloadFile() in C:\FTPDownloader\Renci.SshNet\SftpClient.cs:Zeile 502.     
  
   bei Renci.SshNet.Common.AsyncResult.EndInvoke() in C:\FTPDownloader\Renci.SshNet\Common\AsyncResult.cs:Zeile 93.     
   bei Renci.SshNet.SftpClient.EndDownloadFile(IAsyncResult asyncResult) in C:\FTPDownloader\Renci.SshNet\SftpClient.cs:Zeile 536.     
   bei PortalService.FTPDownloader.Data.cls_sFTPDownloader.DownloadComplete(SftpDownloadAsyncResult oState) in C:\FTPDownloader.Data\FTP Classes\cls_sFTPDownloader.vb:Zeile 123.
Coordinator
Jan 26, 2013 at 1:00 AM

Hey,

 

Thanks for reporting it. I guess now I am at the point where I am getting very rare cases.

I figured out where the problem occurs but I don't know why since file exists and it should download it correctly.

What basically happens is that during the download some exception occurs, which I cannot catch at a moment where, which causes to terminate the thread and dispose of wait handle but this and causes other thread, which tries to use event handle to fail.

 

Bottom line I know what one solution could be but I would like to see if I can find out the reason for first exception at a first place, which causes thread to terminate.

I guess day or two I would commit a solution.

 

Thanks,

Oleg

Coordinator
Jan 26, 2013 at 2:17 PM

Hey,

 

I found the reason behind this and actually surprise how people didn't noticed that before but this error could potentially occur for any async operation where one of them causing error.

 

I just committed a fix so please check it out when you have time and see if you still have a problem.

 

Thanks,

Oleg

Jan 26, 2013 at 6:51 PM

Hi Oleg,

It works like a charm, thank you :-).

Jan 28, 2013 at 10:40 AM
Edited Jan 28, 2013 at 10:40 AM

Hi Oleg,

did you lock something in your threads? Exceptions are gone, but the downloadspeed decreases extremely. 1 Download = 20Mbit/s and 2 Downloads parallel = 2Mbit/s per Download. Before the fix, 2 Downloads went with 10 Mbit/s per Download. It is not network related, testing in Lan at the moment. Do you have an Idea what happened?

Thanks again :-)

Jan 28, 2013 at 1:26 PM

Nerver mind ... my fault! Debugmessages from sftpclient are slowing down everything :-).

Coordinator
Jan 28, 2013 at 3:16 PM

 

Ok,

No problem.

 

I am actually planning to review all debug messages generate by library,

so in release version they would never appear.

 

Thanks,

Oleg

Jan 29, 2013 at 3:24 PM
Edited Jan 29, 2013 at 3:25 PM

Hi Oleg,

found another <Safehandle has been closed> Exception. It is thrown at an another point in your code. It is reproducible by setting the operation timeout to very small limit (e.g. 100ms) and downloading some files parallel. Please have a look at the stacktrace for further investigation:

 

   Renci.SshNet.Common.SshException: Safehandle has been closed. ---> System.ObjectDisposedException: Safehandle has been closed.
   bei Renci.SshNet.Sftp.SubsystemSession.WaitHandle(WaitHandle waitHandle, TimeSpan operationTimeout) in D:\FTPDownloader\Renci.SshNet\SubsystemSession.cs:Zeile 171.
   bei Renci.SshNet.Sftp.SftpSession.RequestRealPath(String path, Boolean nullOnError) in D:\FTPDownloader\Renci.SshNet\Sftp\SftpSession.cs:Zeile 740.
   bei Renci.SshNet.Sftp.SftpSession.GetCanonicalPath(String path) in D:\FTPDownloader\Renci.SshNet\Sftp\SftpSession.cs:Zeile 112.
   bei Renci.SshNet.SftpClient.InternalDownloadFile(String path, Stream output, SftpDownloadAsyncResult asyncResult, Action`1 downloadCallback) in D:\FTPDownloader\Renci.SshNet\SftpClient.cs:Zeile 1392.
   bei Renci.SshNet.SftpClient.<>c__DisplayClassa.b__8() in D:\FTPDownloader\Renci.SshNet\SftpClient.cs:Zeile 522.

   bei Renci.SshNet.Common.AsyncResult.EndInvoke() in D:\FTPDownloader\Renci.SshNet\Common\AsyncResult.cs:Zeile 93.
   bei Renci.SshNet.SftpClient.EndDownloadFile(IAsyncResult asyncResult) in D:\FTPDownloader\Renci.SshNet\SftpClient.cs:Zeile 556.
   bei PortalService.FTPDownloader.Data.cls_sFTPDownloader.DownloadComplete(SftpDownloadAsyncResult oState) in D:\FTPDownloader\FTPDownloader.Data\FTP Classes\cls_sFTPDownloader.vb:Zeile 115.

 

Sorry for disturbing you again. Thanks :-)

Coordinator
Jan 30, 2013 at 2:03 AM

Hey, yea, no problem.

Thanks for letting me know.

Apparently not many people using parallel downloads :):)

 

I will take a look at it again but probably either later this week or next as I am in the middle of other changes now and also start beeing busy with another project at work.

 

Thanks,

Oleg

Jan 30, 2013 at 3:15 PM
Edited Jan 30, 2013 at 3:17 PM

Hey guys,

  I have been seeing sporadic instances of the following exception: "Session operation has timed out" on the connect command.

        If (sPrivateKeyFile.Length > 0) Then
            Dim pk = New PrivateKeyFile(System.IO.File.OpenRead(PrivateKeyFolder() & sPrivateKeyFile), sPassPhraseOrPW)
            conn = New PrivateKeyConnectionInfo(sHost, iPort, sUserID, pk)
        Else
            conn = New PasswordConnectionInfo(sHost, iPort, sUserID, sPassPhraseOrPW)
        End If
        Dim sf = New SftpClient(conn)
        sf.Connect()

I am using the 3.5 version that I believe was built on last Saturday. This is being used in a threaded service, however the most recent error occured when only one thread was processing, so I do not believe it is a thread specific problem.
Other info- this is a private key / passphrase type connection.
I thought I would mention this just in case it is related to the problems being encountered by the original poster.
The problem is very inconsistent. I can reset a failed attempt (try again using the same code/process) and it will work.
Thanks, Mike
edit- the code frame on this panel is hacking up the appearance of this post...
Jan 30, 2013 at 3:33 PM
Edited Jan 30, 2013 at 3:34 PM

One last note, after a bit more reading through theads here, I found it coincidental that the exception message that I am seeing sporadically is the same one mentioned in the 3rd post on this thread:

http://sshnet.codeplex.com/discussions/429447

Sorry if this deviates from the original thread too far, I can create another if you like.  It could very well be that this is a 3.5 issue only.

Thank You

Coordinator
Feb 2, 2013 at 8:07 PM
notrix,

I was trying to recreate this problem but all I am getting is to "Session operation has timed out" exception as it was designed.
Can you may be provide a code that you using where you get "Safehandle has been closed" exception?

mikkoljay,

Did you try to compile it in different VS.NET version as it mentions in other thread?

Thanks,
Oleg
Feb 4, 2013 at 2:57 PM
Hi Oleg,
Yes I tried to compile my own version in VS2008 that just did not seem to ever connect at all. What is interesting though, is that the exact same code would connect if I compiled it via VS2010. After these issues I just went back to using latest binaries that are downloaded from here.
Thank You