Resume SFTP upload after dropped connection

Sep 8, 2011 at 4:51 PM

We are uploading large files via SFTP over an unreliable network connection.  With this library, is it possible to resume an upload if the network connection is dropped and later re-established?

Thanks,
-Robert

Coordinator
Sep 8, 2011 at 4:58 PM

In theory yes,

Practiclly current feature does not exists.

 

What you can do for now is to upload file in chunks using SftpFileStream.

You can open it by calling sftp.Open or other similar method.

 

Then simply keep truck of what chunks of files have been uploaded and simply offset the stream by number of uploaded bytes next time you need to resume upload of the same file.

 

Hope it helps,

 

Thanks,

Oleg

May 25, 2012 at 5:46 AM
Edited May 25, 2012 at 6:20 AM

I have used the above mentioned method to resume file transfer, but i am having problem in resuming file.

I have used one text file to resume file transfer whenever i try to resume file transfer then i am loosing already transferred content, in text file already transferred content was replaced by whitespace and new content is appended to the file...

i have used below code to resume file transfer. Please tell me what i have missed.

        private void ResumeFileUpload(FileStream streamSftpFile file)
        {
            ////client.OffsetSeek = file.Length;
            ////client.BeginResumeFile(stream, "/newtextfile_resume.txt", new AsyncCallback(EndUploadFile), null);
            stream.Seek(file.LengthSeekOrigin.Begin);            
 
            SftpFileStream destStream = client.Open("/newtextfile_resume.txt"FileMode.AppendFileAccess.Write);
 
            ulong offset = 0;
            var buffer = new byte[102400];
            var uploadCompleted = false;
 
            do
            {
                var bytesRead = stream.Read(buffer0buffer.Length);
 
                if (bytesRead < 102400)
                {
                    var data = new byte[bytesRead];
                    Buffer.BlockCopy(buffer0data0bytesRead);
                    using (var wait = new AutoResetEvent(false))
                    {
                        destStream.Write(data0data.Length);
                    }
 
                    uploadCompleted = true;
                }
                else
                {
                    destStream.Write(buffer0buffer.Length);
                }
 
                offset += (uint)bytesRead;
 
            } while (!uploadCompleted);
 
            destStream.Close();
        }

 Here client is a object of SftpClient and parameter stream is FileStreal which i want to resume and file is an object of SftpFile and i am using "Length" property of it to Seek position of stream.

Thanks in advance.

Vijay

Jun 5, 2012 at 8:01 AM
  • if remote ftp file does not exist, open file stream with FileMode.CreateNew
  • if remote ftp file exists, open file stream with FileMode.Append
  • if remote ftp file exists, the local file should skip the remote file length

 

var pci = new PasswordConnectionInfo(_config.Host, _config.Port, _config.User, _config.Password) { Timeout = TimeSpan.FromSeconds(60) };

var ftp = new SftpClient(pci) { KeepAliveInterval = new TimeSpan(0, 1, 0), OperationTimeout = new TimeSpan(3, 0, 0), BufferSize = 1024 * 16, };
ftp.Connect();

const string localPath = @"...";
const string remotepath = @"...";

var remoteExists = ftp.Exists(remotepath);
using (var destStream = ftp.Open(remotepath, remoteExists ? FileMode.Append : FileMode.CreateNew, FileAccess.Write))
{
    using (var localStream = File.Open(localPath, FileMode.Open, FileAccess.Read))
    {
        const int BufferSize = 102400;
        var buffer = new byte[BufferSize];
        var uploadCompleted = false;
        long offset = 0;

        if (remoteExists)
        {
            var attr = ftp.GetAttributes(remotepath);
            offset = attr.Size;

            localStream.Seek(offset, SeekOrigin.Begin);
        }

        while (!uploadCompleted)
        {
            var bytesRead = localStream.Read(buffer, 0, BufferSize);

            destStream.Write(buffer, 0, bytesRead);
            offset += bytesRead;

            uploadCompleted = (bytesRead < BufferSize);
        }
    }
}

Sep 17, 2012 at 1:00 PM

I'm trying the above code, but getting a Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied error on:  

using (var destStream = ftp.Open(remotepath, remoteExists ? FileMode.Append : FileMode.CreateNew, FileAccess.Write))

I was previously using 

ftp.Upload(fs, targetFTPFile);

Where fs is an open filestream to a local file, and targetFTP is the relative path on the FTP server including file name, and this worked except with no support for continuing the upload.

Does this seem to be an issue with the attempted FileMode.CreateNew?  I could attempt a mix of ftp.Upload and then use FileMode.Append if the connection gets dropped.