Carriage Returns Stipped from Downloaded File?

Jan 11, 2013 at 5:56 PM

Hello,  I just stumbled onto this library and I really like it, but my first attempt at using it ran into a snag.  I successfully downloaded a sample text file via the SftpClient, but the downloaded file seems to have had the carriage returns all stripped out, i.e. what used to look like rows now looks like a jumbled continuous string.  If I download the file with another client app like CoreFTP, the file looks "normal" like the original.

Is there a setting or configuration that might get me past this issue?  I did a bit of searching and could not find any similar threads, so I apologize if this has been discussed before.

Thanks, Mike

Coordinator
Jan 11, 2013 at 6:42 PM

Hi,

I can take a look at it.

Can you post a code example that you using?

Also, can you post a code that I can run to create this file on my machine or may be if you get me a test account on your machine to better recreate a problem I could that. You can send me details privately then if you like.

Meanwhile, do you use some setting when you use CoreFTP if so which one? Also what server do you use to connect to.

Also, please not this is SFTP and not FTP library. As far as I know SFTP is completely different from FTP.

The reason you did not find similar thread is because nobody ever experience this problem before.

Thanks,

Oleg

Jan 11, 2013 at 7:03 PM

Thank You for the quick reply Oleg!  Yes I am connecting to an SFTP server.

My simple test file contents looks like this when edited in Notepad:

record number one

record number two

record number three

When I download the file, and open in Notepad, it looks like this:

record number onerecord number tworecord number three

I believe that the \r\n's are being replaced with \r only.

Here is the code I am using:

Imports Renci.SshNet
Imports System
Imports System.Collections

Public Enum SFTP_ConnType
    Unknown
    User_Password
    PrivateKey_PassPhrase
End Enum

Public Module SshSftp
     Public Sub DownLoadDir(ByVal conType As SFTP_ConnType, _
                   ByVal sHost As String, _
                   ByVal sUserID As String, _
                   ByVal sPrivateKeyPath As String, _
                   ByVal sPassPhraseOrPW As String, _
                   ByVal iPort As Integer, _
                   ByVal sDirLocal As String, _
                   ByVal sDirRemote As String, _
                   ByVal sFilter As String, _
                   ByRef files As String(), _
                   ByVal Overwrite As Boolean, _
                   ByVal RemoveFromServer As Boolean)
        If (Not sDirLocal.EndsWith("\")) Then sDirLocal += "\"
        If (Not System.IO.Directory.Exists(sDirLocal)) Then System.IO.Directory.CreateDirectory(sDirLocal)
        Dim sDestFile As String = String.Empty
        Dim bAdded As Boolean = False
        Dim fname As String = String.Empty
        Dim dlist As New List(Of String)()
        Dim iResult As Int32 = 0
        If (sDirRemote.Length = 0) Then sDirRemote = "./"
        If (Not sDirRemote.EndsWith("/")) Then sDirRemote += "/"
        Dim conn As ConnectionInfo = Nothing
        Select Case conType
            Case SFTP_ConnType.PrivateKey_PassPhrase
                Dim pk = New PrivateKeyFile(System.IO.File.OpenRead(sPrivateKeyPath), sPassPhraseOrPW)
                conn = New PrivateKeyConnectionInfo(sHost, iPort, sUserID, pk)
            Case SFTP_ConnType.User_Password
                conn = New PasswordConnectionInfo(sHost, iPort, sUserID, sPassPhraseOrPW)
        End Select
        Dim sf = New SftpClient(conn)

        sf.Connect()
        Dim dirlist = sf.ListDirectory(sDirRemote)
        If (Not Object.ReferenceEquals(dirlist, Nothing)) Then
            For Each f In dirlist
                If Not f.IsDirectory Then
                    bAdded = False
                    fname = f.Name
                    'if a file pattern is specified, we need to see if they match
                    If (sFilter.Length = 0 Or sFilter = "*.*") Then
                        bAdded = True
                    Else
                        If (fname.ToUpper() Like sFilter.ToUpper()) Then
                            bAdded = True
                        End If
                    End If
                    If (bAdded And Not Overwrite) Then
                        bAdded = Not (System.IO.File.Exists(sDirLocal & fname))
                    End If
                    If (bAdded) Then dlist.Add(fname)
                End If
            Next
        End If
        ''Now go get each file we just found
        For Each sfile In dlist
            fname = sDirRemote + sfile
            sDestFile = sDirLocal + sfile
            Dim oOutput = New System.IO.FileStream(sDestFile, System.IO.FileMode.Create)
            sf.DownloadFile(fname, oOutput)
            oOutput.Close()
            oOutput.Dispose()
        Next
        sf.Disconnect()
        files = dlist.ToArray()
    End Sub


The version of CoreFTP that I use is CoreFTP LE v 2.2 build 1751.

The version of the Renci.SshNet.dll that I am using is the 3.5 build.

Thank you for taking the time to reply back to me, I appreciate it.

-Mike

Jan 11, 2013 at 9:03 PM
Edited Jan 11, 2013 at 9:12 PM

I have done a bit more research on this subject, and here is some of what I have found-

These are notes from CoreFTP's Help Manuals:

Site Settings - SSH

Simulate ASCII mode for SFTP -- Applies to "Putty compatible" mode only which does not support ASCII mode by default. Check this option to add a CR to each LF when transferring text files.

</help snippets>

This is very interesting- it seems as though this would be the responsibility of the client (me :) ) to adjust these.  I am going to read more about encoding, etc.  I believe this is starting to explain what is going on.

Thank you

Jan 11, 2013 at 9:43 PM
Edited Jan 11, 2013 at 10:20 PM

One last note, I have just tried this method, and the file downloaded the way that I hoped it would:

        ''Now go get each file we just found
        For Each sfile In dlist
            fname = sDirRemote + sfile
            sDestFile = sDirLocal + sfile
            Dim oOutput = New System.IO.FileStream(sDestFile, System.IO.FileMode.Create)
            Dim thebytes = sf.ReadAllBytes(fname)
            Dim iOffset As Int32 = 0
            oOutput.Write(thebytes, 0, thebytes.Length)
            oOutput.Close()
            oOutput.Dispose()
        Next
        sf.Disconnect()
        files = dlist.ToArray()

I don't know why getting the data into a byte array and writing would produce a different result than the DownloadFile method, but I am happy it is working. 

Edit: This method does leave some concern for very large files, since from what I can tell the ReadAllBytes only gets one chunk up to the MaxBufferSize.

If you have any insights I would appreciate your opinion.

Thank you

Coordinator
Jan 14, 2013 at 9:06 PM

Hi,

 

Thanks for testing it out.

From what I was checking SFTP does not have any special flags to handle ASCII file differently.

I also dont do any special handling of data when I receive it so it may be it a server configuration that does that.

I never heard anybody complaining about this issue before.

Can you try it on different servers and see if you have similar behavior? In case you still want me to take a look it.

 

Thanks,

Oleg