This project is read-only.

HTTP Proxy: Empty Server String and Dynamic Port Forwarding: ipAddress Problem

Oct 28, 2013 at 4:01 PM
I have playing around with sshnet for this few days. And I found some problems when I try Dynamic Port Forwarding with HTTP proxy. Maybe this will help some. So here we go:

In Session.ConnectHTTP() with the version from download link this method cause indefinite loop. This loop seem fixed on the latest version (28765 as I download it), but cause another problem, the method did not catch the server string. My quick fix for this adapted from the first version and modified a little:
 private void ConnectHttp()
        {
            var httpResponseRe = new Regex(@"HTTP/(?<version>\d[.]\d) (?<statusCode>\d{3}) (?<reasonPhrase>.+)$");
            var httpHeaderRe = new Regex(@"(?<fieldName>[^\[\]()<>@,;:\""/?={} \t]+):(?<fieldValue>.+)?");

            var encoding = new Renci.SshNet.Common.ASCIIEncoding();

            //write to http proxy
            this.SocketWrite(encoding.GetBytes(string.Format("CONNECT {0}:{1} HTTP/1.0\r\n", this.ConnectionInfo.Host, this.ConnectionInfo.Port)));
            System.Diagnostics.Debug.WriteLine("Send CONNECT {0}:{1} HTTP/1.0\r\n", this.ConnectionInfo.Host, this.ConnectionInfo.Port);
            //  Sent proxy authorization is specified
            if (!string.IsNullOrEmpty(this.ConnectionInfo.ProxyUsername))
            {
                var authorization = string.Format("Proxy-Authorization: Basic {0}\r\n",
                                                  Convert.ToBase64String(encoding.GetBytes(string.Format("{0}:{1}", this.ConnectionInfo.ProxyUsername, this.ConnectionInfo.ProxyPassword)))
                                                  );
                this.SocketWrite(encoding.GetBytes(authorization));
            }

            this.SocketWrite(encoding.GetBytes("\r\n"));

            HttpStatusCode statusCode = (HttpStatusCode)0;
            //HttpStatusCode statusCode = HttpStatusCode.OK;
            var response = string.Empty;
            var contentLength = 0;

            //while (statusCode != HttpStatusCode.OK)
            while (true)
            {
                this.SocketReadLine(ref response);

                var match = httpResponseRe.Match(response);

                if (match.Success)
                {
                    statusCode = (HttpStatusCode)int.Parse(match.Result("${statusCode}"));
                    //return;
                    continue;
                }

                // continue on parsing message headers coming from the server
                match = httpHeaderRe.Match(response);
                if (match.Success)
                {
                    var fieldName = match.Result("${fieldName}");
                    if (fieldName.Equals("Content-Length", StringComparison.InvariantCultureIgnoreCase))
                    {
                        contentLength = int.Parse(match.Result("${fieldValue}"));
                    }
                    continue;
                }

                //  Read response body if specified
                if (string.IsNullOrEmpty(response) && contentLength > 0)
                {
                    var contentBody = new byte[contentLength];
                    //use socketRead to read from http proxy
                    this.SocketRead(contentLength, ref contentBody);
                }

                // fix "" after OK
                if (string.IsNullOrEmpty(response) && contentLength == 0)
                {
                    return;
                }

                switch (statusCode)
                {
                    case HttpStatusCode.OK:
                        break;
                    default:
                        throw new ProxyException(string.Format("HTTP: Status code {0}, \"{1}\"", statusCode, statusCode));
                }

            }//end while
        }
Then in ForwardedPortDynamic.HandleSocks5() this method throw me exception cause I have bad dns server. My quick fix is disabling the ipAddress parsing and pass the hostname. Should sshnet lookup the hostname locally? Some said that Tor Proxy just pass zero value. And I guest putty did not do dns lookup either, while bitvise give the user an option for this.
...
var addressType = stream.ReadByte();

                IPAddress ipAddress = null;
                byte[] addressBuffer = null;
                var host = "";
                /*
                 * field 5: destination address of 
                4 bytes for IPv4 address
                1 byte of name length followed by the name for Domain name
                16 bytes for IPv6 address
                 * */
                switch (addressType)
                {
                    case 0x01:
                        {
                            addressBuffer = new byte[4];
                            stream.Read(addressBuffer, 0, 4);

                            ipAddress = new IPAddress(addressBuffer);
                            host = ipAddress.ToString();

                        }
                        break;
                    case 0x03:
                        {
                            var length = stream.ReadByte();
                            addressBuffer = new byte[length];
                            stream.Read(addressBuffer, 0, addressBuffer.Length);
                            //should we lookup dns locally?
                            //ipAddress = IPAddress.Parse(new Renci.SshNet.Common.ASCIIEncoding().GetString(addressBuffer));
                           
                            var domainName = new Renci.SshNet.Common.ASCIIEncoding().GetString(addressBuffer);
                            ipAddress = new IPAddress(0x00);
                            host = domainName;

                        }
                        break;
                    case 0x04:
                        {
                            addressBuffer = new byte[16];
                            stream.Read(addressBuffer, 0, 16);

                            ipAddress = new IPAddress(addressBuffer);
                            host = ipAddress.ToString();

                        }
                        break;
                    default:
                        throw new ProxyException(string.Format("SOCKS5: Address type '{0}' is not supported.", addressType));
                }

                /*
                 * field 6: port number in a network byte order, 2 bytes
                */
                var portBuffer = new byte[2];
                stream.Read(portBuffer, 0, portBuffer.Length);
                var port = (uint)(portBuffer[0] * 256 + portBuffer[1]);
                //var host = ipAddress.ToString();

                this.RaiseRequestReceived(host, port);
...
With these I can connect to ssh server with HTTP proxy and start Dynamic Port Forwarding. But then another new problem came up. When I try my browser with this port forwarding, server frequently close its connection throwing An Established connection forcibly closed by remote host thrown from SocketRead().

My first guess is the server get too many connections and forcibly close the connection. Adding Thread.sleep(100) on SendMessage() seem fix the problem when I use polipo before dynamic socks forward, at least it did not closed frequently. But it didn't work with CCProxy. This maybe caused by polipo pipelining mechanism, while CCProxy just send whatever it has. I can see so many SSH_MSG_CHANNEL_OPEN at the same time with CCproxy.

Or maybe there is data corruption especially on heavy connection, because sometime I see the the Channel Message did not orderly sent. Such as this:
SshNet.Logging Verbose: 1 : ReceiveMessage from server: 'ChannelEofMessage': 'SSH_MSG_CHANNEL_EOF : #7'.
SshNet.Logging Verbose: 1 : Received '16:System.Byte[]'.
SshNet.Logging Verbose: 1 : Received '16:System.Byte[]'.
SshNet.Logging Verbose: 1 : ReceiveMessage from server: 'ChannelCloseMessage': 'SSH_MSG_CHANNEL_CLOSE : #7'.
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelCloseMessage': 'SSH_MSG_CHANNEL_CLOSE : #7'.
SshNet.Logging Verbose: 1 : Sent data 'System.Byte[]:48'.
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelEofMessage': 'SSH_MSG_CHANNEL_EOF : #7
I also get frequent Connection aborted by software on your localhost exception from SocketWrite(), and then the _socket closed, thus close the session. Should this exception never came up except we send disconnect method?

Those two new problem also happen when I use socks (in this case i use socks5) proxy. And also happen with the previous version from download link.

These new problem seem not related with previous http proxy or portforwarding problem. Thank You.
Oct 30, 2013 at 12:47 PM
Ok, I think I found the cause of this problem. Its because Channel Message did not orderly sent. From what I've found the server close its connection if we sending late SSH_MSG_CHANNEL_EOF after SSH_MSG_CHANNEL_CLOSE. I'm guessing this happen when EOF and CLOSE are sent in almost the same time, so server confused then close its connection. Because from my understanding, we can send CLOSE without EOF, and after server receive CLOSE message it will terminate its channel, so whatever message we sent to this channel after this should make no difference until we send new OPEN message again, except we deal with broken server. Am I right?

Also I see some message sent over and over when it should already stop, but this seem not causing the server close its connection. But this should not happen, this may cause unpredictable result.
...
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelCloseMessage': 'SSH_MSG_CHANNEL_CLOSE : #3'. -> closed before EOF
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelOpenMessage': 'SSH_MSG_CHANNEL_OPEN : #214'.
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelOpenMessage': 'SSH_MSG_CHANNEL_OPEN : #215'.
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelOpenMessage': 'SSH_MSG_CHANNEL_OPEN : #218'.
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelEofMessage': 'SSH_MSG_CHANNEL_EOF : #3'. -> late EOF
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelOpenMessage': 'SSH_MSG_CHANNEL_OPEN : #216'.
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelOpenMessage': 'SSH_MSG_CHANNEL_OPEN : #217'.
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelOpenMessage': 'SSH_MSG_CHANNEL_OPEN : #219'.
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelOpenMessage': 'SSH_MSG_CHANNEL_OPEN : #220'. -> opening CHANNEL number 220
SshNet.Logging Verbose: 1 : ReceiveMessage from server: 'ChannelOpenConfirmationMessage': 'SSH_MSG_CHANNEL_OPEN_CONFIRMATION : #214'.
SshNet.Logging Verbose: 1 : SendMessage to server 'ChannelDataMessage': 'SSH_MSG_CHANNEL_DATA : #0'. --> this CHANNEL :#0 already closed from early, but this message still sent over and over, the good thing it did not cause the server close its connection, because no late EOF at first CLOSE message sent, even after this,  EOF , CLOSE and DATA sent over and over. But this may cause unpredictable result.
Session SocketRead error: Empty Packet--> Server close its connection
A first chance exception of type 'Renci.SshNet.Common.SshConnectionException' occurred in Shien SSH Client.exe
I still investigating what cause this thing. But until that, or until the developer or anybody have better fix, for now I just try to delaying SSH_MSG_CHANNEL_CLOSE so did not overlap SSH_MSG_CHANNEL_EOF. With these I can use it for regular browsing, and server did not close its connection frequently. I know, the real problem is not fixed yet, and still any possibility of wrong channel message sent that may cause unpredictable result. Well, its better than nothing.
...
            var msgType = message.GetType().Name;
           
            //slowdown close message so not overlap eof
            //ok this is very" dirty fix :)
            if (msgType == "ChannelCloseMessage")
            {
                int d = 100; //still experiment with this value :)
                Thread.Sleep(d);
                System.Diagnostics.Debug.WriteLine("Delaying close message  for {0} miliseconds", d);
            }

            Thread.Sleep(100); //this made quite lot of difference when I use polipo

            this.Log(string.Format("SendMessage to server '{0}': '{1}'.", message.GetType().Name, message.ToString()));

...
Oct 31, 2013 at 5:56 AM
Another alternative:

in Channel.cs
/// <summary>
        /// Sends close channel message to the server
        /// </summary>
        /// <param name="message">Message to send.</param>
        protected void SendMessage(ChannelCloseMessage message)
        {
            //  Send channel messages only while channel is open
            if (!this.IsOpen)
                return;

            //wait for late eof if any
            Thread.Sleep(1000);
            this._session.SendMessage(message);

            //  When channel close message is sent channel considred to be closed
            this.IsOpen = false;
        }
Oct 31, 2013 at 6:01 AM
Edited Oct 31, 2013 at 6:06 AM
Oops... double post...
Coordinator
May 5, 2014 at 8:56 PM
Edited Jun 2, 2014 at 8:04 PM
Sorry for joining this party late, but I don't see how we could be sending the SSH_MSG_CHANNEL_EOF and SSH_MSG_CHANNEL_CLOSE messages in the wrong order for a given channel as messages are sent synchronously.

Can you still reproduce the ordering issue with the latest beta ?

I'll try to find time to look into the SOCKS5 issue after the release of beta 2.