Dealing with "prompts"

Jul 8, 2014 at 3:01 PM
Ive managed to get connected and ive manged to send commands and recieve responses so its working great so far. But whenever I execute a command that has a prompt on the next line my code simply just hangs...

Heres my code:
            PasswordAuthenticationMethod authMethod = new PasswordAuthenticationMethod(username, password);
            ConnectionInfo connectionInfo = new ConnectionInfo(session, username, authMethod);
            vms = new SshClient(connectionInfo);
            vms.Connect();

            SshCommand result = vms.RunCommand(command);
            Console.WriteLine(result.Result);
Any ideas? Its working quite well other than just hanging.
Jul 16, 2014 at 5:08 PM
Edited Jul 16, 2014 at 5:09 PM
@Jelly,

There are many examples from the users here in the discussions to do all kinds of things. There is certainly an example in the discussions, docs, or tests/source to do with what you are trying to get working.

But there are a myriad of things not involved with the SSH library that could be in your way. I mean, one thing could easily be that you are talking to a router or server that requires a CR, LF, or CRLF in order to get your commands. Also, ssh needs to know when the other side is 'done'. As another poster asked, you can do this with a timer, the stream not growing after x time, a timeout, etc.

I wrote a wrapper a while back that reset a timer to 0 whenever there was activity on the stream. If the timer expired, it meant that I wasn't going to wait any longer for any more data, and I proceeded with what I got to that point.

What debugging have you done to see (maybe even on the host) what it might be expecting next, or if it has even given up?

I'm just trying to say, that it is actually more complicated to logon & do commands than most people (me included) realize. That of course means a lot more places to go wrong! :)

So when it works, nobody says anything, but when it fails...!

So if you can provide a bit more detail on the environment, logs/debugging, and pinpointing the place at which the thing breaks down, it makes it a lot easier to offer help. Right now, the code above might just work for me, but I'd be running it on a host not like yours... :)

Not trying to be depressing, just hoping to get more info about your issue! :)

code is poetry.

pat
:)
Jul 17, 2014 at 8:25 AM
Hello Pat,

Thanks for taking the time to look into my problem. I agree theres lots of good information on here and i'm getting little bits from here and there, but im at a stage now where I think I just need the gentle nudge in the right direction.

So here's the situtation... We are using a system called OpenVMS. What i'm attempting to do is to create an automation test suite in C#, its going well so far but everyone still has to manually run batch jobs in openVMS to complete the financial transactions. I dont like having manual steps in my tests! :)

So far i've been able to send commands and read responses no problem, the only issue is when theres a prompt. The issue isnt detecting the prompt I can do that easily its then dealing with the prompt... let me show you my code..
        string myCommand = command;
        myCommand = myCommand.TrimEnd();
        TimeSpan commandTime;
        Stopwatch stopwatch = new Stopwatch();
        string result = "";
        string output = "";
        var cmd = client.CreateCommand(myCommand);

        try
        {
            stopwatch.Start();
            var asynch = cmd.BeginExecute();
            var resultStream = new StreamReader(cmd.OutputStream);

            while (!asynch.IsCompleted)
            {
                commandTime = stopwatch.Elapsed;
                if (commandTime > timeOut)
                {
                    break;
                }
                result = resultStream.ReadToEnd();
                output = output + result;
                Debug.WriteLine(result);
            }
            cmd.CancelAsync();
            cmd.EndExecute(asynch);
            resultArray = output.Split(new char[] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries);
            WebDriver.myLogger.LogComment("Class: " + MethodBase.GetCurrentMethod().DeclaringType.FullName);
            WebDriver.myLogger.LogComment("Method: " + MethodBase.GetCurrentMethod().Name);
            WebDriver.myLogger.LogPass("Executed VMS command: " + myCommand);
        }
        catch (Exception ex)
        {
            WebDriver.myLogger.LogComment("Class: " + MethodBase.GetCurrentMethod().DeclaringType.FullName);
            WebDriver.myLogger.LogComment("Method: " + MethodBase.GetCurrentMethod().Name);
            WebDriver.myLogger.LogFail("Error executing VMS command: " + myCommand + " | " + ex.Message);
        }
        WebDriver.TestLog().LogStep(WebDriver.TestLog().GetTestResult(), "Execute command");


What I need to be able to do is when i detect the prompt... somewhow change my command "cmd" to something like "Text\R" which would input text and a return...
ive not yet worked out how to do this yet because it wont build if i try. I thought maybe I could just cancel the command when it hits the prompt and create a new one, and execute that, but that doesnt work either.

I'm sorry if im not explaining this very clearly, as I said I know how to detect the prompts thats not an issue I just need some way of dealing with them.

In OpenVMS we have DCL's (Digital Command Language) I can use this to run whats known as an ARL (Advanced Reporting Language) and I can do this by creating a text file, filling it with the parameters and then basically saying okay VMS run this ARL and use this text file as your input....Im wondering if I can do that with this? Send a command with a list of parameters to use when it detects prompts....
Jul 17, 2014 at 7:47 PM
@Festive,

I no longer do things this way, because on the systems I am programming against today (Cisco switches, routers, & ASAs), the command prompt is the last thing output after a command. You make it sound like the return string(s) are asynchronous, and the ssh term will see the command prompt returned before the last of the response from the host.

That said, you can monitor text as it is returned, and simply RegEx or .Contains() each batch as they are added into the response as a whole. I have written a 'chunking' wrapper for my commands, as the hosts I'm programming against have a 1kB limit on the command length, and I may need to give 1000's of lines of commands in a single 'action', or 'batch'. My code got complicated quickly as the Cisco equipment will allow multiple sessions from the same user, but the client would not allow multiple SSH/22 connections to the same host-and my program is very multi-threaded.

One thing nobody on SO could solve was how to have multiple threads submit 'work', or 'jobs' as ssh commands, process each serially (one at a time), then (hard part) return the results to each calling thread. That took me a while to get right, and we'll talk about that another day! :)

So let me start with the smallest denominator in my rather complicated system (intuitively called, "SendBigCommand"):

Note: You can convert easily, but VB proves much easier to use and troubleshoot than C#, and the event/regex processing makes it easy to return to your code months later when it is unrecognizable).

[Yes, I write in both, and Java, and Perl, and... so no nasty comments; Just using the right tool for the job]

Public Function SendBigCommand(TheCommands As String, Optional IdleTimeout As Integer = 4) As String
        Dim tsIdleTimeout As TimeSpan = TimeSpan.FromSeconds(IdleTimeout)
        Dim swIdleTimeout As New Stopwatch
        Dim sbResult As New StringBuilder
        Dim sResult As String = ""
        '
        Try
            Using c As SshClient = New SshClient(Me.Host, Me.User, Me.Pass)
                c.Connect() ' only ONE thread can connect at a time!
                If c.IsConnected Then
                    '
                    ' since each command starts from scratch, we need to always get to #enable before we enter the submitted commands...
                    '
                    Dim sGetToEnable As String = "enable" & vbLf & Me.EnablePass & vbLf & "term pager 0" & vbLf
B Dim cmdLastLine As String = Me.GetLastLine(TheCommands)
                    '
                    Dim cmdCombined As String = String.Concat(sGetToEnable, TheCommands & vbLf)
                    log.Debug("cmdCombined: [" & cmdCombined & "]")
                    '
                    Using cmd As SshCommand = c.CreateCommand(cmdCombined)
                        Dim cmdIndex = cmdCombined.IndexOf(cmdLastLine)
                        Dim asynch As System.IAsyncResult = cmd.BeginExecute()
                        Using reader As StreamReader = New StreamReader(cmd.OutputStream)
                            Dim res As String = ""
                            swIdleTimeout.Start()
                            Do While Not asynch.IsCompleted
                                res = reader.ReadToEnd()
                                If String.IsNullOrEmpty(res) Then
                                    If swIdleTimeout.Elapsed.TotalSeconds >= tsIdleTimeout.TotalSeconds Then
                                        log.Debug(String.Format("Exiting from async execution.. idle for {0} seconds", swIdleTimeout.Elapsed.TotalSeconds.ToString("0.00")))
                                        ' cmd.CancelAsync() ' turns asynch.IsCompleted to True; same as doing: (not really, throws exception)
                                        Exit Do
                                    End If
                                    Continue Do
                                Else
                                    sbResult.Append(res)
                                    swIdleTimeout.Restart() ' every time we get something, we re-start the timer...
                                End If
                            Loop
                            ' here after IdleTimeout _or_ rare asynch.IsCompleted()
                        End Using
                        '
                        '=========================================================
                        '
                        ' return anything after the last line of the command (cmdLastLine)
                        '
                        sResult = sbResult.ToString ' convert SB into string
                        log.Debug("sResult: Before [" & sResult & "]")
                        log.Debug("cmdLastLine: " & cmdLastLine)
                        log.Debug("cmdLastLine.Length: " & cmdLastLine.Length)
                        '
                        ' try to remove everything up to the actual response...
                        '
A Dim indexOfLastCharOfCommand As Integer = sResult.IndexOf(cmdLastLine) + cmdLastLine.Length
| '
| ' final string result (sResult):
| '
A sResult = sResult.Substring(indexOfLastCharOfCommand, sResult.Length - indexOfLastCharOfCommand)
                        log.Debug("sResult: After [" & sResult & "]")
                        '
                    End Using
                Else
                    log.Fatal("client could not connect!")
                    Throw New Exception("Client could not connect")
                End If
            End Using
        Catch ex As Exception
            log.Fatal("EXCEPTION(M): " & ex.Message)
            log.Fatal("EXCEPTION(T): " & ex.ToString)
            log.Fatal("EXCEPTION(S): " & ex.StackTrace)
        End Try
        Return sResult
    End Function
I hope this isn't too intimidating. It evolved over time. :)

The observant will notice that (1) I'm not concerning myself with a prompt, and (2) the line: _" ' here after IdleTimeout or rare asynch.IsCompleted()"_ This is because in my situation, I simply NEVER get a signal asynch.IsCompleted-never have. I gave up and moved on by allowing a integer timeout (ToSeconds) that the calling function can set (big commands don't take a lot more than small ones, as it is just idle timeout, not whole operation timeout).

Let em explain the parts that aren't included in the function. The first is GetLastLine(), (B) which I use to trim the response (in my case, the responses include the commands as entered, which is why you see the lines at the A-A. I don't want to process the commands, just the results.
        Public Function GetLastLine(SomeLines As String) As String 
            Dim linesNL() As String
            linesNL = SomeLines.Split(CChar(Environment.NewLine))
            Dim linesLF() As String
            linesLF = SomeLines.Split(CChar(vbLf))
            Dim linesCR() As String
            linesCR = SomeLines.Split(CChar(vbCr))
            Dim linesCRLF() As String
            linesCRLF = SomeLines.Split(CChar(vbCrLf))
            ' each is an array:
            Dim linesArrayArray As String()() = {linesNL, linesLF, linesCR, linesCRLF}

            Dim largest As Integer = linesArrayArray.Max(Function(ar) ar.Count)
            log.DebugEx("largest: {0}", largest)
            ' now which array had than many lines?

            Dim descArray = linesArrayArray.OrderByDescending(Function(c) c.Count)
            log.DebugEx("descArray(0).Count is {0}", descArray(0).Count)
            log.DebugEx("descArray(1).Count is {0}", descArray(1).Count)
            log.DebugEx("descArray(2).Count is {0}", descArray(2).Count)
            log.DebugEx("descArray(3).Count is {0}", descArray(3).Count)

            Dim longestArray As String() = {}

            'For Each element As Integer In numLines
            '    large1 = Math.Max(largest, element)
            'Next

            Dim lastLine As Integer = descArray(0).Count
            Return descArray(0)(lastLine - 1)
        End Function
Now before you guys start laughing, this is how I solved the problem of not knowing whether the commands are given (or even mixed) as nothing, LF, CR, or CRLF at the end of each line. Welcome to Cisco IOS...

That is why you see this above:
Dim linesArrayArray As String()() = {linesNL, linesLF, linesCR, linesCRLF}
You find a better way, I'm all ears!

Suggestion

In the above SendCommand, you will see the following lines:
[...]
Else
    sbResult.Append(res)
    swIdleTimeout.Restart() ' every time we get something, we re-start the timer...
End If
[...]
Every time I am there, it means that a line of test/string (var res) has gobbled up some response. Becuase of this activity, the command swIdleTimeout.Restart() does what it says, and resets the idle timer.

So this is the perfect place to do regex on the last text received from the host (might be multi-line, so prepare for that?), using the string variable 'res' (in my case). If you regex res, and you see a match for your command prompt, or whatever, then you can trigger another action, flip a boolean and wait, or even redirect the output, etc.

Knowing what little I do about your issue, and selecting only fro the code I have written, this seems a good mechanism to inspect the host response as it is coming back to the client. Now be mindful, your host might return ALL of your data from a command in one big string, or it may give it to you line-by-line... Ebanle lots of debugging to see.

Note On Code: I tried to remove as much comments/etc. for clarity.
Jul 17, 2014 at 7:50 PM
Edited Jul 17, 2014 at 7:52 PM
@All,

Well, Codeplex hosed that reply up pretty well.

Let me know if the A & B pointers are confusing...

Also, buried in the above is a subtle 'cry for help' if anyone out there has gotten big data back from a host AND gotten this to work:
Dim asynch As System.IAsyncResult = cmd.BeginExecute()
If asynch.IsCompleted() then
[...]
I'd love to get out of the timeout business, but today, I need it... :(

Always happy to share!

pat
:)
_
NOTE: There also may be a subtlety revolving around a command prompt being RETURNED, and the command prompt being ECHOED..._
Jul 18, 2014 at 8:42 AM
BogusException wrote:
@Festive,

[Yes, I write in both, and Java, and Perl, and... so no nasty comments; Just using the right tool for the job]
Ah Java :) she was my first. I completely agree, use the right tool for the job!

Thanks for taking the time to reply, you've given me a lot that I can use there. As you suspected the prompt text is not the last line I receive, its the last but 1 and the actual prompt is an empty string. But thats fine as I just scan through the last few entries to find it. I guess my issue is with my solution its looking for the response of the command without the capability of issuing further parameters. But I shall solider on with your suggestions!
Jul 18, 2014 at 4:02 PM
@Festive,

I'm far from perfect with this library. If you find a sure-fire way to make asynch.IsComplete() work, I'm your disciple!

Frankly, my suspicion is that SSH, or any client in a 'shell' environment for I/O, really can't have any way of knowing whether the stream response is done.

A good example included the *nix 'tail' command, and any Windows equivalents (I guess ping with lots of iterations would be close). I say this because even after a particular shell returns the prompt, it may not be done yielding content. I hope that makes sense to someone out there... :)

As for Java, you have to respect her. She's STILL the only truly OO language there is. Unfortunately, all high level languages today have to have 'interpreters', which convert source to intermediate language (MSIL is no exception), so virtually no programmer today programs against the CPU/HW they are using. That has brought about a whole other set of problems, where quality of developers as suffered from making programming more like block diagrams and dummy lights. MS has it's own version of this in VS, called LightSwitch...

Anyway, I hope you can make progress, and post back a better way for us to use this library! :)

pat
:)
Jul 22, 2014 at 12:53 PM
Well I thought id made progress, but apparently using the ShellStream fudges up creating a command and trying to read the result. If I create a stream and then use: result = resultStream.ReadToEnd(); it just hangs and doesnt run Asynch like its supposed to. Stupid machine kicks box

Here is how i implemented sending a command and looking for a return:
    public static void SendShellCommand(string command, List<string> passCriteria, TimeSpan timeout)
    {
        stream.Flush();
        var reader = new StreamReader(stream);
        var writer = new StreamWriter(stream);
        int numOfPassCriteriaMet = 0;
        int numOfPassCriteriaExpected = passCriteria.Count;
        bool passCriteriaMet = false;
        string line = "";
        String[] output;
        Stopwatch myWatch = new Stopwatch();

        try
        {
            writer.AutoFlush = true;
            stream.Flush();
            myWatch.Start();
            writer.WriteLine(command);
            line = "";

            while (myWatch.Elapsed < timeout & passCriteriaMet == false)
            {

                line = line + reader.ReadToEnd();
                output = line.Split(new char[] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries);
                resultArray = output;

                foreach (string value in passCriteria)
                {
                    foreach (string outputValue in output)
                    {
                        if (outputValue.Contains(value, StringComparison.CurrentCultureIgnoreCase))
                        {
                            numOfPassCriteriaMet++;
                        }
                        if (numOfPassCriteriaMet == numOfPassCriteriaExpected)
                        {
                            passCriteriaMet = true;
                            break;
                        }
                    }
                    if (passCriteriaMet)
                    {
                        break;
                    }
                }
                Thread.Sleep(100);
            }

            myWatch.Stop();
            myWatch.Reset();

            if (passCriteriaMet)
            {
                WebDriver.myLogger.LogPass("Executed VMS command: " + command);
            }
            else
            {
                WebDriver.myLogger.LogFail("One or more steps failed executing the VMS command: " + command);

                    foreach (string result in resultArray)
                    {
                        string comment = Regex.Replace(result, @"[^\u0000-\u007F]", string.Empty);
                        WebDriver.myLogger.LogComment(comment.Trim());
                    }
            }
        }
        catch(Exception ex)
        {
            myWatch.Stop();
            myWatch.Reset();
            WebDriver.myLogger.LogFail("Error executing VMS command: " + command + " | " + ex.Message);
        }
        WebDriver.TestLog().LogStep(WebDriver.TestLog().GetTestResult(), "Execute command");

    }
Jul 22, 2014 at 3:34 PM
Ive managed to get a version of this working quite robustly, when i've tested it enough i'll post it here! :)

Jelly
Jul 22, 2014 at 5:43 PM
YAY!!!