Memory leak somewhere in library

Oct 18, 2011 at 3:31 PM
Edited Oct 18, 2011 at 3:35 PM

Well, I wish I could be more specific, but this ones has me stumped... its very slight and Im 90% sure its in this library.  Ill describe what im doing and maybe that will help out.

Basically, Im writing a process monitor for windows boxes to be able to see whats going on with processes on a unix box.  Each process on the unix box writes to a log file with its thread count.  Every second I grep the last (newest) line of the log file and pull out its thread count number and then update the listview row for that process' timestamp and thread count.  When only monitoring 1 process, the leak is almost non-existant.  However, I have to connect to 12 different unix servers that each could have several processes on it (it ends up being around 30+ processes).  Now, since I dont want to freak out the unix admins with that many connections per second, I connect once to each server, persist the SshClient object and hold the connection open, and then every second I send 2 Unix Commands (so 2 new SshCommands per second per process, but I .Dispose of them after getting the results.

Any ideas? Over the span of 30 minutes it'll go from 60ish k memory usages to 180k.  While this is slow, this program is a monitor and will remain open for hours on end.  The link is the code to the class that is created for each process and contains the SshClient and timer.  Let me know if you need to see more code.

EDIT: I also have been using the trial version of .NET Memory Proflier and have the saved profile logs of that too if you'd want to see them... I just dont really know how to read them.

using System;
using System.Collections.Generic;
using System.Text.RegularExpressions;
using System.Timers;
using System.Windows.Forms;
using EESQueueCountMonitor.Classes.Support_Classes;
using Renci.SshNet;

namespace EESQueueCountMonitor.Classes
{
    public class TailBackgroundWorker
    {
        #region Static Globals

        private static readonly string unixUserName = "username";
        private static readonly string unixPassword = "password";

        #endregion

        #region Members

        /// <summary>
        /// The timer used to trigger Qcnt updates
        /// </summary>
        public System.Timers.Timer GrepTimer { get; set; }

        /// <summary>
        /// The data for the server and process
        /// </summary>
        public ProcessData ServerData { get; set; }

        private SshClient unixClient = null;
        private ListView currentListView = null;

        #endregion

        #region Constructor

        public TailBackgroundWorker(ProcessData serverData, ListView currListView)
        {
            ServerData = serverData;
            currentListView = currListView;
            GrepTimer = new System.Timers.Timer(1000);
            GrepTimer.Elapsed += new ElapsedEventHandler(OnTimedEvent);
            unixClient = new SshClient(new PasswordConnectionInfo(StringEnum.GetStringValue(ServerData.ServerIP), unixUserName, unixPassword));
            unixClient.Connect();
        }

        ~TailBackgroundWorker()
        {
            GrepTimer.Elapsed -= OnTimedEvent;
            unixClient.Disconnect();
            unixClient.Dispose();
        }

        #endregion

        #region Functions

        private void OnTimedEvent(object source, ElapsedEventArgs e)
        {
            string filepath = "cd the_path" + ServerData.Process.ToLower() + "/";
            string filename = GetNewestFile(filepath);

            string command = filepath + ";grep Qcnt " + filename + " | tail -1;";

            string qCnt = SendUnixCommand(command);

            if (!String.IsNullOrEmpty(qCnt))
                UpdateActiveListView(currentListView, qCnt);
        }

        private string SendUnixCommand(string command)
        {
            string unixResults = String.Empty;

            try
            {                
                SshCommand cmd = unixClient.RunCommand(command);
                unixResults = cmd.Result;
                cmd.Dispose();

                return unixResults;
            }
            catch (Exception ex)
            {
                if (ex is Renci.SshNet.Common.SshException)
                {
                    string err = "There was an error while trying to connect to " + ServerData.ServerIP + ".";
                    MessageBox.Show(err, "Error!", MessageBoxButtons.OK, MessageBoxIcon.Error, MessageBoxDefaultButton.Button1);

                    return unixResults;
                }
                if (ex is Renci.SshNet.Common.SshAuthenticationException)
                {
                    string err = "There was an authentication error, either the username or password is no longer valid.";
                    MessageBox.Show(err, "Error!", MessageBoxButtons.OK, MessageBoxIcon.Error, MessageBoxDefaultButton.Button1);

                    return unixResults;
                }
                else
                {
                    throw;
                }
            }
        }

        private delegate void ListBoxUpdateEventHandler(ListView activeListViewName, string message);

        /// <summary>
        /// Used to allow thread-safe updating of ListViews, this function is called by a
        /// BackgroundWorker to check if we are safe to update the ListView (ie in the UI thread).
        /// If not, it uses the above delegate to Invoke itself on the UI thread and then
        /// carries out its updating of the ListView with the new data retrived by the BackgroundWorker.
        /// </summary>
        /// <param name="ActiveListView">The ListView to write to</param>
        /// <param name="message">The newest set of data to write</param>
        private void UpdateActiveListView(ListView ActiveListView, string message)
        {
            if (ActiveListView.InvokeRequired)
            {
                // This means we're on the wrong thread!  
                // Use BeginInvoke or Invoke to call back on the correct thread.
                ActiveListView.Invoke(
                    new ListBoxUpdateEventHandler(UpdateActiveListView), // the method to call back on
                    new object[] { ActiveListView, message });           // the list of arguments to pass
            }
            else
            {
                QcntMsg nullMsg = new QcntMsg("");
                List<QcntMsg> results = Utilities.FormatUnixResults(message);

                //ActiveListView.BeginUpdate();

                foreach (QcntMsg msg in results)
                {
                    if (msg != nullMsg)
                    {
                        ListViewItem[] listItems = 
                            ActiveListView.Items.Find(ServerData.ServerName + ":" + ServerData.Process, false);

                        foreach (ListViewItem item in listItems)
                        {
                            item.SubItems[0].Text = msg.Time;
                            item.SubItems[4].Text = msg.Qcnt;
                        }

                        listItems = null;
                    }
                }

                if (ActiveListView.Items.Count > 0)
                    ActiveListView.EnsureVisible(ActiveListView.Items.Count - 1);

                //ActiveListView.EndUpdate();

                results.Clear();
            }
        }

        /// <summary>
        /// Gets the Unix folder location for the specifed module due to the fact that
        /// not all module's log file is located in the same location on each server
        /// </summary>
        /// <returns>The Unix file path to the module's application log</returns>
        private string GetNewestFile(string filepath)
        {
            string filename = String.Empty;
            string command = filepath + ";ls -lrt | grep stuff_to_grep | grep -v .gz | tail -1;";

            string result = SendUnixCommand(command);

            string pattern = @"(stuff_to_grep\S+)\n";
            Match match = Regex.Match(result, pattern, RegexOptions.IgnoreCase);

            if (match.Success)
            {
                // Finally, we get the Group value and display it.
                filename = match.Groups[1].Value;
            }
            else
                filename = "ERROR";

            return filename;
        }

        #endregion
    }
}

Oct 18, 2011 at 7:59 PM

Nothing guys? This could potentally be a major issue with the library...

Coordinator
Oct 18, 2011 at 8:04 PM

sorry I didnt reply sooner,

I am just busy with another project, where deadline is approaching fast, so dont have much time to look at it lately.

 

I defentily will take a look at it later, as soon as I am done with this dedaline, which is next month.

 

I remeber I was doing some tests a while back, to ensure that I clean all the memory I use, so I guess the problem came back after many updates.

I will investigate it again as soon as I have time, especially since you have a work case I can test.

 

Sorry about the delay.

 

Thanks,

Oleg

Oct 18, 2011 at 8:43 PM

No worries man, sorry I got a bit uppity, this problem is really bugging me

Oct 19, 2011 at 5:56 AM

Hi

Can you create a dump of the process when you are experiencing this problem? If memory serves me, it should be under the Debug file menu near the bottom in Visual Studio. Or if you are on windows 7 at least, I think you can do this via task manager, but I would prefer it if you did it from within VS.

I will later send a PM with my email and I will investigate your problem. What is your timezone?

A dump will be the most effective way to find leaks.

On 18. okt. 2011, at 22:43, "hershizer33" <notifications@codeplex.com> wrote:

From: hershizer33

No worries man, sorry I got a bit uppity, this problem is really bugging me

Oct 19, 2011 at 5:59 AM
Oh, yes you can attach your logs as well.


On 18. okt. 2011, at 22:43, "hershizer33" <notifications@codeplex.com> wrote:

From: hershizer33

No worries man, sorry I got a bit uppity, this problem is really bugging me

Oct 19, 2011 at 1:35 PM

K Ill get you that dump, and Im in the central time zone

Oct 19, 2011 at 3:34 PM

Well work doesnt allow me to upload from their computers... could you PM me your email and Ill get it to ya?

Coordinator
Oct 19, 2011 at 4:36 PM

Ken,

Thanks for taking a look at it as I am very busy for next 3-4 weeks :(.

Thanks,

Oleg

Oct 19, 2011 at 4:54 PM

hershizer: Message sent!

Oleg: No problem, this might be a really serious issue so I thought I'd take a look at it. Been waiting for my new computer, which will arrive soon! :D My home computer is pretty useless atm :(