Browse Files On Remote Computers

I recently posted on an application that was able to monitor the desktops of multiple remote computers. The network topology is shown below. Notice that the remote computers (User 1-6 below) and the Monitoring Station are behind their own firewalls. This prevents direct communication between the devices. Instead all messages are relayed by a Server.

During this post, I will introduce the ability for the Monitoring Station to browse and download files from the remote computers.

Here is a screen cast of the file browser at work.

 New Commands

The architecture used for the communication between the agent and the server is a message bus that supports a number of commands. The following commands were added to support the file browsing features:

The ‘DirectoryCommandMessage’ is used to send commands from the Server to the Agent (remote computer). This message is used to initiate the following ‘Actions’:

• Fetch Current Data – Fetches the directory data (folders and files) for the current directory.
• Change Directory – Causes the current directory to be changed and new directory data sent to the Server.
• Fetch Content – This fetches content (files or folders) from the remote machine.

The ‘FileDataMessage’ and ‘DirectoryDataMessage’ messages are sent from the Agent (remote computer) to the Server. This messages contain data that was generated as a result of a ‘DirectoryCommandMessage’. The ‘DirectoryDataMessage’ contains the current directory on the Agent and a list of folders and files in that directory. The ‘FileDataMessage’ contains the bytes (‘FileData’ property) of a zip file containing the contents requested.

 System Interaction (Sequence) Diagrams

To better understand the communication, lets looks at the interaction diagrams that capture the file browser features. More details will follow in the code sections. The first diagram captures when the Monitor Station user switches into browse mode for an Agent.

When the Agent starts, it begins polling for new messages (or commands). Many times it does not receive new messages. However, when a new message set (the server sends a queue of commands) is received the Agent acts on the commands and returns results immediately (without waiting for the next poll). The Agent communicates to the Server via a number of web services.

The Monitoring Station application is a web application using the ASP.NET MVC framework. To begin browsing, the user clicks a link and a request is routed to the “Browse” action method on the “Browse” controller. This returns to the browser the “Browse” view. The action method also queues up three new messages for the Agent. The messages are:

• Poll Mode – Switch to poll mode. This allows the Agent to only poll for commands. No image data is posted or collected.
• Poll Rate – Switch to a higher polling rate (500 ms per poll). This allows the fast response times.
• Directory Command Message (DCM) – To fetch the directory info (file and folder list) for the current directory.

The next time the Agent polls it collects the messages in its queue and processes them. As part of processing the DCM, a ‘DirectoryDataMessage’ (DDM) is generated and sent back to the server. As mentioned above, the DDM has a payload that contains the directory info (file and directory list) for the current directory on the Agent’s computer.

In the mean time, the “Browse” view has started a JavaScript interval set to poll the server for new directory info once per second. If no new information is found, then the server returns an empty result (not quite empty but minimal). Once the ‘DirectoryDataMessage’ arrives, new directory info is cached by the server. This is results in new HTML (MVC partial view) being sent to the browser on the next interval timer request.

The following diagrams captures the communication that occurs for changing the current directory and requesting / downloading files. Many of the same elements are in this diagram.

Although not shown, it should be remembered that the Agent is continuously polling (via web service) the Server for new messages (every 500 milliseconds) and the browser is continuously polling (via AJAX) the server for new HTML to render. In between these polling events, other Server communication (either web services or AJAX) are occurring. All of this polling should raise concerns about scalability. The domain model that I am developing for has at most 50 agents so scalability should not be an issue.

Agent Code

Let’s begin examining the code by looking at the new code on the Agent. The following code processes the new ‘DirectoryCommandMessage’ on the Agent:

private void ProcessDirectoryCommandMessage(DirectoryComma
ndMessage message)
{
    switch(message.Command)
    {
        case WebService.Action.FetchCurrentData:
            {
                DirectoryDataMessage ddm = GetDirectoryDataMessage();

                PostMessage(ddm);
                break;
            }
        case WebService.Action.ChangeDirectory:
            {
                if(Directory.Exists(message.CurrentDirectory))
                {
                    _currentDirectory = message.CurrentDirectory;
                    if(!_currentDirectory.EndsWith("\") || !_currentDirectory.EndsWith("/"))
                    {
                        _currentDirectory += "/";
                    }

                    DirectoryDataMessage ddm = GetDirectoryDataMessage();

                    PostMessage(ddm);
                }
                break;
            }
        case WebService.Action.FetchContent:
            {
                ZipHelper zipHelper = new ZipHelper();
                foreach (string file in message.ContentToFetch)
                {
                    if(Directory.Exists(file))
                    {
                        // Directory
                        string folderName = Path.GetFileName(file);
                        zipHelper.AddFolder(file, folderName);
                    }
                    else if(File.Exists(file))
                    {
                        zipHelper.AddFile(file, "");
                    }
                }

                MemoryStream memoryStream = new MemoryStream();
                zipHelper.Save(memoryStream);

                FileDataMessage fdm = new FileDataMessage
                                          {
                                              AgentId = _agentId,
                                              FileName = "files.zip",
                                              FileData = memoryStream.GetBuffer(),
                                              TimeStamp = DateTime.Now
                                          };
                PostMessage(fdm);

                break;
            }
    }
}

private DirectoryDataMessage GetDirectoryDataMessage()
{
    string[] folders = Directory.GetDirectories(_currentDirectory);
    string[] files = Directory.GetFiles(_currentDirectory);
    DirectoryDataMessage ddm = new DirectoryDataMessage
                                   {
                                       AgentId = _agentId,
                                       TimeStamp = DateTime.Now,
                                       CurrentFolder = _currentDirectory,
                                       Folders = new ArrayOfString(),
                                       Files = new ArrayOfString()
                                   };
    ddm.Folders.AddRange(folders);
    ddm.Files.AddRange(files);
    return ddm;
}

The heart of this code is a switch statement that branches the code based upon the ‘Action’ sent in the message.

The first ‘case’ statement processes the ‘FetchCurrentData’ option. This generates DirectoryDataMessage (DDM) using a helper method and then posts that data to the server. The Agent has a private member (_currentDirectory) that keeps track of the current directory.

The second ‘case’ statement does nearly the same except it includes a bit of logic to update the current directory based upon information sent from the Server (indirectly from the Monitoring Station) before generating the DDM.

The final ‘case’ statement handles the request for files. The ‘ContentToFetch’ property is a collection of paths to either folders or files. Each path is added to an archive using a ‘ZipHelper’ (wrapper around DotNetZip library). The bytes for the zip file are then added to a ‘FileDataMessage’ and sent back to the Server.

Web Service (asmx)

The above code shows that the Agent is now posting back to the server two new message types (‘DirectoryDataMessage’ & ‘FileDataMessage’). The following code handles the processing of these two messages on the server:

private static List<BaseMessage> ProcessDirectoryDataMessage(DirectoryDataMessage message)
{
    AgentData agentData = ModelHelpers.FindAgent(message.AgentId);
    List<BaseMessage> msgs = new List<BaseMessage>();

    lock (agentData)
    {
        // Get any messages that are queued.
        //
        msgs.AddRange(agentData.MessagesToDeliver);
        agentData.MessagesToDeliver.Clear();

        // Set the last update time.
        //
        agentData.LastUpdate = message.TimeStamp;

        // Store the directory info data.
        //
        agentData.CurrentDirectory = message.CurrentFolder;
        agentData.Folders = message.Folders;
        agentData.Files = message.Files;

        // Save the agent.
        //
        ModelHelpers.SaveAgent(agentData);

    }

    return msgs;
}

private static List<BaseMessage> ProcessFileDataMessage(FileDataMessage message)
{
    AgentData agentData = ModelHelpers.FindAgent(message.AgentId);
    List<BaseMessage> msgs = new List<BaseMessage>();

    lock (agentData)
    {
        // Get any messages that are queued.
        //
        msgs.AddRange(agentData.MessagesToDeliver);
        agentData.MessagesToDeliver.Clear();

        // Set the last update time.
        //
        agentData.LastUpdate = message.TimeStamp;

        // Store the file content data.
        //
        agentData.FileDataMessage = message;

        // Save the agent.
        //
        ModelHelpers.SaveAgent(agentData);
    }

    return msgs;
}

Notice that each method returns a list of messages. This is the unprocessed message queue that needs to be sent to the agent. The first method processes the ‘DirectoryDataMessage’. The first thing that happens is the Agent’s data is retrieved from an in-memory repository. Next any messages in the queue are added to the returned message list. The last update time is then set. Finally, the data from the ‘DirectoryDataMessage’ is stored. This data is later used by the MVC application to generate browsing views. The second method processes the ‘FileDataMessage’ and follows the same pattern.

ASP.NET MVC

A new controller (and corresponding views) have been added to the Monitoring Station web application to support the file browsing features. I will introduce the action methods, views, and JavaScript in the order they appear in the above sequence diagrams.

Initial Browsing

First the user clicks the “Browse” menu button and the following ‘Index’ action method is called on the ‘Browse’ controller.

public ActionResult Index()
{
    HomeViewModel hvm = new HomeViewModel();
    return View(hvm);
}

This action method uses the same view model as used previously. The view is basic and is shown below in code and a rendered screen capture:

<h2>Folder Browser</h2>

<% foreach (var item in Model.Agents) { %>

    <span class="computer">
        <a href="<%= Url.Action("Browse", "Browse", new { agentId = item.AgentId }) %>">
            <img src="<%= Url.Content("~/Content/computer.png") %>" alt="computer" />
            <%= Html.Encode(item.ComputerName) %> - <%= Html.Encode(item.UserName) %>
        </a>
    </span>

<% } %>    

This view simply creates a list of Agents that have registered with server. Each item in the list is a link to start the file browsing for that remote computer. As can be seen above, the link routes to the “Browse” action method on the “Browse” controller and provides an ID for the Agent of interest. The following is the “Browse” action method.

public ActionResult Browse(Guid agentId)
{
    // Request the folder data from the computer.
    //
    AgentData agentData = ModelHelpers.FindAgent(agentId);

    lock (agentData)
    {
        // Don't send any more images...only poll for commands.
        //
        PollModeMessage pmm = new PollModeMessage(agentData, PollModeMessage.PollModeOptions.PollForCommands);
        agentData.MessagesToDeliver.Add(pmm);

        // Change the polling rate.
        //
        PollIntervalMessage pim = new PollIntervalMessage(agentData, 500);
        agentData.MessagesToDeliver.Add(pim);

        // Fetch the data for the current directory.
        //
        DirectoryCommandMessage dcm = new DirectoryCommandMessage(agentData)
                                          {
                                              Command = DirectoryCommandMessage.Action.FetchCurrentData
                                          };
        agentData.MessagesToDeliver.Add(dcm);

        ModelHelpers.SaveAgent(agentData);
    }

    return View(agentData);
}

The action method above gets an instance of the Agent of interest and then queues up three messages: ‘PollModeMessage’, ‘PollIntervalMessage’ and ‘DirectoryCommandMessage’ (DCM). These messages switch the poll mode to ‘PollForCommands’ (does not send screen shot data), set the polling rate to 500ms, and sends a ‘FetchCurrentData’ action to the Agent. The action method returns the following “Browse” view to the browser:

<h2>Browse <%=Html.Encode(Model.ComputerName) %> - <%= Html.Encode(Model.UserName) %></h2>

<input id="agentId" type="hidden" value="<%= Html.Encode(Model.AgentId) %>" />

<div id="curPath">loading....</div>

<div id="info">&nbsp;</div>

<a href="#" onclick="fetchFiles()">Request</a>&nbsp;<span id="package"></span>

The HTML in the view is simple and consists mostly of three placeholders (‘curPath’, ‘info’ & ‘package’) for dynamically generated HTML. The dynamically generated HTML is requested via AJAX calls. The following JavaScript is used to provide the AJAX requests that fetch the dynamically generated HTML:

$(document).ready(function() {
    setInterval("checkForUpdates()", 1000);
});

var id = $("#agentId").val();
var currentDirectory = "";
function checkForUpdates() {
    var url = "/Monitor/Browse/GetCurrentDirectory?agentId=" + id;
    $.get(url, null, function(data) {
        if (data != "" && data != currentDirectory) {
            currentDirectory = data;

            var url = "/Monitor/Browse/GetDirectoryNav?agentId=" + id;
            $("#curPath").load(url);

            // Need to fetch directory information
            //
            var url = "/Monitor/Browse/GetDirectoryInfo?agentId=" + id;
            $("#info").load(url);
        }
    });
    var url = "/Monitor/Browse/GetPackageList?agentId=" + id;
    $("#package").load(url);
};

Note that I am using jQuery to interact with the DOM elements & events. This is standard best practice to help with better cross-browser support. Let the jQuery team worry about the different nuances between the browser events while I focus on the JavaScript that makes my application work.

The script uses the jQuery ‘document ready’ event to start up an JavaScript interval that calls ‘checkForUpdates’ once every second. The ‘checkForUpdates’ function calls the ‘GetCurrentDirectory’ action method on the ‘Browse’ controller by using the jQuery ‘get’ function. The ‘GetCurrentDirectory’ action method simply returns the current directory as a string and is implemented by the following code:

public string GetCurrentDirectory(Guid agentId)
{
    if(Request.IsAjaxRequest())
    {
        // Request the folder data from the computer.
        //
        AgentData agentData = ModelHelpers.FindAgent(agentId);

        string currentDirectory;
        lock (agentData)
        {
            currentDirectory = agentData.CurrentDirectory;
        }

        return currentDirectory;
    }
    return "";
}

Remember that we have already requested the current directory information from the Agent by issuing a ‘DirectoryCommandMessage’. As soon as that message is processed by the Agent, the Server will have current directory information and this will be returned to the browser.

If the current directory has changed, then the JavaScript above will then use the jQuery ‘load’ function to dynamically inject HTML into the page. The HTML is requested using two action methods (‘GetDirectoryNav’ and ‘GetDirectoryInfo’) on the ‘Browse’ controller. These action methods are shown below:

public ActionResult GetDirectoryNav(Guid agentId)
{
    if (Request.IsAjaxRequest())
    {
        // Request the folder data from the computer.
        //
        AgentData agentData = ModelHelpers.FindAgent(agentId);

        return View("DirectoryNav", agentData);
    }
    return new EmptyResult();
}

public ActionResult GetDirectoryInfo(Guid agentId)
{
    if (Request.IsAjaxRequest())
    {
        // Request the folder data from the computer.
        //
        AgentData agentData = ModelHelpers.FindAgent(agentId);

        return View("DirectoryInfo", agentData);
    }
    return new EmptyResult();
}

Notice that both action methods simply inject the agent data into a view for rendering. Both of these are MVC partial views. The following is the partial view called ‘DirectoryNav’:

<%
string[] parts = Model.CurrentDirectory.Split(new char[]{'\', '/'});
string folder = "";
%>

<ul class="dir-nav">
    <%    foreach (string part in parts)
        {
            folder += part;
            if(!string.IsNullOrEmpty(part))
            {    %>
                <li><a href="#" onclick="javascript:changeDirectory('<%= folder %>');"><%= part %></a></li>
    <%        }
            folder += "/";
            %>

    <%    }    %>
</ul>

This bit of script / HTML builds the bread crumb navigational elements. These elements are simply an un-ordered list with each list element a link that calls a ‘changeDirectory’ JavaScript function with the new folder as the parameter. The HTML is easily styled using CSS and in its most basic form looks like the following:

Each folder in the above path is a link.

The following is the partial view called ‘DirectoryInfo’:

<ul class="directory">

   <%    foreach (string folder in Model.Folders)

        {
            string name = System.IO.Path.GetFileName(folder);    %>

	    <li class="folder">
                <input type="checkbox" name="<%= folder.Replace('\', '/') %>" />
                <a href="#" onclick="changeDirectory('<%= folder.Replace('\', '/') %>');">
                    <%= name %>
                </a>
            </li>

    <%    }

        foreach (string file in Model.Files)

        {
            string name = System.IO.Path.GetFileName(file);        %>

            <li class="file">
                <input type="checkbox" name="<%= file.Replace('\', '/') %>" />
                <span><%= Html.Encode(name) %></span>
            </li>


    <%    } %>

</ul>

The script / HTML above renders out another un-ordered list. This time the list contains the folders (first ‘foreach’ loop) and the files (second ‘foreach’ loop). First notice that the folders are rendered as links and once again wired up to the ‘changeDirectory’ JavaScript function. Each list item (both folders and files) are also rendered with a check box element. This allows folder & files to be selected for download. The partial view is easily styled up to render out as the following:

Finally the ‘checkForUpdates’ JavaScript function calls the ‘GetPackageList’ action method on the ‘Browse’ controller. This method is shown below:

public ActionResult GetPackageList(Guid agentId)
{
    if (Request.IsAjaxRequest())
    {
        // Request the folder data from the computer.
        //
        AgentData agentData = ModelHelpers.FindAgent(agentId);

        return View("PackageInfo", agentData);
    }
    return new EmptyResult();
}

This action method injects the agent data into the ‘PackageInfo’ partial view which adds a ‘Download’ link to the page if a zip file (contained in a ‘FileDataMessage’) is available. This is shown below:

<%    if (Model.FileDataMessage != null)
    {    %>

        <%= Html.ActionLink("Download", "DownloadPackage", new {agentId = Model.AgentId}) %>

<%  } %>

The code above includes all the functionality & implementation for the first interaction diagram.

Changing Directories

The above code wired all the folders (in the navigation header and the folder/file list) to the following ‘changeDirectory’ JavaScript function:

function changeDirectory(folder) {
    var url = "/Monitor/Browse/ChangeDirectory?agentId=" + id + "&folder=" + folder;
    $.get(url, null, null);
}

This function uses the jQuery ‘get’ function to implement an AJAX call to the ‘ChangeDirectory’ action method on the ‘Browse’ controller. This action method is shown here:

public void ChangeDirectory(Guid agentId, string folder)
{
    if (Request.IsAjaxRequest())
    {
        // Request the folder data from the computer.
        //
        AgentData agentData = ModelHelpers.FindAgent(agentId);

        lock (agentData)
        {
            // Fetch the data for the current directory.
            //
            DirectoryCommandMessage dcm = new DirectoryCommandMessage(agentData)
                                              {
                                                  Command = DirectoryCommandMessage.Action.ChangeDirectory,
                                                  CurrentDirectory = folder
                                              };
            agentData.MessagesToDeliver.Add(dcm);

            ModelHelpers.SaveAgent(agentData);
        }
    }
}

This method returns no data to the browser. It simply adds a ‘DirectoryCommandMessage’ to the message queue for the agent with a ‘ChangeDirectory’ action specified along with a new directory. As covered above, the Agent will then respond with a ‘DirectoryDataMessage’ containing updated directory info (files & folders) for the new current directory. This information will then be fetched with the above ‘checkForUpdates’ JavaScript call.

Download Files & Folders

Because of the disconnect between the Monitoring Station and the Agent, file download is a multi-step process for the user. First, the user selects the files / folders of interest and sends a request to the server by clicking the ‘Request’ link:

As shown in the HTML above for the ‘Browse’ view, the ‘Request’ link is wired up to a JavaScript function called ‘fetchFiles’.

function fetchFiles() {
    var vals = "";
    $("input[type=checkbox]:checked").each(function() {
        vals += $(this).attr("name") + ";";
    });
    var url = "/Monitor/Browse/RequestFiles?agentId=" + id + "&files=" + vals;

    $.post(url);
}

The ‘fetchFiles’ uses the jQuery ‘post’ function to initiate an AJAX call to the ‘RequestFiles’ action method on the ‘Browse’ controller. The ‘files’ parameter of the action method is set to a semi-colon separated list of files and folders that have been selected on the form. The ‘RequestFiles’ action method is shown below:

public void RequestFiles(Guid agentId, string files)
{
    if (Request.IsAjaxRequest())
    {
        string[] content = files.Split(';');

        // Request the folder data from the computer.
        //
        AgentData agentData = ModelHelpers.FindAgent(agentId);

        lock (agentData)
        {
            // Fetch the data for the current directory.
            //
            DirectoryCommandMessage dcm = new DirectoryCommandMessage(agentData)
                                              {
                                                  Command = DirectoryCommandMessage.Action.FetchContent,
                                                  ContentToFetch = content
                                              };
            agentData.MessagesToDeliver.Add(dcm);

            ModelHelpers.SaveAgent(agentData);
        }
    }
}

Again, this action method does not return any data to the browser. It queues up another ‘DirectoryCommandMessage’ for the agent with a ‘FetchContent’ action specified along with a list of content to fetch. The agent will then respond (see the previously covered code) with a ‘FileDataMessage’ that contains the bytes of an archive containing all the requested data.

Once the ‘FileDataMessage’ arrives at the server, the ‘checkForUpdates’ JavaScript method will add a ‘download’ link to the page as shown below:

The ‘Download’ link is connected up the ‘DownloadPackage’ action method on the ‘Browse’ controller. This is shown below:

public FileResult DownloadPackage(Guid agentId)
{
    // Request the folder data from the computer.
    //
    AgentData agentData = ModelHelpers.FindAgent(agentId);

    FileStreamResult fsr;

    lock (agentData)
    {
        MemoryStream memoryStream = new MemoryStream(agentData.FileDataMessage.FileData);
        fsr = new FileStreamResult(memoryStream, "application/zip")
                  {
                      FileDownloadName = agentData.FileDataMessage.FileName
                  };

        agentData.FileDataMessage = null;
    }

    return fsr;
}

This action method returns a ‘FileResult’ back to the browser. This is the MVC ActionResult that allows files to be sent to the browser. The above code places the zip file bytes (contained in the ‘FileDataMessage’) into a MemoryStream. This is passed into the FileStreamResult object as a parameter along with the MIME type. Setting the ‘FileDataMessage’ to null will make the ‘Download’ link disappear.

Summary

At this point I have an Agent that can remotely monitor desktop activity and allow file browsing for multiple remote machines regardless of firewall configurations. The only network topology requirement is that the Server must be accessible by both the Agent and the Monitoring Station. I have in mind a couple of more features for this application before releasing the source code. If you have any suggestions for features or comments on the code so far, please leave them below.