Multiple Remote Desktop Viewer (C# / WCF)

Update: Many people have asked me to post the source code for this project. The code posted below is in a ‘prototyping’ phase. So code may not follow best coding standards and will be in severe need of refactoring.

Download

I previously shared a remote desktop viewer implementation. That particular implementation used an adaptive algorithm to send back only the part of the screen that had changed. The remote component hosted a WCF service that could be polled at intervals by the viewer to refresh the image of the remote machine. This implementation was quick and left a bad taste in my mouth for two reasons:

1. It is inefficient to have the viewer poll the remote client. The client knows when it has new information. At times, the viewer was polling and the client would essentially return a message indicating nothing has changed.
2. In the real world, asking the remote machine to host the WCF service creates firewall issues. The remote client would need to ensure the viewer could “see” the WCF service.

A better design is to host the WCF service in the viewer and then have the clients push data only when necessary. This implementation only requires the viewer to expose the WCF service through the firewall. The following image shows the class diagram for components involved in implementing the WCF viewer service:

The WCF service is defined using he IViewerService contract. The three methods are:

• Ping – Used most for troubleshooting.
• PushCursorUpdate – Used to push new cursor content (the mouse has moved).
• PushScreenUpdate – Used to push new screen content (the screen has changed).

The ViewerSession class is used to encapsulate the content for each remote client that connects to the viewer. The ViewerService contains a Dictionary of these sessions. The data flow is as follows:

1. The remote client calls either “push” method via their WCF proxy.
2. The byte array received by the viewer is “unpacked” into image and other metadata.
3. The unpacked data is updated in that client’s ViewerSession.
4. The method UpdateScreenImage is called to merge the screen and cursor content.
5. The OnImageChange event is triggered allowing all listeners to update based upon the new data.

The current implementation of the remote client is a Console application. This is a great test bench implementation. This allows diagnostic data to be written to the screen. The Main method is shown below:

private static ScreenCapture capture = new ScreenCapture();
private static RemoteDesktopServer.ViewerProxy.ViewerServiceClient viewerProxy = new RemoteDesktopServer.ViewerProxy.ViewerServiceClient();
private static Thread _threadScreen = null;
private static Thread _threadCursor = null;
private static bool _stopping = false;
private static int _numByteFullScreen = 1;
static void Main(string[] args)
{
    _threadScreen = new Thread(new ThreadStart(ScreenThread));
    _threadScreen.Start();

    _threadCursor = new Thread(new ThreadStart(CursorThread));
    _threadCursor.Start();

    Console.ReadLine();

    _stopping = true;
    _threadCursor.Join();
    _threadScreen.Join();
}

The ScreenCapture instance provides the features to capture screen and cursor updates. The ViewerServiceClient instance provides the proxy to the WCF service. Updates are pushed to the WCF service via two threads. One is responsible for cursor updates and the other is responsible for screen updates. At times screen updates are fairly bulky. Having a separate cursor thread allows the cursor to be displayed without continuous updates and provides a much smoother viewer experience. The multiple threads do introduce the need for thread safety with respect to the WCF server resources. The following methods provide the push services:

private static void RefreshConnection()
{
    // Get a new proxy to the WCF service.
    //
    viewerProxy = new RemoteDesktopServer.ViewerProxy.ViewerServiceClient();

    // Force a full screen capture.
    //
    capture.Reset();
}

private static void ScreenThread()
{
    Rectangle bounds = Rectangle.Empty;
    // Run until we are asked to stop.
    //
    while (!_stopping)
    {
        try
        {
            // Capture a bitmap of the changed pixels.
            //
            Bitmap image = capture.Screen(ref bounds);
            if (_numByteFullScreen == 1)
            {
                // Initialize the screen size (used for performance metrics)
                //
                _numByteFullScreen = bounds.Width * bounds.Height * 4;
            }
            if (bounds != Rectangle.Empty && image != null)
            {
                // We have data...pack it and send it.
                //
                byte[] data = Utils.PackScreenCaptureData(image, bounds);
                if (data != null)
                {
                    // Thread safety on the proxy.
                    //
                    lock (viewerProxy)
                    {
                        try
                        {
                            // Push the data.
                            //
                            viewerProxy.PushScreenUpdate(data);

                            // Show performance metrics
                            //
                            double perc1 = 100.0 * 4.0 * image.Width * image.Height / _numByteFullScreen;
                            double perc2 = 100.0 * data.Length / _numByteFullScreen;
                            Console.WriteLine(DateTime.Now.ToString() + ": Screen - {0:0.0} percent, {1:0.0} percent with compression", perc1, perc2);
                        }
                        catch (Exception ex)
                        {
                            // Push exception...log it
                            //
                            Console.WriteLine("*******************");
                            Console.WriteLine(ex.ToString());
                            Console.WriteLine("No connection...trying again in 5 seconds");
                            RefreshConnection();
                            Thread.Sleep(5000);
                        }
                    }
                }
                else
                {
                    // Show performance metrics.
                    //
                    Console.WriteLine(DateTime.Now.ToString() + ": Screen - no data bytes");
                }
            }
            else
            {
                // Show performance metrics.
                //
                Console.WriteLine(DateTime.Now.ToString() + ": Screen - no new image data");
            }
        }
        catch (Exception ex)
        {
            // Unhandled exception...log it.
            //
            Console.WriteLine("Unhandled: ************");
            Console.WriteLine(ex.ToString());
        }
    }
}

private static void CursorThread()
{
    // Run until we are asked to stop.
    //
    while (!_stopping)
    {
        try
        {
            // Get an update for the cursor.
            //
            int cursorX = 0;
            int cursorY = 0;
            Bitmap image = capture.Cursor(ref cursorX, ref cursorY);
            if (image != null)
            {
                // We have valid data...pack and push it.
                //
                byte[] data = Utils.PackCursorCaptureData(image, cursorX, cursorY);
                if (data != null)
                {
                    try
                    {
                        // Push the data.
                        //
                        viewerProxy.PushCursorUpdate(data);

                        // Show performance metrics.
                        //
                        double perc1 = 100.0 * 4.0 * image.Width * image.Height / _numByteFullScreen;
                        double perc2 = 100.0 * data.Length / _numByteFullScreen;
                        Console.WriteLine(DateTime.Now.ToString() + ": Cursor - {0:0.0} percent, {1:0.0} percent with compression", perc1, perc2);
                    }
                    catch (Exception ex)
                    {
                        // Push exception...log it.
                        //
                        Thread.Sleep(1000);
                    }
                }
            }
        }
        catch(Exception ex)
        {
            // Unhandled exception...log it.
            //
            Console.WriteLine("Unhandled: ************");
            Console.WriteLine(ex.ToString());
        }

        // Throttle this thread a bit.
        //
        Thread.Sleep(10);
    }
}

The presentation layer of the viewer simply adds a listener to the OnImageChange event that updates the presentation layer with the new data when signaled. The current implementation is a WinForms application. The callback registered to OnImageChange is shown below:

void svc_OnImageChange(Image display, string remoteIpAddress)
{
    lock (display)
    {
        UpdateTabs(display, remoteIpAddress);
    }
}

private delegate void UpdateTabsDelegate(Image display, string remoteIpAddress);
private void UpdateTabs(Image display, string remoteIpAddress)
{
    if (tabControl1.InvokeRequired)
    {
        Invoke(new UpdateTabsDelegate(UpdateTabs), new object[] { display, remoteIpAddress });
    }
    else
    {
        if (!_remoteViews.ContainsKey(remoteIpAddress))
        {
            // Add a new tab
            //
            TabPage page = new TabPage(remoteIpAddress);
            tabControl1.TabPages.Add(page);
        }

        // Add this to or update the dictionary
        //
        _remoteViews[remoteIpAddress] = display;

        // Update the viewer
        //
        pictureBox1.BackgroundImage = _remoteViews[tabControl1.SelectedTab.Text];
    }
}

The clients (multiple) will be calling the service methods asynchronously on a thread different than the UI thread. This requires the use of the InvokeRequired property and the Invoke method. When a new client contacts the viewer, a new tab UI element is created and the display image is added to the dictionary for look up. The background image for the picture box is set to the image for the currently selected tab. The following styles are set in the form’s load method to force the form to double buffer repaints to reduce flickering:

SetStyle(ControlStyles.UserPaint, true);
SetStyle(ControlStyles.AllPaintingInWmPaint, true);
SetStyle(ControlStyles.DoubleBuffer, true); 

The following is a video of the implementation showing multiple remote client connections to the viewer (view in HD)

In the above video, the WCF service is configured to bind to my external IP address. I was attempting to get some real-world connection conditions into the video. Unfortunately, I think my firewall / NAT resolved the address and the packets never reached the internet. Regardless, the video demonstrates 3-4 screen captures per second from both the clients. The default JPEG compression is doing a great job at minimizing the number of bytes transferred. The image quality is a little degraded when compared to normal remote desktop experience.

Overall, I am pleased with the performance. In the next couple of blogs I hope to add some ability to interact with the remote desktop (move mouse, click, type, …).