Simplify asynchronous programming with C# 5 async/await

One of key benefit of asynchronous programming is scalable & responsive applications. On server by eliminating threads blocked for network/disk I/O to complete, you can write scalable application using fewer server resources (threads). Similarly on the client side asynchronous programming helps in designing responsive UI without blocking the UI thread.

Traditionally designing asynchronous programming is hard that involves learning low level operating systems facilities such as POSIX select, epoll on Linux and IO completion ports on Windows. Even though newer platforms (Java and .NET) hide these complexities with nice object oriented API it is still hard to develop asynchronous applications as it involves dealing with callback and state machines.

.NET applications use APM or EAP to write asynchronous applications. Throughout .NET base class library, many classes support using the APM by implementing BeginXXX and EndXXX versions of functions. These methods allow applications to perform I/O operations asynchronously. For example, the FileStream class in System.IO namespace has a BeginXXX and EndReadXXX methods that reads data from a stream asynchronously.

Even with APM, asynchronous application development is still not as easy as synchronous applications because of callback and the state involved. C# 5 introduces new language features to simplify asynchronous programming. It introduces pattern where user can write asynchronous programs with sequential control flow by using async/await keywords. C#  implements this feature using Task Parallel library (TPL) and some compiler magic. If you interested to know more about how these feature implemented see the resource section.

Here is the simple TCP echo server that demonstrates C# async/await pattern .This program handle multiple client connections without creating threads explicitly. If you read the program, it looks like synchronous application with sequential control flow and easy to understand and maintain. Under the cover C# compiler is transforming code to use the asynchronous API with the help of Task parallel library. This transformation is similar to how C# implements yield keyword

 

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Runtime.CompilerServices;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

namespace AsyncEchoServer
{
    public class AsyncEchoServer
    {
        private int _listeningPort;
        public AsyncEchoServer( int port)
        {
            _listeningPort = port;
        }
        ///
<summary>
        /// Start listening for connection
        /// </summary>
        public async void Start()
        {
            IPAddress ipAddre = IPAddress.Loopback;
            TcpListener listener = new TcpListener(ipAddre, _listeningPort);
            listener.Start();
            LogMessage("Server is running");
            LogMessage("Listening on port " + _listeningPort);

            while (true)
            {
                LogMessage("Waiting for connections...");
                try
                {
                    var tcpClient = await listener.AcceptTcpClientAsync();
                     HandleConnectionAsync(tcpClient);
                }
                catch (Exception exp)
                {
                    LogMessage(exp.ToString());
                }

            }

        }
        ///
<summary>
        /// Process Individual client
        /// </summary>
        ///
        ///
        private async void HandleConnectionAsync(TcpClient tcpClient)
        {
            string clientInfo = tcpClient.Client.RemoteEndPoint.ToString();
            LogMessage(string.Format("Got connection request from {0}", clientInfo));
            try
            {
                using (var networkStream = tcpClient.GetStream())
                using (var reader = new StreamReader(networkStream))
                using (var writer = new StreamWriter(networkStream))
                {
                    writer.AutoFlush = true;
                    while (true)
                    {
                        var dataFromServer = await reader.ReadLineAsync();
                        if (string.IsNullOrEmpty(dataFromServer))
                        {
                            break;
                        }
                        LogMessage(dataFromServer);
                        await writer.WriteLineAsync("FromServer-" + dataFromServer);
                    }
                }
            }
            catch (Exception exp)
            {
                LogMessage(exp.Message);
            }
            finally
            {
                LogMessage(string.Format("Closing the client connection - {0}",
                            clientInfo));
                tcpClient.Close();
            }

        }
        private void LogMessage(string message,
                                [CallerMemberName]string callername = "")
        {
            System.Console.WriteLine("[{0}] - Thread-{1}- {2}",
                    callername, Thread.CurrentThread.ManagedThreadId, message);
        }
    }
    class Program
    {
        static void Main(string[] args)
        {
            AsyncEchoServer async = new AsyncEchoServer(51510);
            async.Start();
            Console.ReadLine();
        }
    }
}

Echo Client Sample

class Program
    {
        static void Main(string[] args)
        {
            StartClient(Convert.ToInt32(args[0]));
            Console.ReadLine();
        }

        private static async void StartClient(int port)
        {
            TcpClient client = new TcpClient();
            await client.ConnectAsync(IPAddress.Loopback, port);
            LogMessage("Connected to Server");
            using (var networkStream = client.GetStream())
            using (var writer = new StreamWriter(networkStream))
            using (var reader = new StreamReader(networkStream))
            {
                writer.AutoFlush = true;
                for(int i = 0; i < 10;i++)
                {
                    await writer.WriteLineAsync(DateTime.Now.ToLongDateString());
                    var dataFromServer = await reader.ReadLineAsync();
                    if (!string.IsNullOrEmpty(dataFromServer))
                    {
                        LogMessage(dataFromServer);
                    }
                    
                }
            }
            if (client != null)
            {
                client.Close();
            }
           
        }
        private static void LogMessage(string message, 
                [CallerMemberName]string callername = "")
        {
            System.Console.WriteLine("{0} - Thread-{1}- {2}", 
                callername, Thread.CurrentThread.ManagedThreadId, message);
        }

    }

Resource

1. Visual Studio async home page.

2. DNT TV Screencast which explains async feature in .NET .

3. Jon Skeet multi part screencast on async/await. Part 1 , Part 2

4. Joseph Albahari talk at Tech·Ed Australia 2011

Advertisements

Create Bootable VHD from the Windows 7/Windows 8/Windows 2012 ISO

In my previous blog post I explained how to use windows 7 Boot to VHD feature to dual boot computer without installing OS in different partition.

To use this feature you need to have VHD file with OS installed. This blog post points to resource that allows you to create bootable VHD from Windows 7/Server 2008 R2 and Windows 8/Server 2012 setup media.

Technology used to create VHD is already built into Windows 7 and higher, in the form of Virtual Hard Disk & Windows Imaging Interface API, these API are exported from virtdisk.dll and Wimgapi.dll.

Following blog post explains how to use these built-in API in PowerShell script

http://www.pitorque.de/MisterGoodcat/post/Installing-Windows-8-Developer-Preview-as-bootable-VHD.aspx

Note: PowerShell script referred in the blog post is no longer available , use the alternative link here

Resources

  1. Virtual Hard disk API reference.
  2. Windows Imaging API reference.
  3. Windows 8 Enterprise 90-day evaluation.
  4. New Windows 8 specific  PowerShell script for creating bootable VHD.

Compile and Run .NET program with just .NET framework on your system.

Compile and Run .NET program with just .NET framework on your system.

Did you know that every .NET framework installation also installs VB & C# compiler.

Using these compilers you can compile and run C# or VB.NET program on machine with just .NET framework installed. This can be useful on system where installing full visual studio is not an option.

Here is how to do it

If .NET framework installed on you system, C:\Windows\Microsoft.NET\Framework directory  should list all the versions. Here is what is on my machine.

image

Once you find required .NET version, directory content should list csc.exe and vbs.exe compiler executable. As shown below.

image

Now use C# or VB compiler to compile the program as follows. For example use .NET 2.0 compiler to generate .exe c:\Windows\Microsoft.NET\Framework\v2.0.50727\csc.exe Program.cs

image

Identifying process using TCP port in .NET

Currently I am working on a application  which uses third-party component for remote diagnostics.  This third party component internally uses 8081 tcp port for communication between the process and port was not configurable , which forced our application installer to make sure that no other process in the system is using port 8081 during install. We had to warn user during application install if any process is using required tcp port.

In .Net you can use System.Net.NetworkInformation.IPGlobalProperties to find out whether tcp port in use. But this API does not tell you which process using the port. We wanted to show user not only tcp port is in use , also provide information  about process , such as name , id and path .  We also had to make sure that it works from Windows XP to Windows 7 .

From MSDN documentation, I found that only to get the process id of the process using the tcp port is to use Win32 API . This means I had to use platform invoke(pinvoke) and carefully declare lot of Win32 structure in C#.

Other alternative is to use the netstat command , that provides the same details with appropriate command line switches. I used this technique various times ,when I was working on Linux systems. I decided to do the same here.

I wrote simple helper class which uses netstat and WMI to get the details about the process using the tcp port.
Here is how you use the sample

int port = 8081;
TcpHelperUtil tcpHelper = new TcpHelperUtil();
var details = tcpHelper.GetPortDetails(port);
if (details.Item1)
{
    Console.WriteLine("Port {0} in Use",port);
    Console.WriteLine(details.Item2.ToString());
}else
{
    Console.WriteLine("Port {0} is free ",port);
}

Here is the source code.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Management;

namespace ConsoleApp2
{

    internal class PortDetail
    {
        public int Port { get; set; }
        public int ProcessID { get; set; }
        public string ProcessName { get; set; }
        public string Path { get; set; }
        public override string ToString()
        {
            return string.Format(@" Process Name: {0} ,Process ID: {1} ,
                                    Port: {2} ,\nPath : {3}", ProcessName,
                                    ProcessID, Port, Path);
        }

    }
    /// <summary>
    /// Usage:
    /// int port = 8081
    /// TcpHelperUtil tcpHelper = new TcpHelperUtil();
    /// var details = tcpHelper.GetPortDetails(port);
    /// if (details.Item1)
    /// {
    ///     Console.WriteLine("Port {0} in Use",port);
    ///     Console.WriteLine(details.Item2.ToString());
    /// }else
    /// {
    ///     Console.WriteLine("Port {0} is free ",port);
    /// }
    ///
    /// </summary>
    class TcpHelperUtil
    {
        private const short MINIMUM_TOKEN_IN_A_LINE = 5;
        private const string COMMAND_EXE = "cmd";

        public TcpHelperUtil()
        {

        }

        public Tuple<bool, PortDetail> GetPortDetails(int port)
        {
            PortDetail PortDetail = new PortDetail();
            Tuple<bool, PortDetail> result = Tuple.Create(false, PortDetail);

            // execute netstat command for the given port
            string commandArgument = string.Format("/c netstat -an -o -p tcp|findstr \":{0}.*LISTENING\"", port);

            string commandOut = ExecuteCommandAndCaptureOutput(COMMAND_EXE, commandArgument);
            if (string.IsNullOrEmpty(commandOut))
            {
                // port is not in use
                return result;
            }

            var stringTokens = commandOut.Split(default(Char[]), StringSplitOptions.RemoveEmptyEntries);
            if (stringTokens.Length < MINIMUM_TOKEN_IN_A_LINE)
            {
                return result;
            }

            // split host:port
            var hostPortTokens = stringTokens[1].Split(new char[] { ':' });
            if (hostPortTokens.Length < 2)
            {
                return result;
            }

            int portFromHostPortToken = 0;
            if (!int.TryParse(hostPortTokens[1], out portFromHostPortToken))
            {
                return result;
            }
            if (portFromHostPortToken != port)
            {
                return result;
            }

            PortDetail.Port = port;
            PortDetail.ProcessID = int.Parse(stringTokens[4].Trim());
            Tuple<string, string> processNameAndPath = null;
            try
            {
                processNameAndPath = GetProcessNameAndCommandLineArgs(PortDetail.ProcessID);
                PortDetail.ProcessName = processNameAndPath.Item1;
                PortDetail.Path = processNameAndPath.Item2;
                result = Tuple.Create(true, PortDetail);
            }
            catch (Exception exp)
            {
                Console.WriteLine(exp.ToString());

            }

            return result;

        }
        /// <summary>
        /// Using WMI API to get process name and path instead of
        /// Process.GetProcessById, because if calling process ins
        /// 32 bit and given process id is 64 bit, caller will not be able to
        /// get the process name
        /// </summary>
        /// <param name="processID"></param>
        /// <returns></returns>
        private Tuple<string, string> GetProcessNameAndCommandLineArgs(int processID)
        {
            Tuple<string, string> result = Tuple.Create(string.Empty, string.Empty);
            string query = string.Format("Select Name,ExecutablePath from Win32_Process WHERE ProcessId='{0}'", processID);
            try
            {
                ObjectQuery wql = new ObjectQuery(query);
                ManagementObjectSearcher searcher = new ManagementObjectSearcher(wql);
                ManagementObjectCollection results = searcher.Get();

                // interested in first result.
                foreach (ManagementObject item in results)
                {
                    result = Tuple.Create<string, string>(Convert.ToString(item["Name"]),
                    Convert.ToString(item["ExecutablePath"]));
                    break;

                }
            }
            catch (Exception)
            {

                throw;
            }

            return result;

        }

        /// <summary>
        /// Execute the given command and captures the output
        /// </summary>
        /// <param name="commandName"></param>
        /// <param name="arguments"></param>
        /// <returns></returns>
        private string ExecuteCommandAndCaptureOutput(string commandName, string arguments)
        {
            string commandOut = string.Empty;
            Process process = new Process();
            process.StartInfo.FileName = commandName;
            process.StartInfo.Arguments = arguments;
            process.StartInfo.UseShellExecute = false;
            process.StartInfo.CreateNoWindow = true;
            process.StartInfo.RedirectStandardOutput = true;
            process.StartInfo.RedirectStandardError = true;
            process.Start();

            commandOut = process.StandardOutput.ReadToEnd();
            string errors = process.StandardError.ReadToEnd();
            try
            {
                process.WaitForExit(TimeSpan.FromSeconds(2).Milliseconds);
            }
            catch (Exception exp)
            {

                Console.WriteLine(exp.ToString());
            }
            return commandOut;
        }
    }
}

BTrace : Troubleshooting tool for Java applications

BTrace allows users to dynamically instrument running Java application using special Java annotations. Annotations act as directive to indicate where tracing code should be inserted in the target application.
For example you can insert code (tracing/debug) print all arguments to a method or print information whenever particular method is called along with caller information using stack trace.
All this can be done at runtime without requiring any source code changes to target application or restarting target application.
Here is how it works

At high level you write Java source file using the framework provided by the BTrace which allows you to print arguments ,trace method calls. When user executes external BTrace annotated Java code using BTrace client, it creates and invokes an agent in the target JVM ( through the Attach API ) and then it uses client to communicate with that agent to perform instrumentation.

This technique  is similar to aspect oriented programming  using aspectj where you write aspects  in external file and use aspectj compiler to weave byte code.Advantage of using BTrace is , you are changing byte code in running application.

How to Use BTrace

BTrace provides command line tool and plugin to Visual VM tool ( Visual VM is part of JDK release ).

In this blog post I will show you how to install and use BTrace plugin in Visual VM.

Visual VM is part of JDK since 6.0. You can also download from java.net.

Launch Visual VM and select “Tools\Plugins” option in Menu bar.

In the plugins dialog , select “Available Plugins” tab. One of the option listed is “BTrace workbench”. If it is not listed enter , “BTrace” in search box.

Note:  To get the available list of plugins ,make sure that you are connected to internet and proxy configuration is proper.

image

In the plugin dialog select “BTrace workbench” and install. Restart the Visual VM application so that newly installed plugin is loaded.

Once you restart the Visual VM , it will show all the running java application on your machine. In my case I am running “Notepad” demo application which ships with JDK demos ( JDK_HOME\demos\jfc\Notepad ).

Select the Notepad application , right click and select “Trace application…” from the menu, as show below.

image

Once “Trace application” option is selected following dialog pops up to allow user to enter java code trace “Notepad” application.

image

Now lets trace all the files opened for reading/writing by “Notepad” application. Remove the existing code from the window and copy/paste following code.

This is the sample code from code from BTrace library , which allows to trace file open for read/write.

/* BTrace Script Template */
import com.sun.btrace.annotations.*;
import static com.sun.btrace.BTraceUtils.*;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;

/**
 * This sample prints all files opened for read/write
 * by a Java process. Note that if you pass FileDescriptor
 * to File{Input/Output}Stream or File{Reader/Writer}, 
 * that is not tracked by this script.
 */
@BTrace public class FileTracker {
    @TLS private static String name;

    @OnMethod(
        clazz="java.io.FileInputStream",
        method="<init>"
    )
    public static void onNewFileInputStream(@Self FileInputStream self, 
                                            File f) {
        name = Strings.str(f);
    }

    @OnMethod(
        clazz="java.io.FileInputStream",
        method="<init>",
        type="void (java.io.File)",
        location=@Location(Kind.RETURN)
    )
    public static void onNewFileInputStreamReturn() {
        if (name != null) {
            println(Strings.strcat("opened for read ", name));
            name = null;
        }
    }

    @OnMethod(
        clazz="java.io.FileOutputStream",
        method="<init>"
    )
    public static void onNewFileOutputStream(@Self FileOutputStream self, 
                                              File f, boolean b) {
        name = str(f);
    }

    @OnMethod(
        clazz="java.io.FileOutputStream",
        method="<init>",
        type="void (java.io.File, boolean)",
        location=@Location(Kind.RETURN)
    )
    public static void OnNewFileOutputStreamReturn() {
        if (name != null) {
            println(Strings.strcat("opened for write ", name));
            name = null;
        }
    }
}

Select “Start” option . Now try to open any file in Notepad application or create new file , you will see trace message printed in the output windows shown below.

image

More samples available in BTrace User guide

Using Command Line version

BTrace also ships with command line tool which allows you debug/trace target application. From the command line you can use btrace client tool to trace Java application

btrace <target-java-pid> FileTracker.java

BTrace ships with number of sample application that shows various BTrace features. Download the sample application from the btrace website

Resouces

1. BTrace website

2. BTrace user guide ( user guide as number samples which demonstrate capbaility of BTrace ).

3. BTrace developers guide

4. BTrace wiki.

Monitor process startup/shutdown using WMI & PowerShell

Recently I was working on a project, where I needed to monitor process for startup/shutdown events,  after searching internet  found that it can be easily done using WMI and PowerShell .

WMI (Windows Management Instrumentation) is a technology built in to Windows since Windows 2000, that provides standard interface to manage windows systems. For example it can used to find software products installed in single or multiple machines in network or to verify necessary OS service pack is installed on all machines in the network.

PowerShell provides easy way of accessing WMI functionality using simple scripting language.

Here is how you can monitor for process startup using PowerShell & WMI events

1. Launch PowerShell and enter following command. This command will register for process started event and prints statement in console whenever notepad.exe is executed.

Register-WMIEvent -query “SELECT * FROM Win32_ProcessStartTrace WHERE ProcessName=’notepad.exe'” -SourceIdentifier “testevent” -action { $e = $Event.SourceEventArgs.NewEvent

Write-Host $e.ProcessName,” started ” }

2. Now launch notepad and observer that console has message confirming the same.

3. Once you are done, unregister the event by entering following command

Unregister-Event testevent

4. You can also see list of event subscribers with “Get-EventSubscriber” command.

Some more examples of using WMI & PowerShell

1.Monitor process stop events

Register-WMIEvent -query “SELECT * FROM Win32_ProcessStopTrace WHERE ProcessName=’notepad.exe'” -SourceIdentifier “testevent” -action { $e = $Event.SourceEventArgs.NewEvent

Write-Host $e.ProcessName,” stopped ” }

2. Monitor Windows Service stop/start status. Replace “TargetInstance.Name” value with your service name

Register-WMIEvent -query “Select * From __InstanceOperationEvent within 1 Where TargetInstance ISA ‘Win32_Service’ and TargetInstance.Name=’Fax'” -sourceIdentifier “MyServMonitor” -action { Write-host “Service Name :”,$EventArgs.NewEvent.TargetInstance.Name ,” Service State :”, $EventArgs.NewEvent.TargetInstance.State }

image

You can also do the same event monitoring using WMI & C#


using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Management;

namespace WmiEventTest
{
    class ServiceStatusMonitor
    {
        static void Main(string[] args)
        {

            if (args.Length < 1)
            {
                Console.WriteLine("Usage: ServiceStatusMonitor  ");
                Environment.Exit(0);
            }
            string WMI_EVENT_QUERY = @"SELECT * FROM __InstanceModificationEvent
                WITHIN 1 WHERE TargetInstance ISA 'Win32_Service'";

            string WMI_EVENT_QUERY_WITH_SERVICE_NAME = WMI_EVENT_QUERY
                    + " and TargetInstance.Name = '{0}'";
            WqlEventQuery serviceModificationEvent =
                    new WqlEventQuery(string.Format(WMI_EVENT_QUERY_WITH_SERVICE_NAME, args[0]));
            ManagementEventWatcher eventWatcher =
                    new ManagementEventWatcher(serviceModificationEvent);
            eventWatcher.EventArrived +=
                    new EventArrivedEventHandler(Watcher_EventArrived);
            Console.WriteLine("Waiting for service status change events ...");
            eventWatcher.Start();
            Console.ReadLine();
        }

        static void Watcher_EventArrived(object sender, EventArrivedEventArgs e)
        {
            string eventType = e.NewEvent.ClassPath.ClassName;

            switch (eventType)
            {
                case "__InstanceCreationEvent":

                    Console.BackgroundColor = ConsoleColor.Blue;
                    Console.WriteLine("'{0}' Service created ....",
                            Environment.GetCommandLineArgs()[1]);
                    Console.ResetColor();
                    break;
                case "__InstanceDeletionEvent":

                    Console.BackgroundColor = ConsoleColor.Green;
                    Console.WriteLine("'{0}' Service deleted ....",
                        Environment.GetCommandLineArgs()[1]);
                    Console.ResetColor();
                    break;

                case "__InstanceModificationEvent":

                    Console.BackgroundColor = ConsoleColor.Blue;
                    ManagementBaseObject obj = (ManagementBaseObject)e.NewEvent["TargetInstance"];
                    Console.WriteLine("'{0}' Service Modified ( {1} )",
                        Environment.GetCommandLineArgs()[1], obj["State"]);
                    Console.ResetColor();
                    break;
            }

        }
    }
}

Resources

1. Introduction to WMI

2. Receiving WMI Events

3. PowerShell for event monitoring.

Simple way to try new Windows OS using Boot to VHD

Try out new windows 8 release preview  or  windows server 2012 operating systems without install using Boot to VHD.

Boot to VHD is feature in Windows 7 (higher) system , that allows to boot windows operating system from the VHD file ( virtual hard disk file ). Basically Windows 7 systems boot loader understands , how to boot from VHD file.

Typical way of using boot to VHD feature involves , using command line tools such as diskpart and bcdedit which is cumbersome. Recently I was watching videos from TechEd North America 2012 , in one of the session presenter showed simple way to do the same.

Advantage of using boot VHD  over running in virtual machine ( Virtual Box or VMWare workstation) is  , OS present in VHD will get full access to actual physical hardware instead of virtualized hardware. This will allow OS perform better compared to virtual machine.

Here are the steps,

1. Download Windows Server 2012 VHD file from Microsoft site.   If you want Windows 8 release preview  , you need to create one. I will explain how in the next blog post.

2. Once you have VHD file , use run disk management tool “diskmgmt.msc”.

3. Select “Attach VHD” option from the “Action” Menu.

image

4. “Select/Enter” path to VHD location

image

5. Windows will mount the VHD and assigns drive letter. Note down the drive lettter. In my case it is mount as “G” drive. Now from the explorer you can see the “G” drive.

image

6. Launch command prompt in “Administrator” mode. Type following command to make entry in the windows boot loader

c:\Windows\system32\bcdboot G:\Windows

Note: Replace “G” with the drive letter depending on you system where VHD is mounted.

image

7. Launch “msconfig.exe” from the “Administrator” command prompt to verify that new boot entry is added in the boot loader.

image

You can change the boot order from here.

Note: Replace “G” with the drive letter depending on you system where VHD is mounted

8. If you reboot the system, now you will have option to boot into new OS.

9. Using same steps you can try out new Windows 8 client OS , if you have VHD file.

Resources

1. Windows 2012 RC VHD file

2. Creating Windows 8 Release preview VHD from ISO.

3. Watch “The Busy Developer’s Guide to Virtualization”  talk on Channel9