.NET HttpClient extensibility: Support for UnixDomain Sockets & Named Pipes

.NET HttpClient is main interface for sending HTTP requests and receiving HTTP responses from a resource identified by a URI. Generally this API supported TCP stack. In .NET 5 , library is extended to support other transports such as Unix Domain Sockets & NamedPipes because many web server on Unix supported HTTP server implementations using Unix Domain Sockets . Windows also added Unix domain socket support in Windows 10 ( for more details see here)

ASP.NET Core web server (Kestrel) also added support for UNIX domain sockets and named pipes( from .NET 8 on-wards).

Alternative transport options is introduced in HttpClient SocketsHttpHandler. With this it is possible to connect to server with Unix Domain Socket or Named Pipes.

HttpClient with Unix Domain Socket

SocketsHttpHandler socketsHttpHandler = new SocketsHttpHandler();
// Custom connection callback that connects to Unixdomain Socket
socketsHttpHandler.ConnectCallback = async (sockHttpConnContext, ctxToken) =>
{
    Uri dockerEngineUri = new Uri("unix:///var/run/docker.sock");
    var socket = new Socket(AddressFamily.Unix, SocketType.Stream, ProtocolType.IP);

    var endpoint = new UnixDomainSocketEndPoint(dockerEngineUri.AbsolutePath);
    await socket.ConnectAsync(endpoint, ctxToken);
    return new NetworkStream(socket);
};

// create HttpClient with SocketsHttpHandler
var httpClient  = new HttpClient(socketsHttpHandler);
// make Http Request .

HttpClient with Named Pipe

SocketsHttpHandler socketsHttpHandler = new SocketsHttpHandler();
// Custom connection callback that connects to NamedPiper server
socketsHttpHandler.ConnectCallback = async (sockHttpConnContext, ctxToken) =>
{
    Uri dockerEngineUri = new Uri("npipe://./pipe/docker_engine");
    NamedPipeClientStream pipeClientStream = new NamedPipeClientStream(dockerEngineUri.Host,
                                            dockerEngineUri.Segments[2],
                                            PipeDirection.InOut, PipeOptions.Asynchronous);
    await pipeClientStream.ConnectAsync(ctxToken);
    return pipeClientStream;
};

// create HttpClient with SocketsHttpHandler
var httpClient  = new HttpClient(socketsHttpHandler);
// make Http Request .
        

Complete demonstration sample that uses HttpClient connecting to Docker daemon serving REST API via NamedPipe on Windows & Unix Domain socket on Linux.


using System.IO.Pipes;
using System.Net.Http;
using System.Net.Http.Json;
using System.Net.Sockets;
using System.Runtime.InteropServices;

HttpClient client = CreateHttpClientConnectionToDockerEngine();
String dockerUrl = "http://localhost/v1.41/containers/json";
var containers = client.GetFromJsonAsync<List<Container>>(dockerUrl).GetAwaiter().GetResult();
Console.WriteLine("Container List:...");
foreach (var item in containers)
{
    Console.WriteLine(item);
}


// Create HttpClient to Docker Engine using NamedPipe & UnixDomain
HttpClient CreateHttpClientConnectionToDockerEngine()
{
    SocketsHttpHandler socketsHttpHandler =
        RuntimeInformation.IsOSPlatform(OSPlatform.Windows) switch
        {
            true => GetSocketHandlerForNamedPipe(),
            false => GetSocketHandlerForUnixSocket(),
        };
    return new HttpClient(socketsHttpHandler);

    // Local function to create Handler using NamedPipe
    static SocketsHttpHandler GetSocketHandlerForNamedPipe()
    {
        Console.WriteLine("Connecting to Docker Engine using Named Pipe:");
        SocketsHttpHandler socketsHttpHandler = new SocketsHttpHandler();
        // Custom connection callback that connects to NamedPiper server
        socketsHttpHandler.ConnectCallback = async (sockHttpConnContext, ctxToken) =>
        {
            Uri dockerEngineUri = new Uri("npipe://./pipe/docker_engine");
            NamedPipeClientStream pipeClientStream = new NamedPipeClientStream(dockerEngineUri.Host,
                                                    dockerEngineUri.Segments[2],
                                                    PipeDirection.InOut, PipeOptions.Asynchronous);
            await pipeClientStream.ConnectAsync(ctxToken);
            return pipeClientStream;
        };
        return socketsHttpHandler;
    }
    // Local function to create Handler using Unix Socket
    static SocketsHttpHandler GetSocketHandlerForUnixSocket()
    {
        Console.WriteLine("Connecting to Docker Engine using Unix Domain Socket:");
        SocketsHttpHandler socketsHttpHandler = new SocketsHttpHandler();
        // Custom connection callback that connects to Unixdomain Socket
        socketsHttpHandler.ConnectCallback = async (sockHttpConnContext, ctxToken) =>
        {
            Uri dockerEngineUri = new Uri("unix:///var/run/docker.sock");
            var socket = new Socket(AddressFamily.Unix, SocketType.Stream, ProtocolType.IP);

            var endpoint = new UnixDomainSocketEndPoint(dockerEngineUri.AbsolutePath);
            await socket.ConnectAsync(endpoint, ctxToken);
            return new NetworkStream(socket);
        };
        return socketsHttpHandler;
    }
}

/// <summary>
/// Record class to hold the container information
/// </summary>
/// <param name="Names"></param>
/// <param name="Image"></param>
/// <param name="ImageID"></param>
/// <param name="Command"></param>
/// <param name="State"></param>
/// <param name="Status"></param>
/// <param name="Created"></param>
public record Container(List<String> Names, String Image, String ImageID,
                        String Command, String State, String Status, 
                        int Created);

Conclusion:

.NET HttpClient extensibility allows connecting to HTTP server implemented using non TCP transport such as Unix Domain Socket & Named Pipes.

Reference:

SocketsHttpHandler Extension Points

Docker Engine REST API Reference

Accessing Docker API from .NET Core

Recently I had to access container stats ( CPU & Memory) for the container apps on Linux system with limited access ( only ssh access via multiple intermediate jump servers). We could not install cAdvisor or other UI tools. I tried capturing container stats by running Docker stats command every 5 seconds but the output of the Docker stats less than ideal for analysis like plotting memory/cpu graphs.

So I decided to write simple tool using Docker.DotNet C# SDK . Requirement was to write a self contained utility that can run on Linux & log container stats entries in comma separate format(CSV) for easier analysis using excel.

This tool uses Docker.DotNet , a C# SDK that uses Docker REST API to interact with Docker Daemon and .NET self contained publish profile that produces a platform-specific executable.

Complete source code is available on my GitHub repo ,

Here is the sample run & the output

Output format is has follows ContainerName,CpuPercent,MemUsageInKB,MaxAviMemInKB,MemUsageInPercent,DateTime

Conclusion

The Docker API is a powerful tool that allows you to control Docker from your own applications. By accessing the Docker API from C#, you can automate Docker operations, create your own Docker tools, and integrate Docker with other systems.

Reference

Compile and Run .NET program with just .NET framework on your system.

Compile and Run .NET program with just .NET framework on your system.

Did you know that every .NET framework installation also installs VB & C# compiler.

Using these compilers you can compile and run C# or VB.NET program on machine with just .NET framework installed. This can be useful on system where installing full visual studio is not an option.

Here is how to do it

If .NET framework installed on you system, C:\Windows\Microsoft.NET\Framework directory  should list all the versions. Here is what is on my machine.

image

Once you find required .NET version, directory content should list csc.exe and vbs.exe compiler executable. As shown below.

image

Now use C# or VB compiler to compile the program as follows. For example use .NET 2.0 compiler to generate .exe c:\Windows\Microsoft.NET\Framework\v2.0.50727\csc.exe Program.cs

image

Simple way to share files from between Linux to windows

I often use both Windows and Linux environment at work. Sometimes I need to share files between Linux & windows workstation.

There are couples of ways to share files between Linux & Windows.

  1. Setup Samba server (CIFS share) on your Linux box, which can be accessed from any windows machine. Because In many of the Linux distros Samba server is not installed by default. It requires installing package and setting up configuration file to share folder.
  2. Run Ftp server on you Linux box and use ftp client from windows box to access it. Again this solution may require starting the ftp server on the Linux machine and configuring it to allow access to shares.
  3. Run SSH server on Linux machine and on windows box you use secure copy(scp) or similar SSH client to do file transfer. You can download scp client for windows putty download page.This also requires installing SSH server on Linux box ( for Ubuntu by default SSH server is not installed)

Even though many of the option listed are simple , it requires some effort to setup. You need to repeat the same when you re-install Linux.

Recently I discovered simple and elegant way to achive the same using python built in SimpleHTTPServer module.Advantage of this approach is python is by default installed on all Linux distributions. No need to install any additional package.

Here is how to do it

    1. From the command shell, go to the directory you want to share. For example if you want to share “/usr/home/john/files” .                                                                                                                                                                       $ cd /usr/home/john/files
    2. Run following command to share files. This command starts the webserver on port 8080 with directory browsing enabled .                                                                                                                                                                             $ python -m SimpleHTTPServer 8080
    3. On your windows box point your any Webrowser to http://<linux_machine_ip&gt;:8080 . This will list all the files present on the Linux box . Now select any files download to windows.

Resource