June 23, 2004

Introduction

Although bundled with some Microsoft operating systems (e.g., Windows Server 2003), the .NET platform is not yet a native part of Windows. Most of the Windows applications that users work with every day are still made of unmanaged, Win32 code. Some of these applications, including Internet Explorer, host the .NET Common Language Runtime (CLR) and subsequently run managed code. To say it all, any Win32 application can host the CLR, which is exposed to applications as a plain COM object. In Windows XP and Windows Server 2003, the OS loader is also aware of .NET applications and distinguishes managed programs from traditional, unmanaged executables. While interaction between managed and unmanaged code is definitely possible and fully supported, the core Windows system and the .NET framework remain neatly separated worlds partially unaware of each other.

For example, the Windows shell is not designed to support .NET executables in any special way. It is important to note, though, that this feature doesn’t depend on the .NET Framework capabilities. So don’t expect significantly good news from Whidbey. For something to change, you should wait for the Longhorn timeframe at a minimum.

A sneak preview of the Longhorn-style shell programming can be found in the following article that I wrote for the Longhorn DevCenter. The article demonstrates the various steps needed to build a custom sidebar tile. The sidebar is a sort of custom taskbar and a tile is a small application hosted in it. In Longhorn, all shell components are exposed through base classes. Creating a new Longhorn sidebar tile is as easy as deriving a new class and overriding one method or two. But that’s just the Longhorn’s way of working.

Things are a bit different on today’s Windows operating systems.

There’s no base class to inherit if you want to create a shell or namespace extension for the Windows XP or Windows 2003 Server shell using any version of the .NET Framework. The rub lies in the fact that the Windows shell (specifically, the explorer.exe file) doesn’t know how to handle managed assemblies. The Windows shell only knows how to load COM components; but the .NET COM Interop layer ensures that assemblies can be exposed as COM objects. In the end, Windows shell extensions can be written using C# or Visual Basic .NET, but resulting components must be registered as COM objects to be usable and need a sort of COM proxy that sets up and manages any communication between the shell extension and Windows Explorer.

In this article, I’ll first briefly discuss the COM Interop layer and then attack with the techniques and tricks you need to know to build managed shell extensions. A practical example will complete and complement the explanation.

COM Interop Overview

COM Interop is a double-edged software layer that .NET Framework applications and components use to communicate with COM-based applications and components. The communication is bidirectional—from .NET to COM and from COM back to .NET. The COM Interop consists of two subsystems—the Runtime Callable Wrapper (RCW) and COM Callable Wrapper (CCW).

RCW allows .NET applications to access existing COM components without requiring any modification to the original. RCW is the chief technology that enables managed applications to incorporate COM code. For example, if you use the ADO library from within a Visual Basic .NET application, you are actually relying on a RCW wrapper that Visual Studio .NET creates on the fly for you. The main task of a RCW wrapper is importing all relevant COM types and exposing them through managed counterparts. You create a RCW wrapper for any COM component by using a particular utility named TlbImp.exe and located under the Framework SDK folder.

The utility reads the COM type library information and creates and compiles a .NET class having the same members as the COM original. Each call to the managed members results in a call made to the underlying COM object that the CLR will resolve at run time. The CLR will also take care of marshaling data between COM objects and managed objects as needed.

The TlbImp utility can be manually invoked from the command line or implicitly used through the Visual Studio .NET project interface. In fact, the utility is silently invoked when you choose to add a new reference to your project and pick it up from the COM tab. (See Figure 1.)

The other side of the COM Interop, and the one that is of most interest here, is the Callable COM Wrapper. CCW allows COM applications to access managed objects as easily as they access other COM objects. Another specialized utility (RegAsm.exe) exports the managed types into a type library and exposes the managed component as a traditional COM component by creating the proper registry entries. The COM caller (say, the Windows Explorer) is redirected to a proxy component (mscoree.dll) which, in turn, hosts the CLR and executes any managed code. Again, the CLR takes care of any data marshaling that proves necessary.

In light of the capabilities and features supported by the COM Interop layer, the todo-list for the developers who want to write managed shell extensions can be outlined as follows:

• Create a helper class to import Win32 native structures, types, and interfaces;
• Optionally, create a second helper class to import Win32 native API functions that will simplify certain mandatory tasks;
• Write a managed class that implements any needed COM interface for the shell extension;
• Once you’ve compiled the assembly, register it through the RegAsm.exe utility;

Let’s see how to accomplish these tasks in detail.

Importing Goods from Win32

The most important Win32 definitions you import in a .NET project are the COM interfaces involved with the shell extension of choice. In addition, you bring in some Win32 API functions specifically designed to handle data types the Explorer way. For example, imagine you’re going to write a context menu shell extension. In this case, you might want to import functions like DragQueryFile and InsertMenuItem because they let you respectively extract file names out of a shell memory buffer and create a new menu item. While the same operations could in theory be accomplished using managed code leveraging native Win32 API keeps it easier and less error-prone.

In general, you might want to create a shell extension specific helper class that defines what you need to import to seamlessly implement the extension in .NET. Code below shows a partial example of a similar class targeted to the context menu extension. Have a look at the article’s companion content for the full source code.

namespace ShellExt
{
	public struct MenuItem
	{
		public string Text;
		public string Command;
		public string HelpText;
	}

	// Make these constants 
	public enum MIIM : uint
	{
STATE = 0x00000001,
		ID = 0x00000002,
		SUBMENU = 0x00000004,
            :
	}

	public enum	MF : uint
	{
		INSERT = 0x00000000,
		CHANGE = 0x00000080,
		:
	}

	public enum MFS : uint
	{
		GRAYED =        0x00000003,
:
	}

	public enum CLIPFORMAT : uint
	{
		CF_TEXT = 1,
		CF_BITMAP = 2,
		CF_HDROP = 15,
:
	}

	public enum DVASPECT: uint
	{
		DVASPECT_CONTENT = 1,
:
	}

	public enum TYMED: uint
	{
		TYMED_HGLOBAL = 1,
:
    }

	public enum CMF: uint
	{
		CMF_NORMAL		= 0x00000000,
		:
	}

	// GetCommandString uFlags
	public enum GCS: uint
	{
		:
	}

	[StructLayout(LayoutKind.Sequential)]
	public struct MENUITEMINFO
	{
		public uint cbSize;
		public uint fMask;
		public uint fType;
		public uint fState;
		public int wID;
		public int hSubMenu;
 		public int hbmpChecked;
		public int hbmpUnchecked;
		public int dwItemData;
		public string dwTypeData;
		public uint cch;
		public int hbmpItem;
	}

	[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Unicode)]
	public struct INVOKECOMMANDINFO
	{
	public uint cbSize;
	public uint fMask;
	public uint wnd;
	public int verb;
		[MarshalAs(UnmanagedType.LPStr)]
		public string parameters;
	[MarshalAs(UnmanagedType.LPStr)]
		public string directory;
		public int Show;
	public uint HotKey;
		public uint hIcon;
	}
	:
}

As you can see, a lot of the Win32 constants have been grouped into more manageable enum types and some structures have been imported paying attention to data marshaling. In particular, the [StructLayout] attribute indicates the type of layout of the members in the structure. It can either be sequential or offset-based. The [MarshalAs] attribute indicates how to marshal strings to COM interfaces. The LPStr used above option marshals the .NET string as a pointer to a null-terminated array of ANSI characters. In the source code of this article, you’ll also see a string marshaled through a StringBuilder object, as in Code Snippet 1.

Code Snippet 1
[DllImport("shell32")]
internal static extern uint DragQueryFile(
	uint hDrop,
	uint iFile,
	StringBuilder buffer,
	int cch);
[DllImport("user32")]
internal static extern int InsertMenuItem(
	uint hmenu,
	uint uposition,
	uint uflags,
	ref MENUITEMINFO mii);

You normally opt for a StringBuilder object when a fixed-length character buffer must be passed into unmanaged code to be manipulated. If you simply pass a string you prevent the function from modifying the buffer—a .NET string is immutable. If the string is passed by reference, there is no way to initialize it to a given size. In Win32, many functions require a read/write buffer of a maximum size. In this case, when importing the API to .NET, use the StringBuilder object.

In the Code Snippet 1, you also find the .NET declaration of the InsertMenuItem API function typically used to append a new item to an existing menu. The combined use of these two functions will empower us to create a custom context menu with C# code. The following namespaces are a common presence in any shell extension project.

Code Snippet 2
using System;
using System.Runtime.InteropServices;
using System.Text;

Importing COM Interfaces

Each and every COM interface is characterized by a GUID. Just the GUID is the key attribute to specify when importing a COM interface into a .NET application. In addition, you need to inform the CLR that the interface being declared was previously defined in a COM type library and must indicate the type of the interface to determine how the interface is exposed to COM callers. You fulfill all these requirements using custom attributes. The following code snippet shows how to import the IShellExtInit and IContextMenu interfaces.

Code Snippet 3
[ComImport(),
 InterfaceType(ComInterfaceType.InterfaceIsIUnknown),
 GuidAttribute("000214e8-0000-0000-c000-000000000046")]
public interface IShellExtInit
{
	[PreserveSig()]
	int Initialize(
		IntPtr pidlFolder,
		IntPtr lpdobj,
		uint hKeyProgID);
}

[ComImport(),
 InterfaceType(ComInterfaceType.InterfaceIsIUnknown),
 GuidAttribute("000214e4-0000-0000-c000-000000000046")]
public interface IContextMenu
{
	[PreserveSig()]
	int QueryContextMenu(
		uint hmenu,
		uint iMenu,
		int idCmdFirst,
		int idCmdLast,
		uint uFlags);

 	[PreserveSig()]
	void InvokeCommand(
		IntPtr pici);

[PreserveSig()]
	void GetCommandString(
		int idcmd,
		uint uflags,
		int reserved,
		StringBuilder commandstring,
		int cch);
}

In Code Snippet 3 there are some interop custom attributes that need further explanation.

The [ComImport] attribute simply marks the current interface as a type that was previously defined in a COM type library. Basically, this attribute marks the difference between a .NET native interface and an interface that results from mirroring a COM interface. Applying this attribute is important because the CLR treats types based on COM interfaces differently as for activation and type coercion.

The [InterfaceType] attribute identifies how to expose an interface to COM clients. The ComInterfaceType enumeration lists the options available. (See Table 1.)

By default, interfaces are exposed to COM as dual interfaces. The interface related to shell extensions, instead, have no need to support late binding. They are plain interfaces derived from IUnknown and require the InterfaceIsUnknown attribute.

The role of the [GuidAttribute] attribute is self-explanatory. It assigns an explicit GUID to the interface. In .NET, you don’t have to assign explicit GUIDs to interfaces and classes as they will automatically be generated when needed. Importing an existing COM interface is a different kind of stuff, however. In this case, you must ensure that a particular GUID is used—the GUID that is uniquely assigned to that interface in the Win32 registry. This guarantees that the .NET class implementing that interface can be successfully exposed to COM clients and have its methods transparently invoked irrespective of its .NET implementation. The [GuidAttribute] attribute takes a string that represents the GUID to use.

Last but not least, let’s consider behavior and goal of the [PreserveSig] attribute. In this article, I’m discussing how to write .NET classes that implement certain COM interfaces so that, once properly configured in the system registry, they can be successfully called back by COM client applications. Most methods on COM interfaces return HRESULT values and use pointers to memory buffers to pass return values back to the caller. The retval modifier can be used in the definition of the interface to automatically transform the content of particular out parameter in the return value of the method. This is a pretty common scenario. For this reason, this is also the default behavior that the CLR applies to methods of an exported .NET interface. For example, consider the following .NET interface.

public interface ITryThis
{
	int DoSomething(int n);
}

public interface ITryThis
{
	HRESULT DoSomething(
		[in] int n,
		[out, retval] int * x);
}

When you apply the [PreserveSig] attribute to the signature of a managed method, it will be exposed to COM with the same C# signature.

In summary, to write a shell extension using C# or any other .NET language, you need to come up with a .NET class that implements any required interfaces. Required interfaces are to be .NET interfaces with GUID and methods signature that match those of original COM interfaces. Code Snippet 3 illustrates a class that defines the IShellExtInit and IContextMenu interfaces needed to create a context menu shell extension.

Implementing a Context Menu Shell Extension

A managed class for a shell extension implements the required interfaces and is associated with an explicit and auto-generated GUID. This GUID will be next used to register the .NET class as a COM object in the system registry.

Code Snippet 4
namespace ShellExt
{
  [Guid("auto-gen GUID")]
  public class BatchResultContextMenu: IShellExtInit, IContextMenu
  {
      // Protected members
	protected const string guid = "{auto-gen GUID}";
	protected string m_fileName;
	protected uint m_hDrop = 0;


      // IContextMenu members

      // IShellExtInit members

      // More code here
  }
}

Let’s build a sample context menu shell extension that applies to any file with a .BatchResults extension. To exemplify, such a file would represent the results of some sort of batch process that occurs periodically on the server. Imagine you start the process to work during the night and it generates .BatchResults files with some information. Next morning, you get back at work and as first thing need to examine the contents of the files created overnight. A context menu shell extension can do that for you by looking at the contents of the file and proposing a menu of choices like Reschedule/Retry Now/Cancel if something went bad, or Commit/Rollback if the job completed successfully. All that you have to do is right-click on any .BatchResults file you find and choose the options. Figure 2 gives you an idea of the feature and provides a preview of the shell extension in action.

As you can see in the Code Snippet 4, the shell extension class defines a couple of protected members plus the GUID. The file name member refers to the name of the file that users right-click to start the extension. The m_hDrop member is actually the .NET counterpart of a HDROP handle. In Win32, a HDROP type contains a handle to an internal structure describing the group of files dropped during a drag-and-drop operation. The same format is used to pack the names of the selected files the user right-clicked on. To unpack the data stored in this handle, you need a made-to-measure Win32 API function—DragQueryFile. The handle is retrieved during the execution of the Initialize method of the IShellExtInit interface. This method represents the entry-point in the shell extension.

int IShellExtInit.Initialize(
	IntPtr pidlFolder,
	IntPtr lpdobj,
	uint hKeyProgID)

The key argument is lpdobj which represents a shell created object that implements the IDataObject interface—another COM interface you need to import in your .NET project. The code snippet below shows how to extract HDROP from this object.

Code Snippet 5
IDataObject dataObject
dataObject = (IDataObject) Marshal.GetObjectForIUnknown(lpdobj);
FORMATETC fmt = new FORMATETC();
fmt.cfFormat = CLIPFORMAT.CF_HDROP;
fmt.ptd = 0;
fmt.dwAspect = DVASPECT.DVASPECT_CONTENT;
fmt.lindex = -1;
fmt.tymed = TYMED.TYMED_HGLOBAL;
STGMEDIUM medium = new STGMEDIUM();
dataObject.GetData(ref fmt, ref medium);
m_hDrop = medium.hGlobal;

A lot of COM and Win32 types and constants are used and needed. The good news, though, is that this is just boilerplate code that doesn’t need to be fully understood and let alone modified. Just use it.

The shell extension is initialized whenever the user right-clicks to display the context menu of a certain file. The next step for the shell is invoking the QueryContextMenu on the IContextMenu interface.

In QueryContextMenu you receive a handle to the shell menu and extend it adding as many items and popups as needed. Here are the steps needed to create a custom popup as in Figure 2.

Code Snippet 6
int IContextMenu.QueryContextMenu(
	uint hMenu, uint iMenu, int cmdFirst, int cmdLast, uint uFlags)
{
	// Get the file name to work with
	StringBuilder sb = new StringBuilder(1024);
	Helpers.DragQueryFile(m_hDrop, 0, sb, sb.Capacity + 1);
	m_fileName = sb.ToString();

	// Create and populate the popup menu to insert 
	uint hmnuPopup = Helpers.CreatePopupMenu();
int id=1;
	id = PopulateMenu(hmnuPopup, cmdFirst + id);

	// Add the popup to the context menu
	MENUITEMINFO mii = new MENUITEMINFO();
	mii.cbSize = 48;
	mii.fMask = (uint) MIIM.TYPE | (uint) MIIM.SUBMENU;
	mii.hSubMenu = (int) hmnuPopup;
	mii.fType = (uint) MF.STRING;
	mii.dwTypeData = "Actions";
	Helpers.InsertMenuItem(hMenu, (uint) iMenu, 1, ref mii);

// Append a separator
	MENUITEMINFO sep = new MENUITEMINFO();
	sep.cbSize = 48;
	sep.fMask = (uint) MIIM.TYPE;
	sep.fType = (uint) MF.SEPARATOR;
	Helpers.InsertMenuItem(hMenu, iMenu+1, 1, ref sep);
}

The shell provides you with a HMENU handle to a menu object. This makes harder for you to create and modify the menu using the .NET classes. The code that populates the popup menu is important for one aspect-the ID assigned to the various items.

Code Snippet 7
int PopulateMenu(uint hMenu, int id)
{
	bool done = ProcessBatchResults();
	if (done)
	{
		AddMenuItem(hMenu, "Reschedule", id, 0);
		AddMenuItem(hMenu, "Retry Now", ++id, 1);
		AddMenuItem(hMenu, "Cancel", ++id, 2);
	}
	else
	{
		AddMenuItem(hMenu, "Commit", 100 + id, 0);
		AddMenuItem(hMenu, "Rollback", 100 + (++id), 1);
	}
	return id++;
}

The ProcessBatchResults helper method analyzes the results of the background job and fills out the list of options to offer to the user. Each menu item must have a unique ID so that the InvokeCommand method can properly handle it when clicked. Since the code here generates two different menus depending on runtime conditions, using a different offset for IDs is a must. I’ve chosen to start menu IDs from 0 or 100 depending on the results of the batch—the value of the “done” Boolean variable in Code Snippet 7.

To handle the click on a shell extension item, you attach some code to the InvokeCommand method of the IContextMenu interface.

Code Snippet 8
void IContextMenu.InvokeCommand (IntPtr pici)
{
	Type t = Type.GetType("ShellExt.INVOKECOMMANDINFO");
	INVOKECOMMANDINFO ici;
	ici = (INVOKECOMMANDINFO) Marshal.PtrToStructure(pici, t);
	switch (ici.verb-1)
	{
		case 0:
			RescheduleJob();
			break;
		case 1:
			RetryNowJob();
			break;
		case 2:
			CancelJob();
			break;
		case 100:
			CommitJob();
			break;
		case 101:
			RollbackJob();
			break;
	}
}

In the sample code you find with this article, each of the above methods pops up a message box. (See Figure 3.)

Registration and Deployment

So you now have a fresh assembly that exposes a class with a few COM interfaces. What’s the next step? You must register the class with the system registry so that it looks like a COM object first, and a shell extension next. The regasm.exe tool can create all necessary entries to configure the assembly as a COM object.

regasm.exe yourassembly.dll

What about the shell specific entries? Each shell extension type requires different registry settings. And also shell extensions of the same type may need to enter a different set of changes. A context menu shell extension must be associated with a particular file type (say, .BATCHRESULTS extension) meaning that the registry must also contain references to that file type. Let’s see the steps to accomplish to register the shell extension with .BATCHRESULTS files.

You make sure that the .BATCHRESULTS node exists in the registry’s HKEY_CLASSES_ROOT (HKCR) node. If it doesn’t exist, you create it and set its default value to an arbitrary string—typically, BatchResultsFile. Next, you create a new entry under HKCR and name it after this arbitrarily chosen string. The BatchResultsFile node will contain the details of the shell configuration for .BATCHRESULTS files.

Kindly enough, the regasm.exe tool offers to call a static method on your shell extension class to configure the registry. You define a couple methods like below.

[System.Runtime.InteropServices.ComRegisterFunctionAttribute()]
static void RegisterServer(String str1)
{
	RegistryKey root;
	RegistryKey rk;

	// Register .BATCHRESULTS as a known file type
	:

	// Register as a shell extension for .BATCHRESULTS files
	root = Registry.ClassesRoot;
	rk = root.CreateSubKey("...\\ContextMenuHandlers\\BatchResults");
	rk.SetValue("", guid.ToString());
	rk.Close();

	// Set as an approved shell extension
	root = Registry.LocalMachine;
	rk = root.OpenSubKey("...\\Shell Extensions\\Approved", true);
	rk.SetValue(guid.ToString(), "BatchResults shell extension");
	rk.Close();
}
[System.Runtime.InteropServices.ComUnRegisterFunctionAttribute()]
static void UnregisterServer(String str1)
{
	// Unregister as an approved shell extension
	:

	// Removes shell extension from the BatchResultsFile entry
	:

	// Removes .BATCHRESULTS entries
	:
}

In light of the behavior of regasm.exe all that you have to do to successfully deploy the shell extension is calling regasm.exe passing it the name of the assembly.

Figure 4 shows the registry entries for the COM object representing the shell extension. The server behind the COM object is mscoree.dll—the bridge between COM and CLR--and the reference to the actual assembly is maintained through the assembly and class name. The shell extension assembly must be placed in a folder where mscoree.dll can reach it—for example, you can copy it to the global assembly cache (GAC).

Development and Testing Tips

Debugging shell extensions has never been fun. The main problem is that Explorer holds a copy of the previous DLL which either prevents you from overwriting the existing file or just keeps the old file in memory until the shell restarts. My favorite step-by-step procedure is as follows:

• Compile the assembly
• Remove the assembly from the GAC
• Restart the shell (possibly without logging off or restarting the system)
• Copy the new assembly to the GAC
• Test the shell extension

To restart the shell you can use the following C++ code wrapped up in a quick Win32 executable.

void SHShellRestart(void)
{
	HWND hwnd;
	hwnd = FindWindow("Progman", NULL );
	PostMessage(hwnd, WM_QUIT, 0, 0 );
	ShellExecute(NULL, NULL, "explorer.exe", NULL, NULL,
SW_SHOW );
	return;
}

This code will refresh the taskbar and kill all opened Explorer windows. All other applications are not affected by this event. Restarting the shell process, of course, frees all currently loaded shell extensions.

Summary

This article demonstrated how to create a Windows shell extension using C# code and the .NET Framework. Although the .NET Framework doesn’t provide any special support for it, the task didn’t prove that hard—just a bit boring, isn’t it?

A managed shell extension is a .NET class marked with an explicit GUID and implementing any needed COM interface. A COM interface, in turn, is a .NET interface with the same signature and GUID. To register a .NET assembly to look like a COM object, you use the regasm.exe utility bundled with the .NET Framework SDK.

Figures

Figure 1—Adding a COM reference to your Visual Studio .NET project results in the creation of a new RCW wrapper through the TlbImp.exe utility



Figure 2—The BatchResults shell extension looks at the contents of the file and provides a list of suggested actions (View larger image)



Figure 3—The user clicked to execute an action on the batch results file as suggested by the shell extension



Figure 4—The registry settings for the COM object that represents to the Explorer’s eyes our managed shell extension (View larger image)

Author Bio

Dino Esposito is a trainer and consultant based in Rome, Italy. Member of the Wintellect team, Dino specializes in ASP.NET and ADO.NET and spends most of his time teaching and consulting across Europe and the United States. Prolific author, Dino writes the "Cutting Edge" column for MSDN Magazine and contributes to the Microsoft’s ASP.NET DevCenter and several other magazines. Recent books are Programming Microsoft ASP.NET (Microsoft Press, 2003), Applied XML Programming with the .NET Framework (Microsoft Press, 2002), and the upcoming Introducing ASP.NET 2.0, always from Microsoft Press. When not writing or teaching, Dino is likely speaking at industry events such as Microsoft TechEd, DevConnections and WinDev.

Authors

Dino Esposito is a trainer and consultant for Wintellect based in Rome, Italy. He runs the Cutting Edge column for MSDN Magazine and is a regular contributor to MSDN News and SQL Server Magazine. Before becoming a full-time author, a full-time consultant and a full-time trainer, Dino was a full-time employee and worked day and night for Andersen Consulting focusing on the very first real-world implementations of DNA systems. Dino also has extensive experience developing commercial Windows-based software, especially for the photography world, and was part of the team who designed and realized one of the first European image online databanks. To get in touch, just write to dinoe@wintellect.com. To meet in person, just attend leading conferences such as Microsoft TechEd, VSLive!, VS DevCon, WinSummit or Wrox WebDev Conference.