February 9, 2004

Download the code for this article

Whether you are developing at the enterprise level or for the smallest venues, IT is all about automation, finding ways to allow these infernal machines to do things better, faster, or in many cases, ‘at all’ than the human alternatives. Suppose, for example, that your company receives critical files by way of FTP, either from a mainframe FTP dump or via the internet from trading partners, in the Pre-.NET world RAD developers for the Microsoft Windows platform (OK, VB Developers) were limited to using either devices like the NT scheduler or launching background applications in the Startup folder, or worse yet, relying on human intervention to “see” certain conditions and react to them. The introduction of the .Net framework has introduced a new tool into the VB developer’s toolbox, Windows Services (of course, this is available to C# developers as well, and has been available to C++ developers for years). In all of the “hoopla” around Web services, windows services have been largely an unheralded feature of the .Net framework, but represent a real opportunity to get a lot of bang for your buck in development efforts.

It almost goes without saying that relying on human intervention is problematic at best (from sick days to vacation time, not to mention distractions and other “human factors”, this system is just not reliable). The Windows native NT Scheduler and other scheduler programs suffer from their own reliability issues, not to mention that they have to be explicitly launched and have to run in a specific user context, which requires maintaining significant overhead, just to support a few operations. Applications run in the Startup folder; well to begin with, they require someone to actually log in, and further, stay logged in, to start and run. In response to these challenges, Windows Services have the following benefits:

• Windows Services are Launched and controlled by the OS - While you can start, stop, and pause services from the service control manager or from the command line, one of the points of building an application as a service is that it can be controlled by the OS, and, therefore, doesn’t require operator input.



• Windows Services Operate in the “Background” and are “Long Running” - The kind of functionality that we tend to place in services should operate with direct user input or interface and should run throughout the lifecycle of the system.



• Windows Services Run in their own context - A service is launched within specific user credentials, which “should” be different than those of the customary user of a system. This creates a level of isolation between applications running in the foreground and the service. This keeps errors at the user layer, or even at the service layer, from interfering with other operations and spares the system of significant operating overhead from having to run within a specific user context.

Of course, just being a “service” is of no intrinsic value, rather it is in the “task” that your service performs that any application gains value in your operation. Fortunately, the .Net framework provides several classes and objects that lend themselves to utilization by a service to accomplish useful functions. For our demonstration, we will look at the system.io.filewatcher and system.threading. (By no means do I wish to imply that the use of these assemblies is limited to use in services, only that they are very useful within services.) Additionally, we will go into a detailed look at how a service should be deployed and managed. Finally, because services, like any other application, generally require some level of debugging, we will cover how to attach the debugger to a running service.

As with any other application, there are several design “best” practices that should be followed to ensure a smooth running service. Services tend to fall into three categories: Watchers, Listeners, and Workers. Watchers and listeners tend to operate in a similar fashion; they only differ in the trigger mechanism. A watcher has a polling mechanism (either explicitly or implicitly) to watch for some external stimulus (like a file appearing in a directory), while a listener has a mechanism (such as a port listener) waiting for something external to act explicitly on that service to set it in motion (as IIS, through the World Wide Web Publishing Service, listens on Port 80 for a request for web pages).

Worker services are the heavy lifters which “do the work” that the service is built around. As a general rule design listener/watcher services to do as little as possible so that the service actually launches a separate worker process or service (as the World Wide Web Publishing Service hands off requests for ASP.NET pages to the ASP.NET worker process which in turn, does the work of building the httpResponse and sending it back to the requestor). This architecture accomplishes two important goals: first, better use of system resources, since the service is made more lightweight if it does not have the extra overhead of managing the (usually) more complex and larger worker routines which can be left out of active memory until they are truly “needed”. Second, improved availability. With both watchers and listeners, you don’t want to handle the same request more than once, but you also want to ensure that you can handle every request. Having a more lightweight service makes recycling the watcher/listener faster and more effective (when you review the code samples with this article you will notice that the first action after the watcher is triggered is to start a new watcher). Worker services should be reserved for those occasions and situations where both high availability and fast response time is critical (such as SQL server). These are generally services for which a physical server can and should be a dedicated device (again like SQL or Exchange).

Scenario: A legacy mainframe system, connected by WAN, and not under our direct control, periodically does an FTP dump to a directory on our LAN. The files do not come in at a particular time or interval and each file needs to be handled separately and differently. The files do appear with a given naming pattern (filename + timestamp + .doc). The files we are interested in are:

• daily010104.dat: This file contains fixed length data which needs to be parsed and updated to an SQL database.
• weekly010104.dat: This file contains a summary report of activity, which needs to be formatted and sent to several managers.
• monthly010104.dat: This file also contains a summary report of activity, which needs to be formatted and sent to several managers.
• whenever010104.dat: This file contains a list of errors that need to be reviewed by appropriate staff members and summarized for managers.

Design Goals

The following goals are to be incorporated into the solution design, first we need to ensure Real-time processing; these files need to be processed as soon as they arrive so that the data and reports can be worked with by management in a timely manner. Second, we want a system that will require Minimal Human Intervention; these processes should be accomplished without the need for a staff member to have to constantly watch the download directory to see when the files arrive and have to react to their arrival. Next we want to make this economical by Using existing infrastructure; this process should not require the addition of new hardware or purchased software. (Of course, we are assuming you already own VS.NET and are running the .NET framework) and finally we want a solution that is Extensible; the solution needs to allow for new download management routines to be added to this system.

Design Plans

In order to accomplish the goals laid out we will build the following applications:

• FTPwatch.exe: A windows service designed to respond to the filesystem events. It will then launch auxiliary applications that will, in turn, do the heavy lifting.
• wkrDaily.exe: This process will contain the specific algorithms to parse the daily file and insert the results into SQL.
• wkrReports.exe: This process will open the report and format it for local consumption, as well as send it on to its destinations.
• wkrErrors.exe: This process will parse the errors, and send them to the data entry person responsible for the error as well as sending a summary report to the data entry manager.

Creating your service project

• Launch your Visual Studio.Net Development Environment and select “New Project” on the Start page.

• Name the project “FTPwatcher” (of course, you can leave the default name of “Service1” or name it “Fred” but that is hardly a “good practice.”) and open the project. (Click “OK”)



• VS.Net will create the basic project for you by adding an AssemblyInfo.vb and Service1.vb to your project. Now this is one of those (few) times when VS.Net helps you “too much” because the service that is created in your project is Service1.vb and a class, called (amazingly) Service1 is “pre-built” for you. However, the Name of the service that will eventually appear in your Service Control Manager is related to this and so it should be renamed. This is accomplished with the following steps:
• Open the Service1.vb assembly, the designer view will appear and populate the Property Box.
• In the Property box, change the property for ServiceName to FTPwatch.



• In the Solution Explorer, right-click on the filename Service1.vb and select rename. Change the name of the file to FTPwatch.vb.



• Change your view to the “code-view” for the file [now] FTPwatch.vb



• Rename the Class to FTPwatch (1a)



• In the Sub Main where the ServicesToRun array is declared (we will discuss this shortly) to change “service1” to a type of FTPwatch (1b)



• Change the property assignment for “me.servicename” to FTPwatch.(1c)

Imports System.ServiceProcess
Public Class Service1  '(1a)Change to FTPwatch Here
    Inherits System.ServiceProcess.ServiceBase

#Region " Component Designer generated code "

    Public Sub New()
        MyBase.New()

        ' This call is required by the Component Designer.
        InitializeComponent()

        ' Add any initialization after the InitializeComponent() call

    End Sub

    'UserService overrides dispose to clean up the component list.
    Protected Overloads Overrides Sub Dispose(ByVal disposing As Boolean)
        If disposing Then
            If Not (components Is Nothing) Then
                components.Dispose()
            End If
        End If
        MyBase.Dispose(disposing)
    End Sub

    ' The main entry point for the process
    <MTAThread()> _
    Shared Sub Main()
        Dim ServicesToRun() As System.ServiceProcess.ServiceBase

        ' More than one NT Service may run within the same process. To add
        ' another service to this process, change the following line to
        ' create a second service object. For example,
        '
        '   ServicesToRun = New System.ServiceProcess.ServiceBase () {New Service1, New MySecondUserService}
        '
        ServicesToRun = New System.ServiceProcess.ServiceBase () {New Service1} '(1b)Change to FTPwatch Here

        System.ServiceProcess.ServiceBase.Run(ServicesToRun)
    End Sub

    'Required by the Component Designer
    Private components As System.ComponentModel.IContainer

    ' NOTE: The following procedure is required by the Component Designer
    ' It can be modified using the Component Designer.
    ' Do not modify it using the code editor.
    <System.Diagnostics.DebuggerStepThrough()> _
    Private Sub InitializeComponent()
        components = New System.ComponentModel.Container()
        Me.ServiceName = "FTPWatch" '(1c)Change to FTPwatch Here
    End Sub

#End Region

    Protected Overrides Sub OnStart(ByVal args() As String)
        ' Add code here to start your service. This method should set things
        ' in motion so your service can do its work.
    End Sub

    Protected Overrides Sub OnStop()
        ' Add code here to perform any tear-down necessary to stop your service.
    End Sub

End Class

Now let us next go back to the Property Box (which can be viewed while you have the designer open). There are several properties here which you should understand:

• AutoLog: This is set to “True” by default and causes the start, stop, pause events to be sent to the system application event log.



• CanHandlePowerEvents: This is set to “False” by default and is most useful if you are developing a service that is intended to be run on a laptop computer. When set to true you can design event handlers to respond to the standard power events that occur (such as switching to battery power. See MSDN OnPowerEvent description for samples and explanations about how to use this and what the powerevents can be handled)



• CanPauseAndContinue: Setting this property to true allows your service to (guess what?) Pause and Continue. The important fact here is that, by default this is set to false, and if you want your service to be able to handle a Pause command from the service manager, you must set this to true. Then, of course, you have to write a handler to tell the service what to do when the pause command is received.

• CanStop: At first you might be inclined to say “Well Duh” but this property is set by default to True so that the Service manager Stop Event can be passed to your service and enables your service to have an OnStop event handler. This allows your service to be stopped separately from the system. If you are designing a service to run on a client machine and (perhaps) pass messages (like error logs) to a central depository, you may not want your service to be “stoppable” at the client.



• CanShutDown: This property does not, regardless of its setting, allow or disallow the service from shutting down; rather, it directs the service control manager to send a specific event into your service when the user attempts to shut down the system. The OnShutDown event allows your code to do whatever final clean-up might be needed for continued smooth operation, such as release connections or even inform other servers on the network that this instance of the service is going down. It is important to remember that this event is invoked only during an explicit shut down and should not do anything that might interfere with the actual shut down process.

While the default “service1.vb” file that is created for you in VS.NET only contains default handlers for “OnStart(ByVal args() As String)” and “OnStop()” Each of the properties listed above (except Autolog) expose one or default event handlers which can be added to your service, without the need for a “handles” statement. Since there are event handlers for all of these events in the System.ServiceProcess.ServiceBase class, event handlers for any of these events MUST be declared as Protected Overrides

For our purposes we will be setting “CanPauseAndContinue” & “CanShutdown” to True so that we can react to those events. But we will discuss those in further detail later. For now, all we need to do is change these properties and add the following subroutines to the FTPwatch class:

Protected Overrides Sub OnPause()
End Sub
Protected Overrides Sub OnContinue()
End Sub
Protected Overrides Sub OnShutDown()
End Sub

Now that we have our class file in place let us examine its contents starting at the top. We start by Importing System.ServiceProcess. The System.ServiceProcess exposes the base functionality that is required to build services, in particular the ServiceProcess.ServiceBase class that our FTPwatcher class is derived from. In the Public Sub New the base class must be initialized first then the class must call the InitializeComponent() subroutine. Here, the code to set the properties we discussed earlier is executed as part of the service object creation.

A peculiarity that you will notice is that this object contains both a “Sub Main” and a “Sub New”. We have seen the path created when we create a new instance of the object is created. Now we will look at what happens when the object is executed via the Sub Main. The ServicesToRun array is created and then initialized with an object of the type of the local service. If you wished to run multiple services within the same process, you can add those types to the array at this point. The array is then passed into the Shared Run method of the ServiceBase class. This is the point at which the service is actually available; though the OnStart method has not been called, the service is functionally idle, waiting for the OnStart event to be invoked by the service control manager. Up to this point we have been looking at how the OS and the .NET framework put in motion the underlying pieces that make a service work. Now that these foundations are in place, it is here that we start to build our functionality into the service class.

Here is where good design practices become both very important and challenging. While we could simply start a new filesystemwatcher right here, this creates certain problems. The first is recycling the watcher, so that it will work for more than one instance (one file system event), until the whole service was restarted. The second is that we would have difficulty managing the pauses and continue commands. The third “challenge” here is one of resources: “How much system time do we want to dedicate to this activity?”

If, however, we create a new thread and put our worker process on that, and assign the priority of that thread to “low” (which is sufficient for our purposes, on “low” priority the watcher will “only” look at the specified directory several times per second) all of these issues become easily manageable. Since we need access to the thread throughout the application, we need to declare it in the general declarations for the class as Dim myThread As System.Threading.Thread. Then we create an instance of the thread in the OnStart event as myThread = New System.Threading.Thread(AddressOf StartMonitor), where StartMonitor is the subroutine which creates the fileSystemWatcher and sets it in motion. Similarly the FileSystemWatcher needs to be in the general declarations so that it can be managed across the scope of the class as well.

Private Sub Monitor()
        myThread = New Thread(AddressOf StartMonitor)
        With myThread
            .Priority = ThreadPriority.Lowest
            .Start()
        End With
    End Sub
    'We delagate our thread to start here...
    Private Sub StartMonitor()
        myFSW = New FileSystemWatcher("C:\WatchMe") 'Of course you can
	 			imagine dynamic ways to assign the target directory....
        myLog.WriteEntry("Watching")
        With myFSW
            .IncludeSubdirectories = False
            .EnableRaisingEvents = True
            AddHandler .Created, AddressOf OnCreated
            .WaitForChanged(WatcherChangeTypes.Created)
        End With
    End Sub

With these elements in place the next piece we need is the handler for the Created Event. First we need a means to manage the relationship between the Filename and the action to be taken. After parsing the filename, we will use the app.config file to fetch the program name that we need to execute.

Private Sub OnCreated(ByVal sender As Object, ByVal e As FileSystemEventArgs)
        'start monitoring for the next file...
        Monitor()
        Dim FileName As String = DemoFileName(e.Name)
        Dim ExecProg As String = ConfigurationSettings.AppSettings(FileName)
        Dim LogMsg As String = e.Name & " created @ " & TimeStamp() & " " & vbCrLf & _
        "Executing " & ExecProg & " on " & FileName
        myLog.WriteEntry(LogMsg)
        'This is the code you might use to execute the worker program.
        'If ExecProg <> "" Then
        '    Call Shell(ExecProg)
        'End If
        'this Archives the specific file into a subdirectory called archive
        File.Move(e.FullPath, e.FullPath.Replace(e.Name, "archive\" & e.Name.Replace(".dat", ".arc")))
        'this ends the activites of this thread and leaves the new watcher running.
        Thread.CurrentThread.Join()
End Sub

The Call Shell(ExecProg) piece (remarked out in this demo) shows you how to offload the heavy lifting to a separate application, allowing your service to return its attention to watching the file system for the next event.

Installing the service

Services don't execute as normal processes do — instead, they are launched by the ServiceControlManager at system startup (if the service is flagged as "Automatic"). The SCM has to be told of the new service, and doing so is done either by going through a set of Win32 APIs to install the service, or by creating a VisualStudio.NET installer.

1. Return to Design view for your service.
   1. Click the background of the designer to select the service itself, rather than any of its contents.
   2. In the Properties window, click the Add Installer link in the gray area beneath the list of properties. (alternately, you can right click on the service designer and select “Add Installer”)
   3. By default, a component class containing two installers is added to your project. The component is named ProjectInstaller, and the installers it contains are the installer for your service and the installer for the service's associated process. By default the design view for the project installer appears.
   1. In the Design view for ProjectInstaller, and click ServiceInstaller1.
   2. In the Properties window, set the ServiceName property to your service name. (You need not change the “(Name)” Property here, this will remain a hidden object is, therefore, irrelevant)
   3. Set the StartType property to Automatic.
   4. In the designer, select ServiceProcessInstaller1. Set the Account property to LocalService. This will cause the service to be installed and to run on a local service account. (Use the other accounts with caution, as they run with higher privileges and increase your risk of attacks from malicious code.)
2. To build your service project
   1. In Solution Explorer, right-click your project and select Properties from the shortcut menu. The project's Property Pages dialog box appears.
   2. In the left pane, select the General tab in the Common Properties folder. From the Startup object list, choose yourservicename. Click OK.
   3. Press CTRL+SHIFT+B to build the project.
   
   Now that the project is built, it can be deployed. A setup project will install the compiled project files and run the installers needed to run the Windows service. To create a complete setup project you will need to add the project output, MyNewService.exe, to the setup project and then add a custom action to have MyNewService.exe installed.
   
3. To create a setup project for your service
   1. On the File menu, point to Add Project
   2. Choose New Project.
   3. In the Project Types pane, select the Setup and Deployment Projects folder.
   4. In the Templates pane, select Setup Project.
   5. Name the project MyServiceSetup. (A setup project is added to the solution.)
   6. Add the output from the Windows Service project, yourservicename.exe, to the setup.
   1. To add yourservicename.exe to the setup project
   2. In Solution Explorer, right-click MyServiceSetup,
   3. Point to Add, then choose Project Output.
   4. The Add Project Output Group dialog box appears.
   5. yourservicename is selected in the Project box.
   6. From the list box, select Primary Output, and click OK.
   (A project item for the primary output of yourservicename is added to the setup project.)
4. Now add a custom action to install the yourservicename.exe file.
   1. In Solution Explorer, right-click the setup project.
       
   2. Point to View, and then choose Custom Actions.
      (The Custom Actions editor appears.)
       
   3. In the Custom Actions editor, right-click the Custom Actions node and choose Add Custom Action.
      (The Select Item in Project dialog box appears.)
       
   4. Double-click the Application Folder in the list box to open it.
       
   5. Select Primary Output from yourservicename(Active), and click OK.
      (The primary output is added to all four nodes of the custom actions — Install, Commit, Rollback, and Uninstall.)
      
5. Finally, Build the installer
   1. In Solution Explorer, right-click the MyServiceSetup project and choose Build.
   
   
   2. NOTE: If you have to edit your service code, you must first, rebuild the service project, THEN rebuild the installer. (rebuild or build for the solution will not work for this purpose)
      
6. To install the Windows Service
   1. On your development box To install yourservicename.exe, right-click the setup project in the Solution Explorer and select Install.
   2. On another system: Execute the service installer.msi created in the Bin director of the installer project.

Debugging the service

Debugging a service has traditionally been a tricky part of service development — because it doesn't run as a normal console application, and because a service is typically launched by the ServiceControlManager at system startup, under a least-privileged account, debugging the service in VisualStudio requires the developer to attach to the running process, and then start and stop the service as necessary (or do what actions are necessary to trigger the desired functionality). Fortunately, VisualStudio.NET makes this fairly simple.

To Debug the service

1. Start the service running on your development box.
2. Prior to building/rebuilding your service, set a break point at some point that will execute after start-up.
3. In VS.NET, Select the Debug Menu.
4. Select Process
   (the Process dialog box will open)
5. Select the yourservicename process and attach.
6. If there is something you can do to “invoke” the process, do so, the code segment will “appear” in VS.NET and you can step-through using the F10 (or F8 key, in the VB keyboard configuration)

Conclusion

Now that we have designed, deployed and debugged our service it is ready to go into production. Of course all we have here is a very simple service, but with a little bit of tweaking it can be used to handle files arriving by FTP or any other means, in your environment. More importantly, what you have is a template for building other services that can automate processes on your network.

Windows Services represent a tremendous opportunity to create “fire-and-forget” applications that can automate almost any process where human intervention is time consuming, unreliable, and most importantly not required. Many key critical parts of the Windows operating system, including but not limited to IIS and ASP.NET, run as Services, for precisely the same reasons.

Authors

Cos Callis has been programming in BASIC since 1976 when he was introduced to his uncle's TRS-80 Model I (4K of RAM and a cassette player for storage) and has been an "enthusiast" ever since. He currently works as a Senior Programmer/Analyst for Delta Dental of Oklahoma and is the Program Director for the Oklahoma City .NET Developers Group. In 2000 he and Daniel Clausen formed HyperL, Inc. which does independent consulting and development work.