June 2, 2004

Introduction

Sometimes, it's necessary to remember the roots, the place where it all began. The World Wide Web and electronic mail have become very common to our life, and it's our duty to tell the story behind their creation to our children and grandchildren. More importantly, as the importance of a technology increases, so does its usage--and thus the need that we'll need to understand what's happening at the protocol level in order to debug and/or diagnose problems. To do that, we need to understand how to work with sockets at the most primitive level, and to speak and understand the SMTP and HTTP protocols "in the raw". Fortunately, Internet protocols aren't all that hard to understand. As I took a closer look at SMTP and HTTP for the first time, I was surprised how simple they are.

You might be a bit scared of such low-level exploration. Maybe you know the Z39.50 protocol, which often is used by libraries to offer an interface for searching their inventory of books and articles. This protocol is very tricky and almost completely incomprehensible. However, internet protocols are deliberately written to be human readable, making it relatively easy to know their basics. The .NET Framework is a very good friend to explore the mysteries of these socket based protocols. Going through an email server sample will address both sides: SMTP basics and .NET Framework programming.

The following email server will be able to handle multiple connections for receiving emails. In order to keep it simple, only a subset of SMTP commands will be implemented, but it’s easy to extend and finish the software.

SMTP Basics

Before starting coding, some background information about the Simple Mail Transport Protocol (SMTP) is necessary. SMTP is a fully human readable, text-based protocol described in RFC 821 [1]. That makes it easier and you are able to communicate with a email server via good old Telnet. Here's a sample dialog between a client (marked with "C:") and a server (marked with "S:") sending an email:

  C: HELO myComputerName
  S: 250 smtp.theserverside.net Hello myComputerName [120.42.42.1] 
  C: MAIL FROM: me@TheServerSide.net
  S: 250 smtp.TheServerSide.net <me@TheServerSide.net> is syntactically 
  correct 
  C: RCPT TP: you@TheServerSide.net
  S: 250 <you@TheServerSide.net> verified 
  C: DATA 
  S: 354 Enter message, ending with "." on a line by itself 
  C: Date: 1 April 04 10:30:42
  C: From: me@TheServerSide.net
  C: To: you@TheServerSide.net
  C: Subject: Say Hello
  C: Hello my friend! 
  C: And good bye! 
  C: . 
  S: 250 OK ID=1B3alH-0004ue-00 
  C: QUIT 
  S: 221 smtp.TheServerSide.net closing connection 

As you can see, the server always responds with a status code (250, 421, ...). All text behind is undefined and can be set free by the server’s programmer. But step by step:
First of all, the client sends a "HELO" followed by its computer name. The server normally responds with code 250 which always indicates a positive answer. Please have a look at Common SMTP Reply Codes box for more reply codes. The text behind 250 needn’t be interpreted because, as said above, it’s up to programmers what they append. Because it's useful for a manual telnet session, your server should give some information too. Don’t forget the geeks out there using Telnet as their primary email client.

  C: HELO myComputerName
  S: 250 smtp.TheServerSide.net Hello myComputerName [120.42.42.1] 

You have to send a HELO first because it's a kind of an initial handshake between both client and server. Now, the client is able to send commands to the server. In order to send an email, the client has to send the sender's address first. This is done by the MAIL FROM: command:

  C: MAIL FROM: me@TheServerSide.net
  S: 250 smtp.TheServerSide.net <me@TheServerSide.net> is syntactically 
  correct 

The MAIL FROM:-command expects the sender's email address as a parameter. The Server checks its syntax and returns 250 if it's ok, otherwise a 501 status. Next, the client specifies the recipient's address by using the RCPT TO: command and passing the email address as a parameter:

  C: RCPT TO: you@TheServerSide.net
  S: 250 <you@TheServerSide.net> verified 

After that, client starts the data part with DATA command. This command has no parameters but signals that the next lines of text sent by client contain the mail message. Th server answers with a status 354. The data session ends with a single “.” followed by a carriage return (CR). The server answers with the status code and often returns its own unique identifier for this message. The data part has a special field for meta information. An email client takes a subject, sender’s address, timestamp etc. of a message out of this data part, not from the SMTP commands. Commands are used for communication between the client and server and are not of part of the message. That’s important to understand! – What does it mean? Now, even specifying the sender’s address via the Mail From-command won’t display it in an email client. Therefore, you have to use fields inside as specified in RFC 822 [2] and build the mail header. Only three are required: Date, From and To. But think about Subject: as an unofficial fourth.

  C: DATA
  S: 354 Enter message, ending with "." on a line by itself 
  C: Date: 1 April 04 10:30:42
  C: From: me@TheServerSide.net
  C: To: you@TheServerSide.net
  C: Subject: Say Hello
  C: Hello my friend! 
  C: And good bye! 
  C: . 
  S: 250 OK ID=1B3alH-0004ue-00

Last but not least you should close the connection using the Quit command and again, the server returns its status code.

  C: QUIT
  S: 221 smtp.TheServerSide.net closing connection

Common SMTP Reply Codes

SMTP Commands used by this sample

Framework Classes

The .NET Framework provides classes for Socket-Programming in its Namespace System.Net.Sockets . Besides this, common helper classes like IPAddress are stored in System.Net and Streams for reading and writing data in System.IO. A socket server listens on the IP port where it provides the service; an SMTP server uses port 25. That is the job of the TcpListener class. You can specify an IP Address and port number on which your server should listen. Calling the AcceptSocket() method will block the application flow and returns once a client has connected. Then it returns a Socket object which contains the established connection. In order to support multiple connections, the server should start a new thread handling this client connection and then recall AcceptSocket() for new connections. Using the returned socket object, you are able to cerate a NetworkStream object and with it a StreamReader and StreamWriter object. It's the best way to operate with separate reader and writer objects. The StreamReader has a method ReadLine(), that returns data from client as a string. In order to send data back to client, choose the StreamWriter's method WriteLine(). These five classes TcpListener, Socket, NetworkStream, StreamReader and StreamWriter are the fundamentals of all socket applications. A very simple, not multi connection-compliant sample looks like this:

using System.IO;
using System.Net;
using System.Net.Sockets;

// Create TcpListener on Port 8080: 
TcpListener tcpListener = new TcpListener(IPAddress.Any, 8080);                 
tcpListener.Start(); 

// Wait for clie
nt connection: 
Socket socket = tcpListener.AcceptSocket(); 

// Create streams: 
NetworkStream networkStream = new NetworkStream(socket); 
StreamReader reader = new StreamReader(networkStream); 
StreamWriter writer = new StreamWriter(networkStream); 

// Enabled AutoFlush options sends data to client immediately: 
writer.AutoFlush = true; 

// Send data to client and read its answer message: 
writer.WriteLine("Hello Client!"); 
string answer = reader.ReadLine(); 

// Close connections: 
reader.Close(); 
writer.Close(); 
networkStream.Close(); 
socket.Close(); 

// Stop listener: 
tcpListener.Stop(); 

.NET Classes for Socket Programming

Email Server

But a real email server is more than the small script above - it must handle multiple connections, catche errors and last but not least "speak" SMTP. You can divide it into two parts: One handles incoming client connections (the Server class) and another does the SMTP communication between the client and server (the SMTPProcessor class).

The Server class is doing part one and its implementation is listed in Listing 1. It has two main methods: public method Start() and the private method StartListen(). Start() creates a new TcpListener object first and initializes it with a given IP address and port as specified via the constructor. In order not to block the caller of the Server class, a new thread will be started immediately. That’s important to give control back to main application. The server has a Stop() method which can be called by the application to stop listening. The current implementation is very strict and does a hard abort on the TcpListener object regardless of whether any client is connected; however, this can be done better using flags to find out if there is an active client connection to wait for. Start() calls the private method StartListen() in a new thread. This method calls the AcceptSocket() of TcpListener first. AcceptSocket() blocks until a new client has connected. Once a new connection is established, StartListen() creates a new SMTPProcessor instance. This class is doing part two: communication between the client and server. For this object a new thread will be created because the server should be able to return more connections in parallel. Now SMTProcessor handles the client and server SMTP talk in a separate thread and the server can loop back to AcceptSocket(), and wait for next client.

Listing 1: Implementation of Server Class

public class Server {
	
  private readonly int _port;
  private readonly IPAddress _serverAddress;
  private TcpListener _tcpListener;
	
  /// <summary>
  /// Initialize server to listen on given address and port
  /// </summary>
  /// <param name="serverAddress">IP Address to listen on</param>
  /// <param name="port">Port to listen on</param>
  public Server(IPAddress serverAddress, int port) {
    _port = port;
    _serverAddress = serverAddress;
  }

  /// <summary>Start server listening</summary>
  public void Start() {			
    try {
      _tcpListener = new TcpListener(_serverAddress, _port);			
      _tcpListener.Start();

      Console.WriteLine("Server Ready - Listening for new connections ...");
      Thread thread = new Thread(new ThreadStart(StartListen));
      thread.Start();
    }
    catch(Exception e) {
      Console.WriteLine("Error while listening :" + e.ToString());
    }
  }

  /// <summary>Stop server listening</summary>
  public void Stop() {
    _tcpListener.Stop();
  }      

  /// <summary>
  /// Wait for client connections and start new communication thread
  /// </summary>
  private void StartListen() {
    try {
      while(true) {		
	  // AcceptSocket blocks until new connection has established
        Socket socket = _tcpListener.AcceptSocket();					
        socket.Blocking = true; // Blocks until a operation has completed	
        
        if(socket.Connected) {
          Console.WriteLine("Client connected: {0}", socket.RemoteEndPoint);

          // Create new SMTPProcessor and start in new thread:
          SMTPProcessor smtpProcessor = new SMTPProcessor(socket);
          Thread thread = new Thread(new ThreadStart(smtpProcessor.Process)); 
          thread.Start();					
        }				
       }
    }
    catch (SocketException ex) {
      // An exception will be thrown when tcpListerner.Stop() is called	
      Console.WriteLine("SocketException: {0}", ex);
    }		
  }	
}

As you can see, the Server class has no knowledge about SMTP. You can take it as a generic socket server class not only for an email server. But SMTPProcessor is specialized in understanding and speaking SMTP. See the implementation in Listing 2. This class has more code but it’s also not rocket science. And again, you will find two main methods: The Process() method is the only public one. It’s called by the server class to start handling the SMTP communication. The parameter socket contains an established connection and is “ready to be used”. First, Process() will create one stream for reading and one for writing. Also it creates a file stream where the message will be stored to. The filename is generated by a GUID and is therefore unique. Then Process() starts a loop where commands from the client will be read, then evaluated and at the very least, a server response will be sent back to client. The evaluation of each command is done by the second method, EvaluateCommand(). What should be implemented is the SMTP dialog as described in the first part of this article. Of course, the length of EvaluateCommand() has reached its limit and should be split up into more methods when you implement more commands. But for a sample it’s easiest to find all in one place. I think it’s really not necessary to talk about each line of code because it contains the simplest read and write operations. Therefore, I suggest you take the code and work on it yourself.

Listing 1: Implementation of SMTPProcessor Class

public class SMTPProcessor {
	
  private const string _mailFolder = @"C:\temp\";
  private Socket _socket;
  private bool _quitRequested = false;
  private bool _dataPartStarted = false;
  private Guid _messageID = Guid.NewGuid();
  private StreamWriter _outputStream;

  /// <summary>
  /// Initializes object for given socket.
  /// </summary>
  /// <param name="socket">Socket with established connection</param>
  public SMTPProcessor(Socket socket) {
    _socket = socket;			
  }

  /// <summary>Starts communication with client.</summary>
  public void Process() {
		
    string clientMessage = string.Empty;
    string serverMessage = string.Empty;

    NetworkStream networkStream = new NetworkStream(_socket);
    StreamReader streamReader = new StreamReader(networkStream);
    StreamWriter streamWriter = new StreamWriter(networkStream);

    _outputStream = File.CreateText(_mailFolder + _messageID.ToString() +       
                                    ".txt");
		
    try {			
      // All writes will be done immediately and not cached:
      streamWriter.AutoFlush = true;

      // Send welcome message first:
      string welcome = "220 " + System.Environment.MachineName + " 
                       TheServerSide.NET Demo Mail Service ready at " + 
                       DateTime.Now.ToString();

      streamWriter.WriteLine(welcome);

      // Start loop and handle commands:
      while (!_quitRequested) {
        clientMessage = streamReader.ReadLine();					
        serverMessage = EvaluateCommand(clientMessage);	
							
        if (serverMessage != "")
          streamWriter.WriteLine(serverMessage);							
      }
    }
    catch (Exception ex) {
      Console.WriteLine("An Exception occured: " + ex.ToString());
    }
    finally {
      streamWriter.Close();
      streamReader.Close();
      networkStream.Close();
      _socket.Close();
      _outputStream.Close();
    }
  }

  /// <summary>
  /// Evaluates incoming request and returns server's command.
  /// </summary>
  /// <param name="clientMessage">Command to evaluate.</param>
  /// <returns>Message to send to client</returns>
  private string EvaluateCommand(string clientMessage) {
    string serverMessage = string.Empty;

    // *** DATA ***
    if (clientMessage.ToUpper() == "DATA") {
      _dataPartStarted = true;
      serverMessage = "354 Please start mail input; end with <CRLF>.<CRLF>";
    }	
    else if (_dataPartStarted  && clientMessage == ".") {
      _dataPartStarted  = false;								
      serverMessage = "250 Mail queued for delivery. MessageId: " + 
                      _messageID.ToString();
    }
    else if (_dataPartStarted ) {
      _outputStream.WriteLine(clientMessage);
    }			
	
      // *** HELO ***
    else if (clientMessage.ToUpper().StartsWith("HELO ") || 
             clientMessage.ToUpper() == "HELO") {				
      string serverAddress = 
      serverMessage = "250 " + System.Environment.MachineName + " Hello [" +                   
      Dns.Resolve(System.Environment.MachineName).AddressList[0].ToString() + 
      "]";				
    }
		
      // *** QUIT ***				
    else if (clientMessage.ToUpper() == "QUIT") {
      serverMessage = "221 Closing connection. Good bye!";
      _quitRequested = true;
    }

      // *** MAIL FROM ***				
    else if (clientMessage.ToUpper().StartsWith("MAIL FROM:")) {
      string mailFrom = clientMessage.Substring(clientMessage.IndexOf(":")+1);
      _outputStream.WriteLine(clientMessage);
      serverMessage  = "250 2.1.0 " + mailFrom + "....Sender OK";				
    }

      // *** RCPT TO ***				
    else if (clientMessage.ToUpper().StartsWith("RCPT TO:")) {
      string mailTo = clientMessage.Substring(clientMessage.IndexOf(":") + 1);
      _outputStream.WriteLine(clientMessage);
      serverMessage  = "250 2.1.5 " + mailTo;
				
    }
    return serverMessage;
  }
}

Starting the Server

Now all classes are built and the server is ready to be started. This is done in a very few lines of code and needn’t any comments. You will find the whole application for download at [3].

class ServerApp {    
  [STAThread]
  static void Main(string[] args) {
    Server server = new Server(System.Net.IPAddress.Any, 25);
    server.Start();		

    Console.WriteLine("Press enter to stop server");
    string input = Console.ReadLine();
    server.Stop();			
  }
}

Sometimes it’s good to have a starting point for new technologies and this article should be the one for socket programming. This email server sample can easily be extended. Maybe you need to process emails automatically, e.g. for a help desk application or you want to write your own list server. Take this sample and build it up! – A POP3 and HTTP server or your own email client will also be based on these fundamentals of socket programming. Combine the parts as needed, lookup RFCs specifying the details and explore the internet on the server side.

Resources

[1] http://www.ietf.org/rfc/rfc0821.txt
[2] http://www.ietf.org/rfc/rfc0822.txt
[3] Download the Sample Code from this Article

Authors

Sebastian Weber is Software Engineer at Platinion GmbH in Germany, A Company of The Boston Consulting Group. He's specialized in building .NET-based applications. Besides this, he's known as author and speaker in the field of .NET and Microsoft Server technologies. Sebastian can be contacted via http://weblogs.asp.net/SebastianWeber.