How IT Works: How Simple is SMTP?

Article
09/08/2016

How IT Works How Simple is SMTP?

R'ykandar Korra'ti

SMTP, the Simple Mail Transfer Protocol, carries the electronic mail of the world. While other message transferring systems exist—some more efficient, some better at specific tasks, some privately owned and some public—none have won the widespread public acceptance of the venerable SMTP, first defined in RFC 821, all the way back in 1982.

But how does SMTP work? The "Simple" part of the name is pure truth in advertising. A message can be moved from one SMTP server to another using one of the simplest TCP/IP protocols (line-oriented telnet) and just a few simple text commands. One SMTP server makes a telnet-like connection to TCP port 25 on the other; it then sends commands and data, receiving numeric responses.

This process can be simulated with any telnet client. In fact, in the early days of the grand network unification of ARPAnet, Bitnet, Edunet, and several other major networks, a lot of testing was done this way, with developers connecting to an SMTP server and typing commands at it, just to see what would happen. To do this yourself, run telnet from a command prompt and connect to an online SMTP mail server on TCP port 25. The server will send back a response that looks a lot like this:

C:\> telnet newmoon.microsoft.net 25
Trying 127.0.0.1...
Connected to newmoon.microsoft.net.
Escape character is ‘^]’.
220 newmoon.microsoft.net Microsoft ESMTP MAIL Service, Version: 
6.0.3790.1830 ready at  Sun, 15 May 2005 22:56:26 –0700

As easily as that, the client machine is ready to talk to that SMTP server as if it were an SMTP peer. Type commands, and they’ll be processed and responded to with numeric response codes.

These three-digit coded responses, in this case, 220, may look familiar to Webmasters. The first digit indicates success or failure. 1 indicates a success requiring confirmation, 2 indicates complete success, and 3 indicates success so far, but that more input is expected. 4, the first failure code, is used for failures that may be temporary, such as the mail account existing but being locked; 5 indicates that the failure condition is permanent, such as a syntax error in the command. The second and third digits provide further details, but at the top level, all that matters is that 1xx, 2xx, and 3xx are good, and 4xx and 5xx indicate problems. (For further information on mail response codes, read RFC 1893.)

Issuing Commands

EHLO (or HELO) identifies the server initiating the connection (see the sidebar "The Difference Between HELO and EHLO?" to understand the difference between the two). It also tells the receiving server to clear its slate. The format is:

EHLO <network name of connecting machine>

The parameter is mandatory on all recent SMTP servers, despite the fact that all modern servers will also use DNS to identify your actual domain and machine name, if possible. It will also log the IP address before responding with another three-digit response code (250, in most cases), and a greeting message:

250 newmoon.microsoft.net Hello [127.0.0.1]

The receiving machine has now identified the sending server and is ready to continue.

The sending machine must next provide the mail’s individual sender—that might be the person on the From: line, but it often isn’t; it could, for example, be a mailing list software package, a redirector, or some other agent. This is envelope data, rather than header data:

MAIL FROM: <email address>

The e-mail address must be in a valid format, and with any modern e-mail server, must indicate a valid domain. Invalid MAIL FROM: addresses will be rejected, thusly:

MAIL FROM: sajkdflaef
553 5.5.4 sajkdflaef... Domain name required for sender address sajkdflaef
MAIL FROM: asfe@asefsad.asf
553 5.1.8 asfe@asefsad.asf... Domain of sender address asfe@asefsad.asf does not exist
MAIL FROM: sample@microsoft.net
250 2.1.0 sample@microsoft.net... Sender ok

Next, the address (or list of addresses) to receive the messages must be specified, as arguments to the RCPT TO: command:

RCPT TO: <email address>

Only one e-mail address may be specified with this command, but the command may be repeated as many times as needed with different addresses as parameters:

RCPT TO: sample@microsoft.net
250 2.1.5 sample@microsoft.net... Recipient ok
RCPT TO: test@microsoft.net
250 2.1.5 test@microsoft.net ... Recipient ok

Once all addresses have been specified, it’s time to send the actual mail! This includes the actual header as seen by the mail client, then a blank line, then the message body and any attachments. Describing all that is well beyond the scope of this column (see RFC 2822, for starters) but the command to tell the server that it’s time for the actual message isn’t. It’s one of the simplest SMTP commands:

DATA

There are no parameters for this command. Barring error, the server will reply:

354 Enter mail, end with "." on a line by itself

(Note a 3xx success code: "successful so far, more input expected.") Then the message header and body should be sent, followed by a period on a line by itself indicating message end. If all goes well, the server will respond to the standalone period with this response code:

250 2.0.0 Message accepted for delivery

And the message is on its way! The connecting server can then start another message (with another MAIL FROM: command), send other SMTP commands, or end the session by sending a QUIT command on a line by itself:

QUIT
221 2.0.0 microsoft.net closing connection

QUIT must prompt a success error code and connection drop by the server. And that’s all you need to move a piece of mail.

However, there are several other standard commands available: NOOP, RSET, VRFY, and EXPN. These are neither optional nor obsolete, though they are of somewhat limited use in the modern corporate environment. For a complete list of all commands, including obsolete verbs, see RFC 2821, RFC 1123, and RFC 821, available via MSN® Search. Remember that later RFCs supersede earlier ones, and that some commands have been made obsolete, like the TURN, SEND, SOML, and SAML commands described in RFC 821. The simplest of these commands is NOOP, useful mostly in testing to avoid timeouts; it does nothing and can generate only a successful response, with no change in state, as shown here:

NOOP
250 2.0.0 OK

RSET clears the MAIL FROM: and RCPT TO: address stacks, throwing out any message delivery in progress, and resetting the connection back to the state it was in just after HELO or EHLO. The command should never cause the server to disconnect or require a new EHLO/HELO exchange. It takes no parameters:

RSET
250 2.0.0 Reset state

The Difference Between HELO and EHLO

EHLO indicates that the client supports at least one of several possible extensions that are not part of the basic SMTP specification. HELO indicates that it does not, and that only the minimum SMTP set will be used when talking to this server. Many of these extensions are common and standardized; others may be local to the particular SMTP host only, in which case, the extended commands must begin with the letter X. All extended commands the server supports must be listed as part of the server’s response to EHLO, one per line, each line prefixed with code 250. Typical uses of extended commands are to transfer binary information in non-encoded form, and to tunnel another protocol through SMTP.

The command VRFY allows clients to validate individual e-mail addresses. Many postmasters disabled this command early in the spam wars, since some spammers began using it to clean their mailing lists. Support in some form, however, is still required by the SMTP specification. For servers supporting it as described in the RFC, the format is:

VRFY <email address>

VRFY submitted with a valid address returns a canonical expansion of the e-mail address given, complete with friendly name. An invalid address, by contrast, triggers an error code 550:

VRFY dara@microsoft.net
250 2.1.5 Dara Korra’ti <dara@microsoft.net>

VRFY bogus-address@microsoft.net
550 5.1.1 bogus-address@microsoft.net ... User unknown

Finally, EXPN historically provided a way to expand a mailing list into its component members. This allowed e-mail clients and servers to identify all the members of such a list. As can be easily imagined, this command, too, was an early casualty of the spam wars. Independently of that, it also became less and less useful as mailing list management software changed, and became less integrated with the MTA (or Message Transfer Agent). Many third-party mailing list management software packages do not provide mail servers with ways of supporting EXPN correctly at all. However, servers that do support it as intended should return all the e-mail addresses of the list members, each prefixed with a success code, or a 550 failure if the list name doesn’t exist:

EXPN mylist
250 2.1.5 sample1@microsoft.net
250 2.1.5 sample2@microsoft.net
250 2.1.5 sample3@microsoft.net

EXPN bogus-list
550 5.1.1 bogus-list ... User unknown

Collectively, these commands complete the minimum required set for SMTP servers. Simple, isn’t it?

R'ykandar Korra'ti, postmaster for a small co-op ISP, lives near Seattle with her partner Anna. Having previously shipped mail products at Microsoft, she is now looking at grad school in a CS-related field so esoteric it doesn’t really have a name. Potential faculty advisors can reach her at darako@murkworks.net.

Additional resources