Microsoft Exchange 2013 introduces a number of fundamental changes in architecture compared to Exchange 2010 and 2007. In this article, we’ll have a look at Exchange 2013 Mail Flow and its differences from prior versions of Microsoft Exchange, and review how messages are routed through Exchange 2013 to their final destination. Finally, we’ll trace the complete path of a message through all the different services of Exchange 2013 and talk about Exchange message queues.
Exchange 2013 introduces a number of important architectural changes compared to previous versions of Microsoft Exchange. While some of these changes only affect internal mail flow, some others can and do affect the delivery of messages. Either way, these changes must be taken into consideration if you are writing applications running on the Exchange platform.
In order to understand how e-mail flows in the new architecture presented in Exchange 2013, one must first understand the new server roles available in Exchange 2013. In its latest edition, Microsoft Exchange Server has dropped support for all server roles except just two:
A number of things were removed from Exchange 2013, encapsulated or reassigned to different entities. Let’s see what these are.
MAPI services are no longer offered. In this edition of Exchange Server, MAPI connections are encapsulated using RPC-over-HTTPS, enabling clients from the Internet to securely access a Microsoft Exchange Server without having to log into a VPN first.
One more thing that is missing from Exchange 2013 is Edge Server. Services previously provided by the Edge Server are now delivered via the Client Access Server. However, if your legacy infrastructure is highly dependent on the use of the Edge Server, there is nothing to prevent you from using a copy of Edge Server from an Exchange 2007 or 2010 installation within your Exchange 2013 infrastructure.
Architectural changes in Exchange 2013 make Client Access Server and Mailbox Server less dependent on one another compared to previous versions. The Mailbox server now handles all processing and activities persisting to mailboxes, while the Client Access server handles client authentication and redirection and serves as a proxy. This is logical, as an active database copy of a mailbox physically resides on the Mailbox Server. Delegating all mailbox activities to a Mailbox server ensures that all data rendering takes place on a local copy of the mailbox database, reducing the number of interactions between the Mailbox and Client Access servers and eliminating concerns of version compatibility between the two servers.
In Exchange 2013, mail flows through the Transport Pipeline. In this edition, the Transport Pipeline consists of the following three services:
This service runs on all Client Access servers. It handles all external SMTP traffic, acting as a stateless proxy for all inbound and outbound connections. The Front End Transport service has the following properties:
The Transport service runs on all Mailbox servers, taking place of the Hub Transport server role that was available in previous versions of Exchange and removed in Exchange 2013. The Transport service handles all SMTP traffic for the organization. The Transport Service has the following properties:
The Mailbox Transport Service runs on all Mailbox servers. It and consists of two separate services:
As I already mentioned, the Exchange 2013 installation no longer includes the Edge Server. As a result, email messages from the Internet can be handled in one of the following ways:
Obviously, Microsoft intends Exchange customers to make use of the Client Access Server. In this scenario, external emails enter the Exchange organization through a Receive connector in the Front End Transport service, and are then routed to the Transport service on a Mailbox server.
Unlike Exchange 2010 and 2007, Exchange 2013 can receive emails from the Internet in its default configuration. The Client Access server comes equipped with a pre-configured Receive Connector enabling organizations to receive emails from the Internet. The Receive Connector is named “Default Frontend ”, which already accepts anonymous connections by allowing “Anonymous Users” to connect:
There are four ways for a message within the organization to enter the Transport service on a Mailbox server:
The Transport Service itself consists of the following components and processes:
The following diagram summarizes everything described above, clearly showing the components and the relationships between them.
The role of Transport Agents has not changed since the previous version of Exchange. Transport Agents are custom extensions that process email messages passing through the transport pipeline.
Organizations can create their own custom agents, or use Transport Agents supplied by Microsoft or a third-party software vendor.
Transport Agents can be used, for example, to intercept all messages received via the SMTP protocol and convert their format into plain text, remove all attachments or add a legal disclaimer at the bottom of each message. Note that Transport Agents have full access to all email messages that they encounter. Exchange applies no sandboxing, no rules and no restrictions on agents’ behavior. Transport Agents obtained from an unknown source may present a security or stability threat to the entire Exchange organization. A good advice would be only using Transport Agents obtained from a known, trusted source or created – and tested – within the organization to fulfill a particular purpose.
There are several types of Transport Agents available: SmtpReceiveAgent, RoutingAgent and DeliveryAgent. Exchange 2013 comes with a number of built-in transport agents. Most of these agents are invisible and cannot be managed. Transport Agents that are visible can be viewed with the Get-TransportAgent cmdlet:
We have already reviewed the changes in Exchange 2013 general architecture and the Mail Flow in particular as compared to Exchange 2007 and 2010. Now let’s have a look at mail routing and categorization in Exchange 2013. This part will be mostly theoretical, so it has lots of text and few pictures. You are encouraged to read this part if you want to understand how messages are routed through Exchange 2013 to their final destination. If you don’t want the theory, you may skip directly to the last part where we will trace the complete path of a message through all the different services of Exchange 2013.
Messages are routed to their final destination with the Transport service which present on all mailbox servers. In order to decide on how to route a particular message, the Transport server must perform categorization, which is an important component of the Transport service. Categorization processes all incoming mail, determining actions to perform based on their final destination. As described in Part I of this series, Exchange 2013 implemented a number of architectural changes. Due to these changes, the Transport service is now hosted on all Mailbox servers. As a result, the routing process becomes more intelligent. The routing process is now fully aware of Database Availability Groups (DAG). DAG membership is used as a routing boundary.
What does that mean in practice? Using the Active Directory site as the primary routing boundary becomes inefficient if the DAG spans multiple Active Directory sites. For this reason, if a Mailbox server belongs to a certain DAG, message routing becomes closely aligned with the DAG.
Of course, if a Mailbox server does not belong to a DAG, Exchange 2013 continues using Active Directory site membership as a routing boundary, just as in previous versions of Exchange.
Due to the architectural changes made in Exchange 2013, the routing process changes significantly compared to earlier versions of Microsoft Exchange.
All messages received by the Transport service must be categorized for future routing. The categorization process resolves the recipient first. After resolving the recipient, the ultimate destination of the message can be determined. After that, Exchange performs message routing by determining the path to the final destination, which is also called the Routing Destination. The Routing Destination is determined as follows:
Exchange 2013 introduces Delivery Groups. In Exchange 2013 architecture, each routing destination has one or several associated transport servers (either an Exchange 2013 Mailbox server or an Exchange 2010 Hub Transport server). These transport servers are called Delivery Groups. Delivery groups are responsible for routing the message to its ultimate destination.
To make things even more complicated, the transport servers in the delivery group can be of different versions. If the routing destination is a mailbox database, the version of transport servers in the delivery group is the same version of Exchange as the mailbox database. If, however, the routing destination is a connector or a distribution group expansion server, the delivery group may consist of a mixture of Exchange 2013 Mailbox servers and Exchange 2010 Hub Transport servers.
Exchange 2013 defines 5 types of delivery groups:
Being part of one delivery group does not automatically exclude membership in other groups. For example, an Exchange 2013 Mailbox server can be a member of a DAG and, at the same time, can be the source server of a scoped Send connector. Аor mailbox databases in the DAG, this Mailbox server belongs to the routable DAG delivery group. For the scoped Send connector it serves as a connector source server delivery group.
When delivering a message to a remote delivery group, Exchange 2013 must determine the correct routing path in order to be able to perform the task. The algorithm has not changed since version 2010:
OK, enough theory! Now when you understand how Exchange 2013 uses Routing Destination and Delivery Groups to route messages to their correct destination, you are ready for the next part. At this point, we’ll trace a message through all the hoops of Exchange 2013 mechanisms, discuss how the Front End Transport and Mailbox Transport services route messages, and have a closer look at Queues.
The Front End Transport service is executed on all Client Access servers. This service handled all inbound and outbound SMT traffic originating from or addressed to outside of Exchange organization, serving as a stateless proxy.
To process outgoing messages, the Transport service on Mailbox servers communicates with the Front End Transport Service by invoking Send connectors. The Front End Transport service proxies outgoing messages when the FrontEndProxyEnabled parameter on an applicable Send connector in the Exchange Management Shell is set to $True, or when the “Proxy through client access server” option is selected in the Exchange Administration Center:
To process incoming messages, the Front End Transport service attempts to identify a healthy Transport service on a Mailbox server to receive the message. As the Front End Transport service does not queue messages, it assumes the email service is not available and fails to deliver a message if no single healthy Transport service is found.
Message recipients are resolved to their corresponding mailbox databases by querying Active Directory. The Front End Transport service looks up the delivery group and associated routing information for each mailbox database. The delivery group can be a Routable DAG, a Mailbox delivery group or an Active Directory site.
Depending on the number and type of message recipients, Front End Transport service can do one of the following actions (Table 1):
|Messages with a single mailbox recipient||A Mailbox server in the target delivery group is selected. The preference is given to the closest Mailbox server based on the proximity of the Active Directory site.|
|Messages with multiple mailbox recipients||The first 20 recipients are used to select a Mailbox server in the closest delivery group based on the proximity of the Active Directory site. A single Mailbox server is always selected as message bifurcation does not occur in Front-End Transport.|
|Message with no mailbox recipients||A random Mailbox server is selected in the local Active Directory site.|
The Mailbox Transport Delivery service delivers incoming mail by receiving SMTP messages from the Transport service and connecting to the local mailbox database via Remote Procedure Call (RPC).
Outgoing messages are retrieved from the local mailbox database via RPC by the Mailbox Transport Delivery service. The messages are then submitted to the Transport service via SMTP.
The Mailbox Transport Service is stateless, and does not queue messages locally.
What happens when a user sends an email message? First, the message recipients are resolved to corresponding mailbox databases by the Mailbox Transport Submission service by querying Active Directory. The associated routing information is retrieved by looking up the delivery group for each mailbox database. The delivery group can be a Routable DAG, a Mailbox delivery group or an Active Directory site..
Depending on the number and type of recipients, the Mailbox Transport Submission service performs one of the actions described in Table 1.
As mentioned before, the Mailbox Transport Delivery is stateless and does not queue any messages locally. As a result, messages are accepted or rejected for delivery to a local mailbox database on the spot, based on whether or not the recipient resides in an active copy of a local mailbox database.
If the recipient does not reside in an active copy of a local mailbox database, the Mailbox Transport Delivery service fails to deliver the message, providing an instant non-delivery response to the Transport service. This could happen if, for example, the active copy of the mailbox database was recently moved to a different server and the message was incorrectly routed by the Transport service to a Mailbox server holding an inactive copy of the mailbox database.
If this happens, the non-delivery response generated by the Mailbox Transport Delivery can be one of the following:
Now let’s have a closer look at Front End Transport and Mailbox Transport services route messages. We’ll have a look at how messages travel through all the services and processes by tracing the complete path of an email.
Similar to the previous version, Exchange 2013 generates Delivery Reports that can be used to track messages sent or received in an Exchange 2014 organization. Note that we can only trace messages sent using Outlook or Outlook Web App, but not via POP or IMAP.
However, Exchange 2013 offers an even better way to trace messages via Message Tracking Logs. As you already know, Exchange 2013 no longer provides a dedicated Hub Transport Server role. For this reason, the TransportServer cmdlet is replaced with two new cmdlets:
So let’s trace a message sent from the internal user Test1 with a mailbox on server MBX1 to the internal user Test2 with a mailbox on server MBX2. The following cmdlet must be executed in order to obtain information about this particular message. Note that we could use either one of the two cmdlets described above.
Get-TransportService | Get-MessageTrackingLog –ResultSize Unlimited –Start “01/09/2013” –Sender email@example.com -Recipients firstname.lastname@example.org –MessageSubject “Mail flow example” | Sort TimeStamp | Select ClientHostname, ServerHostname, ConnectorId, Source, EventId
The cmdlet returns the names of the Mailbox servers possibly involved in the mail flow. Analyzing the output of the cmdlet, we can make the diagram Figure 2:
Finally, we end up with the following diagram:
Finally, let’s talk about the last thing we have left: the queues. Exchange 2013 implements several architectural changes compared to earlier versions of Exchange. Exchange 2007/2010 Edge servers do have queues, while in Exchange 2013 most services involved in mail flow do not queue messages. Out of the three services, only the Transport service maintains message queues. The Front End Transport service and the Mailbox Transport service do not provide local queuing.
What does it mean in practice? If you are using Client Access server as your primary email gateway, you must clearly understand that if there are no mailbox servers available to receive messages, the sender’s server will be issued a delivery failure notification immediately. For regular mail this may not pose a critical problem as most email servers will make several attempts to deliver a message (typically during the 24, 48 or 72 hour period). However, with applications or devices other than mail servers this may lead to message failures.
If you have a custom Receive Connector for applications, printers, or other devices in an Exchange 2013 organization, you may want to create that Receive Connector on your Mailbox servers instead of the Client Access servers.
This article concludes our series discussing mail flow in Exchange 2013. We covered all the architectural changes introduced into the flow of messages in Exchange 2013, traced an email message from start to finish through all processes and services participating in Exchange 2013 Mail Flow. Finally, we reviewed Queues in Exchange 2013, noting their limitations and suggesting a workaround for users of custom Receive Connectors.