Sent and Received Time on a Message and EWS

This one has come up a couple of times for me over the last couple of weeks so I thought I'd put together a post to expand on the Subject a little.

The easiest place to start before talking about Exchange is to look at a ordinary MIME message and its associated headers. In a MIME message there is one Message date  header http://tools.ietf.org/html/rfc4021#section-2.1.1 which outlines "Specifies the date and time at which the creator of the message indicated that the message was complete and ready to enter the mail delivery system" which is a way of saying that its the Sent Time. eg In a Message

Subject: Re: Oh no
Date: Thu, 14 Aug 2014 18:44:52 +1000
Message-ID: <nk0h69wn6waj8s32rnc3kma0.1408005882691@email.android.com>

 As the Message traverses various MTA's along the way to its final destinations, Received headers with dates are added to the Transport Headers  of a message indicating the date\time a particular MTA's processed the message eg

Received: from BY2PR03MB459.namprd03.prod.outlook.com (10.141.141.147) by
 DM2PR03MB463.namprd03.prod.outlook.com (10.141.85.20) with Microsoft SMTP
 Server (TLS) id 15.0.859.15 via Mailbox Transport; Wed, 29 Jan 2014 22:06:05
 +0000

When the Message finally arrives at is destination and is delivered by the Store to a users Mailbox two MAPI properties will be created to  reflect the Sent Time and also the Date Time the message was delivered by the store (which should match (most recent) received header in the message). With POP3 and and some POP downloaders this is where the date can get a little offset and not represent the real time of message delivery. But sticking to Exchange the following two props get set

PR_MESSAGE_DELIVERY_TIME which in EWS is also represented by the Strongly typed DateTimeReceived property

PR_CLIENT_SUBMIT_TIME property which in EWS is represented by the Strongly typed DateTimeSent property.

Exchange when it stores these dates like with other dates will store these in UTC format. When your using Outlook with the default views it will use the PR_MESSAGE_DELIVERY_TIME for every Mail folder other then the Sent Items where the view will use the PR_CLIENT_SUBMIT_TIME.

So whenever you importing EML's and you see an unexpected Received (or Sent) by date the first thing to check is the Transport Headers and look at the most recent received header. If your missing these headers then that maybe why your dates aren't what you expect.

In EWS you can access the Transport Headers via the InternetMessageHeaders strongly typed property or you can use the PR_TRANSPORT_MESSAGE_HEADERS extended properties. Depending on the version of Exchange you are using there can be issues with the strongly typed property so you should read http://msdn.microsoft.com/EN-US/library/office/hh545614(v=exchg.140).aspx . Because of the size of these properties this information will only be returned when you use a GetItem's operation

The other thing you can do with these message dates is track the amount of time it took from submit to the delivery of a message eg the following script will use EWS to grab both the PR_MESSAGE_DELIVERY_TIME  and PR_CLIENT_SUBMIT_TIME MAPI properties and use those to calculate the delivery time is also parses the Sent Header datetime and creates a CSV of the output. I've put a download of this script here the code looks like

1. ## Get the Mailbox to Access from the 1st commandline argument  
2.   
3. $MailboxName = $args[0]  
4.   
5. ## Load Managed API dll    
6.   
7. ###CHECK FOR EWS MANAGED API, IF PRESENT IMPORT THE HIGHEST VERSION EWS DLL, ELSE EXIT  
8. $EWSDLL = (($(Get-ItemProperty -ErrorAction SilentlyContinue -Path Registry::$(Get-ChildItem -ErrorAction SilentlyContinue -Path 'Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Exchange\Web Services'|Sort-Object Name -Descending| Select-Object -First 1 -ExpandProperty Name)).'Install Directory') + "Microsoft.Exchange.WebServices.dll")  
9. if (Test-Path $EWSDLL)  
10.     {  
11.     Import-Module $EWSDLL  
12.     }  
13. else  
14.     {  
15.     "$(get-date -format yyyyMMddHHmmss):"  
16.     "This script requires the EWS Managed API 1.2 or later."  
17.     "Please download and install the current version of the EWS Managed API from"  
18.     "http://go.microsoft.com/fwlink/?LinkId=255472"  
19.     ""  
20.     "Exiting Script."  
21.     exit  
22.     }  
23.     
24. ## Set Exchange Version    
25. $ExchangeVersion = [Microsoft.Exchange.WebServices.Data.ExchangeVersion]::Exchange2010_SP2    
26.     
27. ## Create Exchange Service Object    
28. $service = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService($ExchangeVersion)    
29.     
30. ## Set Credentials to use two options are availible Option1 to use explict credentials or Option 2 use the Default (logged On) credentials    
31.     
32. #Credentials Option 1 using UPN for the windows Account    
33. $psCred = Get-Credential    
34. $creds = New-Object System.Net.NetworkCredential($psCred.UserName.ToString(),$psCred.GetNetworkCredential().password.ToString())    
35. $service.Credentials = $creds        
36.     
37. #Credentials Option 2    
38. #service.UseDefaultCredentials = $true    
39.     
40. ## Choose to ignore any SSL Warning issues caused by Self Signed Certificates    
41.     
42. ## Code From http://poshcode.org/624  
43. ## Create a compilation environment  
44. $Provider=New-Object Microsoft.CSharp.CSharpCodeProvider  
45. $Compiler=$Provider.CreateCompiler()  
46. $Params=New-Object System.CodeDom.Compiler.CompilerParameters  
47. $Params.GenerateExecutable=$False  
48. $Params.GenerateInMemory=$True  
49. $Params.IncludeDebugInformation=$False  
50. $Params.ReferencedAssemblies.Add("System.DLL") | Out-Null  
51.   
52. $TASource=@' 
53.   namespace Local.ToolkitExtensions.Net.CertificatePolicy{ 
54.     public class TrustAll : System.Net.ICertificatePolicy { 
55.       public TrustAll() {  
56.       } 
57.       public bool CheckValidationResult(System.Net.ServicePoint sp, 
58.         System.Security.Cryptography.X509Certificates.X509Certificate cert,  
59.         System.Net.WebRequest req, int problem) { 
60.         return true; 
61.       } 
62.     } 
63.   } 
64. '@   
65. $TAResults=$Provider.CompileAssemblyFromSource($Params,$TASource)  
66. $TAAssembly=$TAResults.CompiledAssembly  
67.   
68. ## We now create an instance of the TrustAll and attach it to the ServicePointManager  
69. $TrustAll=$TAAssembly.CreateInstance("Local.ToolkitExtensions.Net.CertificatePolicy.TrustAll")  
70. [System.Net.ServicePointManager]::CertificatePolicy=$TrustAll  
71.   
72. ## end code from http://poshcode.org/624  
73.     
74. ## Set the URL of the CAS (Client Access Server) to use two options are availbe to use Autodiscover to find the CAS URL or Hardcode the CAS to use    
75.     
76. #CAS URL Option 1 Autodiscover    
77. $service.AutodiscoverUrl($MailboxName,{$true})    
78. "Using CAS Server : " + $Service.url     
79.      
80. #CAS URL Option 2 Hardcoded    
81.     
82. #$uri=[system.URI] "https://casservername/ews/exchange.asmx"    
83. #$service.Url = $uri      
84.     
85. ## Optional section for Exchange Impersonation    
86.     
87. #$service.ImpersonatedUserId = new-object Microsoft.Exchange.WebServices.Data.ImpersonatedUserId([Microsoft.Exchange.WebServices.Data.ConnectingIdType]::SmtpAddress, $MailboxName)   
88.   
89. # Bind to the Inbox Folder  
90. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)     
91. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)  
92.   
93. $PR_MESSAGE_DELIVERY_TIME = new-object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition(0x0E06, [Microsoft.Exchange.WebServices.Data.MapiPropertyType]::SystemTime)  
94. $PR_CLIENT_SUBMIT_TIME = new-object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition(0x0039, [Microsoft.Exchange.WebServices.Data.MapiPropertyType]::SystemTime)  
95. $PR_TRANSPORT_MESSAGE_HEADERS = new-object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition(0x007D,[Microsoft.Exchange.WebServices.Data.MapiPropertyType]::String);  
96. $psPropset= new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::IdOnly)    
97. $psPropset.Add($PR_CLIENT_SUBMIT_TIME)  
98. $psPropset.Add($PR_MESSAGE_DELIVERY_TIME)  
99. $psPropset.Add($PR_TRANSPORT_MESSAGE_HEADERS)  
100. $psPropset.Add([Microsoft.Exchange.WebServices.Data.ItemSchema]::Subject)  
101. #Define ItemView to retrive just 1000 Items      
102. $ivItemView =  New-Object Microsoft.Exchange.WebServices.Data.ItemView(1000)    
103. $ivItemView.PropertySet = $psPropset  
104. $rptCollection = @()  
105. $fiItems = $null      
106. do{      
107.     $fiItems = $service.FindItems($Inbox.Id,$ivItemView)      
108.     [Void]$service.LoadPropertiesForItems($fiItems,$psPropset)    
109.     foreach($Item in $fiItems.Items){      
110.         $Headers = $null;  
111.         $ClientSubmitTime = $null  
112.         $DeliveryTime = $null  
113.         [Void]$Item.TryGetProperty($PR_CLIENT_SUBMIT_TIME,[ref]$ClientSubmitTime)  
114.         [Void]$Item.TryGetProperty($PR_MESSAGE_DELIVERY_TIME,[ref]$DeliveryTime)  
115.         if($Item.TryGetProperty($PR_TRANSPORT_MESSAGE_HEADERS,[ref]$Headers)){  
116.             $slen = $Headers.ToLower().IndexOf("`ndate: ")  
117.             if($slen -gt 0){  
118.                 $elen = $Headers.IndexOf("`r`n",$slen)  
119.                 $TimeSpan =  NEW-TIMESPAN –Start $ClientSubmitTime –End $DeliveryTime   
120.                 $rptobj = "" | select Subject,HeaderDate,DELIVERY_TIME,SUBMIT_TIME,Diff  
121.                 $rptobj.Subject = $Item.Subject  
122.                 $parsedDate = $Headers.Substring(($slen+7),($elen-($slen+7)))                 
123.                 $rptobj.HeaderDate = [DateTime]::Parse($parsedDate).ToLocalTime()  
124.                 $rptobj.DELIVERY_TIME = $DeliveryTime.ToLocalTime()  
125.                 $rptobj.SUBMIT_TIME = $ClientSubmitTime.ToLocalTime()  
126.                 $rptobj.Diff = [Math]::Round($TimeSpan.TotalMinutes,0)  
127.                 $rptCollection += $rptobj  
128.             }  
129.         }        
130.     }      
131.     $ivItemView.Offset += $fiItems.Items.Count  
132.     Write-Host ("Processed " + $ivItemView.Offset + " of " + $fiItems.TotalCount)  
133. }while($fiItems.MoreAvailable -eq $true)   
134. $rptCollection | Export-Csv -NoTypeInformation -Path "c:\Temp\$mailboxName-mTimes.csv"  
135. Write-Host("Exported to c:\Temp\$mailboxName-mTimes.csv")