Skip to main content

EWS Managed API and Powershell How-To series Part 3 - Items

This is Part3 in my continuing EWS Managed API and Powershell how to series, in this post im going to look at using Mailbox Items in Exchange Web Services. The single most important thing obviously within a mailbox is the content and one of Exchanges main strengths is the richness and flexibility of the content and item types that it can store and process.  So in this post im going to cover the complexities of processing the different Items that are stored in a mailbox in different way to solve various everyday problems.

Items, Objects and Properties

Some of the Items you might find in an Exchange Mailbox include Email, Contacts, Calendar Appointments, Meeting, Tasks and other custom Item types. The data that constitutes an Exchange Item for example the Subject of an email message is stored as one or more properties in the Exchange Mail-Store associated with that Item. Looking at the Subject example a little more deeply the underlying Property for the subject is PidTagSubject http://msdn.microsoft.com/en-us/library/cc815720.aspx  .  This sounds easy but with Exchange Items there aren't just one level of properties for example an Email will generally have a collection of properties for the Item, then every recipient is a separate object with its own properties and then possible one more attachments with their own properties on each attachment.

This maybe a little too much detail but it is important to consider when you need to do more complex things with EWS for the most part the Managed API abstracts out these underlying properties into Strongly type objects for example EmailMessage, Contact or Appointment. For a quick example lets look at two ways of accessing the same property. Eg to access the Subject using the Strongly typed Subject Property then using the Extended Mapi property PidTagSubject. To put that another way its just two ways of accessing the same piece of stored data.

  1. #Define Property Set  
  2. $psPropertySet = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties)  
  3. $PidTagSubject = new-object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition(0x0037,[Microsoft.Exchange.WebServices.Data.MapiPropertyType]::String);  
  4. $psPropertySet.Add($PidTagSubject)  
  5.   
  6. $ItemId = New-Object Microsoft.Exchange.WebServices.Data.ItemId($ewsid.UniqueId)  
  7. $msMessage = [Microsoft.Exchange.WebServices.Data.EmailMessage]::Bind($service,$ItemId,$psPropertySet)    
  8. "Strong Type : " + $msMessage.Subject  
  9. $SubjectVal = $null  
  10. if($msMessage.TryGetProperty($PidTagSubject,[ref]$SubjectVal)){  
  11.     "Extended Property : " + $SubjectVal  
  12. }  
 Accessing an Items data

When you access an Item with EWS your getting data from one or more properties on that Item in EWS there are two operations you can use do to this. FindItems will let you search for items and retrieve for example the subject,size and received data of multiple items however FindItems wont return things like the Body or the recipients on the messages for this you need to use a GetItem request which is abstracted as the Load Method in the Managed API or if you want to do this on multiple messages you can use LoadPropertiesForItems which does a batch GetItem request. Lets look at some examples

This first example will show the Last 10 Items in your Inbox and list the Received Date, Subject and Size using FindItems

  1. #Bind to the Inbox  
  2. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)   
  3. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)  
  4.  
  5. #Define ItemView to retrive just 10 Items  
  6. $ivItemView =  New-Object Microsoft.Exchange.WebServices.Data.ItemView(10)  
  7.   
  8. $fiItems = $service.FindItems($Inbox.Id,$ivItemView)  
  9. foreach($Item in $fiItems.Items){  
  10.     "RecivedDate : " + $Item.DateTimeReceived   
  11.     "Subject     : " + $Item.Subject   
  12.     "Size        : " + $Item.Size  
  13. }  
This second example shows how to enumerate every Item in a folder you should page through 1000 items  at a time to cater for throttling for example

  1. #Bind to Inbox  
  2. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)   
  3. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)  
  4.  
  5. #Define ItemView to retrive just 1000 Items  
  6. $ivItemView =  New-Object Microsoft.Exchange.WebServices.Data.ItemView(1000)  
  7. $fiItems = $null  
  8. do{  
  9.     $fiItems = $service.FindItems($Inbox.Id,$ivItemView)  
  10.     foreach($Item in $fiItems.Items){  
  11.         "RecivedDate : " + $Item.DateTimeReceived   
  12.         "Subject     : " + $Item.Subject   
  13.         "Size        : " + $Item.Size  
  14.     }  
  15.     $ivItemView.Offset += $fiItems.Items.Count  
  16. }while($fiItems.MoreAvailable -eq $true)  
As i mentioned before there are a number of properties such as the message recipients and the message body that would not be returned by the following examples. So if we where looking at the first example where the last 10 Items in a mailbox are returned if you wanted to show the recipients these Messages where sent to you would need to do a GetItem request on each message. This could be achieved using the Load Method on each item but a better method is to use the LoadItemsFromProperties method which will batch the GetItem requests together. An example of this looks like

  1. #Bind to Inbox  
  2. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)   
  3. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)  
  4.   
  5. #Define the properties to get  
  6. $psPropset = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties)  
  7. #Define ItemView to retrive just 10 Items  
  8. $ivItemView =  New-Object Microsoft.Exchange.WebServices.Data.ItemView(10)  
  9. $fiItems = $service.FindItems($Inbox.Id,$ivItemView)  
  10. [Void]$service.LoadPropertiesForItems($fiItems,$psPropset)  
  11. foreach($Item in $fiItems.Items){  
  12.     "RecivedDate : " + $Item.DateTimeReceived   
  13.     "Subject     : " + $Item.Subject   
  14.     "Size        : " + $Item.Size  
  15.     "Recipients  : " + $Item.ToRecipients  
  16. }  
Searching for Items

Generally when your dealing with user mailboxes your going to be accessing folders that have a lot of Items, users are notoriously bad at managing their own data so you may end up with folders with 20-100 thousand items so being able to effectively search and filter items is very important. Exchange offers a number of different methods to search and filter data which I've written about in the past in http://msdn.microsoft.com/en-us/library/hh148195%28v=exchg.140%29.aspx so I wont go back over this. But I'll show a number of examples that use AQS to search the content index on a Exchange server.

In the first example it will show all the emails that have a PDF attachment in the Inbox and what those attachments are called paged 100 at a time.

  1. #Bind to Inbox    
  2. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)     
  3. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)    
  4.    
  5. #Define the properties to get    
  6. $psPropset = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties)    
  7. #Define AQS Search String  
  8.   
  9. $AQSString = "System.Message.AttachmentContents:*.pdf"  
  10.   
  11. #Define ItemView to retrive just 1000 Items    
  12. $ivItemView =  New-Object Microsoft.Exchange.WebServices.Data.ItemView(1000)    
  13. $fiItems = $null    
  14. do{    
  15.     $fiItems = $service.FindItems($Inbox.Id,$AQSString,$ivItemView)    
  16.     [Void]$service.LoadPropertiesForItems($fiItems,$psPropset)  
  17.     foreach($Item in $fiItems.Items){      
  18.         "Subject     : " + $Item.Subject  
  19.         foreach($Attachment in $Item.Attachments){  
  20.             if($Attachment.Name.Substring($Attachment.Name.Length-4).ToLower() -eq ".pdf"){  
  21.                 $Attachment.Name  
  22.             }  
  23.         }  
  24.     }    
  25.     $ivItemView.Offset += $fiItems.Items.Count    
  26. }while($fiItems.MoreAvailable -eq $true)    
There are a lot of combinations that will work with AQS filters some of which I've shown before in other posts http://gsexdev.blogspot.com.au/2010/08/using-exchange-search-and-aqs-with-ews.html . To use any of these examples with the following above sample just change the AQSString

eg to search for Items between 5-10 MB

"System.Size:5mb..10mb"

Or From a particular users

"System.Message.FromAddress:(fred@onmicrosoft.com OR amanda@onmicrosoft.com)" 

With a specific Subject

"System.Subject:FootBall"

The other way of searching for Items is to use SearchFilters which offer the greatest flexibility in terms of what you can search against but come at the cost of having to build a temporary restriction on the folder you search which depending on the number of Items in the folder and the complexity of the search may work well or not.

There are a few different types of Search filters basic logic operator type filters like isEqual, Greatorthan and Less than give standard type functionality. One example of using one of these is to search for Mail that arrived Today

  1. #Bind to Inbox    
  2. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)     
  3. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)    
  4.    
  5. #Define the properties to get    
  6. $psPropset = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties)    
  7.   
  8. $SearchFilter = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+IsGreaterThan([Microsoft.Exchange.WebServices.Data.ItemSchema]::DateTimeReceived,[system.DateTime]::Now.AddDays(-1));  
  9.   
  10. #Define ItemView to retrive just 1000 Items    
  11. $ivItemView =  New-Object Microsoft.Exchange.WebServices.Data.ItemView(1000)    
  12. $fiItems = $null    
  13. do{    
  14.     $fiItems = $service.FindItems($Inbox.Id,$SearchFilter,$ivItemView)    
  15.     #[Void]$service.LoadPropertiesForItems($fiItems,$psPropset)  
  16.     foreach($Item in $fiItems.Items){      
  17.         "Received    : " + $Item.DateTimeReceived  
  18.         "Subject     : " + $Item.Subject  
  19.           
  20.     }    
  21.     $ivItemView.Offset += $fiItems.Items.Count    
  22. }while($fiItems.MoreAvailable -eq $true)   
One type of useful searchFilter is the Not searchfilter which basically allows you to negate any SearchFilter you create for example if you want to show any Items in the Inbox that weren't IPM.Note (normal email message) you could use

  1. #Bind to Inbox    
  2. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)     
  3. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)    
  4.    
  5. #Define the properties to get    
  6. $psPropset = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties)    
  7.   
  8. $SearchFilter = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+IsEqualTo([Microsoft.Exchange.WebServices.Data.ItemSchema]::ItemClass,"IPM.Note");  
  9. $NotFilter = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+Not($SearchFilter)  
  10. #Define ItemView to retrive just 1000 Items    
  11. $ivItemView =  New-Object Microsoft.Exchange.WebServices.Data.ItemView(1000)    
  12. $fiItems = $null    
  13. do{    
  14.     $fiItems = $service.FindItems($Inbox.Id,$NotFilter,$ivItemView)    
  15.     #[Void]$service.LoadPropertiesForItems($fiItems,$psPropset)  
  16.     foreach($Item in $fiItems.Items){      
  17.         "Received    : " + $Item.DateTimeReceived  
  18.         "Subject     : " + $Item.Subject  
  19.         "ItemClass   : " + $Item.ItemClass  
  20.     }    
  21.     $ivItemView.Offset += $fiItems.Items.Count    
  22. }while($fiItems.MoreAvailable -eq $true)    
Calendar Items

Calendar Items are one of the more complex types in Exchange because of the functionality that surrounds appointments such as having recurring Appointments, exceptions, and deleted exceptions. With a recurring Appointment you have one underlying master object and the recurrence pattern is stored as a property on the Master Item and exceptions are stored as hidden attachments. Because of this reason when your accessing Calendar Items you need to use Calendar Paging which tells exchange to expand the recurring appointment so what is returned to you is a separate object for each meeting occurrence. For example to show you appointment for the next week


  1. #Bind to Calendar    
  2. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Calendar,$MailboxName)     
  3. $Calendar = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)    
  4.   
  5. #Define Date to Query 
  6. $StartDate = new-object [System.DateTime]::Now  
  7. $EndDate = new-object [System.DateTime]::Now.AddDays(7)  
  8.   
  9. #Define the calendar view  
  10. $CalendarView = New-Object Microsoft.Exchange.WebServices.Data.CalendarView($StartDate,$EndDate,1000)    
  11. $fiItems = $service.FindAppointments($Calendar.Id,$CalendarView)    
  12. foreach($Item in $fiItems.Items){      
  13.     "Start    : " + $Item.Start  
  14.     "Subject     : " + $Item.Subject  
  15. }    
Binding to Items Directly

To Bind to an Item directly you need to know its EWSid while this is a uniqueid it does change if the items are moved/copied between folders so as a general rule this is not a good Identifier to store externally. EWS does offer the ability to convert other identifiers to a EWSid which allows some interoperability between a Mapi EntryID from OOM/RDO or CDO code or from the Exchange Management Shell cmdlet which return different types of identifiers for example the Public Folder cmdlets will return the HexEntryID for folder which you can then convert to EWSid.

An Example of converting a HexEntryID to EWSid looks like

  1. $aiItem = New-Object Microsoft.Exchange.WebServices.Data.AlternateId  
  2. $aiItem.Mailbox = $MailboxName  
  3. $aiItem.UniqueId = "00000000EA21EB937E13F24195E1C55F12CB95B60100C34C9620940D4548BAC97277B49175AD0000000DE0040000"  
  4. $aiItem.Format = [Microsoft.Exchange.WebServices.Data.IdFormat]::HexEntryId;  
  5. $ewsid = $service.ConvertId($aiItem, [Microsoft.Exchange.WebServices.Data.IdFormat]::EWSId)  
  6. $ewsid  
Once you have this Id you can then just Bind to an Item for example to bind to an Email

  1. $ItemId = New-Object Microsoft.Exchange.WebServices.Data.ItemId($ewsid.UniqueId)
  2. $msMessage = [Microsoft.Exchange.WebServices.Data.EmailMessage]::Bind($service,$ItemId)
Different Folder Collections

Folders in Exchange has 3 collections where Items can be stored firstly you have the normal Items collection where all the normal items are stored. The Associated Items collection where the folders FAI (Folder Associated Items) are stored and the Deleted Items collection which is the dumpster v1 of the folder in Exchange 2010 this works a little different if you have Single Item Recovery enabled and dumpster v2 is in operation

The Associated Items Collection is useful if your going to be working on scripts that modify configuration data or if you want to store a custom item of your own that is hidden from the user. An example of a script that enumerates every Item in the FAI collection of the Inbox looks like

  1. #Bind to Inbox    
  2. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)     
  3. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)    
  4.   
  5. #Define ItemView to retrive just 1000 Items    
  6. $ivItemView =  New-Object Microsoft.Exchange.WebServices.Data.ItemView(1000)   
  7. #Define Assoicated Traversal  
  8. $ivItemView.Traversal = [Microsoft.Exchange.WebServices.Data.ItemTraversal]::Associated;  
  9. $fiItems = $null    
  10. do{    
  11.     $fiItems = $service.FindItems($Inbox.Id,$ivItemView)    
  12.     #[Void]$service.LoadPropertiesForItems($fiItems,$psPropset)  
  13.     foreach($Item in $fiItems.Items){      
  14.         "Received    : " + $Item.DateTimeReceived  
  15.         "Subject     : " + $Item.Subject  
  16.         "ItemClass   : " + $Item.ItemClass  
  17.     }    
  18.     $ivItemView.Offset += $fiItems.Items.Count    
  19. }while($fiItems.MoreAvailable -eq $true)    

That's it for Part 1 of Items in the next post I'll cover all the operations you can perform on Items such as Creating, Copying, Moving, Deleting, Importing and Exporting.

Popular posts from this blog

Testing and Sending email via SMTP using Opportunistic TLS and oAuth in Office365 with PowerShell

As well as EWS and Remote PowerShell (RPS) other mail protocols POP3, IMAP and SMTP have had OAuth authentication enabled in Exchange Online (Official announcement here ). A while ago I created  this script that used Opportunistic TLS to perform a Telnet style test against a SMTP server using SMTP AUTH. Now that oAuth authentication has been enabled in office365 I've updated this script to be able to use oAuth instead of SMTP Auth to test against Office365. I've also included a function to actually send a Message. Token Acquisition  To Send a Mail using oAuth you first need to get an Access token from Azure AD there are plenty of ways of doing this in PowerShell. You could use a library like MSAL or ADAL (just google your favoured method) or use a library less approach which I've included with this script . Whatever way you do this you need to make sure that your application registration  https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-

How to test SMTP using Opportunistic TLS with Powershell and grab the public certificate a SMTP server is using

Most email services these day employ Opportunistic TLS when trying to send Messages which means that wherever possible the Messages will be encrypted rather then the plain text legacy of SMTP.  This method was defined in RFC 3207 "SMTP Service Extension for Secure SMTP over Transport Layer Security" and  there's a quite a good explanation of Opportunistic TLS on Wikipedia  https://en.wikipedia.org/wiki/Opportunistic_TLS .  This is used for both Server to Server (eg MTA to MTA) and Client to server (Eg a Message client like Outlook which acts as a MSA) the later being generally Authenticated. Basically it allows you to have a normal plain text SMTP conversation that is then upgraded to TLS using the STARTTLS verb. Not all servers will support this verb so if its not supported then a message is just sent as Plain text. TLS relies on PKI certificates and the administrative issue s that come around certificate management like expired certificates which is why I wrote th

The MailboxConcurrency limit and using Batching in the Microsoft Graph API

If your getting an error such as Application is over its MailboxConcurrency limit while using the Microsoft Graph API this post may help you understand why. Background   The Mailbox  concurrency limit when your using the Graph API is 4 as per https://docs.microsoft.com/en-us/graph/throttling#outlook-service-limits . This is evaluated for each app ID and mailbox combination so this means you can have different apps running under the same credentials and the poor behavior of one won't cause the other to be throttled. If you compared that to EWS you could have up to 27 concurrent connections but they are shared across all apps on a first come first served basis. Batching Batching in the Graph API is a way of combining multiple requests into a single HTTP request. Batching in the Exchange Mail API's EWS and MAPI has been around for a long time and its common, for email Apps to process large numbers of smaller items for a variety of reasons.  Batching in the Graph is limited to a m
All sample scripts and source code is provided by for illustrative purposes only. All examples are untested in different environments and therefore, I cannot guarantee or imply reliability, serviceability, or function of these programs.

All code contained herein is provided to you "AS IS" without any warranties of any kind. The implied warranties of non-infringement, merchantability and fitness for a particular purpose are expressly disclaimed.