Skip to main content

Migrating your Mailbox searches in EWS to the Graph API Part 1 Filters and Search Folders

This is part one of a two part post where I'm going to look at how you can migrate any searches you are doing in EWS to the Graph API. In this first part I'm going to cover SearchFilters (from EWS) and Search-Folders as they have been around the longest and in part 2 I'll look at Searches which has  some new functionality in beta in the Graph.

Lets start by looking at how you might be doing searches in EWS at the moment

  • Search Filters (restrictions) in a FindItem Request that can be run against a Folder or Search Folder
  • QueryString (KQL) in a FindItem Request that can be run against a Folder or Search Folder
  • SearchFolder with a FindItem Request
  • eDiscovery via SearchMailbox which has now been depreciated in Office 365 and no longer supported
Search Filters (Restrictions)

If you have used the EWS Managed API to build your application you use the SearchFilter class which creates a underlying restriction in EWS https://docs.microsoft.com/en-us/exchange/client-developer/web-service-reference/restriction.  The term Restriction came from the Exchange ROP's protocol which is what MAPI uses to talk to the Exchange Store.

In the Microsoft Graph the language you talk in regards to filtering is OData  
OData (Open Data Protocol) is an ISO/IEC approved, OASIS standard that defines a set of best practices for building and consuming RESTful APIs ref https://www.odata.org/
OData filters are therefore a standard that anybody implementing the protocol (like Microsoft have done with the Graph API) should adhere to.

Lets look at some examples (from this blog) of SearchFilters I've used and how they can be converted to Graph oData Filters.

Easy - the easiest query to make is against one of the strongly typed properties like Subject or Sender eg in EWS you might have a search filter like this

SearchFilter SubjectFilter = new SearchFilter.IsEqualTo(ItemSchema.Subject, "Subject");

in the Graph this would just look something like this (applied just against the Inbox)

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?$filter=subject eq 'test'

This is an Equality search some other things you can do is

Startswith (which you couldn't actually do in EWS)

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?$filter=startswith(Subject,'test')

Sub-String Searches

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?$filter=Contains(Subject,'test')

With the later two searches if you have a folder with a large item count then these aren't going to perform like an equality or a Content Index Search would and its a possibility that they will timeout the first time you run the query. (Then on subsequent queries may succeed this is due to the way Exchange applies temporary restrictions for dynamic searches).

Medium - The most used search filter in EWS for me is that date restriction where you want to restrict the emails returned to a certain time frame so an EWS Search Filter like the following

$Sfgt = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+IsGreaterThan
([Microsoft.Exchange.WebServices.Data.ItemSchema]::DateTimeReceived, $Startdatetime)
$Sflt = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+IsLessThan
([Microsoft.Exchange.WebServices.Data.ItemSchema]::DateTimeReceived, $Enddatetime)
$sfCollection = new-object Microsoft.Exchange.WebServices.Data.SearchFilter
+SearchFilterCollection([Microsoft.Exchange.WebServices.Data.LogicalOperator]::And);
$sfCollection.add($Sfgt)
$sfCollection.add($Sflt)


In the Graph this would look like

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$filter=(receivedDateTime gt 2020-03-24T13:01:50Z) AND (receivedDateTime lt 2020-03-25T12:59:50Z)

(Just watch the date formatting)

Hard - If your using Extended properties in your SearchFilters then you need to include the Extended property definition in the Filter and use the lambda any or all expression

The first Query looks for Items based on the underlying Message Class (which isn't exposed as a strongly typed property in the Graph)

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$filter=singleValueExtendedProperties/any(ep:ep/id eq 'String 0x001a' and ep/value eq 'IPM.Note')

Another useful filter is to be able to find Messages where a particular property has been set eg this filter looks for messages that have the In-Reply-To header set (So Repsonses only)

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$filter=singleValueExtendedProperties/any(ep: ep/id eq 'String 0x1042' and ep/value ne null)

For non String properties for instance the Message size you need to make sure you cast the value to the Json datatype. Eg to find all messages that are larger the 10 MB you could use

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$filter=singleValueExtendedProperties/any(ep:ep/id eq 'Integer 0x0E08' and cast(ep/value, Edm.Int32) gt 1048576)

In EWS when you did a search that returns a large number of Items that was paged you got back also the total number of items that matched your search query. I've used this in the past to get statistical information about email with a filter without needing to page all the results. The Graph offers the same thing using the $count query parameter eg if I change the above query to find all messages above 1 MB in my mailbox and include the $count parameter this is what I get for my mailbox



Another example of this maybe to look at the count of messages in 2018

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$count=true&$filter=(receivedDateTime gt 2018-01-01) AND (receivedDateTime lt 2019-01-01)

And finally one last example is for pidTagHasAttachments because I know myself and others use this (because it gives a different value than the strongly type hasAttachments for various reasons)

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$filter=singleValueExtendedProperties/any(ep:ep/id eq 'Boolean 0x0E1B' and cast(ep/value, Edm.Boolean) eq true)

Hopefully I've covered off enough examples here for anybody stuck with syntax to be able to get their head around it. If you do have problems try posting a question into stack overflow here

SearchFolders

SearchFolders give you a way of creating a Virtual Folder that represent an underlying restriction (or Search) that can span one folder or the whole mailbox. While they are more suited to static type searches if you have ever used the mapi fiddler inspector https://github.com/OfficeDev/Office-Inspectors-for-Fiddler to look at what Outlook is doing under the covers when you do a Search, you can see that Outlook uses Searchfolders to provide a more functional search for dynamic queries.

Another example that is used in the Microsoft graph is the me/Messages endpoint which is a Searchfolder that provides access to all the mail folders in a mailbox.

In EWS when you create a SearchFolder you specify a SearchFilter for that folder to be based on. With the Graph similarly you can create a SearchFolder based on a OData filter which I've detailed above. So looking at something topically if you wanted to create a SearchFolder to show all the Email which had a subject of Coronavirus you could use

{
  "@odata.type": "microsoft.graph.mailSearchFolder",
  "displayName": "Coronavirus Email",
  "includeNestedFolders": true,
  "sourceFolderIds": ["AQMkADYA…."],
  "filterQuery": "contains(subject, 'Coronavirus')"
}

One thing that is mentioned in the SearchFolder documentation https://docs.microsoft.com/en-us/graph/api/resources/mailsearchfolder?view=graph-rest-1.0 for the Graph to be aware of is

  1. Search folders expire after 45 days of no usage.
  2. There are limits on the number of search folders that can be created per source folder. When this limit is breached, older search folders are deleted to make way for new ones.
So if you are going to use SearchFolders in your application you will need to make sure you have some appropriate management logic. Searchfolders are pretty powerful like EWS the Graph only implements a subset of what can be done in MAPI so if you are trying to reproduce what you see is possible in Outlook you may not be able to do this with Graph (or EWS).

Popular posts from this blog

Export calendar Items to a CSV file using EWS and Powershell

Somebody asked about this last week and while I have a lot of EWS scripts that do access the Calendar I didn't have a simple example that just exported a list of the Calendar events with relevant information to a CSV file so here it is.

I've talked on this one before in this howto  but when you query the calendar folder using EWS you need to use a CalendarView which will expand any recurring appointments in a calendar. There are some limits when you use a calendarview in that you can only return a maximum of 2 years of appointments at a time and paging will limit the max number of items to 1000 per call. So if you have a calendar with a very large number of appointments you need to break your query into small date time blocks. In this example script I'm just grabbing the next 7 days of appointments if you want to query a longer period you need to adjust the following lines (keeping in mind what I just mentioned)

#Define Date to Query
$StartDate = (Get-Date)
$EndDate = (Ge…

EWS Managed API and Powershell How-To series Part 1

I thought I'd start the year with a series of posts that goes back over the basics of using the EWS Managed API from Powershell and provides a modular remarked example that you can easily cut and paste to build your own scripts. Along the way in this series I'll show a whole bunch of examples around specific things.

As a starting point for versions this will be Powershell Version 2.0  and the EWS Managed API 1.1 (which will soon change to 1.2 once released) http://www.microsoft.com/download/en/details.aspx?id=13480.

The starting point for any EWS script your going to write is connecting to Exchange for which there are three important pieces of information you will need. Firstly you need to know the version of Exchange your running in this script its going to be held in the following variable

$ExchangeVersion = [Microsoft.Exchange.WebServices.Data.ExchangeVersion]::Exchange2010_SP1

Other valid values for Exchange 2007 would be

$ExchangeVersion = [Microsoft.Exchange.WebServices.…

Microsoft Teams Private Chat History Addin for Outlook

Being somebody who is transitioning across from Skype for Business to Teams one of things I missed the most (and found the most frustrating) is the lack of the ability in Outlook and OWA to view the conversation history from Online meetings and private chats in Microsoft Teams. This is especially frustrating when you have an external meeting and your sent an IM that contains some vital information for what you need to do. This information is tracked in your mailbox for compliance reasons in the Teams Chat folder but this folder is hidden so it not accessible to the clients and must be extracted by other means eg. Most people seem to point to doing a compliance search if you need this data https://docs.microsoft.com/en-us/microsoftteams/security-compliance-overview .

Given that the information is in my mailbox and there shouldn't be any privacy concern around accessing it I looked at a few ways of getting access to these TeamsChat messages in OWA and Outlook the first way was using …
All sample scripts and source code is provided by for illustrative purposes only. All examples are untested in different environments and therefore, I cannot guarantee or imply reliability, serviceability, or function of these programs.

All code contained herein is provided to you "AS IS" without any warranties of any kind. The implied warranties of non-infringement, merchantability and fitness for a particular purpose are expressly disclaimed.