Thursday, April 26, 2018

Looking at the Email sentiment analysis properties in Office365 with EWS and REST

Sentiment analysis which means looking at a block of text and telling if this is Negative or Positive (okay oversimplified but the Wikipedia page has a good basic introduction into this https://en.wikipedia.org/wiki/Sentiment_analysis) has been one of the standout applications of Machine learning for the last couple of years. If your a observer of what MAPI properties are available on Messages in your Inbox in Office365 like me you may have noticed a new one appear recently on some messages which is the Sentiment analysis of the body Text of an email eg


Before I go any further everything I'm talking about in this post is from observation only and none of this as far as I know is documented anywhere. The entity extraction part of this is documented and is a relative new feature that can be used in Addin's (or other applications) https://docs.microsoft.com/en-us/outlook/add-ins/match-strings-in-an-item-as-well-known-entities . Entity extraction is one of those cool power up features that allows you to write more useful Addins with a smaller amount of code and coding effort(eg if you had to parse this stuff out yourself and do the analysis it would be a lot of time/effort/testing etc)  so this type of machine learning extraction is just another evolution of this.

What you find in this named property (when it is set which is not always the case) is a JSON payload with the emotiveprofile returned from the ML process. If you going to use the in EWS or REST because of the size of the payload you will need to do a GetItem (in EWS) or a single Item GET in REST. For performance reasons when you do this with either of these API's you should take advantage of the Batching feature available. Batching is a little new in the Graph API and allows you to retrieve up to 20 items in a single call, in EWS there is no limit but realistically you should keep you calls to under 50 to ensure you don't get throttled see .

I've put together two samples of using this property the first is issuing EWS, for this I've modified the Generic Folder Item enum script I recently posted. To returned the sentiment property using this script just use the -ReturnSentiment switch. For REST I've modified my Exch-Rest library to be able to return this property which included implementing batching in the library to give the best performance.  What both of these scripts do is request the Extended property relating to the sentiment analysis and then parse the JSON using the ConvertFrom-Json cmdlet. The data is then published as first class properties on the Item with the polarity (eg Positive or Negative) published as a separate property.


Some examples of these in action

EWS - Source Script https://github.com/gscales/Powershell-Scripts/blob/master/GenericFolderAndItem.ps1

Invoke-GenericFolderItemEnum -MailboxName gscales@datarumble.com -FolderPath \Inbox -Credentials $creds -MaxCount 5 -ReturnSentiment -FullDetails | Where-Object {@_.Sentiment -ne $null} |  Select Subject,Sentiment,EmotiveProfile | fl


REST - Source Exch-REST Module https://www.powershellgallery.com/packages/Exch-Rest and https://github.com/gscales/Exch-Rest
https://github.com/gscales/Exch-Rest
The -ReturnSentiment switch in Get-EXRLastInboxEmail, Get-EXRWellknowFolderItems and Get-EXRFolderItems will return the extracted sentiment and EmotiveProfile from the messages


Get-EXRLastInboxEmail -MailboxName jcool@datarumble.com -Focused -ReturnSentiment | Select Subject,Sentiment,EmotiveProfile | fl





No comments: