Skip to main content

Looking at the Email sentiment analysis properties in Office365 with EWS and REST

Sentiment analysis which means looking at a block of text and telling if this is Negative or Positive (okay oversimplified but the Wikipedia page has a good basic introduction into this https://en.wikipedia.org/wiki/Sentiment_analysis) has been one of the standout applications of Machine learning for the last couple of years. If your a observer of what MAPI properties are available on Messages in your Inbox in Office365 like me you may have noticed a new one appear recently on some messages which is the Sentiment analysis of the body Text of an email eg


Before I go any further everything I'm talking about in this post is from observation only and none of this as far as I know is documented anywhere. The entity extraction part of this is documented and is a relative new feature that can be used in Addin's (or other applications) https://docs.microsoft.com/en-us/outlook/add-ins/match-strings-in-an-item-as-well-known-entities . Entity extraction is one of those cool power up features that allows you to write more useful Addins with a smaller amount of code and coding effort(eg if you had to parse this stuff out yourself and do the analysis it would be a lot of time/effort/testing etc)  so this type of machine learning extraction is just another evolution of this.

What you find in this named property (when it is set which is not always the case) is a JSON payload with the emotiveprofile returned from the ML process. If you going to use the in EWS or REST because of the size of the payload you will need to do a GetItem (in EWS) or a single Item GET in REST. For performance reasons when you do this with either of these API's you should take advantage of the Batching feature available. Batching is a little new in the Graph API and allows you to retrieve up to 20 items in a single call, in EWS there is no limit but realistically you should keep you calls to under 50 to ensure you don't get throttled see .

I've put together two samples of using this property the first is issuing EWS, for this I've modified the Generic Folder Item enum script I recently posted. To returned the sentiment property using this script just use the -ReturnSentiment switch. For REST I've modified my Exch-Rest library to be able to return this property which included implementing batching in the library to give the best performance.  What both of these scripts do is request the Extended property relating to the sentiment analysis and then parse the JSON using the ConvertFrom-Json cmdlet. The data is then published as first class properties on the Item with the polarity (eg Positive or Negative) published as a separate property.


Some examples of these in action

EWS - Source Script https://github.com/gscales/Powershell-Scripts/blob/master/GenericFolderAndItem.ps1

Invoke-GenericFolderItemEnum -MailboxName gscales@datarumble.com -FolderPath \Inbox -Credentials $creds -MaxCount 5 -ReturnSentiment -FullDetails | Where-Object {@_.Sentiment -ne $null} |  Select Subject,Sentiment,EmotiveProfile | fl


 REST - Source Exch-REST Module https://www.powershellgallery.com/packages/Exch-Rest and https://github.com/gscales/Exch-Rest
https://github.com/gscales/Exch-Rest
The -ReturnSentiment switch in Get-EXRLastInboxEmail, Get-EXRWellknowFolderItems and Get-EXRFolderItems will return the extracted sentiment and EmotiveProfile from the messages

 

Get-EXRLastInboxEmail -MailboxName jcool@datarumble.com -Focused -ReturnSentiment | Select Subject,Sentiment,EmotiveProfile | fl



Labels:

Popular posts from this blog

Testing and Sending email via SMTP using Opportunistic TLS and oAuth in Office365 with PowerShell

As well as EWS and Remote PowerShell (RPS) other mail protocols POP3, IMAP and SMTP have had OAuth authentication enabled in Exchange Online (Official announcement here ). A while ago I created  this script that used Opportunistic TLS to perform a Telnet style test against a SMTP server using SMTP AUTH. Now that oAuth authentication has been enabled in office365 I've updated this script to be able to use oAuth instead of SMTP Auth to test against Office365. I've also included a function to actually send a Message. Token Acquisition  To Send a Mail using oAuth you first need to get an Access token from Azure AD there are plenty of ways of doing this in PowerShell. You could use a library like MSAL or ADAL (just google your favoured method) or use a library less approach which I've included with this script . Whatever way you do this you need to make sure that your application registration  https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-
Read more

The MailboxConcurrency limit and using Batching in the Microsoft Graph API

If your getting an error such as Application is over its MailboxConcurrency limit while using the Microsoft Graph API this post may help you understand why. Background   The Mailbox  concurrency limit when your using the Graph API is 4 as per https://docs.microsoft.com/en-us/graph/throttling#outlook-service-limits . This is evaluated for each app ID and mailbox combination so this means you can have different apps running under the same credentials and the poor behavior of one won't cause the other to be throttled. If you compared that to EWS you could have up to 27 concurrent connections but they are shared across all apps on a first come first served basis. Batching Batching in the Graph API is a way of combining multiple requests into a single HTTP request. Batching in the Exchange Mail API's EWS and MAPI has been around for a long time and its common, for email Apps to process large numbers of smaller items for a variety of reasons.  Batching in the Graph is limited to a m
Read more

How to test SMTP using Opportunistic TLS with Powershell and grab the public certificate a SMTP server is using

Most email services these day employ Opportunistic TLS when trying to send Messages which means that wherever possible the Messages will be encrypted rather then the plain text legacy of SMTP.  This method was defined in RFC 3207 "SMTP Service Extension for Secure SMTP over Transport Layer Security" and  there's a quite a good explanation of Opportunistic TLS on Wikipedia  https://en.wikipedia.org/wiki/Opportunistic_TLS .  This is used for both Server to Server (eg MTA to MTA) and Client to server (Eg a Message client like Outlook which acts as a MSA) the later being generally Authenticated. Basically it allows you to have a normal plain text SMTP conversation that is then upgraded to TLS using the STARTTLS verb. Not all servers will support this verb so if its not supported then a message is just sent as Plain text. TLS relies on PKI certificates and the administrative issue s that come around certificate management like expired certificates which is why I wrote th
Read more
All sample scripts and source code is provided by for illustrative purposes only. All examples are untested in different environments and therefore, I cannot guarantee or imply reliability, serviceability, or function of these programs.

All code contained herein is provided to you "AS IS" without any warranties of any kind. The implied warranties of non-infringement, merchantability and fitness for a particular purpose are expressly disclaimed.