Thursday, July 19, 2018

Expanded Folder Age and Attachment reporting for Exchange / Office 365 Mailboxes with EWS and Powershell

Folder Item numbers, size's and Attachment Item Numbers, types and sizes are one of the Mailbox statistics that is useful when your trying to look at how and why a mailbox is growing or why your mailbox performance might be declining if you have a large number of folder items. With the recent increases in Mailbox size quotas  in Office365 it can be easy to become quite indifferent to how quickly your mailbox might be growing. For example if you have 100GB quota the lazy mindset (which I admit to being a part of) can be to just keep everything you get via mail (or stack it and rack it). If your users are not able to take advantage of cloudy attachments (or the users refuse to change their use habits) the nature of expanding file sizes means a sharper growth curve for Folder Items and Attachments.

The traditional way of tracking Mailbox sizes is to use Get-MailboxStatistics and Get-MailboxFolderStatistics for which there are many good scripts around that give a good overview of what is in a Mailbox or Folder. For a deeper dive into what  is in messages in a particular folder where you are seeing growth then using a Mailbox API like EWS (or REST) can be useful. In the past I've posted a few scripts for doing Attachment and Item level statistics which where in need of some updating which is what I've done with this post

Script Approach 

For this script I've used EWS as it is universal across OnPremise and Office365 from Exchange 2010 up. I do have two different versions of this script however because Office365 and Exchange 2016 allows you to have reference attachments (cloudy attachments) so some slightly different code is needed and the latest compiled version of the EWS Managed API which is only available from GitHub is needed so statistics can be compiled successfully for these type of Attachments. For the Item level processing side of the script it will process every item in a folder firstly using the EWS FindItems operation which will return a minimal results set of Items and Sizes and then for those items with Attachments another batch operation is used to get the metadata for the attachments which will include the Attachment name (which we can then derive the attachment type from using the file extension name) and the size of the attachment. If its a cloudy attachment then this is processed separately. I haven't included any code to call out to OneDrive to get the filesize for the cloudy attachments but its possible to do this with some extra code (generally you would want to do this asynchronously while processing the Exchange attachments).

The Results the script produces is the following html report

The script writes its result sets out in JSON, I've then used the Tabulator JavaScript library which can turn JSON datasets into well formatted tables and offer sorting options so its quite easy to create a more user friendly results set then just a plain CSV file which I usually use. Currently the script will report on just one particular folder as I envisaged that it would be best used in conjunction with other scripts (that utilize Get-MailboxStatistics and Get-MailboxFolderStatistics eg take your pick from ) where you identify a particular Mailbox or  folder you want to investigate more thoroughly as this script can take some time to run. I also have two defined modes where you can process Inline Attachments (which are primary just image files) or skip the processing of Inline attachments which will make the script run a lot faster because of the number of Messages that have inline attachments.

Some Examples of running the script on the Inbox folder without processing Inline attachments

Import-Module .\AttachmentStats.ps1
Invoke-MailboxFolderAttachmentReport -MailboxName gscales@dataru -FolderPath \Inbox -Verbose

Or with processing Inline attachments

Import-Module .\AttachmentStats.ps1
Invoke-MailboxFolderAttachmentReport -MailboxName gscales@dataru -FolderPath \Inbox -IncludeInline -Verbose
The report will be saved to the same path as the script module is located.

The regular version of the script can be found on GitHub

The cloudy version of the script can be found on GitHub here 

The cloudy version will require the following EWS Managed API version to be located in the same directory as the script module.

The Cloudy version will produce a separate table of cloudy attachment stats eg

Hire me - If you would like to do something similar to this or anything else you see on my blog I'm currently available to help with any Office365, Exchange or Active Directory related development work or scripting, please contact me at too big or small).

Wednesday, June 27, 2018

Building a Microsoft Teams Tab Application that uses the Graph API and Exchange OOF data

Over the years I've created a few different versions of In/Out boards like this that have used Exchange FreeBusy information to present the In/Out status for Users. In this post I am going to look at how you can create the same thing in Microsoft Teams using a Team's tab application that will call some Microsoft Graph endpoints that will first get the members of a particular Team, then there OOF MailTip to determine if they have an AutoResponse set and finally get there userphoto. The end results is a board that looks like this

Team's Tab Applications

Team's tab apps are very simular to the Add in's framework used across other Office365 products that utilize JavaScript and html to provide a consistent approach across desktop and web clients. However because teams is fairly new the underlying client SDK isn't as feature rich or mature as you would find for example with Exchange/Outlook. Some examples of this is there is no way to get the members of a Team using the client SDK (v1.0). Also the Authentication methods to get the token to use against other workloads is not as seamless and in some cases relies on Auth popups that can be blocked by web browsers.

Getting Started

To get started with developing and using your own custom tab applications you first need to enable side loading of Apps in the Office365 Admin portal ref .The important part is "Sideloading is how you add an app to Teams by uploading a zip file directly to a team. Sideloading lets you test an app as it's being developed. It also lets you build an app for internal use only and share it with your team without submitting it to the Teams app catalog in the Office Store. "

You also will need somewhere to host your pages a good free solution is GitHub pages  or Azure if you have a subscription or credits via MSDN etc.

An Application registration is needed which is used in both the Teams manifest and it allows you to assign what rights your Team app will have to other workloads in the Graph API. To Create a app registration see . This sample app uses a v1 registration


Because this app accesses various graph endpoints different oauth Grants need to be allowed eg
Admin Consent 

Once you have your application registration setup with the correct user grants for what you plan to access because some of these permissions may require Administrative Consent you will need to consent to its use within your tenant. The easiest way to do this is in a Web Browser, eg post the following where you replace the Client_Id with the application Id from the registration you just created.  

Installing a Custom tab app

Once you have met all the prerequisites you are now ready to upload the App into teams and enable it on your desired channel. To do this you need to use the "upload a custom app" link which is available when you click ManageTeam-Apps tab see

(Note if you don't see  the "upload a custom app" check that you have side loading of apps enabled in your tenant config)

What you upload here is a Zip file containing your manifest file and two images that you manifest refers to for eg
   "icons": {
    "outline": "Outline32.png",
    "color": "Colour192.png"

For this sample this is located in

These icons are what is used in the UI to represent your application.

How the code works

Because the code uses the Graph API to get the Team Members, AutoResponse Mailtip and Profile picture the first thing it must do is get a Token to use against the Graph endpoint. With this sample it uses the JavaScript ADAL which allows you to do a silent Authentication if a cached token is already available otherwise a popup is used to authenticate the user (in most cases the user doesn't need to enter anything the popup will showup and be able to use the currently logged on creds). The Authentication flow through tab applications is explained in detail here .

Once a Token has been acquired the requests to the Graph are relatively straight forward, for performance reasons the requests are preformed asynchronously so the app can be as responsive to the user as possible.


Because this was proof of concept sample its not optimized for performance, eg for things like getting the user photo if you had a very large group then this could potentially take quite a while to get and render all these images (eg you need to start implementing paging and caching things in blob storage to improve performance and user response).

All the code for this sample is available in GitHub

Its also hosted in GitHub pages if you have a development tenant and want to test it (you will need to first use the admin consent URL listed above).

This is just a sample to show what you can do within the teams client to bring in data from other workloads. Other things you could do with this is provide a reporting interface directly within a Teams channels for the data provided by the Graph endpoint who's functionality is constantly growing.

Hire me - If you would like to do something similar to this or anything else you see on my blog I'm currently available to help with any Office365, Exchange or Active Directory related development work or scripting please contact me at too big or small).

Tuesday, June 19, 2018

Using Message and Adaptive Cards with Powershell in Email and Microsoft Teams in Office365

Cards are "a user-interface (UI) container for short or related pieces of information ref" which there are currently two formats for, MessageCards and AdativeCards . The underlying data format is a JSON object that describes the content to be rendered by a supporting client. Eg here is a basic sample

So they are a simple way of presenting information in consistent manner which  then makes deploying that same UI experience across multiple client forms factors (mobile,web,desktop) a lot easier.  Cards also allow you to define actions, so then the cards themselves can become a small app or part of an integration piece of a Line of Business workflow etc. Actionable Cards in Outlook have the ability to link with an Outlook Addin's see which starts to really extend what you can use cards for. This means your no longer just stuck in the limited feature set of Cards but you can use the richness available in the Outlook Addin framework.

Email Security  

Cards rely on JSON (JavaScript Object Notation) and JavaScript to provide the functionality that they do. Messaging people that have been around for a while would know that JavaScript (and other active Content) in Email bodies hasn't been allowed for a very long time because of the risk of malicious code execution. Therefor senders of Actionable cards must be registered with Microsoft and Approved using the following portal . If you just want to send Actionable messages within your own tenant you would registered your App within your tenant but your Actionable Messages would not have any context outside of your tenant (or other peoples Actionable Messages within your own).

Client Support 

 If you are a student of Technology cards are a good study into cloud cadences and just how quickly standards can come and go now but also how hard it can be to get all your software wheels turning to keep pace. For example MessageCards are now considered legacy however as of this post June 2018 both the Outlook desktop  and the Team's clients don't support adaptive cards in their current general releases.  Eg for Adaptive Cards support in Teams you will need the developer preview and in the Outlook Desktop client you need to be on the Monthly Channel with version 1085. This is neither good nor bad but if you find Adaptive Cards the exact thing you need to solve your problem you should be aware of client support status before you start your planning. 


For this post I've created both a MessageCard PowerShell sample that can use be used in either Outlook or Teams and a Adaptive Cards sample that can be used primary in OWA at the moment. The to produce just the MessageCard can be found and the Adaptive Card

Microsoft Teams - In/Out - Out of Office Teams Message card

This script will produce the following output to a Teams Channel using a incoming WebHook to receive the Messages

This example uses my Exch-REST module available from the PowerShell Gallery and GitHub . This contains all the code to first get the Group associated with a particular team, get the members of that groups mail Addresses, make the Graph Mail tips request and then package the results as JSON message which can then be posted to a WebHook.

To Setup a WebHook for the script to post to on a Team see

 So lets look at how this all fits together

Connect-EXRMailbox -mailbox
$GroupName = "A Team" $Group = Get-EXRModernGroups -GroupName $GroupName $Members = Get-EXRGroupMembers -GroupId $
The above code first gets the ModernGroup which is associated with the Team you are working with using the Teams DisplayName. It then uses the GroupId that is returned to make a request to Get the Members of that group.

$OOFMailboxCol = @() $mtHash = @{} foreach($Member in $Members){     if(![String]::IsNullOrEmpty($Member.mail)){         $OOFMailboxCol += $Member.mail         $mtHash.Add($Member.mail,$Member.displayName)     }  }
This code creates a collection of Mailboxes to use the Get-EXRMailTips operation and also creates a hashtable to allow us to map the email address in the Mailtips response back to the DisplayName because by default MailTips only gives you back the EmailAddress.

$Mailtips = Get-EXRMailTips -Mailboxes $OOFMailboxCol -tips "automaticReplies"
$FactsColl = ($Mailtips | select @{Name='EmailAddress';Expression={$mtHash[$_.emailAddress.address]}},@{Name='OOF Message';Expression={$val =($_.automaticReplies.message -replace '<[^>]+>','').Trim();if([String]::IsNullOrEmpty($val)){"In"}else{"Out : " + $val}}})
$WebhookAddress = ""
Invoke-WebRequest -Uri $WebhookAddress -Method Post -ContentType 'Application/Json' -Body (New-EXRMessageCard -Facts $FactsColl -Summary "Team OOF Status" -Title "Team OOF Status")
The first line of the above code makes the MailTips request for the addresses that are passed into it. The next line builds a Collection of Facts which is a Messagecard element that display a two column table. I've used expressions to clean up the results as the MailTips response is a HTML value that needs to be formatted back to text for the card and I've added some In/Out prefixes based on the response. The last line of code is what submits the Message card to the configured webhook address in the $WebhookAddress variable.

The Full Sourcecode for this script can be found

Adaptive Card to show the Office365 Service health status


As I mentioned previously if your going to send an Adaptive Card as an email you need to be registered in the portal and include the originator Id you receive in your adaptive Card. For testing what you can do is send an Adaptive Card to yourself (from yourself) and without the originator property being set and it will appear okay. But if you where to send it to someone else the content will be stripped from the message.


One of the other things that you can get from the endpoint that I talked about in this post is the Service Health information about each of the different workloads in Office365. In this example I use the Management API to produce the following adaptive card example eg

Adaptive Cards have a lot more formatting options for example you can have a column set and use different font colors. To make the script as dynamic as possible I do the color switching using a hashtable. This example uses my Exch-REST module available from the PowerShell Gallery and GitHub  . The script does the following

$To = "" Connect-EXRMailbox -MailboxName "" Connect-EXRManagementAPI -UserName "" $StatusArray = Get-EXRMCurrentStatus | Select-Object WorkLoad,Statusdisplayname,StatusTime 
In the above code it makes a connection to the Mailbox and ManagementAPI and then builds an collection for the service health information.

$ColorSwitchHash = @{} $ColorSwitchHash.Add("Service degradation","attention") $ColorSwitchHash.Add("Normal service","good")
This code sets up a HashTable that will be used to display a different color for the service status column based on its status. Note the Adaptive Cards support a subset of the MarkDown format so attention (red), good (green) etc.

$Card = New-EXRAdaptiveCard -Columns $StatusArray -ColorSwitchColumnNumber 1 -ColorSwitchHashTable $ColorSwitchHash -ColorSwitchDefault warning Send-EXRAdaptiveCard -To $To -Subject "Office 365 Service Status" -AdaptiveCard $Card
The last part of the code generates the AdatpiveCard, the ColorSwitchColumnNumber is a zero based array to match with the collection that is passed in. What Send-EXRAdaptiveCard does is wraps the JSON from the Card in the following script which is necessary when you send a message.

<script type="application/adaptivecard+json"></script>

Hire me - I'm currently available to help with any Exchange\Office365 related development work or scripting please contact me at (nothing too big or small). I'd also be interested in hearing from any companies that want to sponsor open source projects around Exchange and Office365 development.

Wednesday, June 13, 2018

Using the Office 365 Management Activity API from Powershell to audit Exchange and Office365

The Office 365 Management Activity API is a REST endpoint that can be used to access audit events from user, admin, system, and policy actions and events in Azure and Office365 workloads (its been around for a while first appeared in 2015 in preview). Within Office365 there are many ways of accessing this type of information the Primary UI these days for looking at auditing is the Unified Audit logs available in the Office365 Portal

The advantage of using the Management Activity API over the portal is that  allows you to create more tailored reporting by accessing the raw data over a period of time (so writing your own user trend reporting) using a scalable API like REST (which if you have a lot of data to query is superior to using WinRM solutions using something like the Exchange Management Shell cmdlets). It also includes WebHooks interfaces that can be used to trigger other workloads to process particular Audit events on a continuing basis. (Although you should note the documentation states that this API isn't meant to be used for real time alerting as there are other Microsoft provided endpoint and services to provide this functionality).
How it works

The Management Activity API uses a subscription based approach, the mechanics are relatively simple eg 

  • You create a Subscription for the Workload you want to access the Audit event for.(eg Exchange)
  • The backend then produces contentblobs which are aggregates of those audit logs and then makes them available to access. If you have registered a WebHook for your subscription then your endpoint will be notified that there is a new ContentBlob available for you to process.
  • Once you no longer require the subscription you can then Stop that subscription.

One point to note is that the Management Activity API doesn't actually enable auditing for any of the workloads it just make available data from the Auditing that is already enabled. Eg for instance Exchange Mailbox Auditing isn't turned on by default so if you did the above process to create a subscription for Exchange you would never see any ContentBlobs generated for Mailbox Auditing events unless you first went and enabled auditing on the Mailboxes you wish to audit and then these events would be available within your subscription. This is the same for other workload as well, some workloads like AzureAD have some auditing enabled by default but you should always check your auditing configuration to make sure you have turned on the particular auditing setting for the particular workload your interested in before using this API.


With the Management Activity API the subscription mechanism has the ability to send a WebHook notification to any Webhook endpoint you configure. One easy to use webhook example is Azure RunBooks so using something like this your subscription could trigger a RunBook which would then process the ContentBlob that has just been made available and apply any Logic or custom report you need. The good thing with RunBooks is all the underlying service parts of the code are done for you and you can just plug in your custom script to do what you need.


The prerequisites for this API is that you need to have an Application registration see created that has been  given the oAuth Grants to access the API. You need to generate your access tokens for the resource url (so for example you can't use a token you have generated against the Graph endpoint you need to specifically request this ResourceURL). The last thing is the Account you then use to the access the API needs to have rights to the Audit data. (If you choose to use certificate authentication and a daemon type app this would get the rights through the different oAuth Grants for applications).

Putting it to use

A really straight forward way of using this API from PowerShell is to make use of Invoke-WebRequest and there is a good document here on doing this . Another easier and more functional approach is to use my Exch-REST module which is available from the PowerShell Gallery and GitHub which now has some cmdlets and plumbing to enable use of this API. The module can now handle caching tokens from multiple resources (eg the Graph and Management API) so you can then combine operations from both API's which is where you can then start to build more powerful tailored reports. Eg if your processing the Content Blobs for particular Mailboxes and you want more information about that mailbox to include in a report you can use the Graph to easily access that. Or if you wanted to check on a Message that was deleted by a user to see if it had any attachments that could contain sensitive content you could use the Graph API to reach into the Mailbox and access the deleted message from the Recoverable Items folders etc.

Using Exch-REST to create and access subscription content

Connecting and Generating the Access Token 

To Connect and generate an Accesstoken to use against the Management API use Connect-EXRManagementAPI

I've created a default app registration that you can use for testing (number 5) that just has access to the Management API Oauth Grants but would recommend as usually that you create your own App registration  see so you control what rights the code will have. One other important thing to note is the subscription are per appid, so if you create a subscription with say the above AppId and then create your own ApplicationId at a later point and use that you won't be able to see/view/stop the subscriptions you created with the pervious appId unless your login using that pervious ApplicationId.

Once you have a token you can then use the Management API cmdlets for example to show the current subscriptions use Get-EXRMSubscriptions

If this is the first time you are using this API it will just show a blank list. To create a new subscription use the New-EXRMSubscription cmdlet with the switch for the workload you want to use eg to create a New AzureADSubscription use the -AzureAd switch

other workload switch are

-General (other workloads like Teams are included here)

If you want to use WebHooks there are specific parameters for this that allows you to specify the necessary Webhook details.

Once you have created the subscription the content blobs don't get created straight away the documentation say that this will take about 6-24 hours for content to start to become available. Once they are available you can access them using the Get-EXRMSubscriptionContent cmdlet eg to get the Exchange content for the last 24 hours

Get-EXRMSubscriptionContent -Exchange -StartTime (Get-Date).AddDays(-1) -EndTime (Get-Date)

The data you get back for this operation contains the contenturi of the blob that you can then access using the SubscriptionContentBlob operation. I have a separate cmdlet that can be used to retrieve the blob eg

Get-EXRMSubscriptionContentBlob -ContentURI
However I found the useability of doing it this way to not be so great so I included a switch in the Get-EXRMSubscriptionContent cmdlet so you can specify to return the contentblobs eg

Get-EXRMSubscriptionContent -Exchange -StartTime (Get-Date).AddDays(-1) -EndTime (Get-Date) -returnContentBlobs
eg this will the return the contentblob with each content entry

You can then process the Contentblob property and the data within anyway you like, here is one example that produces a Client report using the client agent data for the last 24hours

$Last24Results = Get-EXRMSubscriptionContent -Exchange -StartTime (Get-Date).AddDays(-1) -EndTime (Get-Date) -returnContentBlobs
$BlobEntries = foreach($ContentEntry in $Last24Results){$ContentEntry.ContentBlob}
$BlobEntries | Where-Object{$_.ClientInfoString -ne $null} | select  CreationTime,Operation,ClientInfoString,Use rId | fl

Or you could just look at things like the MoveToDeletedItems events and get more information

Once your finished with a Subscription and you no longer want to process content blobs you can stop the subscription using Invoke-EXRMStopSubscription eg to stop the Exchange Subscription use

Invoke-EXRMStopSubscription  -Exchange

The combinations of what you can do are just limited to your own imagination or particular audit bucket you need to fill. If you have need for a developer/scripter or just someone for to help out for anything exchange/office365 related I'm available to take on work at the moment so please contact me at (nothing too big or small). I'd also be interested in hearing from any companies that want to sponsor open source projects around Exchange and Office365 development.

The Exch-REST module is available from the PowerShell Gallery and GitHub