Skip to main content

Using the Office 365 Management Activity API from Powershell to audit Exchange and Office365

The Office 365 Management Activity API is a REST endpoint that can be used to access audit events from user, admin, system, and policy actions and events in Azure and Office365 workloads (its been around for a while first appeared in 2015 in preview). Within Office365 there are many ways of accessing this type of information the Primary UI these days for looking at auditing is the Unified Audit logs available in the Office365 Portal https://support.office.com/en-us/article/Search-the-audit-log-in-the-Office-365-Security-Compliance-Center-0d4d0f35-390b-4518-800e-0c7ec95e946c#PickTab=HowTo

The advantage of using the Management Activity API over the portal is that  allows you to create more tailored reporting by accessing the raw data over a period of time (so writing your own user trend reporting) using a scalable API like REST (which if you have a lot of data to query is superior to using WinRM solutions using something like the Exchange Management Shell cmdlets). It also includes WebHooks interfaces that can be used to trigger other workloads to process particular Audit events on a continuing basis. (Although you should note the documentation states that this API isn't meant to be used for real time alerting as there are other Microsoft provided endpoint and services to provide this functionality).
How it works

The Management Activity API uses a subscription based approach, the mechanics are relatively simple eg 


  • You create a Subscription for the Workload you want to access the Audit event for.(eg Exchange)
  • The backend then produces contentblobs which are aggregates of those audit logs and then makes them available to access. If you have registered a WebHook for your subscription then your endpoint will be notified that there is a new ContentBlob available for you to process.
  • Once you no longer require the subscription you can then Stop that subscription.

One point to note is that the Management Activity API doesn't actually enable auditing for any of the workloads it just make available data from the Auditing that is already enabled. Eg for instance Exchange Mailbox Auditing isn't turned on by default so if you did the above process to create a subscription for Exchange you would never see any ContentBlobs generated for Mailbox Auditing events unless you first went and enabled auditing on the Mailboxes you wish to audit https://support.office.com/en-us/article/enable-mailbox-auditing-in-office-365-aaca8987-5b62-458b-9882-c28476a66918 and then these events would be available within your subscription. This is the same for other workload as well, some workloads like AzureAD have some auditing enabled by default but you should always check your auditing configuration to make sure you have turned on the particular auditing setting for the particular workload your interested in before using this API.

Webhooks

With the Management Activity API the subscription mechanism has the ability to send a WebHook notification to any Webhook endpoint you configure. One easy to use webhook example is Azure RunBooks https://docs.microsoft.com/en-us/azure/automation/automation-webhooks so using something like this your subscription could trigger a RunBook which would then process the ContentBlob that has just been made available and apply any Logic or custom report you need. The good thing with RunBooks is all the underlying service parts of the code are done for you and you can just plug in your custom script to do what you need.

Prerequisites 

The prerequisites for this API is that you need to have an Application registration see created that has been  given the oAuth Grants to access the API. You need to generate your access tokens for the resource url manage.office.com (so for example you can't use a token you have generated against the Graph endpoint you need to specifically request this ResourceURL). The last thing is the Account you then use to the access the API needs to have rights to the Audit data. (If you choose to use certificate authentication and a daemon type app this would get the rights through the different oAuth Grants for applications).

Putting it to use

A really straight forward way of using this API from PowerShell is to make use of Invoke-WebRequest and there is a good document here on doing this https://msdn.microsoft.com/en-us/office-365/troubleshooting-the-office-365-management-activity-api . Another easier and more functional approach is to use my Exch-REST module which is available from the PowerShell Gallery https://www.powershellgallery.com/packages/Exch-Rest and GitHub https://github.com/gscales/Exch-Rest which now has some cmdlets and plumbing to enable use of this API. The module can now handle caching tokens from multiple resources (eg the Graph and Management API) so you can then combine operations from both API's which is where you can then start to build more powerful tailored reports. Eg if your processing the Content Blobs for particular Mailboxes and you want more information about that mailbox to include in a report you can use the Graph to easily access that. Or if you wanted to check on a Message that was deleted by a user to see if it had any attachments that could contain sensitive content you could use the Graph API to reach into the Mailbox and access the deleted message from the Recoverable Items folders etc.

Using Exch-REST to create and access subscription content

Connecting and Generating the Access Token 

To Connect and generate an Accesstoken to use against the Management API use Connect-EXRManagementAPI


I've created a default app registration that you can use for testing (number 5) that just has access to the Management API Oauth Grants but would recommend as usually that you create your own App registration  see so you control what rights the code will have. One other important thing to note is the subscription are per appid, so if you create a subscription with say the above AppId and then create your own ApplicationId at a later point and use that you won't be able to see/view/stop the subscriptions you created with the pervious appId unless your login using that pervious ApplicationId.

Once you have a token you can then use the Management API cmdlets for example to show the current subscriptions use Get-EXRMSubscriptions


If this is the first time you are using this API it will just show a blank list. To create a new subscription use the New-EXRMSubscription cmdlet with the switch for the workload you want to use eg to create a New AzureADSubscription use the -AzureAd switch


other workload switch are

-Exchange
-SharePoint
-General (other workloads like Teams are included here)
-DLP

If you want to use WebHooks there are specific parameters for this that allows you to specify the necessary Webhook details.

Once you have created the subscription the content blobs don't get created straight away the documentation say that this will take about 6-24 hours for content to start to become available. Once they are available you can access them using the Get-EXRMSubscriptionContent cmdlet eg to get the Exchange content for the last 24 hours

Get-EXRMSubscriptionContent -Exchange -StartTime (Get-Date).AddDays(-1) -EndTime (Get-Date)

The data you get back for this operation contains the contenturi of the blob that you can then access using the SubscriptionContentBlob operation. I have a separate cmdlet that can be used to retrieve the blob eg


Get-EXRMSubscriptionContentBlob -ContentURI https://manage.office.com...
However I found the useability of doing it this way to not be so great so I included a switch in the Get-EXRMSubscriptionContent cmdlet so you can specify to return the contentblobs eg


Get-EXRMSubscriptionContent -Exchange -StartTime (Get-Date).AddDays(-1) -EndTime (Get-Date) -returnContentBlobs
eg this will the return the contentblob with each content entry


You can then process the Contentblob property and the data within anyway you like, here is one example that produces a Client report using the client agent data for the last 24hours


$Last24Results = Get-EXRMSubscriptionContent -Exchange -StartTime (Get-Date).AddDays(-1) -EndTime (Get-Date) -returnContentBlobs
$BlobEntries = foreach($ContentEntry in $Last24Results){$ContentEntry.ContentBlob}
$BlobEntries | Where-Object{$_.ClientInfoString -ne $null} | select  CreationTime,Operation,ClientInfoString,Use rId | fl

Or you could just look at things like the MoveToDeletedItems events and get more information


Once your finished with a Subscription and you no longer want to process content blobs you can stop the subscription using Invoke-EXRMStopSubscription eg to stop the Exchange Subscription use

Invoke-EXRMStopSubscription  -Exchange

The combinations of what you can do are just limited to your own imagination or particular audit bucket you need to fill. If you have need for a developer/scripter or just someone for to help out for anything exchange/office365 related I'm available to take on work at the moment so please contact me at gscales@msgdevelop.com (nothing too big or small). I'd also be interested in hearing from any companies that want to sponsor open source projects around Exchange and Office365 development.

The Exch-REST module is available from the PowerShell Gallery https://www.powershellgallery.com/packages/Exch-Rest and GitHub https://github.com/gscales/Exch-Rest


Popular posts from this blog

Testing and Sending email via SMTP using Opportunistic TLS and oAuth in Office365 with PowerShell

As well as EWS and Remote PowerShell (RPS) other mail protocols POP3, IMAP and SMTP have had OAuth authentication enabled in Exchange Online (Official announcement here ). A while ago I created  this script that used Opportunistic TLS to perform a Telnet style test against a SMTP server using SMTP AUTH. Now that oAuth authentication has been enabled in office365 I've updated this script to be able to use oAuth instead of SMTP Auth to test against Office365. I've also included a function to actually send a Message. Token Acquisition  To Send a Mail using oAuth you first need to get an Access token from Azure AD there are plenty of ways of doing this in PowerShell. You could use a library like MSAL or ADAL (just google your favoured method) or use a library less approach which I've included with this script . Whatever way you do this you need to make sure that your application registration  https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-

How to test SMTP using Opportunistic TLS with Powershell and grab the public certificate a SMTP server is using

Most email services these day employ Opportunistic TLS when trying to send Messages which means that wherever possible the Messages will be encrypted rather then the plain text legacy of SMTP.  This method was defined in RFC 3207 "SMTP Service Extension for Secure SMTP over Transport Layer Security" and  there's a quite a good explanation of Opportunistic TLS on Wikipedia  https://en.wikipedia.org/wiki/Opportunistic_TLS .  This is used for both Server to Server (eg MTA to MTA) and Client to server (Eg a Message client like Outlook which acts as a MSA) the later being generally Authenticated. Basically it allows you to have a normal plain text SMTP conversation that is then upgraded to TLS using the STARTTLS verb. Not all servers will support this verb so if its not supported then a message is just sent as Plain text. TLS relies on PKI certificates and the administrative issue s that come around certificate management like expired certificates which is why I wrote th

The MailboxConcurrency limit and using Batching in the Microsoft Graph API

If your getting an error such as Application is over its MailboxConcurrency limit while using the Microsoft Graph API this post may help you understand why. Background   The Mailbox  concurrency limit when your using the Graph API is 4 as per https://docs.microsoft.com/en-us/graph/throttling#outlook-service-limits . This is evaluated for each app ID and mailbox combination so this means you can have different apps running under the same credentials and the poor behavior of one won't cause the other to be throttled. If you compared that to EWS you could have up to 27 concurrent connections but they are shared across all apps on a first come first served basis. Batching Batching in the Graph API is a way of combining multiple requests into a single HTTP request. Batching in the Exchange Mail API's EWS and MAPI has been around for a long time and its common, for email Apps to process large numbers of smaller items for a variety of reasons.  Batching in the Graph is limited to a m
All sample scripts and source code is provided by for illustrative purposes only. All examples are untested in different environments and therefore, I cannot guarantee or imply reliability, serviceability, or function of these programs.

All code contained herein is provided to you "AS IS" without any warranties of any kind. The implied warranties of non-infringement, merchantability and fitness for a particular purpose are expressly disclaimed.