Tuesday, June 19, 2018

Using Message and Adaptive Cards with Powershell in Email and Microsoft Teams in Office365

Cards are "a user-interface (UI) container for short or related pieces of information ref" which there are currently two formats for, MessageCards and AdativeCards . The underlying data format is a JSON object that describes the content to be rendered by a supporting client. Eg here is a basic sample


So they are a simple way of presenting information in consistent manner which  then makes deploying that same UI experience across multiple client forms factors (mobile,web,desktop) a lot easier.  Cards also allow you to define actions, so then the cards themselves can become a small app or part of an integration piece of a Line of Business workflow etc. Actionable Cards in Outlook have the ability to link with an Outlook Addin's see which starts to really extend what you can use cards for. This means your no longer just stuck in the limited feature set of Cards but you can use the richness available in the Outlook Addin framework.

Email Security  

Cards rely on JSON (JavaScript Object Notation) and JavaScript to provide the functionality that they do. Messaging people that have been around for a while would know that JavaScript (and other active Content) in Email bodies hasn't been allowed for a very long time because of the risk of malicious code execution. Therefor senders of Actionable cards must be registered with Microsoft and Approved using the following portal . If you just want to send Actionable messages within your own tenant you would registered your App within your tenant but your Actionable Messages would not have any context outside of your tenant (or other peoples Actionable Messages within your own).

Client Support 

 If you are a student of Technology cards are a good study into cloud cadences and just how quickly standards can come and go now but also how hard it can be to get all your software wheels turning to keep pace. For example MessageCards are now considered legacy however as of this post June 2018 both the Outlook desktop  and the Team's clients don't support adaptive cards in their current general releases.  Eg for Adaptive Cards support in Teams you will need the developer preview and in the Outlook Desktop client you need to be on the Monthly Channel with version 1085. This is neither good nor bad but if you find Adaptive Cards the exact thing you need to solve your problem you should be aware of client support status before you start your planning. 

Examples

For this post I've created both a MessageCard PowerShell sample that can use be used in either Outlook or Teams and a Adaptive Cards sample that can be used primary in OWA at the moment. The to produce just the MessageCard can be found https://github.com/gscales/Powershell-Scripts/blob/master/CreateMessageCard.ps1 and the Adaptive Card https://github.com/gscales/Powershell-Scripts/blob/master/CreateAdaptiveCard.ps1

Microsoft Teams - In/Out - Out of Office Teams Message card

This script will produce the following output to a Teams Channel using a incoming WebHook to receive the Messages



This example uses my Exch-REST module available from the PowerShell Galleryhttps://www.powershellgallery.com/packages/Exch-Rest and GitHub https://github.com/gscales/Exch-Rest . This contains all the code to first get the Group associated with a particular team, get the members of that groups mail Addresses, make the Graph Mail tips request and then package the results as JSON message which can then be posted to a WebHook.

To Setup a WebHook for the script to post to on a Team see https://docs.microsoft.com/en-us/microsoftteams/platform/concepts/connectors#setting-up-a-custom-incoming-webhook


 So lets look at how this all fits together



Connect-EXRMailbox -mailbox mailbox@domain.com
$GroupName = "A Team" $Group = Get-EXRModernGroups -GroupName $GroupName $Members = Get-EXRGroupMembers -GroupId $Group.id
The above code first gets the ModernGroup which is associated with the Team you are working with using the Teams DisplayName. It then uses the GroupId that is returned to make a request to Get the Members of that group.



$OOFMailboxCol = @() $mtHash = @{} foreach($Member in $Members){     if(![String]::IsNullOrEmpty($Member.mail)){         $OOFMailboxCol += $Member.mail         $mtHash.Add($Member.mail,$Member.displayName)     }  }
This code creates a collection of Mailboxes to use the Get-EXRMailTips operation and also creates a hashtable to allow us to map the email address in the Mailtips response back to the DisplayName because by default MailTips only gives you back the EmailAddress.



$Mailtips = Get-EXRMailTips -Mailboxes $OOFMailboxCol -tips "automaticReplies"
$FactsColl = ($Mailtips | select @{Name='EmailAddress';Expression={$mtHash[$_.emailAddress.address]}},@{Name='OOF Message';Expression={$val =($_.automaticReplies.message -replace '<[^>]+>','').Trim();if([String]::IsNullOrEmpty($val)){"In"}else{"Out : " + $val}}})
$WebhookAddress = "https://outlook.office.com/webhook/cc1b7a"
Invoke-WebRequest -Uri $WebhookAddress -Method Post -ContentType 'Application/Json' -Body (New-EXRMessageCard -Facts $FactsColl -Summary "Team OOF Status" -Title "Team OOF Status")
The first line of the above code makes the MailTips request for the addresses that are passed into it. The next line builds a Collection of Facts which is a Messagecard element that display a two column table. I've used expressions to clean up the results as the MailTips response is a HTML value that needs to be formatted back to text for the card and I've added some In/Out prefixes based on the response. The last line of code is what submits the Message card to the configured webhook address in the $WebhookAddress variable.

The Full Sourcecode for this script can be found https://github.com/gscales/Exch-Rest/blob/master/Samples/SendOOFMessageCard.ps1


Adaptive Card to show the Office365 Service health status

Pre-requisites   

As I mentioned previously if your going to send an Adaptive Card as an email you need to be registered in the portal and include the originator Id you receive in your adaptive Card. For testing what you can do is send an Adaptive Card to yourself (from yourself) and without the originator property being set and it will appear okay. But if you where to send it to someone else the content will be stripped from the message.

Example

One of the other things that you can get from the manage.office.com endpoint that I talked about in this post is the Service Health information about each of the different workloads in Office365. In this example I use the Management API to produce the following adaptive card example eg



Adaptive Cards have a lot more formatting options for example you can have a column set and use different font colors. To make the script as dynamic as possible I do the color switching using a hashtable. This example uses my Exch-REST module available from the PowerShell Galleryhttps://www.powershellgallery.com/packages/Exch-Rest and GitHub https://github.com/gscales/Exch-Rest  . The script does the following


$To = "gscales@datarumble.com" Connect-EXRMailbox -MailboxName "gscales@datarumble.com" Connect-EXRManagementAPI -UserName "gscales@datarumble.com" $StatusArray = Get-EXRMCurrentStatus | Select-Object WorkLoad,Statusdisplayname,StatusTime 
In the above code it makes a connection to the Mailbox and ManagementAPI and then builds an collection for the service health information.


$ColorSwitchHash = @{} $ColorSwitchHash.Add("Service degradation","attention") $ColorSwitchHash.Add("Normal service","good")
This code sets up a HashTable that will be used to display a different color for the service status column based on its status. Note the Adaptive Cards support a subset of the MarkDown format so attention (red), good (green) etc.



$Card = New-EXRAdaptiveCard -Columns $StatusArray -ColorSwitchColumnNumber 1 -ColorSwitchHashTable $ColorSwitchHash -ColorSwitchDefault warning Send-EXRAdaptiveCard -To $To -Subject "Office 365 Service Status" -AdaptiveCard $Card
The last part of the code generates the AdatpiveCard, the ColorSwitchColumnNumber is a zero based array to match with the collection that is passed in. What Send-EXRAdaptiveCard does is wraps the JSON from the Card in the following script which is necessary when you send a message.


<script type="application/adaptivecard+json"></script>



Hire me - I'm currently available to help with any Exchange\Office365 related development work or scripting please contact me at gscales@msgdevelop.com (nothing too big or small). I'd also be interested in hearing from any companies that want to sponsor open source projects around Exchange and Office365 development.

Wednesday, June 13, 2018

Using the Office 365 Management Activity API from Powershell to audit Exchange and Office365

The Office 365 Management Activity API is a REST endpoint that can be used to access audit events from user, admin, system, and policy actions and events in Azure and Office365 workloads (its been around for a while first appeared in 2015 in preview). Within Office365 there are many ways of accessing this type of information the Primary UI these days for looking at auditing is the Unified Audit logs available in the Office365 Portal https://support.office.com/en-us/article/Search-the-audit-log-in-the-Office-365-Security-Compliance-Center-0d4d0f35-390b-4518-800e-0c7ec95e946c#PickTab=HowTo

The advantage of using the Management Activity API over the portal is that  allows you to create more tailored reporting by accessing the raw data over a period of time (so writing your own user trend reporting) using a scalable API like REST (which if you have a lot of data to query is superior to using WinRM solutions using something like the Exchange Management Shell cmdlets). It also includes WebHooks interfaces that can be used to trigger other workloads to process particular Audit events on a continuing basis. (Although you should note the documentation states that this API isn't meant to be used for real time alerting as there are other Microsoft provided endpoint and services to provide this functionality).
How it works

The Management Activity API uses a subscription based approach, the mechanics are relatively simple eg 


  • You create a Subscription for the Workload you want to access the Audit event for.(eg Exchange)
  • The backend then produces contentblobs which are aggregates of those audit logs and then makes them available to access. If you have registered a WebHook for your subscription then your endpoint will be notified that there is a new ContentBlob available for you to process.
  • Once you no longer require the subscription you can then Stop that subscription.

One point to note is that the Management Activity API doesn't actually enable auditing for any of the workloads it just make available data from the Auditing that is already enabled. Eg for instance Exchange Mailbox Auditing isn't turned on by default so if you did the above process to create a subscription for Exchange you would never see any ContentBlobs generated for Mailbox Auditing events unless you first went and enabled auditing on the Mailboxes you wish to audit https://support.office.com/en-us/article/enable-mailbox-auditing-in-office-365-aaca8987-5b62-458b-9882-c28476a66918 and then these events would be available within your subscription. This is the same for other workload as well, some workloads like AzureAD have some auditing enabled by default but you should always check your auditing configuration to make sure you have turned on the particular auditing setting for the particular workload your interested in before using this API.

Webhooks

With the Management Activity API the subscription mechanism has the ability to send a WebHook notification to any Webhook endpoint you configure. One easy to use webhook example is Azure RunBooks https://docs.microsoft.com/en-us/azure/automation/automation-webhooks so using something like this your subscription could trigger a RunBook which would then process the ContentBlob that has just been made available and apply any Logic or custom report you need. The good thing with RunBooks is all the underlying service parts of the code are done for you and you can just plug in your custom script to do what you need.

Prerequisites 

The prerequisites for this API is that you need to have an Application registration see created that has been  given the oAuth Grants to access the API. You need to generate your access tokens for the resource url manage.office.com (so for example you can't use a token you have generated against the Graph endpoint you need to specifically request this ResourceURL). The last thing is the Account you then use to the access the API needs to have rights to the Audit data. (If you choose to use certificate authentication and a daemon type app this would get the rights through the different oAuth Grants for applications).

Putting it to use

A really straight forward way of using this API from PowerShell is to make use of Invoke-WebRequest and there is a good document here on doing this https://msdn.microsoft.com/en-us/office-365/troubleshooting-the-office-365-management-activity-api . Another easier and more functional approach is to use my Exch-REST module which is available from the PowerShell Gallery https://www.powershellgallery.com/packages/Exch-Rest and GitHub https://github.com/gscales/Exch-Rest which now has some cmdlets and plumbing to enable use of this API. The module can now handle caching tokens from multiple resources (eg the Graph and Management API) so you can then combine operations from both API's which is where you can then start to build more powerful tailored reports. Eg if your processing the Content Blobs for particular Mailboxes and you want more information about that mailbox to include in a report you can use the Graph to easily access that. Or if you wanted to check on a Message that was deleted by a user to see if it had any attachments that could contain sensitive content you could use the Graph API to reach into the Mailbox and access the deleted message from the Recoverable Items folders etc.

Using Exch-REST to create and access subscription content

Connecting and Generating the Access Token 

To Connect and generate an Accesstoken to use against the Management API use Connect-EXRManagementAPI


I've created a default app registration that you can use for testing (number 5) that just has access to the Management API Oauth Grants but would recommend as usually that you create your own App registration  see so you control what rights the code will have. One other important thing to note is the subscription are per appid, so if you create a subscription with say the above AppId and then create your own ApplicationId at a later point and use that you won't be able to see/view/stop the subscriptions you created with the pervious appId unless your login using that pervious ApplicationId.

Once you have a token you can then use the Management API cmdlets for example to show the current subscriptions use Get-EXRMSubscriptions


If this is the first time you are using this API it will just show a blank list. To create a new subscription use the New-EXRMSubscription cmdlet with the switch for the workload you want to use eg to create a New AzureADSubscription use the -AzureAd switch


other workload switch are

-Exchange
-SharePoint
-General (other workloads like Teams are included here)
-DLP

If you want to use WebHooks there are specific parameters for this that allows you to specify the necessary Webhook details.

Once you have created the subscription the content blobs don't get created straight away the documentation say that this will take about 6-24 hours for content to start to become available. Once they are available you can access them using the Get-EXRMSubscriptionContent cmdlet eg to get the Exchange content for the last 24 hours

Get-EXRMSubscriptionContent -Exchange -StartTime (Get-Date).AddDays(-1) -EndTime (Get-Date)

The data you get back for this operation contains the contenturi of the blob that you can then access using the SubscriptionContentBlob operation. I have a separate cmdlet that can be used to retrieve the blob eg


Get-EXRMSubscriptionContentBlob -ContentURI https://manage.office.com...
However I found the useability of doing it this way to not be so great so I included a switch in the Get-EXRMSubscriptionContent cmdlet so you can specify to return the contentblobs eg


Get-EXRMSubscriptionContent -Exchange -StartTime (Get-Date).AddDays(-1) -EndTime (Get-Date) -returnContentBlobs
eg this will the return the contentblob with each content entry


You can then process the Contentblob property and the data within anyway you like, here is one example that produces a Client report using the client agent data for the last 24hours


$Last24Results = Get-EXRMSubscriptionContent -Exchange -StartTime (Get-Date).AddDays(-1) -EndTime (Get-Date) -returnContentBlobs
$BlobEntries = foreach($ContentEntry in $Last24Results){$ContentEntry.ContentBlob}
$BlobEntries | Where-Object{$_.ClientInfoString -ne $null} | select  CreationTime,Operation,ClientInfoString,Use rId | fl

Or you could just look at things like the MoveToDeletedItems events and get more information


Once your finished with a Subscription and you no longer want to process content blobs you can stop the subscription using Invoke-EXRMStopSubscription eg to stop the Exchange Subscription use

Invoke-EXRMStopSubscription  -Exchange

The combinations of what you can do are just limited to your own imagination or particular audit bucket you need to fill. If you have need for a developer/scripter or just someone for to help out for anything exchange/office365 related I'm available to take on work at the moment so please contact me at gscales@msgdevelop.com (nothing too big or small). I'd also be interested in hearing from any companies that want to sponsor open source projects around Exchange and Office365 development.

The Exch-REST module is available from the PowerShell Gallery https://www.powershellgallery.com/packages/Exch-Rest and GitHub https://github.com/gscales/Exch-Rest