Friday, January 11, 2019

Create a Microsoft Teams Group Calendar tab application using the Graph API and FullCalendar JavaScript library

Group calendars have always been one of the big asks for in any group collaboration programs back from Lotus Notes to Microsoft Exchange and now Microsoft Teams. There are a few ways of getting a Group calendar working in Teams, one is hosting the OWA web apps see or some other people advocate using a SharePoint calendar and hosting that similarly.

Here is a different method you can use by taking advantage of being able to call the Graph API in a  Tab application.The getSchedule Graph action (still currently in beta) allows you to query up to 62 days of Freebusy information on up to 100 calendars in a single call this makes it a good option for this type of application. So as long as users can view each others calendars or have detailed freebusy permissions the action should return the level of detail required for a Group calendar.   The other thing you can do with the Graph API is get the users photo and build a nice legend for the Group calendar also. To make this visually appealing you need a good calendar display library which there has been a few over the years but a recent one that is really nice is FullCalendar https://fullcalendar.io/ . It ticks all the boxes it looks great, its easy to use and its free to use. To make the calendar appointments from graph appear in the calendar is as easy as building and array from the return JSON from the graph and throwing in a little random color code to break this up and then stitching in the user photos as they are returned asynchronously from the server to build the legend.


 Here are some screenshots of the Tab in action using Graph data




    Daily view


Weekly view


List view


I've put together a separate GitHub repository for all the code that is required for this app https://github.com/gscales/TeamsGroupCalendar and put some detailed install instruction in the Readme in Github.

I'm currently looking for work either contract or fulltime so if you need a creative developer with lots of energy to write C#,JS, NodeJS, Azure or Lambda functions, Messaging DevOps or PowerShell scripts then please contact me  at gscales@msgdevelop.com

Tuesday, January 08, 2019

Converting Folder and ItemIds from the Exchange Management Shell and Audit Log entries using PowerShell and the Graph API in Exchange Online

First a little news about Exchange Identifiers that you may have missed (its not often that something like this changes so its rather exciting)

When you access an item in an Exchange Mailbox store whether its OnPrem or in the Cloud you use the Identifier of the particular item which will vary across whatever API your using. Eg

MAPI - PR_EntryId eg NameSpace.GetItemFromID(EntryId)
EWS -  EWSId eg EmailMessage.Bind(service,ewsid)
Rest -   RestId  eg https://graph.microsoft.com/v1.0/me/messages('restid')
The advice over the years has always been its not a good idea to store these Id's in something like a database because they change whenever and Item is moved. Eg if an Item is moved from the Inbox to a Subfolder in the Inbox it will received a different Id so whatever you have stored in your database suddenly becomes invalid and its not easy to reconcile this. However a new feature that has appeared in Exchange Online in Beta with the Graph API is immutableId's see https://docs.microsoft.com/en-us/graph/outlook-immutable-id the idea behind this is that this Id doesn't change regardless of which folder the item is moved to (or even if its deleted). While it still in Beta at the moment this is a good feature to use going forward if your building synchronization code. Along with immutableId's an operation to Translate Id's between the EntryId, EWS and REST formats is now available in beta in the Graph which is great if your looking to Migrate your MAPI or EWS apps to use the Graph API https://github.com/microsoftgraph/microsoft-graph-docs/blob/master/api-reference/beta/api/user-translateexchangeids.md 

As Audit records are a hot topic of discussion this week with this post from Microsoft another Identifier format you see when using the Exchange Management Shell cmdlets like Get-MailboxFolderStatics is something like



or in an ItemId in a  AuditLog Record like



With these Id's there are just a base64 encoded version of the EntrydId with a leading and trailing byte. So to get back to the Hex version of the Entryid you might be familiar with from a Mapi Editor you can use something like the following



$HexEntryId = [System.BitConverter]::ToString([Convert]::FromBase64String($_.FolderId.ToString())).Replace("-","").Substring(2)  
$HexEntryId =  $HexEntryId.SubString(0,($HexEntryId.Length-2))

This would turn something like

RgAAAAC+HN09lgYnSJDz3kt9375JBwB1EEf9GOowTZ1AsUKLrCDQAAAAAAENAAB1EEf9GOowTZ1AsUKLrCDQAALbJe1qAAAP

Into

00000000BE1CDD3D9606274890F3DE4B7DDFBE490700751047FD18EA304D9D40B1428BAC20D000000000010D0000751047FD18EA304D9D40B1428BAC20D00002DB25ED6A0000

Just having the Id in whatever format isn't much good unless you can do something with it, so I've created a simple Graph script that uses the new user-translateexchangeids.md operation to allow you to translate this Id into an Id that would be useable in other Graph requests. I've create a basic ADAL script version an posted it here on my GitHub https://github.com/gscales/Powershell-Scripts/blob/master/translateEI.ps1

A quick Demo of it in use eg Translate a RestId into an EntryId


Invoke-TranslateExchangeIds -SourceId "AQMkADczNDE4YWE..." -SourceFormat restid -TargetFormat entryid
By default the operation returns a urlsafe base64 encoded results (with padding) so in the script I decode this to the HexEntryId which I find the most useful.

I've also cater for allowing you to post a HexEntryId and the script will automatically encode that for the operations eg


Invoke-TranslateExchangeIds -SourceHexId "00000000BE1CDD3D9606274890F3DE4B7DDFBE49..." -SourceFormat entryid -TargetFormat restid
And it also caters for the encoded EMS format and will strip the extra bytes and covert that eg

Invoke-TranslateExchangeIds -SourceEMSId  $_.FolderId.ToString() -SourceFormat entryid -TargetFormat restid
I've also added this to my Exch-Rest module which is available from the PowerShell Gallery and GitHub which is useful if you want to do some following type things. eg if you wanted to bind to the folder in question you could use


$folderId = Invoke-EXRTranslateExchangeIds -SourceEMSId  $_.FolderId.ToString() -SourceFormat entryid -TargetFormat restid
Get-EXRFolderFromId -FolderId $folderId
Need help with anything I've talked about in this post or need somebody to write C#,JS, NodeJS, Azure or Lambda functions, Messaging DevOps or PowerShell scripts then I'm available now for freelance/contract or fulltime work so please drop me an Email at gscales@msgdevelop.com

Friday, January 04, 2019

Using the Skype for Business UCWA API in a Microsoft Teams Tab application to show the Skype Conversation history

One of the things you maybe considering in the new year is migrating from Skype for Business to Microsoft Teams. In this post I'm going to demonstrate how you can use the UCWA api (which is the REST API you can use to talk to a Skype for Business server either in Office365 or OnPrem) to access Skype for Business from within the Teams Client via a Teams Tab application. (For those unacquainted with UCWA this the API that is used to Access Skype within OWA).
Why would you want to do this ? its one way of easing migration friction by providing a different level of interoperability (outside of using both clients) and also a way of adding functionality into the Teams client that isn't there currently.  In this post I'm going to look at showing the users Skype conversation history, while this information is also stored in a users Mailbox and also accessible via the Graph API, in this app I'm going to use the UCWA API to access the conversation logs via the Skype for Business Online servers and also the conversation transcripts. What you end up with is a Teams  tab that looks like the following



and the Transcripts like (this is POC so mind the formatting)



Using UCWA from a Teams Tab Application

There isn't much difference between using the UCWA API in a Teams Tab application then  from using it in any other application, however UCWA does present some challenges around authentication because of the way the discovery process works.  For a quick recap for those unacquainted please read https://docs.microsoft.com/en-us/skype-sdk/ucwa/developingucwaapplicationsforsfbonline . As part of that process you need to get an AccessToken to make a discovery request to find the SK4B pool server to use and then get another AccessToken for the pool server. So when using the Teams Tab Silent authentication flow you need to execute this twice (which is different and more time consuming then say a normal Graph type application)

Getting the Conversation History in UCWA

Once you have logged into UCWA you need to configured the session to enabled the conversation history as its disabled by default.  This involves doing a Put request against the application resource. with the if/match header set to the ETag. You then need to acknowledge the event this will generate and once that is done your UCWA session will be ready to go, you then just need to query the communication resource to get the links required to access the ConversationLogs from the server. In the sample app I'm only accessing the last 50 items from the server as this is only a POC anyway. When it comes to access the conversation transcripts this requires a Batch request to make it efficient (the max batch size in SK4B in 100) so using a page size of 50 keeps this all working okay. A brief overview of the requests required to access the Conversation history.

  • 1 Get request to get the Conversation Logs which is a list with a link to each of the Conversation Entries
  • Batch Get Request for each of the Conversation Entries which gives back a detail history of each conversation (minus the actual Transcript of the conversation but you do get limited Message preview).
  • If you want the full conversation transcript you use the link from the conversation history to access the Transcript. (in the sample when you click the transcript Cell in the Table it makes this request to the SK4B server to get the Transcript and presents that in a separate Div on the page),
Installing and using this Tab Application 

Like any Teams Tab application it must be hosted somewhere, I'm hosting it out of my GitHub site so the configuration file located in https://github.com/gscales/gscales.github.io/blob/master/TeamsUCWA/app/Config/appconfig.js has the following configuration to ensure it point to the hosted location

const getConfig = () => {
   var config = {
        clientId : "eed5c282-249f-46f3-9e18-bde1d0091716",
        redirectUri : "/TeamsUCWA/app/silent-end.html",
        authwindow :  "/TeamsUCWA/app/auth.html",
 hostRoot: "https://gscales.github.io",
   };
   return config;
}

Also the manifest file https://github.com/gscales/gscales.github.io/blob/master/TeamsUCWA/TabPackage/manifest.json  has setting that point to hosted that need to be changed if its hosted elsewhere (just search and replace gscales.github.io)

Application Registration 

To use the UCWA API you need to create an application registration with the following oAuth grants



the applicationId for this registration should then be used to replace the clientid in the appconfig.js . The application registration should use the silent-end.html page as the redirect for authentication. Then the last thing you need to do is make sure that the ApplicationId has been consented to in your Organization eg 


https://login.microsoftonline.com/common/adminconsent?client_id=08401c36-6179-4bbe-9bcc-d34ff51d828f
  
Side Loading - To use custom tab applications you first need to enable side loading of Apps in the Office365 Admin portal ref .The important part is  "Sideloading is how you add an app to Teams by uploading a zip file directly to a team. Sideloading lets you test an app as it's being developed. It also lets you build an app for internal use only and share it with your team without submitting it to the Teams app catalog in the Office Store. "

As this is a custom application you need to use the "upload a custom app" link which is available when you click ManageTeam-Apps tab see

(Note if you don't see  the "upload a custom app" check that you have side loading of apps enabled in your tenant config)

What you upload here is a Zip file containing your manifest file and two images that you manifest refers to for eg
{
   "icons": {
    "outline": "Outline32.png",
    "color": "Colour192.png"
  },

For this sample this is located in https://github.com/gscales/gscales.github.io/blob/master/TeamsUCWA/TabPackage/manifest.json

All the code for this post is located in GitHub https://github.com/gscales/gscales.github.io/tree/master/TeamsUCWA

Need help with anything I've talked about in this post or need somebody to write C#,JS, NodeJS, Azure or Lambda functions or PowerShell scripts then I'm available now for freelance/contract or fulltime work so drop me an Email at gscales@msgdevelop.com

Monday, December 17, 2018

ZAP (Zero-hour auto purge) Junk email reporting for Office365 using EWS and REST

Zero-hour auto purge is one of the features of Office365 that will detect malicious and Spam emails and move them to the Junk email folder for any email that has breached the first level defences and has been delivered to users mailboxes. There is a good description of how it works here but basically when the service learns a particular message was malicious/spam it can retrospectively detect and eliminate/move any simular messages that arrived previously and weren't detected.

This is a good and much need feature as no AntiSpam or Malware solution is perfect (no matter what the vendor say) so there will always be the case where thing slip through. But this very fact is what causes an exposure point where the potentially malicious email sits in the Inbox of end user up until the time its gets zapped. What I wanted to present in this post is a few ways you can measure the amount of the time you may have been vulnerable for and show some methods you can use to look more at messages and it's potential malicious content.

How to detect messages that have been zapped in a EWS and REST script

There is a good reference article for this here , what happens when a Message is Zapped and moved to the Junk Email folder is a Internet Message Headers is added which will also create a underlying Extended property see



We can use  this in a EWS or REST script to do some reporting on. In EWS we can use an Exists Search filter on Messages in the JunkEmail folder to find just messages where this property has been set meaning that these messages have been zapped

 $ZapInfo = new-object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition([Microsoft.Exchange.WebServices.Data.DefaultExtendedPropertySet]::Common, "X-Microsoft-Antispam-ZAP-Message-Info", [Microsoft.Exchange.WebServices.Data.MapiPropertyType]::String)
 $Sfexists = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+Exists($ZapInfo) 

In the Graph API we can also do something simular using the following filter

$filter=singleValueExtendedProperties/any(ep: ep/id eq 'String {00062008-0000-0000-c000-000000000046} Name X-Microsoft-Antispam-ZAP-Message-Info' and ep/value ne null) 

What this does is returns us a collection of Messages that have been zapped, I went a step further in my script and put a DateTime filter around this as well (but technically the Junk Email folder has a default retention period of 30 days so it shouldn't really have a large volume of email). Once you have messages return if you check the datetime the message was received and then using the Time-Span function in PowerShell calculate the TimeSpan  against last modified time (which should have been the time the Message was Zapped and moved to the JunkEmail folder) this will give you a good indication of the time that these messages sat in the Inbox of the user (this becomes your vulnerability period). You can also look at the read setting of the Email to determine if the user had actually read the Message that was Zapped. I've gone a another step further in my reporting script to also do some Antispam analysis of the email headers using some code I previously wrote so you can also look the DKIM,DMARC etc values this mail received as well. So in the end what my reporting script does is checks for Zapped messages in the Junk Email folder for a specific time period and then produces a report on the actually exposure time and relevant information around this so it can be further evaluated. The output of the report is something like this

I've put the EWS script that can do this report on GitHub https://github.com/gscales/Powershell-Scripts/blob/master/ZapStatistics.ps1 To run a Report on a particular mailbox use

 Get-ZapStatistics  -MailboxName mailbox@datarumble.com -startdatetime (Get-Date).AddDays(-14)  | Export-Csv -Path c:\Reports\mailboxName.csv -NoTypeInformation

I've also added the same type of script to my Exch-Rest Module so you can do the same thing using the Microsoft Graph API

 Get-EXRZapStatistics  -MailboxName mailbox@datarumble.com -startdatetime (Get-Date).AddDays(-14)  | Export-Csv -Path c:\Reports\mailboxName.csv -NoTypeInformation

The module is available from the PowerShell Gallery or GitHub

DevOps - Looking at deeper analysis and proactive measure

I thought I'd start including a DevOps section in some of my posts to show how you can delve a bit deeper to look what we are reporting on and some potential proactive measures you might be able to put in place to earn such a DevOps tag. In this section I'm assuming you know enough about development to know your way around objects,methods,properties etc. I'm going to be using my Exch-Rest module because its the best tool I have to do this and its also a good way to improve the module itself (and its free).

Analysis 

So first lets just look at one of the Messages that have been zapped to do this just pull the Messages into a collection like

$Messages =  Get-EXRZapStatistics  -MailboxName mailbox@datarumble.com -startdatetime (Get-Date).AddDays(-14)

Then dump out the first message in the Collection

$Messages[0]

This give us something like



So we have a whole bunch of interesting information about the source that is trying to essentially attack or steal the users credentials. We have

Source SMTP server IP Address in the CIP and SPF Values
We have the Reverse DNS PTR (which tells us the sending server country location)
The HostName of the SMTP server which does resolve in DNS to the same IP Address as the PTR record but the HostName doesn't match
The SCL value of 1 which is pretty low

I can see the message was Read by the user (which isn't good if this is a user). So the next thing that might be useful is looking at the content of the messages to see if it has any attachments. We can do this using the InternetMessageId and the following

$message = Find-EXRMessageFromMessageId -MailboxName gscales@datarumble.com -MessageId "<57a196603977b4b3d9e0680d3119d816 baseny-wroclaw.pl="">"

This gives us some information like the following


So this tells us right away there where no Attachments so that is one less thing to worry about it also shows the they tried to fool the target user by putting in a different email address in the SenderName then the actual senders Email address as they where trying to make it appear as if it was a system generated message that came from Microsoft itself.

So now we know that there where no attachments it probably just has a link they want the user to click. To get more information on links in the body of the Message we can use something like the following

$expandedEmail = Get-EXREmailBodyLinks -MailboxName gscales@datarumble.com  -InternetMessageId "<57a196603977b4b3d9e0680d3119d816 baseny-wroclaw.pl="">"

and then we can view the links in the email like


Bingo there you have an attempt to fake the domain name to make it look like something that a user may have used before so they might feel confident entering their details into etc. At this point you might want to trawl through what ever logs you have available to see if a user did actually visit that URL and take some urgent action if they did.

Being Proactive

So what I've gone through above are some manual steps above to show that the while you can be assured that the cloud is doing its job to a certain extent there are some extra measures you can take to keep a closer a eye on what potential these malicious email are trying to do and catch any unsuspecting users before something like this becomes a bigger problem for you. The cloud isn't set and forget and if your not doing some extra checks on what is happening in the service your not being as an effective for the company you work for (or on behalf of) as you can be. Some other quick DevOps ideas

  • Setup a Azure Runbook that uses Certificate Authentication and AppOnly tokens that will report on the Zapmessages and do some further automated analysis on a Daily or Weekly basis.
  • Look for correlations based on the information you have in the Zapped messages, if your being targeted by someone there is a good chance that location data, Ipaddress from one attacked maybe used in subsequent ones. So apply those patterns to look at your Message Tracking logs which will give you details about SenderIp ect.
Hire me - If you would like to do something similar to this or anything else you see on my blog I'm currently available to help with any Office365,Microsoft Teams, Exchange or Active Directory related development work or scripting, please contact me at gscales@msgdevelop.com (nothing too big or small).   




Monday, December 10, 2018

Updates to the Exch-Rest PowerShell Module to support PowerShell Core, Azure Cloud Shell and more ADAL integration options

I've had some time recently to do some much needed updates to my Exch-Rest module so it now supports both Azure Cloud Shell and PowerShell Core on Linux (tested on RHEL,CentOS, Debian and Ubuntu). So now you can logon to an Office365 Mailbox using this Module with Powershell on Linux and send Email or a Skype for Business Message or do some mailbox reporting eg


The requirements on Linux is you need to be using the latest version of PowerShell core installed as per https://docs.microsoft.com/en-us/powershell/scripting/setup/installing-powershell-core-on-linux?view=powershell-6 .This ensures that all the required .net Core libraries will be available as older version of .Net core didn't have some of the libraries I'm using and I didn't want to backport for older versions.  Also because there are no Linux forms to interact with for authentication you need to pass in the credentials to use via a PSCredential and the code will use the password grant to get the Token eg
$cred = Get-Credential -UserName gscales@datarumble.com
connect-exrmailbox -MailboxName gscales@datarumble.com -Credential $Cred
Azure Cloud Shell

As Cloud Shell is a browser based version of  PowerShell core running on Linux the same connection method of using the credentials as above is needed.

ADAL Integration

I've also added full integration with the ADAL library for authentication so in addition to the native dependency free script methods the module now distributes the ADAL libraries and supports Authentication using that library as well as Token refreshes for scripts that run over an hour etc (using Acquiretokenasync in the ADAL). This supports the following scenarios such as
To use the ADAL libraries for Logon use the following 

connect-exrmailbox -MailboxName gscales@datarumble.com -useADAL

To use the Never Prompt to use the ADAL Cache
connect-exrmailbox -MailboxName gscales@datarumble.com -useADAL -Prompt Never
For connecting using the currently logged on credentials use
connect-exrmailbox -MailboxName gscales@datarumble.com -useADAL -useLoggedOnCredentials -AADUserName gscales@datarumble.com

(The -AADUserName variable is optional but usually required read the GitHub link in the second bullet point)

For those who want to do something simular and are using EWS you will need something like the following to get the AccessToken using ADAL in a normal PowerShell Script.

    $ClientId = "d3590ed6-52b3-4102-aeff-aad2292ab01c"
    $ResourceURI = "https://outlook.office365.com"
    $EndpointUri = 'https://login.microsoftonline.com/common'
    $Context = New-Object Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext($EndpointUri)
    $AADCredential = New-Object "Microsoft.IdentityModel.Clients.ActiveDirectory.UserCredential" -ArgumentList "username@domain.com"
    $authResult = [Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContextIntegratedAuthExtensions]::AcquireTokenAsync($Context, $ResourceURI, $ClientId, $AADcredential)
    if ($authResult.Result.AccessToken) {
        $token = $authResult.Result
    }
    elseif ($authResult.Exception) {    
        throw "An error occured getting access token: $($authResult.Exception.InnerException)"    
    }
    return $token

This also goes along with support for using the Module in an Azure Runbook which was added last month.

The Exch-REST Module is available from the PowerShell Gallery https://www.powershellgallery.com/packages/Exch-Rest and GitHub https://github.com/gscales/Exch-Rest

A big thankyou to all those people who provided feedback on using the module hopefully these upgrades make it easier to use in more scenarios.

Hire me - If you would like to do something similar to this or anything else you see on my blog I'm currently available to help with any Office365,Microsoft Teams, Exchange or Active Directory related development work or scripting, please contact me at gscales@msgdevelop.com (nothing too big or small).