Friday, March 27, 2020

Migrating your Mailbox searches in EWS to the Graph API Part 1 Filters and Search Folders

This is part one of a two part post where I'm going to look at how you can migrate any searches you are doing in EWS to the Graph API. In this first part I'm going to cover SearchFilters (from EWS) and Search-Folders as they have been around the longest and in part 2 I'll look at Searches which has  some new functionality in beta in the Graph.

Lets start by looking at how you might be doing searches in EWS at the moment

  • Search Filters (restrictions) in a FindItem Request that can be run against a Folder or Search Folder
  • QueryString (KQL) in a FindItem Request that can be run against a Folder or Search Folder
  • SearchFolder with a FindItem Request
  • eDiscovery via SearchMailbox which has now been depreciated in Office 365 and no longer supported
Search Filters (Restrictions)

If you have used the EWS Managed API to build your application you use the SearchFilter class which creates a underlying restriction in EWS https://docs.microsoft.com/en-us/exchange/client-developer/web-service-reference/restriction.  The term Restriction came from the Exchange ROP's protocol which is what MAPI uses to talk to the Exchange Store.

In the Microsoft Graph the language you talk in regards to filtering is OData  
OData (Open Data Protocol) is an ISO/IEC approved, OASIS standard that defines a set of best practices for building and consuming RESTful APIs ref https://www.odata.org/
OData filters are therefore a standard that anybody implementing the protocol (like Microsoft have done with the Graph API) should adhere to.

Lets look at some examples (from this blog) of SearchFilters I've used and how they can be converted to Graph oData Filters.

Easy - the easiest query to make is against one of the strongly typed properties like Subject or Sender eg in EWS you might have a search filter like this

SearchFilter SubjectFilter = new SearchFilter.IsEqualTo(ItemSchema.Subject, "Subject");

in the Graph this would just look something like this (applied just against the Inbox)

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?$filter=subject eq 'test'

This is an Equality search some other things you can do is

Startswith (which you couldn't actually do in EWS)

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?$filter=startswith(Subject,'test')

Sub-String Searches

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?$filter=Contains(Subject,'test')

With the later two searches if you have a folder with a large item count then these aren't going to perform like an equality or a Content Index Search would and its a possibility that they will timeout the first time you run the query. (Then on subsequent queries may succeed this is due to the way Exchange applies temporary restrictions for dynamic searches).

Medium - The most used search filter in EWS for me is that date restriction where you want to restrict the emails returned to a certain time frame so an EWS Search Filter like the following

$Sfgt = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+IsGreaterThan
([Microsoft.Exchange.WebServices.Data.ItemSchema]::DateTimeReceived, $Startdatetime)
$Sflt = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+IsLessThan
([Microsoft.Exchange.WebServices.Data.ItemSchema]::DateTimeReceived, $Enddatetime)
$sfCollection = new-object Microsoft.Exchange.WebServices.Data.SearchFilter
+SearchFilterCollection([Microsoft.Exchange.WebServices.Data.LogicalOperator]::And);
$sfCollection.add($Sfgt)
$sfCollection.add($Sflt)


In the Graph this would look like

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$filter=(receivedDateTime gt 2020-03-24T13:01:50Z) AND (receivedDateTime lt 2020-03-25T12:59:50Z)

(Just watch the date formatting)

Hard - If your using Extended properties in your SearchFilters then you need to include the Extended property definition in the Filter and use the lambda any or all expression

The first Query looks for Items based on the underlying Message Class (which isn't exposed as a strongly typed property in the Graph)

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$filter=singleValueExtendedProperties/any(ep:ep/id eq 'String 0x001a' and ep/value eq 'IPM.Note')

Another useful filter is to be able to find Messages where a particular property has been set eg this filter looks for messages that have the In-Reply-To header set (So Repsonses only)

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$filter=singleValueExtendedProperties/any(ep: ep/id eq 'String 0x1042' and ep/value ne null)

For non String properties for instance the Message size you need to make sure you cast the value to the Json datatype. Eg to find all messages that are larger the 10 MB you could use

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$filter=singleValueExtendedProperties/any(ep:ep/id eq 'Integer 0x0E08' and cast(ep/value, Edm.Int32) gt 1048576)

In EWS when you did a search that returns a large number of Items that was paged you got back also the total number of items that matched your search query. I've used this in the past to get statistical information about email with a filter without needing to page all the results. The Graph offers the same thing using the $count query parameter eg if I change the above query to find all messages above 1 MB in my mailbox and include the $count parameter this is what I get for my mailbox



Another example of this maybe to look at the count of messages in 2018

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$count=true&$filter=(receivedDateTime gt 2018-01-01) AND (receivedDateTime lt 2019-01-01)

And finally one last example is for pidTagHasAttachments because I know myself and others use this (because it gives a different value than the strongly type hasAttachments for various reasons)

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages?
$filter=singleValueExtendedProperties/any(ep:ep/id eq 'Boolean 0x0E1B' and cast(ep/value, Edm.Boolean) eq true)

Hopefully I've covered off enough examples here for anybody stuck with syntax to be able to get their head around it. If you do have problems try posting a question into stack overflow here

SearchFolders

SearchFolders give you a way of creating a Virtual Folder that represent an underlying restriction (or Search) that can span one folder or the whole mailbox. While they are more suited to static type searches if you have ever used the mapi fiddler inspector https://github.com/OfficeDev/Office-Inspectors-for-Fiddler to look at what Outlook is doing under the covers when you do a Search, you can see that Outlook uses Searchfolders to provide a more functional search for dynamic queries.

Another example that is used in the Microsoft graph is the me/Messages endpoint which is a Searchfolder that provides access to all the mail folders in a mailbox.

In EWS when you create a SearchFolder you specify a SearchFilter for that folder to be based on. With the Graph similarly you can create a SearchFolder based on a OData filter which I've detailed above. So looking at something topically if you wanted to create a SearchFolder to show all the Email which had a subject of Coronavirus you could use

{
  "@odata.type": "microsoft.graph.mailSearchFolder",
  "displayName": "Coronavirus Email",
  "includeNestedFolders": true,
  "sourceFolderIds": ["AQMkADYA…."],
  "filterQuery": "contains(subject, 'Coronavirus')"
}

One thing that is mentioned in the SearchFolder documentation https://docs.microsoft.com/en-us/graph/api/resources/mailsearchfolder?view=graph-rest-1.0 for the Graph to be aware of is

  1. Search folders expire after 45 days of no usage.
  2. There are limits on the number of search folders that can be created per source folder. When this limit is breached, older search folders are deleted to make way for new ones.
So if you are going to use SearchFolders in your application you will need to make sure you have some appropriate management logic. Searchfolders are pretty powerful like EWS the Graph only implements a subset of what can be done in MAPI so if you are trying to reproduce what you see is possible in Outlook you may not be able to do this with Graph (or EWS).

Friday, March 13, 2020

Automating opening a Search-Mailbox result in Excel using EWS

While the Search-Mailbox cmdlet is now depreciated in Exchange Online, OnPrem its still used a fair bit and also does still have some use in the cloud for specific tasks. I've been using it this week a fair bit for various testing tasks and one pain I found when doing a lot of repeated searches in logging mode is each time to have to go in, open the results message in the discovery search mailbox and download the attachment with the log file, unzip and open it in Excel. So I came up with a way of automating this in powershell which turned out to be pretty simple but effective.

First off the only information you need to get the Results Message gets returned in the Target Folder property of the Search results eg.


 The TargetFolder value tells you what folder in the discovery Search mailbox the results are stored in and the DateTime value that will be in the subject of the Results Message.

So in EWS you can use FindFolder to Find that Folder (using a Split on "\" which will work as long as you don't put that in the displayName) and then FindItem can be used to find the results Item eg.

        $ivItemView = New-Object Microsoft.Exchange.WebServices.Data.ItemView(1)  
        $SfSearchFilter = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+ContainsSubstring([Microsoft.Exchange.WebServices.Data.ItemSchema]::Subject, $Subject) 
        $findItemResults = $Folder.FindItems($SfSearchFilter, $ivItemView)
        if ($findItemResults.Items.Count -eq 1) {
            return $findItemResults.Items[0]
        }
        else {
            throw "No Item found"
        }

Once you have the Results Message you can download the Attachment using some code like this

        if ($SearchResultItem.HasAttachments) {
            $SearchResultItem.Load();
            foreach ($Attachment in $SearchResultItem.Attachments) {
                $Attachment.Load()

I then save it to the default downloads directory using


                $downloadDirectory = (New-Object -ComObject Shell.Application).NameSpace('shell:Downloads').Self.Path
                $fileName = ($downloadDirectory + "\" + $ItemPath.SubString(1).Replace("/", "-").Replace(":", "-") + "-" + $Attachment.Name.ToString())
                $fiFile = new-object System.IO.FileStream($fileName, [System.IO.FileMode]::Create)
                $fiFile.Write($Attachment.Content, 0, $Attachment.Content.Length)
                $fiFile.Close()

and finally open the ZipFile, Extract the csv datastream from the Archive and save that as a File in the Downloads Directory and then open that file in Excel.


                if ($FileName.contains(".zip")) {                    
                    $Zip = [System.IO.Compression.ZipFile]::OpenRead($FileName)
                    try {
                        foreach ($file in $Zip.Entries) {
                            if ($file.Name.contains(".csv")) {
                                $ms = New-Object System.IO.MemoryStream
                                $ZipStream = $file.Open()
                                $ZipStream.CopyTo($ms);
                                $outputfile = $FileName.replace("zip", "")
                                [System.IO.File]::WriteAllBytes($outputfile, $ms.ToArray())
                                Invoke-Item $outputfile
                                
                            }
                        }
                    }
                    catch {
                        Write-Host $_.ScriptStackTrace
                    }
                    $Zip.Dispose()

An example of using this is

$SearchResult = Search-Mailbox -id meganb -TargetFolder Search1 -SearchQuery from:glen -TargetMailbox "DiscoverySearchMailbox{D919BA05-46A6-415f-80AD-7E09334BB852}@xxx.onmicrosoft.com" -LogOnly -LogLevel Full

then

Get-SearchMailboxResultsToExcel -MailboxName "DiscoverySearchMailbox{D919BA05-46A6-415f-80AD-7E09334BB852}@M365x680608.onmicrosoft.com" -SearchResultPath $SearchResult.TargetFolder -Verbose

or if you want to use ModerAuth (You will need the Adal dll in the same directory)

Get-SearchMailboxResultsToExcel -MailboxName "DiscoverySearchMailbox{D919BA05-46A6-415f-80AD-7E09334BB852}@M365x680608.onmicrosoft.com" -ModernAuth -SearchResultPath $SearchResult.TargetFolder -Verbose

I've put a download of this script on GitHub https://github.com/gscales/Powershell-Scripts/blob/master/Get-SearchMailboxResultsToExcel.ps1


Friday, January 24, 2020

Export calendar Items to a CSV file using Microsoft Graph and Powershell

For the last couple of years the most constantly popular post by number of views on this blog has been Export calendar Items to a CSV file using EWS and Powershell closely followed by the contact exports scripts. It goes to show this is just a perennial issue that exists around Mail servers, I think the first VBS script I wrote to do this type of thing was late 90's against Exchange 5.5 using cdo 1.2.

Now it's 2020 and if your running Office365 you should really be using the Microsoft Graph API to do this. So what I've done is create a PowerShell Module (and I made it a one file script for those that are more comfortable with that format) that's a port of the EWS script above that is so popular. This script uses the ADAL library for Modern Authentication (which if you grab the library from the PowerShell gallery will come down with the module). Most EWS properties map one to one with the Graph and the Graph actually provides better information on recurrences then EWS did. Where extended properties where used in the EWS script the equivalent is used in the Graph. (The only real difference is the AppointmentState property which is a strongly typed property in EWS but I had to use the Extended property in the Graph).

Just a couple of things if your new to Microsoft Graph scripts and Modern Authentication that you need to know

1. You need an Approved Azure Application registration to use this (or any script that is going to access the Graph). The Microsoft walk-throughs https://docs.microsoft.com/en-us/graph/auth-register-app-v2 are pretty good at describing how to do this. Specific config I recommend you use

"https://login.microsoftonline.com/common/oauth2/nativeclient" as the redirectURL (this is part of the Suggested Redirect URIs for public clients (mobile, desktop)).

2. Permission for the above



You only need the following permissions for this script to work, Calendar.Read gives you rights to the calendar the account that is being used and Calendar.Read.Shared gives you read access to any calendars that the account being used has been granted access to (eg via delegation, admin portal or add-mailboxpermission). 

Then you just need to copy the Application (client) ID guid from the overview screen in the Applicaiton Registration  and use that as in the -clientId paraemter in the Export-GCECalendarToCSV cmdlet.

I've included a demo multi tenant app registration as the default in the module that just has these rights which you can use for testing but I would always recommend you create you own.

You can install the module which will give you access to the Export-GCECalendarToCSV and Export-GCECalendar cmdlets from the Powershell gallery https://www.powershellgallery.com/packages/MSGraph-ExportCalendar/  (see the instruction on that page).

Or if you want to take the script and modify it yourself its located on GitHub https://github.com/gscales/Powershell-Scripts/blob/master/MSGraph-ExportCalendar/functions/Export-GCECalendarToCSV.ps1

Simple example of exporting the last 7 days of calendar appointment to csv

Export-GCECalendarToCSV -MailboxName gscales@datarumble.com -StartTime (Get-Date).AddDays(-7) -EndTime (Get-Date) -FileName c:\temp\Last7.csv



Friday, January 10, 2020

Using Azure device code authentication on a arduino iot 33 and getting the Teams presence from the Microsoft Graph

A while ago I published this post on accessing the Graph directly from an Arduino, this made use of the "resource owner password credentials grant" (meaning it used a hard coded username and password). Once you have enabled MFA (multi factor authentication) on an account this grant no longer works because you have no ability to provide the other factors for the Authentication to succeed.  For devices like Arduino's or most IOT devices that have very limited UI capabilities this is where device code authentication can be used.

The way Device Code Authentication works is instead of posting the user credentials to the token endpoint to get an access token, you make a post first to the /v2.0/devicecode endpoint which will then give you a specific user code to use to authenticate with on another device. You then visit http://microsoft.com/devicelogin (on a pc or mobile device) enter the user code and authenticate as the required user doing any extra MFA authentication. In the meantime the limited UI device polls the Token Endpoint and once authentication has been completed(on the external device) instead of the endpoint returning a pending error the poll results will be a normal Access token (and refresh token) that can then be used to access any Graph resources you have access to.

Visually on the Serial port here is what the whole process looks like on the Arduino

The last part of this code makes a request to get the Presence from Microsoft Teams which was introduced into beta in the Microsoft Graph in December see https://docs.microsoft.com/en-us/graph/api/resources/presence?view=graph-rest-beta.

So putting this all together you can make a simple Teams presence light with a circuit like (circuit is for demonstration purposes only)


and processing the Presence result you can get returned from the Graph using the code I've referenced below

A few notes on Device code Authentication, its important when you setup your App Registration in the Azure Portal that you mark your registration as public "Treat application as a public client" eg



Device code requests must be made against the Tenant endpoint (so you can't use the common endpoint). In the code I've included discovery code that gets the tenant specific endpoint to use based on the domain name stored in the Secrets file.

Also if your reading this because your following the documentation for Device code on https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-device-code and you can't get it to work there is an issue with the payload information in the document. Where device_code is used as a parameter name in the payload in the documentation it should just be code with your device code as the value.

I've put the sketch which contains the code I've used for Device Code authentication and grabbing the presence from the Microsoft Graph on GitHub here https://github.com/gscales/MS-Graph-Arduino/tree/master/MSGraph-Presence please refer to my previous article on details on getting you code up and running on an Arduino Iot33 which include downloading the SSL certs to the device which is required (also flash the firmware).

A couple of notes on the code because the Json parsing library I used can't handle the access token response I needed to manually parse the token out (which is a little frustrating) but is one of the chanllendges of working with Arduino's and dealing with the issues that limited memory causes. 

Monday, December 09, 2019

Update to ExchangeContacts Module to support Modern Auth,Exporting all Contacts to a VCF file (or CSV) ,NON_IPM root folder,hidden contact folders and dumpster exports

I've done some updating of my ExchangeContacts PowerShell module to support the following

  1. Modern Authentication in Office365 (distributing the ADAL dll with this module)
  2. Compiled and distributed the latest version of the EWS Managed API from GitHub with the module
  3. New cmdlet Export-EXCContacts that supports exporting all contacts in a Folder to a single VCF File
  4. New cmdlet Export-EXCContacts that supports exporting all contacts to a CSV file (this was already possible with the ExportFolder cmdlet but this is a slightly enhanced format)
  5. New cmldet Export-EXCRootContacts lets you export the Non_IPM Subtree folders that contain contacts. (Some of these are created by the Office365 substrate process) for example mycontacts, AllContacts, ContactSearch folders etc. Include dedup code based on Email Address in this cmdlet
  6. This is already supported but I wanted to show how you can export the Hidden Contacts Folder likes Recipient Cache, Gal and Organizational Contacts
  7. New cmdlet Get-EXCDumpsterContacts get the contacts that are in the RecoverableItems Deletions or Purges Folder
  8. New cmdlet Export-EXCDumpsterContacts Exports the contacts that are in the RecoverableItems Deletions or Purges Folder to a single VCF or csv file

Using Modern Authentication

As Basic Authentication in EWS is going away soon in Office365 I've enabled Modern Auth for this module using the ADAL dll which gets distributed via the bin directory in the Module. I didn't enabled it by default because it would cause issues with OnPrem Exchange so to use Modern Auth you just need to use the -ModernAuth switch. You can still pass in the PSCredential object with the -ModernAuth switch and oAuth will still be used vai the username and password grant to allow for silent auth. There is also provision to pass in your own client id for custom app registrations with the -ClientId parameter eg a simple example for using ModernAuth is 



Get-EXCContact -MailboxName gscales@datarumble.com
 -EmailAddress what@what.com -ModernAuth
Export-EXCContacts

Export-EXCContacts supports exporting all the contacts from any folder in a Mailbox to a single VCF file or a CSV File. (EWS provides the VCF nativly for Mailbox contacts so this cmlet hanldes streaming them out to a single file). Eg here are some examples

Exporting to a single VCF

Export-EXCContacts -Folder "\Contacts" -MailboxName gscales@datarumble.com
 -ModernAuth -FileName c:\temp\exp.vcf
or to a CSV


Export-EXCContacts -Folder "\Contacts" -MailboxName gscales@datarumble.com
 -ModernAuth -FileName c:\temp\exp.csv -ExportAsCSV
Export-EXCRootContacts

Export-EXCRootContacts supports exporting contacts from the NON_IPM_Subtree folders in a Mailbox. Typically folders here are created by either a Client like Outlook or OWA, other Office365 substrate process (eg Microsoft Teams) or other third party apps where they want the data to be hidden from the user. Examples of these folders would be Allcontacts, mycontacts etc. I've added this more for educational and diag purposes and Included some dedup code to deduplicate exports based on the EmailAddress. An example of export the AllContacts Folder to a CSV file


Export-EXCRootContacts -MailboxName gscales@datarumble.com -FolderName AllContacts -FileName c:\temp\allContacts.csv
-ExportAsCSV -ModernAuth

Get-EXCDumpsterContacts


This cmdlet will query either the RecoverableItemsDeletions or RecoverableItemsPurges folders in a Mailbox (Dumpster v2 folder) and it will get any contacts that exist in these folders and return them as EWSContact objects. (You can then process them further eg copy,move etc)

eg


Get-EXCDumpsterContacts -MailboxName gscales@datarumble.com -ModernAuth -Purges
or purges




Get-EXCDumpsterContacts -MailboxName gscales@datarumble.com -ModernAuth 
Export-EXCDumpsterContacts

This cmdlet builds on Get-EXCDumpsterContacts and allows you to export what is returned to either a single VCF file or a CSV file. (same logic as Export-EXCContacts)



Export-EXCDumpsterContacts -MailboxName gscales@datarumble.com -ExportAsCSV
 -FileName c:\temp\dumpsterDeletions.csv
or purges


Export-EXCDumpsterContacts -MailboxName gscales@datarumble.com -purges -ExportAsCSV
 -FileName c:\temp\dumpsterPurges.csv


Exporting  hidden Contacts folders

One last thing I wanted to demonstrate with this module is the ability to export the Hidden contact folders in your mailbox, if you have ever peeked at the Contacts folder subfolder hierarchy in a MAPI editor like MFCmapi there are a number of Hidden folders eg


Folders like Recipient Cache, Gal Contacts and Organizational Contacts folder all serve different client specific tasks (that do go wrong sometimes). So you can use this module to export the contacts in these folders to a CSV for any troubleshooting, migration or personal interest needs.

Here are some examples of exporting contacts from those folders to a csv file


Export-EXCContacts -Folder "\Contacts\Organizational Contacts"
-MailboxName gscales@datarumble.com -ModernAuth -FileName c:\temp\exp.csv 
-ExportAsCSV

The new module can be found on the Powershell Gallery https://www.powershellgallery.com/packages/ExchangeContacts/1.6.0.0 and the source is available here on GitHub https://github.com/gscales/Powershell-Scripts/tree/master/EWSContacts/Module

Thursday, November 28, 2019

Creating a year at a glance Calendar (in Excel) from aggregated Shared Calendars in Exchange Online within a Microsoft Teams Tab app using the Microsoft Graph

Calendaring formats in Messaging Clients tend to all follow much the same approach whether its Outlook, Outlook on the Web or Mobile, Gmail or Microsoft Teams.  Like email data, calendar data can be random and complex in its volume and type (recurring appointments etc) so a simple year at a glance calendar for someone designing a mass market client is hard to do well for all the data types and volumes that you could encounter, therefor its not something you see in mail clients by default (lets face it who wants to support that).

In the absence of year at a glance calendars  I was surprised to see people using Excel to create yearly aggregated calendars in Microsoft Teams for events (for data that already existed in shared Calendars). But more surprisingly is that it actually kind of worked well when there wasn't a lot of data that needed to be shown. The one thing that sprang to my mind was if you could automate this it  would be really good for people who use the Birthday calendar feature in Outlook, simple Company events calendars and also public holidays calendars especially when you want to aggregate multiple countries public holidays in a simple spreadsheet to help people like me who work across multiple regions and then share that within a team.

So I thought I'd set out to build a simple Microsoft Teams Tab application that could create an aggregated Spreadsheet of events from any calendar (or calendars) in a Office365 Mailbox that was shared to a Particular Microsoft Teams (Group) using the Graph API to get the Calendar Data from the Mailboxes and also using the Graph API to build the Excel workbook using the workbook functionality that the graph has. The result is then stored in a OneDrive File and provided back to the user in a iFrame as and embedded Excel Online spreadsheet. And the end result looks something like this (this is the result of having a Shared Mailbox with the Holiday calendars added/imported for Australia, US and the UK and that Mailbox being Shared to the Group/Teams)


How it works

Like the other Teams Tab apps I've written it takes advantage of using the Teams tab silent Auth method documented here . Once the code has acquired an Access Token to access the Graph it can get to work.

Configuration 

For this application to work I needed to be able to store the configuration of the calendars I wanted to aggregate . As the app is written in JS the easiest form of config file was a straight JSON file like the following
{
    "Calendars": [
        {
            "CalendarEmailAddress": "mb1@datarumble.com",
            "CalendarName": "Australia holidays",
            "CalendarDisplayName": "Australia"
        },
        {
            "CalendarEmailAddress": "mb1@datarumble.com",
            "CalendarName": "United States holidays",
            "CalendarDisplayName": "United States"
        }
    ]
}


And then I just required a way of storing and retrieving the file (a todo would be to create a nice form to allow people to create and edit the config but if I had time ...). The Teams client Sdk (and tab apps) don't have any provision for storing custom configuration, properties or pretty much anything configuration related so I just went for putting the file in the Channel document library as a starting point. So next I just needed some Graph code to grab the contents of that file. In JS the easiest way i found to do this was like this

From the Teams Context interface you can get the GroupId and ChannelName where you tab is executing so you can the construct the following URL that can be used in the Get against the MS Graph.

v1.0/groups/" + GroupId + "/drive/root:/" + channelName + "/ExcelCalendarConfig.json

The Graph documentation points to using the /content  endpoint to download the contents of a file, I have used this before in .NET (and node.js) and it works okay, it returns a 302 response with a Location header that can be followed to the SharePoint site. In client side JS its a lot messier so I found it easier to do this

CCDriveItem = await GenericGraphGet(Token,CalendarConfigURL);        
var CCFetch = await fetch(CCDriveItem["@microsoft.graph.downloadUrl"]);

So the @microsoft.graph.downloadUrl is a short-lived URL for the file that doesn't need authentication. So its easy to just do a Get and then use fetch on this url to return the JSON back to the code and I don't have to wade through a bunch of URL follow and cors issues with ajax and fetch

Template

One of the things that the Graph API can't do is create a new Excel file from scratch so you have to have an existing file you want to create a session with or some people recommend a number of different libraries to create the file. An easy solution for this one for me was to create a blank Excel file with no metadata and include that in with the webfiles so I could just copy it to OneDrive as a template file (overwriting any existing older file that may have been there) and then use that.

Storing the Result File 

One other problem for this project was where to store the end result file, at first I just used the SharePoint library associated with the Teams Channel but there where problems around the file becoming locked easily if two people ran it simultaneously. I also wanted to be able to run this with the least amount of permission as possible so the users App Folder (for this Tab app) seemed like the best spot as a starting point which is what the following code handles.


let AppDrive = await GenericGraphGet(Token,"https://graph.microsoft.com/v1.0/me/drive/special/approot");
let FileData = await ReadTemplate();
var fileName = "Calendars.xlsx";
var UploadURL = "https://graph.microsoft.com/v1.0/me/drive/special/approot:/" + fileName + ":/content";
let NewFile = await CreateOneDriveFile(Token,UploadURL,FileData);    

Getting the Calendars

Getting the Calendars was probably the easiest task, from the config file the CalendarName property is used to find the Folder from the Mailbox you want to access the data from. The query of the Calendar is then done for a the current years data using a CalendarView (which will expand any recurring calendar appointments). To aggregate the calendar data that was retrieved into orderable lists I used multiple Map objects in JS,loop iterations and arrays so I get an ordered list of events that are aggregated by first the Month and then day within the Month.

Building the Spreadsheet 

To build the spreadsheet in the output format that I wanted (which mirrored what I saw users doing manually) I had to first insert the data, then merge the month rows so I only had 1 row per month. Then format the merge so the text was aligned correctly and had the correct formatting. And then lastly was to Autofit the columns so the spreadsheet displayed correctly to users. So this required a lot of separate request to the Graph API to do which at first ran a little slowly. Then came Batching

Batching

Batching really is a Godsend when it comes to performance with a task like this, for example my original code had around 40-50 individual request to get the data and formatting done and with batching it was reduced to around 6 (and I was being a little conservative and could have reduced this). The big tip for using batching with the WorkBook endpoint is that you need to make sure you include the workbook session id with ever request (just not the batch request). If you don't you will get a lot of EditModeCannotAcquireLockTooManyRequests  which the documentation,the error (and the internet in general) aren't really helpful in pointing out why this happened.

Displaying it back to the Teams tab

This turned out to be one of the hardest problems to solve and is one of the outstanding issues with this in Teams anyway. I used an Iframe and generated and embeed link (which is what you get when you use Share-embed in Excel Online). This does work okay in the browser as long as you already have a login to your personal OneDrive (token in the Cache) else you will be prompted to logon to SharePoint. In the Desktop client this logon is a problem so instead of opening within the Tab in the desktop client, if it detects the Desktop client it launchs a new browser tab (which you may or may not need to logon to SharePoint to view).  This was a little disappointing but probably something I'll have a fix for soon (If anybody has any suggestions I'm all ears)

GitHub Repo for this App

I have a hosted version of this Tab App on my GitHub pages on https://gscales.github.io/TeamsExcelCalendar/  and there is repo version inside the Aggregation engine repo https://github.com/gscales/ExcelCalendarAggregate/tree/master/TeamsExcelCalendar with a Readme that details the installation process

Building on the Aggregation engine

Because I kind of enjoy taking things and running with them I have some plans of using the Calendar to Excel aggregation engine in a few different formats. The first will be a Simple powershell script so you can do the same thing but all from with an Automation context so if your interested in this but don't want a Teams tab app watch this space.