Wednesday, February 22, 2012

EWS Managed API and Powershell How-To series Part 4 - Items Part 2

Continuing on from my last post in this how to series on Items this post is going to look at how you can go about creating, deleting, moving, copying,exporting different Items and we will also look at the important batch operations which help when you need to perform actions on multiple Items at the same time.

As I have mentioned previously in this series when your dealing with Exchange Objects your dealing with Rich data-types that vary greatly eg  If you compare the properties on an Email object with that of a Task while they share a set of common properties what makes a Task a Task is its unique properties such as Percent complete, Start Time etc which have all been dutifully documented in this Exchange Protocol document. As this is the first I've mentioned this I would strongly recommend you check out the Open Specification Development Center which is a really brilliant resource for any reference information you need about using Exchange data types or EWS in general.

Creating Items

 With EWS when you create Items you use the Strongly Typed class for that particular Item you want to create for example Email, Appointment, Tasks etc. The exception for this is when you have an object Type where this in no Strong Type such as Notes (or StickyNotes), Journals Items etc for these you use the MessageType class and then change the ItemClass property to the IPM. type you want. Lets look at a few examples the first sample shows you how to Send an Email and the Second how to create an Appointment.

  1. # Create Email Object  
  2. $EmailMessage = New-Object Microsoft.Exchange.WebServices.Data.EmailMessage -ArgumentList $service  
  3. #Set the Subject  
  4. $EmailMessage.Subject = "Subject of Email blah"  
  5. #Set the Body of Messsage  
  6. $EmailMessage.Body = "Message Body Blah blah"  
  7. #Add Recipients  
  8. $EmailMessage.ToRecipients.Add("targetemail@domain.com")  
  9. #Send Message SendandSave will save the Message to the Sender Sent Item Folder  
  10. $EmailMessage.SendAndSaveCopy()  
Creating an Appointment

  1. ##Create a Calendar Appointment  
  2.   
  3. $Appointment = New-Object Microsoft.Exchange.WebServices.Data.Appointment -ArgumentList $service  
  4. #Set Start Time  
  5. $Appointment.Start = [system.DateTime]::Now.AddDays(7)  
  6. #Set End Time  
  7. $Appointment.End = [system.DateTime]::Now.AddDays(7).AddHours(1)  
  8. #Set Subject  
  9. $Appointment.Subject = "Drink Coffee with Friend"  
  10. #Set the Location  
  11. $Appointment.Location = "Good Coffee Shop"  
  12. #Set any Notes  
  13. $Appointment.Body = "Dont forget to mention X and Y"  
  14. #Create Appointment will save to the default Calendar  
  15. $Appointment.Save()  
Okay now lets look at what to do when you need to create an Item where there isn't a Strong Type. The first thing you need to know is all the underlying Extended Mapi properties that make this Item work the most accurate source for this is the Exchange protocol documents for example for a Note this would this. The other way is to look at an example Item with a Mapi Editor like OutlookSpy or MFCMapi (personally i use both) where you can see all the underlying extended properties there's a good sample of doing this here. Here's a sample for creating a StickyNote

  1. #Create Sticky Note  
  2.   
  3. #First we use the EmailMessage Class (MessageType) this ensures we can change the ItemClass  
  4. $snStickyNote = New-Object Microsoft.Exchange.WebServices.Data.EmailMessage -ArgumentList $service  
  5. #Set the Subject of the Note  
  6. $snStickyNote.Subject = "Stick Note Subject"  
  7. #Change the ItemClass to IPM.StickNote 
  8. $snStickyNote.ItemClass = "IPM.StickyNote"  
  9. #Set the Text body of the Note  
  10. $snStickyNote.Body = "First Line `r`nNext Line"  
  11. #Start Note specific Extended properties   
  12. #Define the Guid for the Notes Named properties  
  13. $noteGuid = new-object Guid("0006200E-0000-0000-C000-000000000046")  
  14. #Set the Colour of the Note  
  15. $snColour = New-Object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition($noteGuid,35584, [Microsoft.Exchange.WebServices.Data.MapiPropertyType]::Integer)  
  16. #Set the Height of the Note  
  17. $snHeight = New-Object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition($noteGuid,35586, [Microsoft.Exchange.WebServices.Data.MapiPropertyType]::Integer)  
  18. #Set the Width of the Note  
  19. $snWidth = New-Object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition($noteGuid,35587, [Microsoft.Exchange.WebServices.Data.MapiPropertyType]::Integer)  
  20. #Postion from Left  
  21. $snLeft = New-Object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition($noteGuid,35588, [Microsoft.Exchange.WebServices.Data.MapiPropertyType]::Integer)  
  22. #Postion from Top  
  23. $snTop = New-Object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition($noteGuid,35589, [Microsoft.Exchange.WebServices.Data.MapiPropertyType]::Integer)  
  24. #Set Note props  
  25. $snStickyNote.SetExtendedProperty($snColour,3)  
  26. $snStickyNote.SetExtendedProperty($snHeight,200)  
  27. $snStickyNote.SetExtendedProperty($snWidth,166)  
  28. $snStickyNote.SetExtendedProperty($snLeft,80)  
  29. $snStickyNote.SetExtendedProperty($snTop,80)  
  30. #Save the Item to the Notes Folder  
  31. $snStickyNote.Save([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Notes)  
Creating Items in Public Folders or custom Mailbox folders

To create these type of objects in Public folders there is no real difference to a normal mailbox folder other then in the save method eg $snStickyNote.Save() you need to use the EWSid of the public folder in question see Part two in the how to series for information on how to get the EWSid of the folder.

Copy and Moving Items between folders

With EWS you have the ability to move Items between folders in a mailbox or copy them between folders or between different Mailboxes the later can be useful if your looking to replicate one folder to the other. With the Managed API there is a simple Copy or Move Method on an Item that takes the EWS FolderID of the folder where you want to copy\move the Item to. For a simple sample lets look at a script that the finds a Message based on its subject then demonstrates first ly copy and then a move to a folder called destination folder located in the Inbox


  1. #Move and Copy sample  
  2.   
  3. #AQSString  
  4.   
  5. $AqsString = "Subject:=`"Test Mail Move`""  
  6.   
  7. #Bind to Inbox    
  8. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)     
  9. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)    
  10.   
  11. #Get the ID of the folder to move to  
  12. $fvFolderView =  New-Object Microsoft.Exchange.WebServices.Data.FolderView(1000)  
  13. $fvFolderView.Traversal = [Microsoft.Exchange.WebServices.Data.FolderTraversal]::Shallow;  
  14. $SfSearchFilter = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+IsEqualTo([Microsoft.Exchange.WebServices.Data.FolderSchema]::DisplayName,"MoveDestination")  
  15. $findFolderResults = $Inbox.FindFolders($SfSearchFilter,$fvFolderView)  
  16.   
  17. #Define ItemView to retrive just 1000 Items    
  18. $ivItemView =  New-Object Microsoft.Exchange.WebServices.Data.ItemView(1000)  
  19. $fiItems = $null    
  20. do{    
  21.     $fiItems = $service.FindItems($Inbox.Id,$AqsString,$ivItemView)    
  22.     #[Void]$service.LoadPropertiesForItems($fiItems,$psPropset)  
  23.     foreach($Item in $fiItems.Items){      
  24.         # Copy the Message  
  25.         $Item.Copy($findFolderResults.Folders[0].Id)  
  26.         # Move the Message  
  27.         $Item.Move($findFolderResults.Folders[0].Id)  
  28.     }    
  29.     $ivItemView.Offset += $fiItems.Items.Count    
  30. }while($fiItems.MoreAvailable -eq $true)    
This type of script works okay when you only need to copy or move one item at a time however when you want to move multiple items its better to use the Batch methods which essentially batch the individual move or copy requests into one request. This can be useful when you are doing archiving, cleanup or deletion type scripts. The following sample will create a folder under the Inbox called January2011 and then query for all the items in the Inbox from January2011 using a AQS string it will then do a batch move of these items to the January2011 folder.

  1. #Batch Move Sample   
  2.   
  3. #AQSString  
  4.   
  5. $Range = "01/01/2012..01/30/2012"    
  6. $AQSString = "System.Message.DateReceived:" + $Range   
  7.   
  8. #Bind to Inbox    
  9. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)     
  10. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)    
  11.   
  12. #Create the folder to move to  
  13. $NewFolder = new-object Microsoft.Exchange.WebServices.Data.Folder($service)    
  14. $NewFolder.DisplayName = "January2011"    
  15. $NewFolder.Save($Inbox.Id)    
  16.   
  17. #Define ItemView to retrive just 1000 Items    
  18. $ivItemView =  New-Object Microsoft.Exchange.WebServices.Data.ItemView(1000)  
  19. $fiItems = $null    
  20. do{   
  21.     #Create the collection of ItemIds to perform batch request  
  22.     $Itemids = [activator]::createinstance(([type]'System.Collections.Generic.List`1').makegenerictype([Microsoft.Exchange.WebServices.Data.ItemId]))    
  23.     $fiItems = $service.FindItems($Inbox.Id,$AQSString,$ivItemView)    
  24.     #[Void]$service.LoadPropertiesForItems($fiItems,$psPropset)  
  25.     foreach($Item in $fiItems.Items){   
  26.         "Processing : " + $Item.Subject  
  27.         $Itemids.add($Item.Id)  
  28.     }    
  29.     $Result = $service.MoveItems($Itemids,$NewFolder.Id)  
  30.     [INT]$Rcount = 0    
  31.     foreach ($res in $Result){    
  32.         if ($res.Result -eq [Microsoft.Exchange.WebServices.Data.ServiceResult]::Success){    
  33.             $Rcount++    
  34.         }    
  35.     }    
  36.     $Rcount.ToString() + " Items moved successfully"    
  37.     $ivItemView.Offset += $fiItems.Items.Count    
  38. }while($fiItems.MoreAvailable -eq $true)    
Deleting Items

Deleting Items works the same as Copy or Move with the addition of passing the delete mode you wish for example to delete the Item instead of copying/Moving it you would use something like

  1.         $Item.Delete([Microsoft.Exchange.WebServices.Data.DeleteMode]::SoftDelete)
There is a Batch Delete sample in part 2 of this series.

 Exporting and Importing Items

One frequently asked question about EWS is how can i export to a PST or MSG file and unfortunately the answer is you can't these are Office file formats so if you want to use them you need to use an Office API like the OOM or my recommendation would be to use Redemption/RDO http://www.dimastr.com/redemption/home.htm. (Or you can use the Exchange Management Shell and the export-... cmdlets).

However there are a few methods you can use to Export and Import Items in other formats one format you can use for Email Messages is to use the Mime content of a message which is essentially the RFC822 stream of the message (or more or less the stream you would see submitted to and from the MTA over SMTP). Using the Mime Content is only good for Email messages and it wont include any custom Mapi properties or message flags so it would not be a full fidelity export of a message. Here's a sample of how to export the last message received to an EML file.

  1. #Eml Export sample  
  2.   
  3. #Define PropertySet  
  4. $psPropset = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.ItemSchema]::MimeContent)   
  5. #Bind to the Inbox  
  6. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)     
  7. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)  
  8. #Set the ItemView to only return 1 Email  
  9. $ivItemView =  New-Object Microsoft.Exchange.WebServices.Data.ItemView(1)  
  10. #Define PropertySet to load MimeContent  
  11. $psPropset = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.ItemSchema]::MimeContent)   
  12. $fiItems = $service.FindItems($folderid,$ivItemView)  
  13. foreach($itItem in $fiItems.Items){  
  14.     $itItem.load($psPropset)  
  15.     $fileName = "C:\temp\exportedmail.eml"  
  16.     $fiFile = new-object System.IO.FileStream($fileName, [System.IO.FileMode]::Create)   
  17.     $fiFile.Write($itItem.MimeContent.Content, 0,$itItem.MimeContent.Content.Length)  
  18.     $fiFile.Close()   
  19. }  
To Import an EML file you essentially upload the MIME content you either saved from the export script or other EML file from another email client. To make sure the message doesn't appear as a draft you need to set the SentFlag on the message. For example to upload an EML file

  1. #Upload Eml Sample  
  2.   
  3. $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)     
  4. $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)  
  5. #Crete Email Object  
  6. $emUploadEmail = new-object Microsoft.Exchange.WebServices.Data.EmailMessage($service)  
  7. #Read File  
  8. [byte[]]$bdBinaryData1 =  get-content -encoding byte "C:\temp\exportedmail.eml"  
  9. #Set Mime Content in Message  
  10. $emUploadEmail.MimeContent = new-object Microsoft.Exchange.WebServices.Data.MimeContent("us-ascii"$bdBinaryData1);  
  11. #Set Sent Message Flags which means message wont appear as a Draft  
  12. $PR_Flags = new-object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition(3591, [Microsoft.Exchange.WebServices.Data.MapiPropertyType]::Integer);  
  13. $emUploadEmail.SetExtendedProperty($PR_Flags,"1")  
  14. $emUploadEmail.Save($Inbox.Id)  
The other way you can export and Import Items with EWS on Exchange 2010 is to use the new ExportItems and UploadItems operations. These operations export items in a Opaque data stream that contains all the Store properties of the underlying Item which make its a full fidelity copy of that underlying Item. There is nothing in the Managed API to allow you to use these operations so you need to use either WSDL proxy code or raw SOAP for more information have a look at these two links


http://msdn.microsoft.com/en-us/library/hh135142%28v=exchg.140%29.aspx

and

http://gsexdev.blogspot.com.au/2011/02/exporting-and-uploading-mailbox-items.html

That's it for this post in the next post I'll look at the special operations in EWS to Get the OOFStatus, FreeBusy and create Inbox Rules.

29 comments:

drunkenbaker said...

Keep these coming, Glen, you're one of the few people writing about EWS that knows exactly what they are talking about.

Kevin said...

I have learned so much from your blog and am charged with the possibilities.

Thanks!

My Eyes Now Hurt said...

Possibly the worst colour scheme on a website i have ever seen. Makes it almost impossible to read!

Bill Bohlen said...

THANK YOU SO MUCH!

A few years back, after implemeting a mail archival system, I wrote a MAPI/CDO based script to delete all mail items older than 45 days from our 2003 mailboxes and e-mail the users a CSV report of their messages purged. Your EWS blog posts were exactly what I needed to begin converting the script to work with 2010 mailboxes.

JoeN said...

Hi, had the a version of these scripts working fine to find and delete a certain item type with in mailboxes.

Then a week or so later I now get this on the script...

Exception calling "Bind" with "2" argument(s): "The request failed. The underlying connect
error occurred on a send."

Has anyone any ideas on this one?

been searching and reading, but have not found any relationship to EWS connection to this yet?

Joris said...

Hi Glen,

Thanks for the interesting articles!

I'm trying to find a way to automatically detach attachments of mails with a specific subject. Server side rules can't do this, so I was looking at EWS. Can you show me how to do this ?

Glen said...

Have a look at http://gsexdev.blogspot.com.au/2010/01/writing-simple-scripted-process-to.html

Cheers
Glen

Tomas said...

Hi Glen,
Thanks for nice articles.

I would like ask if is possible recovery deleted Public Folder through EWS?

Thanks.

//Tomas

Glen Scales said...

No I'd suggest you have a look at Exfolders.

On 2010 it maybe possible if you enumerate through the softdeleted collection of a public folder and then try to copy the source out but I've never tried it.

Cheers
Glen

Tomas said...

Yep, I tried but without success.
Anyway, thanks a lot.

//Tomas

JR said...

Glen, is there an Exchange 2007 alternative to the ExportItems/UploadItems operations available in 2010+? It'd be great to save rich data types via EWS. Thanks for the informative articles.

Glen Scales said...

No you can only get the MIME data with EWS in 2007 if you want rich types the only option is MAPI

Cheers
Glen

Steve said...

When I try to upload a EML file with an attachment it takes up all of my Memory... Do I need to change something?

Donoho said...

I've uploaded .eml with and without attachments successfully. In my experience, this problem occurs when the attachment is corrupt.

This line of code churns and consumes all memory available until the process is killed.
[byte[]]$bdBinaryData1 = get-content -encoding byte "C:\temp\exportedmail.eml"

As a test I opened the .eml in question in notepad++ and in Outlook Express 6. OE6 opend the message and listed 3 attachments, 1 of them named ATT0002.dat. I searched for the file names in np++ and found 2 out of the 3. The third however was supposed to be a .png file. Been googling for solutions for dyas now, so I thought I'd come back to the source.

Glen, do you know of a way to either test attachment validity, or to skip over problems such as this? I wish I knew what OE6 was doing. I'd replicate that.

P.S. Thanks for the great info Glen

Glen Scales said...

I've seen problems with Get-Content in the past so I would suggest you trying us the .NET FileIO class directly eg use

[byte[]]$bdBinaryData1 = [System.IO.File]::ReadAllBytes("C:\temp\exportedmail.eml")

Cheers
Glen

Donoho said...

Glen, Thank You!

That change made All the difference. The difference in processing speed alone is (in my environment) phenomenal. That said, it (successfully) tore through the messages Get-Content was having problems with.

Anonymous said...

Is there a way to import .eml files into a subfolder instead of the root INBOX. I would like to keep the folder structure on my email accounts.

Glen Scales said...

Yes so the line

$emUploadEmail.Save($Inbox.Id)

determines where the Item is saved, you just need to replace the Inbox.Id with the Id of the other folder. You'll need to find the other folderId using something like

function FolderIdFromPath{
param (
$FolderPath = "$( throw 'Folder Path is a mandatory Parameter' )"
)
process{
## Find and Bind to Folder based on Path
#Define the path to search should be seperated with \
#Bind to the MSGFolder Root
$folderid = new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::MsgFolderRoot,$MailboxName)
$tfTargetFolder = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)
#Split the Search path into an array
$fldArray = $FolderPath.Split("\")
#Loop through the Split Array and do a Search for each level of folder
for ($lint = 1; $lint -lt $fldArray.Length; $lint++) {
#Perform search based on the displayname of each folder level
$fvFolderView = new-object Microsoft.Exchange.WebServices.Data.FolderView(1)
$SfSearchFilter = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+IsEqualTo([Microsoft.Exchange.WebServices.Data.FolderSchema]::DisplayName,$fldArray[$lint])
$findFolderResults = $service.FindFolders($tfTargetFolder.Id,$SfSearchFilter,$fvFolderView)
if ($findFolderResults.TotalCount -gt 0){
foreach($folder in $findFolderResults.Folders){
$tfTargetFolder = $folder
}
}
else{
"Error Folder Not Found"
$tfTargetFolder = $null
break
}
}
if($tfTargetFolder -ne $null){
return $tfTargetFolder.Id.UniqueId.ToString()
}
}
}
#Example use
$fldId = FolderIdFromPath -FolderPath "\Inbox\test"
$SubFolderId = new-object Microsoft.Exchange.WebServices.Data.FolderId($fldId)
$SubFolder = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$SubFolderId)

Cheers
Glen

Anonymous said...

Thank you very much! I kept trying to find a way get the sub-folder ID but couldn't get it. I was missing a few steps you have.

However, my inbox worked perfectly, but when i try to import another standard user, i get the error. "Excepting calling "Bind" with "2" arguments(s): "Id is malformed"

this error happens when running the Bind at the end of your code.

Thanks,
Kevin

Anonymous said...

I couldn't edit my comment, but i forgot to mention that it's not finding the folder when using any other user but myself. the user created the exchange account and has what i believe is the top permissions. I can't seem to do anything when using my creds. against another user. Any tips?

-Kevin

Glen Scales said...

You might want to test your permission using the EWSEditor http://ewseditor.codeplex.com/ that will tell you if its a script problem or rights issue.

Cheers
Glen

Anonymous said...

I found I didn't have the permissions I needed. Got everything working for me.

However I am unable to create sub folders for other users. I did some research and could only find it wasn't possible using the new-mailboxfolder. Is there another script that can do this?

Thanks for all your help,

-Kevin

Glen Scales said...

As long as you have rights (or EWS Impersonation rights) then you can use something like http://gsexdev.blogspot.com.au/2013/01/creating-new-calendarcontactstasks-or.html

Cheers
Glen

Anonymous said...

Thanks Glen,

With you help, my script now runs through and imports everyone's emails over from our old server to exchange.

-Kevin

Anonymous said...

Thank you Glen, your blogs on EWS and poweshell have really helped a whole lot!

Anonymous said...

Hi Glen, many thanks for this blogs that give me the chance to learn more about PS and EWS, but i have not learn enough. So my question: How can i search a subfolder and find unread Mails and when Mail unread move the item to the inbox? Cheers Heiko

Anonymous said...

Hi Glen, i found the solution in your blog :-)

Cheers Heiko

Thomas Konietzko said...

Hi Glen,
i have problems to save an appointment to a public folder.
$appointment.Save($FolderID)
Error:microsoft.exchange.webservices.data.ServiceResponseException: Meeting requests and cancellations can't be sent for calendar items located in public folders.

-- Thomas

Praveen Giri said...

Hi Glen,
After getting the binary data stream I have a requirement to display useful information out of it like message body, to , from, cc addressess. Is there a way to extract info from the data we got from ExportItems ?