Sunday, September 21, 2008

WizBang Exchange 2007 Message Tracking Powershell Gui Version 1

Unlocking the secrets from the depths of the Message Tracking Logs is an ever recurring theme on this blog and in general an important area of Exchange Server Management. In this latest incarnation we boldly go where no Message Tracking application has gone before as well as the normal aggregation, graphing and exports bits and pieces. The new functionality this script introduces expands on post I was talking about a couple of weeks ago which is Using Exchange Web Services to Enhance Exchange Message Tracking. So what we end up is a GUI that does aggregation and filtering and then the ability to look at the content of a particular message by using EWS to find the message based on its message-ID and then being able to see the Subject,Body and then export attachments or the whole message as an EML file. As this is version one i haven't quite go around to making the message search do a Deleted Item Scan (Dumpster Dive) and there is also an issue with Sent Items where the Clients in using Outlook in Cache mode. This is because when you are using Outlook in Cached Mode Items saved to the Sent Items receive a different MessageID which is described in this KB. So this mean you cant find messasges that where sent if the client is using Outlook in Cache mode (if the mesage was sent internally you would be able to get if from the Inbox of the recipient).

This script produces a Multi-Tabbed Winform with different functionality provided by each tab

  1. Tab1 - Setup tab this allows you to specify the parameter for the search of the message tracking logs
  2. Tab2 - Graphs and Summarised tables this tab shows some Google chart's graphs of the Last 6 hours of Message Tracking activity. Graphs of the total Aggregates of Internal/External and Sent/Received Messages. Last hour top 5 Message Sent/Receivers and Organization Totals
  3. Tab3 - Email Summaries per Internal User for Internal Sent/Receive, External Sent/Received and the ability to show the section of the Message Tracking logs for those particular messages. Also Find these message from these filtered Tracking logs via EWS and the Tab5 message Find function. (expot to csv for filtering and aggregated logs)
  4. Tab4 - Raw Message Tracking view
  5. Tab5 - EWS Message Find tab This use EWS to find messages based on message ID invoked from Tab 3 (show message button). Allows export of attachments and message to eml.
To do the search of the Message Tracking logs the script using the get-messagetrackinglogs Exchange powershell cmdlet to return all the Logs entries and then filters based on Send and Receive Entries. To ensure that a log entry is not counted twice the recipients of a message are expanded and a Hastable is used to ensure that the combination of MessageID-Sender and Recipient isn't counted more then once. Being version one this process hasn't been well test so there are probably lots of bugs with this method but so far it seems to work okay.

The graphing functions are provided using Google charts online service and a winform picture control which is the method I've described in the past. The one issue with the method is that it does require that you have a direct Internet connection from the client (eg NAT etc). If your going out through proxies then generally you may see that the graphs don't draw properly this is because the Picture control cant deal with Internet proxies.

The aggregate functions are all done through using a combination of hashtable and custom powershell objects.

The EWS Find functionality is done using the EWS powershell library which i've been using a fair bit and changing and adding more bits and pieces too. The library will handle Autodiscover and allow for both delegate and impersonation access method. The Form allows for all of these combinations and also allows you to specify overrides for both authentication and the URL for the CAS server to use (you need to use the full url to the EWS if you going to use eg https://servername/ews/exchange.asmx).

To use EWS find message function you need a copy of the EWSUtil.dll in c:\temp you can download this from here. You also need rights on the mailboxes you are searching either via delegation or Impersonation.

I've put a download of the script here the code itself is a little large to post.

Thursday, September 18, 2008

Adding Entries to the Safe Sender,Safe Recipients and Blocked Senders in Exchange 2007 via a script workaround

The following is an extension to the Junk Mail enable script for 2007 i posted here. Using the same OWA automation workaround this extends the ability of this script to also add entries to the Safe Senders, Safe Recipients and Blocked Sender by re-using the cmds from OWA 2007.

To add these entires the following XML is posted to /ev.owa?oeh=1&ns=JunkEmail&ev=Add

<params><iLT>1</iLT><sNE>" & emEmailAddress & "</sNE><fFrmOpt>1</fFrmOpt></params>

The <iLT>1</iLT> is the only things that changes eg

<iLT>0</iLT> Adds to Safe Senders
<iLT>1</iLT> Adds to Safe Recipients
<iLT>2</iLT> Adds to Blocked Senders

Make sure you read the original post I've linked above which documents the objects used and the potential issue around using this type of method which is just a quick workaround.

I've put a download of this script here the code looks like

snServername = "servername"
mnMailboxname = "mailboxname"
domain = "domain"
strpassword = "password"

Targetmailbox = "mailbox@domain.com"

strusername = domain & "\" & mnMailboxname
szXml = "destination=http://" & snServername & "/owa/&flags=0&username=" & strusername
szXml = szXml & "&password=" & strpassword & "&SubmitCreds=Log On&forcedownlevel=0&trusted=0"


set req = createobject("MSXML2.ServerXMLHTTP.6.0")
req.Open "post", "http://" & snServername & "/owa/auth/owaauth.dll", False
req.SetOption 2, 13056
req.send szXml

reqhedrarry = split(req.GetAllResponseHeaders(), vbCrLf,-1,1)
for each ent in reqhedrarry
wscript.echo ent
Next

Call UpdateJunk(Targetmailbox)
call AddToSafeSenders(Targetmailbox,"address@domain.com")
call AddToBlockedSenders(Targetmailbox,"address@domain.com")
call AddToSafeRecipients(Targetmailbox,"address@domain.com")

Sub UpdateJunk(mbMailbox)

xmlstr = "<params><fEnbl>1</fEnbl></params>"
req.Open "POST", "http://" & snServername & "/owa/" & mbMailbox & "/ev.owa?oeh=1&ns=JunkEmail&ev=Enable", False
req.setRequestHeader "Content-Type", "text/xml; charset=""UTF-8"""
req.setRequestHeader "Content-Length", Len(xmlstr)
req.send xmlstr
wscript.echo req.status
reqhedrarry = split(req.GetAllResponseHeaders(), vbCrLf,-1,1)
for each ent in reqhedrarry
wscript.echo ent
Next
If InStr(req.responsetext,"name=lngFrm") Then
wscript.echo "Mailbox has not been logged onto before via OWA"
'Create a regular expression object
Dim objRegExp
Set objRegExp = New RegExp

objRegExp.Pattern = "<option selected value=""(.*?)"">"
objRegExp.IgnoreCase = True
objRegExp.Global = True

Dim objMatches
Set objMatches = objRegExp.Execute(req.responsetext)
If objMatches.count = 2 then
lcidarry = Split(objMatches(0).Value,Chr(34))
wscript.echo lcidarry(1)
tzidarry = Split(objMatches(1).Value,Chr(34))
wscript.echo tzidarry(1)
pstring = "lcid=" & lcidarry(1) & "&tzid=" & tzidarry(1)
req.Open "POST", "http://" & snServername & "/owa/" & mbMailbox & "/lang.owa", False
req.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
req.setRequestHeader "Content-Length", Len(pstring)
' req.SetRequestHeader "cookie", reqCadata
req.send pstring
if instr(req.responsetext,"errMsg") then
wscript.echo "Permission Error"
else
wscript.echo req.status
If req.status = 200 and not instr(req.responsetext,"errMsg") Then
Call UpdateJunk(mbMailbox)
Else
wscript.echo "Failed to set Default OWA settings"
End if
end if

Else
wscript.echo "Script failed to retrieve default values"
End if
Else
wscript.echo "Junk Mail Setting Updated"
End if
End sub

Sub AddToSafeSenders(mbMailbox,emEmailAddress)

xmlstr = "<params><iLT>1</iLT><sNE>" & emEmailAddress & "</sNE><fFrmOpt>1</fFrmOpt></params>"
req.Open "POST", "http://" & snServername & "/owa/" & mbMailbox & "/ev.owa?oeh=1&ns=JunkEmail&ev=Add", False
req.setRequestHeader "Content-Type", "text/xml; charset=""UTF-8"""
req.setRequestHeader "Content-Length", Len(xmlstr)
req.send xmlstr
wscript.echo req.status
reqhedrarry = split(req.GetAllResponseHeaders(), vbCrLf,-1,1)
for each ent in reqhedrarry
wscript.echo ent
Next
wscript.echo req.responsetext

end Sub

Sub AddToBlockedSenders(mbMailbox,emEmailAddress)

xmlstr = "<params><iLT>2</iLT><sNE>" & emEmailAddress & "</sNE><fFrmOpt>1</fFrmOpt></params>"
req.Open "POST", "http://" & snServername & "/owa/" & mbMailbox & "/ev.owa?oeh=1&ns=JunkEmail&ev=Add", False
req.setRequestHeader "Content-Type", "text/xml; charset=""UTF-8"""
req.setRequestHeader "Content-Length", Len(xmlstr)
req.send xmlstr
wscript.echo req.status
reqhedrarry = split(req.GetAllResponseHeaders(), vbCrLf,-1,1)
for each ent in reqhedrarry
wscript.echo ent
Next
wscript.echo req.responsetext

end Sub

Sub AddToSafeRecipients(mbMailbox,emEmailAddress)

xmlstr = "<params><iLT>3</iLT><sNE>" & emEmailAddress & "</sNE><fFrmOpt>1</fFrmOpt></params>"
req.Open "POST", "http://" & snServername & "/owa/" & mbMailbox & "/ev.owa?oeh=1&ns=JunkEmail&ev=Add", False
req.setRequestHeader "Content-Type", "text/xml; charset=""UTF-8"""
req.setRequestHeader "Content-Length", Len(xmlstr)
req.send xmlstr
wscript.echo req.status
reqhedrarry = split(req.GetAllResponseHeaders(), vbCrLf,-1,1)
for each ent in reqhedrarry
wscript.echo ent
Next
wscript.echo req.responsetext

end Sub

Monday, September 08, 2008

Setting the OWA themeid via a script in Exchange 2007

This is a workaround for lack of the ability to set the OWA themeid programatically for those interested in the Technical side this setting is held in a FAI Message with a message class of "IPM.Configuration.OWA.UserOptions" in the Non_IPM_subtree of a mailbox. The setting is held as part of XML string in the 0x7C070102 binary mapi property on this item. Because there is no documentation for the structure of this property it makes setting and reverse engineering this risky and potentially error prone. One realtivly easy workaround to do this is to use some OWA automation code to use the method OWA uses to set this property. I've used this type of thing before to enable junkemail filtering on 2007 in OWA see

Using the MSXML2.ServerXMLHTTP.6.0 object this object is included with the Microsoft XML Parser (MSXML) and is a better choice for this script because it firstly supports the ability to ignore any SSL errors that might happen (eg self signed Certs, Alternate names etc) and it also handles dealing with the Forms Based Authentication cookie without the necessity to add additional code. The code still needs to perform the synthetic forms logon this works similar to 2003 with a few URL tweaks.

Dealing with the Language form for users who have never logged on to OWA before. Because this script is simulating a user in OWA if the mailbox your trying to set the junk email settings has never been logged onto before in OWA then the default language form will be presented to the user (or the script in this case) asking the person to choose there timezone and language. What this script does to cater for this is that it looks for this form in the response if it finds the form it then uses 2 regular expressions to parse the default values from the form and then posts these values. One problem this creates is that the user will nolonger be shown this form when they first logon anymore this may or may not be a problem for you. If you need for this form to still appear at first logon there are two options the first is to delete the OWA storage object in the root of the mailbox using WebDAV or Neil Hobson posted another method on the Exchange Blog.

The operation of the script is pretty simple after logging in it tries to post to the following URL QueryString in the Target Mailbox “ev.owa?oeh=1&ns=Options&ev=SaveGenOpts” with a body

<params><anrFst>0</anrFst><thmId>& themeID &</thmId><optAcc>0</optAcc></params>

The themeID is the interger of the ThemeID to set within 2007 the following intergers represent the default themes

0 Seattle Sky
1 Carbon Black
2 Xbox Theme
3 Zune Theme

To use this script you need to hard code the username and password of a user that has been give delegate access to the target mailbox using something like this in the Exchange Command Shell

Add-MailboxPermission –Identity ‘Mailbox’ –User ‘User’ –AccessRights FullAccess

Or has been given Send As and Receive As rights on the target mailbox. Basically the user needs to be able to open the target mailbox as another mailbox in OWA you can test if the account you want to use is going to be work by testing this yourself in OWA. Its also important that the account that you want to use has logged onto OWA once as well this is because the Language form would be presented to this user the first time the user tries to logon to OWA while the script caters for this for the target user it doesn’t do it for the source so this would cause a timeout error in the case the source user has never logged on to OWA. So before using the script you need to configure the following variables

snServername = "ServerName"
mnMailboxname = "mailbox"
domain = "domain"
strpassword = "password"

In the snServername variable make sure you use the servername of your CAS server this may or may not be different from your mailbox server.

The following variable is for the target mailbox you want to set the junkemail setting on this should be the primary SMTP address of the target mailbox

Targetmailbox = "user@domain.com"

The scripts output is pretty verbose you should see the full response headers outputted for each request the script make this helps if you ever need to diagnose why the script isn’t working

I’ve put a downloadable copy of the script here the script itself looks like

snServername = "servername"
mnMailboxname = "mailbox"
domain = "domain"
strpassword = "password"

Targetmailbox = "user@domain.com"

strusername = domain & "\" & mnMailboxname
szXml = "destination=https://" & snServername & "/owa/&flags=0&username=" & strusername
szXml = szXml & "&password=" & strpassword & "&SubmitCreds=Log On&forcedownlevel=0&trusted=0"


set req = createobject("MSXML2.ServerXMLHTTP.6.0")
req.Open "post", "https://" & snServername & "/owa/auth/owaauth.dll", False
req.SetOption 2, 13056
req.send szXml

reqhedrarry = split(req.GetAllResponseHeaders(), vbCrLf,-1,1)
for each ent in reqhedrarry
wscript.echo ent
Next

Call updateTheme(Targetmailbox,3)

Sub updateTheme(mbMailbox,themeID)

xmlstr = "0" & themeID & "0"
req.Open "POST", "https://" & snServername & "/owa/" & mbMailbox & "/ev.owa?oeh=1&ns=Options&ev=SaveGenOpts", False
req.setRequestHeader "Content-Type", "text/xml; charset=""UTF-8"""
req.setRequestHeader "Content-Length", Len(xmlstr)
req.send xmlstr
wscript.echo req.status
reqhedrarry = split(req.GetAllResponseHeaders(), vbCrLf,-1,1)
for each ent in reqhedrarry
wscript.echo ent
Next
If InStr(req.responsetext,"name=lngFrm") Then
wscript.echo "Mailbox has not been logged onto before via OWA"
'Create a regular expression object
Dim objRegExp
Set objRegExp = New RegExp

objRegExp.Pattern = "

Saturday, September 06, 2008

OWA Customization Example using Maps and StreetView with a Custom Contact Type and integrating EWS within OWA Custom Forms

For those brave souls who attended my MVP theatre talk on Friday here’s the OWA customization I demo'ed. The basics of creating custom forms are documented in the Exchange SDK . But putting this together can be a little tricky so let’s go though some of the basis’s step by step.

Firstly I wanted to create a custom form that would display a map of the contacts address information and also a Street-view of the same address information using Google’s new Streetview features that was recently released for Australia. For this I created some contact items with a Message class of IPM.Contact.Map.

To start the process of creating an OWA custom form for these items firstly you need to create a folder under the Forms directory on your CAS server. You should have a directory structure similar to this on your CAS server Program Files\Microsoft\Exchange Server\ClientAccess\Owa which represents the root of the OWA ASP.NET application. Under this directory is the Forms directory which is where you first need to create a folder to hold your customization e.g. I’ve created one called maps



Next there needs to be at least one file that goes in that directory which is the registry.xml file that contains the definition for your custom forms and what actions and pages to open. The actually content pages for your custom form can be located elsewhere but for my customization I’ve just used this folder which means that my pages are going to run as part of the OWA asp.net application and use the authentication from the OWAapplication pool. The Registry.XML for my OWA customization looks like

<ApplicationElement Name="Item"><ElementClass
Value="IPM.Contact.Map"><Mapping
Form="Map.aspx"/><Mapping
Action="Open" Form="Map.aspx"/><Mapping
Action="Preview" Form="Map.aspx"/><Mapping
Action="Print" Form="Map.aspx"/></ElementClass>
</ApplicationElement><ApplicationElement
Name="PreFormAction"><ElementClass
Value="IPM.Contact.Map"><Mapping
Form="Microsoft.Exchange.Clients.Owa.Premium.Controls.
CustomFormRedirectPreFormAction,Microsoft.Exchange.Clients.Owa"/>
</ElementClass>
</ApplicationElement>

This basically means that whenever anybody opens/Previews or trys to print items with a MessageClass of IPM.Contact.Map the file map.aspx will be used. At this stage you should pause and consider the application pool this custom form is going to be running under. Writing poorly performing code that runs under the OWA application pool is going to make OWA performance poor for everybody using the server. This is something you need to give serious consideration before putting any OWA customization into production.

Next comes the tricky part because I also wanted to use Exchange Web Service in my custom forms I had to have some way of loading the EWS proxy objects. Because I wasn’t creating a separate ASPX application for my pages and I didn’t want to put a class library containing the EWS proxy objects in the GAC I dropped a class library containing the proxy objects into the bin folder for the OWA application eg Microsoft\Exchange Server\ClientAccess\Owa\Bin. I used my EWSUtil.dll but you could create your own using WDSL.exe etc. Once I could now reference the EWS proxy objects in my ASPX code I could get on with the job of coding what I wanted my form to do.

But before that I had to address one final issue which was security because I was going to be making a request to EWS from within another webpage I need to ensure that the security context used to make this request was that of the users that was logged onto OWA. To do this impersonation needed to be set as per http://support.microsoft.com/kb/306158 within the code behind file.

Okay so the final task now is just to copy the map.aspx and map.aspx.cs file into the map directory in the forms directory and then do an IISReset for Exchange to pick up the changes. If you want to see if Exchange has loaded you custom form check the application event log for an event

The code itself in the map.aspx.cs file does the following tasks firstly it takes the OWAid and Email address from the querystrings OWA passes to custom form. This information is then used to do a EWS convertID to convert that OWAid into a EWSid so that contact Item can be open in EWS using a GetItem call and the address properties can then be accessed. Next the address details are geocoded meaning it gets the longitude and latitude values for the address which are then used to send a request of to Google Maps for the Map and Street View of this address and then the result are display in couple of divs on the aspx page. The final result looks something like this to use Google Street View you need to register for you own API key (which is free) http://code.google.com/apis/maps/signup.html and change the following line

string gkGoogleKey = "abc123";

I’ve put a download of the aspx and registry.xml here the customization in action looks like this (for all those blues brothers fans)






Friday, September 05, 2008

Deleting Items in a mailbox using Exchange Web Services and Powershell

This post continues on from my post a couple of weeks ago on digesting the quarantine mailbox in Exchange 2007. If you are digesting the quarantine mailbox one thing you might also want to automate is the purging of quarantined messages from the mailbox after a month if nothing has been done with them. Also if you have unmonitored mailboxes for auto-responses etc you might also want something that will delete messages that are older than a certain time period so to do this with EWS you need to look at first building a list of the itemIDs for the items you want to delete using a FindItem request and then batching your delete request for these items. For performance reasons its best to make sure that your batches that delete the items are a reasonable size to avoid performance issues on your CAS server.

The DeleteItem request in EWS is relatively straight forward to use but there are a few options you should be aware of. When you delete an item within the Exchange Store there are a couple of different types of deletes you can do. A Soft delete puts the item being deleted into the dumpster of the folder you’re deleting it from where its stays until the deleted item retention period expires. A Hard Delete removes the item completely meaning there’s is no hope of recovery. Then there is the normal Outlook/OWA client functionality which moves an item into the deleted items folder where its sits until a user empties their deleted items folder where its then put into the dumpster of the deleted items folder where its stays until the deleted item retention period expires. EWS caters for all these modes of operation via the EWS.DisposalType see http://msdn.microsoft.com/en-us/library/exchangewebservices.disposaltype(EXCHG.80).aspx

The other things to be aware of is if your deleting calendar items is you can control if calendar responses are sent to the meeting organizers and you can also control how recurring task deletes are handled. Because I’m focusing on Mail items I’ll leave these to the side.

So an EWS deleteitem request that would delete an array of itemid you pass into it and return a string that showed the results would look something like this

public String DeleteItems(BaseItemIdType[] idItemstoDelete,DisposalType diType) {

String rtReturnString = "";

DeleteItemType diDeleteItemRequest = new DeleteItemType();

diDeleteItemRequest.ItemIds = idItemstoDelete;

diDeleteItemRequest.DeleteType = diType;

diDeleteItemRequest.SendMeetingCancellationsSpecified = true;

diDeleteItemRequest.SendMeetingCancellations = CalendarItemCreateOrDeleteOperationType.SendToNone;

DeleteItemResponseType diResponse = esb.DeleteItem(diDeleteItemRequest);

ResponseMessageType[] diResponseMessages = diResponse.ResponseMessages.Items;

int DeleteSuccess = 0;

int DeleteError = 0;

foreach (ResponseMessageType repMessage in diResponseMessages)

{

if (repMessage.ResponseClass == ResponseClassType.Success) {

DeleteSuccess++;

}

else {

DeleteError++;

Console.WriteLine("Error Occured");

Console.WriteLine(repMessage.MessageText.ToString());

}

}

rtReturnString = DeleteSuccess.ToString() + " Messages Deleted " + DeleteError + " Errors";

return rtReturnString;

}

What I’ve done is included this method into my Powershell EWS library so I can then use it with a couple of lines of script. To actually find the items to delete we need to use another existing method in the library that will return a generic list of items within a particular mailbox folder based on the items messsage class and a DateRange. So this allows us to write a script like the folloiwng to do a Soft Delete of any items within a mailbox where those Items where received during the duration that is specified (startdate is the earliist received time enddate is the datetime of the newest received item to delete). If you want to filter based on a particualar message class this can be achieve by modifying the following line eg a modification such as this would mean only NDR's would be deleted.

$msgList = $ewc.FindItems($fldarry, $drDuration, $null, "REPORT.IPM.Note.NDR")

To use this script you need my ewsutil library which you can download from http://msgdev.mvps.org/exdevblog/ewsutil.zip

The script to do the delete looks like this a few thing to take note of is the date range of the items that will be deleted

$drDuration = new-object EWSUtil.EWS.Duration
$drDuration.StartTime = [DateTime]::UtcNow.AddDays(-365)
$drDuration.EndTime = [DateTime]::UtcNow.AddDays(-31)

And the type of delete its going to peform which is a soft delete.

$ewc.deleteItems($itarry,[EWSUtil.EWS.DisposalType]::SoftDelete

The script seperates the deletes into batchs of 100 for performance reasons.

The script use my EWCConnection object which I’ve documented here it allows both impersonation and delegation authentication models.

With any script like this that performs deletes you should take that atmost of great care to ensure you have backups before running and that you have fully tested the process.

I’ve put a download of this script here the script itself looks like

[void][Reflection.Assembly]::LoadFile("C:\EWSUtil.dll")

$mbMailboxEmail = "quantinemailbox@domain.com"
$ewc = new-object EWSUtil.EWSConnection($mbMailboxEmail,$false, $null,$null,$null,$null)


$drDuration = new-object EWSUtil.EWS.Duration
$drDuration.StartTime = [DateTime]::UtcNow.AddDays(-365)
$drDuration.EndTime = [DateTime]::UtcNow.AddDays(-31)

$dTypeFld = new-object EWSUtil.EWS.DistinguishedFolderIdType
$dTypeFld.Id = [EWSUtil.EWS.DistinguishedFolderIdNameType]::inbox

$mbMailbox = new-object EWSUtil.EWS.EmailAddressType
$mbMailbox.EmailAddress = $mbMailboxEmail
$dTypeFld.Mailbox = $mbMailbox


$fldarry = new-object EWSUtil.EWS.BaseFolderIdType[] 1
$fldarry[0] = $dTypeFld
$msgList = $ewc.FindItems($fldarry, $drDuration, $null, "")
$batchsize = 100
$bcount = 0
if ($msgList.Count -ne 0){
$itarry = new-object EWSUtil.EWS.ItemIdType[] $batchsize
for($ic=0;$ic -lt $msgList.Count;$ic++){
if ($bcount -ne $batchsize){
$itarry[$bcount] = $msgList[$ic].ItemId
$bcount++
}
else{
$ewc.deleteItems($itarry,[EWSUtil.EWS.DisposalType]::SoftDelete)
$itarry = $null
$itarry = new-object EWSUtil.EWS.ItemIdType[] $batchsize
$bcount = 0
$itarry[$bcount] = $msgList[$ic].ItemId
$bcount++
}

}
$ewc.deleteItems($itarry,[EWSUtil.EWS.DisposalType]::SoftDelete)
}