Wednesday, January 24, 2007

Exporting a Mailbox larger then 2 GB and spanning it across multiple PST files with a script

*update* I suggest checking out Michael Smith's RDO port of this script which is a much more developed examples http://theessentialexchange.com/blogs/michael/archive/2009/10/16/exporting-mailboxes-larger-than-2-gb-on-an-exchange-server.aspx . *update*

I have a small customer with an old server that recently had a staff member leave who had a very large mailbox over 5 GB. Now these guys are pretty tight still using Exchange 2000 and Office 2000 and don’t want to upgrade software or hardware so getting rid of this mailbox out the Exchange store while still giving people occasionally access to this mailbox was desirable and also challenging. Because all the clients are using Outlook 2000 I’m stuck with the 2 GB PST non-Unicode file limit which Exmerge is similarly afflicted with. Using Exmerge with date filters was one possible solution but I decided I’d rather just do a liner export where mail was exported one item at a time and would just span to the next PST when the space was exhausted in one. This did prove a little changeling at first but I did manage to come up with a script that worked using a combination of CDO 1.2 and RDO (Redemption Data Objects) which is part of Dmitry Streblechenko excellent Outlook Redemption library. RDO provides the ability to create PST files on the fly and also provides the same functionality as CDO 1.2. I did have some problems copying different types of objects with both libraries so I found using a combination of both allowed me to work around the problems I did have and successfully copy most objects from a mailbox.

How does it work?

The script is broken up into multiple subs and functions that all perform different tasks the first task that needs to be done is to logon to the mailbox and create the first PST file. Once that PST file is created the delete-items folder is located and a mapping is created using a dictionary object that maps the entryID of the mailboxe’s deleted items to the EntryID of the deleted items in the PST file. The script then loops through every folder in the Mailbox and then recreates this hierarchy in the PST file. This will happen every time the createpst function is called and ensures that the same folder structure is in each of the spanned PST files. EntryID mappings are created in the dictionary object for each folder as is used later on in the script to process each the contents of each folder and map the items that are being copied in to the right folder in the PST file. The CreatePST function uses the enumfolder function to enumerate any subfolders and the ProcessFolderRoot and ProcessFolderSub to create the folders in the PST and update the dictionary object.

Once the PST is setup the script then performs another enumeration on the folders in the mailbox and processes the items in each of the folder collections in the processitems sub. This sub loops though each item in a folder one at a time and calculates what the size of the PST file will be after copying the current item into the file based on the size of the item and the current file size of the PST file returned form the File System Object. If the new file size will be over a configure threshold value a new PST file is created by calling the CreatePST function. The destination folder object is reset and the script then resumes using the CDO 1.2 message object’s copyto function. If the cdo copyto function fails the RDO copyto function is tried as a fallback (this is a work around for some issues I had with both libraries). If the processitems sub detects the current folder is a contacts folder RDO is used to do the copy to stop the script from hitting the CDO security prompt issues. To track the progress of the script debugging information is written to the command prompt and also to a file in the same directory as the PST file. After each folder is processed the number of items in both the source and destination are written to the log file so this can be checked for debugging purposes.

How to use the script

The script requires CDO 1.2 (from Exchange, Outlook or the Standalone version) and Redemption. The script has a number of parameters you need to configure first


tnThreshold = 1800

This is the threshold value for the size of the spanned PST’s I found using 1800 to be the most effective threshold and made sure that the pst files where always under 2 GB. If you want to make the PST smaller eg maybe you want to fit the spanned PST's onto multiple CD’s you could reduce this value it doesn’t work out to be an exact size so you need to make sure you give yourself a buffer.

servername = "servername"

mbMailbox = "mailbox"


These variables should be obvious

bfBaseFilename = "expMailbox"

pfFilePath = "c:\temp\"


The base file name is what the exported PST will be called there will also be have a number appended to the file name to specify the number in the span set.


Note: This script is defiantly not a replacement for Exmerge or any other backup or export methods you have. (eg if I had Outlook 2003 at this client I would have used this instead). I can’t verify the accuracy or consistency of this script when exporting a mailbox so it’s use at your own risk and do your own testing. (Any always make sure you have backed up what ever you’re exporting on a different media). For me it seemed to work fine exporting my 5 GB mailbox to 3 pst file which can be accessed from the old outlook 2000 clients okay when ever needed.

I’ve put a downloadable copy of this script here the script itself looks like

Set doDictionaryObject = CreateObject("Scripting.Dictionary")
Set fso = CreateObject("Scripting.FileSystemObject")
Set fso = CreateObject("Scripting.FileSystemObject")
set RDOSession = CreateObject("Redemption.RDOSession")
tsize = 10
tnThreshold = 1800
servername = "ServerName"
mbMailbox = "Mailbox"
bfBaseFilename = "expMailbox"
pfFilePath = "c:\temp\"
fnFileName = ""
PST = ""
pstroot = ""
IPMRoot = ""
pfPstFile = ""
fNumber = 0
set wfile = fso.opentextfile(pfFilePath & bfBaseFilename & ".txt",2,true)
RDOSession.LogonExchangeMailbox mbMailbox,servername
Set dfDeletedItemsFolder = RDOSession.GetDefaultFolder(3)
CreateNewPst()

wscript.echo fnFileName
wscript.echo "Enumerate Messages"
for miLoop = 1 to IPMRoot.Folders.count
ProcessItems(IPMRoot.Folders(miLoop))
if IPMRoot.Folders(miLoop).Folders.count <> 0 then
call Enumfolders(IPMRoot.Folders(miLoop),PstRootFolder,2)
end if
next


function Enumfolders(FLDS,RootFolder,ltype)
for fl = 1 to FLDS.Folders.count
if ltype = 1 then
call ProcessFolderSub(FLDS.folders(fl),RootFolder)
else
ProcessItems(FLDS.folders(fl))
end if
wscript.echo FLDS.folders(fl).Name
if FLDS.folders(fl).Folders.count <> 0 then
if ltype = 1 then
call Enumfolders(FLDS.folders(fl),FLDS.folders(fl).EntryID,1)
else
call Enumfolders(FLDS.folders(fl),FLDS.folders(fl).EntryID,2)
end if
end if
next
End function

Function CreateNewPst()

doDictionaryObject.RemoveAll
fNumber = fNumber + 1
fnFileName = pfFilePath & bfBaseFilename & "-" & fNumber & ".pst"
set PST = RDOSession.Stores.AddPSTStore(fnFileName, 1, "Exported MailBox-" & now())
set pstroot = RDOSession.GetFolderFromID(PST.IPMRootFolder.EntryID,PST.EntryID)
For Each pstfld In PstRoot.folders
If pstfld.Name = "Deleted Items" Then
If fNumber = 1 Then
doDictionaryObject.add dfDeletedItemsFolder.EntryID, pstfld.EntryID
wscript.echo "Added Deleted Items Folder"
End if
End if
next
set IPMRoot = RDOSession.Stores.DefaultStore.IPMRootFolder
for fiLoop = 1 to IPMRoot.Folders.count
if IPMRoot.Folders(fiLoop).Name <> "Deleted Items" then
PstRootFolder = ProcessFolderRoot(IPMRoot.Folders(fiLoop),PST.IPMRootFolder.EntryID)
if IPMRoot.Folders(fiLoop).Folders.count <> 0 then
call Enumfolders(IPMRoot.Folders(fiLoop),IPMRoot.Folders(fiLoop).EntryID,1)
end If
Else
if IPMRoot.Folders(fiLoop).Folders.count <> 0 then
call Enumfolders(IPMRoot.Folders(fiLoop),IPMRoot.Folders(fiLoop).EntryID,1)
end if
end if
next
Set pfPstFile = fso.GetFile(fnFileName)

end function

function ProcessFolderRoot(Fld,parentfld)

set CDOPstfld = RDOSession.GetFolderFromID(parentfld,PST.EntryID)
wscript.echo fld.Name
Set newFolder = CDOPstfld.Folders.ADD(Fld.Name)
ProcessFolder = newfolder.EntryID
newfolder.fields(&H3613001E) = Fld.fields(&H3613001E)

doDictionaryObject.add Fld.EntryID,newfolder.EntryID
end Function

function ProcessFolderSub(Fld,parentfld)

set CDOPstfld = RDOSession.GetFolderFromID(doDictionaryObject.item(parentfld),PST.EntryID)
wscript.echo fld.Name
Set newFolder = CDOPstfld.Folders.ADD(Fld.Name)
ProcessFolder = newfolder.EntryID
newfolder.fields(&H3613001E) = Fld.fields(&H3613001E)

doDictionaryObject.add Fld.EntryID,newfolder.EntryID
end function

Sub ProcessItems(Fld)

If Fld.fields(&H3613001E) = "IPF.Contact" Then
set dfDestinationFolder = RDOSession.GetFolderFromID(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
wscript.echo dfDestinationFolder.Name
wfile.writeLine("Processing Folder : ") & dfDestinationFolder.Name
for fiItemloop = 1 to Fld.items.count
on error resume next
pfPredictednewSize = formatnumber((pfPstFile.size + Fld.items(fiItemloop).size)/1048576,2,0,0,0)
if err.number <> 0 Then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo Fld.items(fiItemloop).EntryID
wscript.echo "Subect of Item:"
wscript.echo Fld.items(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(Fld.items(fiItemloop).EntryID )
Wfile.writeline("Subect of Item:")
Wfile.writeline(Fld.items(fiItemloop).Subject)
err.clear
end if
If Int(pfPredictednewSize) >= Int(tsize) Then
Wscript.echo "10 MB Exported"
tsize = tsize + 10
End if
If Int(pfPredictednewSize) >= Int(tnThreshold) Then
wfile.writeLine("New PST about to be created - Destination - Number of Items : " & dfDestinationFolder.messages.count)
CreateNewPst()
set dfDestinationFolder = RDOSession.GetFolderFromID(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
call Fld.items(fiItemloop).copyto(dfDestinationFolder)
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo Fld.items(fiItemloop).EntryID
wscript.echo "Subect of Item:"
wscript.echo Fld.items(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(Fld.items(fiItemloop).EntryID )
Wfile.writeline("Subect of Item:")
Wfile.writeline(Fld.items(fiItemloop).Subject)
err.clear
end if
else
call Fld.items(fiItemloop).copyto(dfDestinationFolder)
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo Fld.items(fiItemloop).EntryID
wscript.echo "Subect of Item:"
wscript.echo Fld.items(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(Fld.items(fiItemloop).EntryID )
Wfile.writeline("Subect of Item:")
Wfile.writeline(Fld.items(fiItemloop).Subject)
err.clear
end if
End if
on error goto 0
Next
wfile.writeLine("Source - Number of Items : " & Fld.fields(&h36020003) & " Destination - Number of Items : " & dfDestinationFolder.items.count)
else
set CDOSession = CreateObject("MAPI.Session")
CDOSession.MAPIOBJECT = RDOSession.MAPIOBJECT
Set objInfoStore = CDOSession.GetInfoStore(PST.EntryID)
set srcFld = CDOSession.GetFolder(Fld.EntryID)
wfile.writeLine("Processing Folder : ") & srcFld.Name
set dfDestinationFolder = CDOSession.GetFolder(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
wscript.echo dfDestinationFolder.Name
for fiItemloop = 1 to srcFld.messages.count
on error resume next
pfPredictednewSize = formatnumber((pfPstFile.size + srcFld.messages(fiItemloop).size)/1048576,2,0,0,0)
if err.number <> 0 Then
Wscript.echo "Error Processing Item in " & srcFld.messages(fiItemloop).Name
wscript.echo "EntryID of Item:"
wscript.echo srcFld.messages(fiItemloop).id
wscript.echo "Subect of Item:"
wscript.echo srcFld.messages(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & srcFld.messages(fiItemloop).Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).id)
Wfile.writeline("Subect of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).Subject)
err.clear
rem Try to Copy with RDO
Set rdosrc = RDOSession.GetMessageFromID(srcFld.messages(fiItemloop).Id)
rdosrc.copyto(dfDestinationFolder)
if err.number <> 0 Then
Wscript.echo "Also Failed RDO Copy"
wfile.writeline("Also Failed RDO Copy")
Else
Wscript.echo "Copied with RDO Okay"
wfile.writeline("Copied with RDO Okay")
End if
err.clear
end if
If Int(pfPredictednewSize) >= Int(tsize) Then
Wscript.echo "10 MB Exported"
tsize = tsize + 10
End if
If Int(pfPredictednewSize) >= Int(tnThreshold) Then
wfile.writeLine("New PST about to be created - Destination - Number of Items : " & dfDestinationFolder.messages.count)
CreateNewPst()
set CDOSession = CreateObject("MAPI.Session")
CDOSession.MAPIOBJECT = RDOSession.MAPIOBJECT
Set objInfoStore = CDOSession.GetInfoStore(PST.EntryID)
set dfDestinationFolder = CDOSession.GetFolder(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
Set cpymsg = srcFld.messages(fiItemloop).copyto(dfDestinationFolder.ID)
cpymsg.update
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo srcFld.messages(fiItemloop).Id
wscript.echo "Subect of Item:"
wscript.echo srcFld.messages(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).id)
Wfile.writeline("Subect of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).Subject)
err.clear
rem Try to Copy with RDO
Set rdosrc = RDOSession.GetMessageFromID(srcFld.messages(fiItemloop).Id)
rdosrc.copyto(dfDestinationFolder)
if err.number <> 0 Then
Wscript.echo "Also Failed RDO Copy"
wfile.writeline("Also Failed RDO Copy")
Else
Wscript.echo "Copied with RDO Okay"
wfile.writeline("Copied with RDO Okay")
End if
err.clear
end if
Else
Set cpymsg = srcFld.messages(fiItemloop).copyto(dfDestinationFolder.ID)
cpymsg.update
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo srcFld.messages(fiItemloop).id
wscript.echo "Subect of Item:"
wscript.echo srcFld.messages(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).id)
Wfile.writeline("Subect of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).Subject)
err.clear
rem Try to Copy with RDO
Set rdosrc = RDOSession.GetMessageFromID(srcFld.messages(fiItemloop).Id)
rdosrc.copyto(dfDestinationFolder)
if err.number <> 0 Then
Wscript.echo "Also Failed RDO Copy"
wfile.writeline("Also Failed RDO Copy")
Else
Wscript.echo "Copied with RDO Okay"
wfile.writeline("Copied with RDO Okay")
End if
err.clear
end if
End if
on error goto 0
Next
wfile.writeLine("Source - Number of Items : " & srcFld.fields(&h36020003) & " Destination - Number of Items : " & dfDestinationFolder.messages.count)
End if
end sub

Wednesday, January 17, 2007

Sending Attachments via the Exchange Web Services in Exchange 2007

Anyone who has every tried to send an attachment using WebDAV will know that it is a real pain compared with CDOSYS/CDOEX where it was generally just one line of code. The new Exchange Web Services fall somewhere in the middle of these two while not a simple 1 line of code the functionality is there it is just not immediately easy to work out how to do it. Here are two methods I’ve found that work if you want to send a message with an attachment (or just create an item or attach something to an already existing item). Note there are probably some other methods you can use but hopefully this is a little more helpful then just the XML in the current SDK. The first method is to create an item in a folder (eg the drafts folder) using a createitem operation then using the itemid and changeid returned from the create item request use the createattachment operation to attach the file to the new item you created. Then using the new changeid returned after the createattachment operation use a senditem operation to send the Message. The other method is to use a Createitem operation and set the MessageDisposition to SendAndSaveCopy which means the item that is created in this operation will also be sent at the same time. When using this method if you want to send a message with an attachment you need to post a MIME encoded copy of the message you want to send (with the attachment included). To MIME encode a message and its attachments you need to use a MIME Encoder/Parser. While in .NET 2.0 there is the Systen.NET.Mail and MIME classes these namespaces don’t give you direct access to the MIME stream of a message (you can post the message to a local directory and then reopen the file from the file system but this isn’t a very practial method). Fortunalty the good old CDOSYS/CDOEX still makes the best tried and tested MIME parser around and is more the adequate for the task.

I’ve created a couple of C# samples of both methods that use the Webservices proxy object and also a sample for Powershell and VBS that use CDOSYS to get a Base64 encoded version of the MIME Encoded message and then post this XML.

I’ve put a downloadable copy of all the samples here the create,attach send C# sample looks like

using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.IO;
using System.Xml.Serialization;
using ewsCreateAttachSend.mgne12v2;

namespace ewsCreateAttachSend
{
class Program
{
static void Main(string[] args)
{
Program coControlobject = new Program();
ExchangeServiceBinding ewsServiceBinding = new ExchangeServiceBinding();
ewsServiceBinding.Credentials = new NetworkCredential("username", "password", "domain");
ewsServiceBinding.Url = @"https://servername/EWS/exchange.asmx";
MessageType emMessage = new MessageType();
emMessage.Subject = "Test Attachment Send";
emMessage.Body = new BodyType();
emMessage.Body.BodyType1 = BodyTypeType.Text;
emMessage.Body.Value = "Blah,Blah";
emMessage.ItemClass = "IPM.Note";
emMessage.ToRecipients = new EmailAddressType[1];
emMessage.ToRecipients[0] = new EmailAddressType();
emMessage.ToRecipients[0].EmailAddress = "recipient@domain.com";
emMessage.Sensitivity = SensitivityChoicesType.Normal;
ItemIdType iiCreateItemid = coControlobject.CreateDraftMessage(ewsServiceBinding, emMessage);
iiCreateItemid = coControlobject.CreateAttachment(ewsServiceBinding, "c:\\file.ext", iiCreateItemid);
coControlobject.SendMessage(ewsServiceBinding,iiCreateItemid);
}
private ItemIdType CreateDraftMessage(ExchangeServiceBinding ewsServiceBinding,MessageType emMessage) {
ItemIdType iiItemid = new ItemIdType();
CreateItemType ciCreateItemRequest = new CreateItemType();
ciCreateItemRequest.MessageDisposition = MessageDispositionType.SaveOnly;
ciCreateItemRequest.MessageDispositionSpecified = true;
ciCreateItemRequest.SavedItemFolderId = new TargetFolderIdType();
DistinguishedFolderIdType dfDraftsFolder = new DistinguishedFolderIdType();
dfDraftsFolder.Id = DistinguishedFolderIdNameType.drafts;
ciCreateItemRequest.SavedItemFolderId.Item = dfDraftsFolder;
ciCreateItemRequest.Items = new NonEmptyArrayOfAllItemsType();
ciCreateItemRequest.Items.Items = new ItemType[1];
ciCreateItemRequest.Items.Items[0] = emMessage ;
CreateItemResponseType createItemResponse = ewsServiceBinding.CreateItem(ciCreateItemRequest);
if (createItemResponse.ResponseMessages.Items[0].ResponseClass == ResponseClassType.Error)
{
Console.WriteLine("Error Occured");
Console.WriteLine(createItemResponse.ResponseMessages.Items[0].MessageText);
}
else
{
ItemInfoResponseMessageType rmResponseMessage = createItemResponse.ResponseMessages.Items[0] as ItemInfoResponseMessageType;
Console.WriteLine("Item was created");
Console.WriteLine("Item ID : " + rmResponseMessage.Items.Items[0].ItemId.Id.ToString());
Console.WriteLine("ChangeKey : " + rmResponseMessage.Items.Items[0].ItemId.ChangeKey.ToString());
iiItemid.Id = rmResponseMessage.Items.Items[0].ItemId.Id.ToString();
iiItemid.ChangeKey = rmResponseMessage.Items.Items[0].ItemId.ChangeKey.ToString();
}

return iiItemid;
}
private ItemIdType CreateAttachment(ExchangeServiceBinding ewsServiceBinding,String fnFileName,ItemIdType iiCreateItemid) {
ItemIdType iiAttachmentItemid = new ItemIdType();
FileStream fsFileStream = new FileStream(fnFileName, System.IO.FileMode.Open, System.IO.FileAccess.Read);
byte[] bdBinaryData = new byte[fsFileStream.Length];
long brBytesRead = fsFileStream.Read(bdBinaryData, 0, (int)fsFileStream.Length);
fsFileStream.Close();
FileAttachmentType faFileAttach = new FileAttachmentType();
faFileAttach.Content = bdBinaryData;
faFileAttach.Name = fnFileName;
CreateAttachmentType amAttachmentMessage = new CreateAttachmentType();
amAttachmentMessage.Attachments = new AttachmentType[1];
amAttachmentMessage.Attachments[0] = faFileAttach;
amAttachmentMessage.ParentItemId = iiCreateItemid;
CreateAttachmentResponseType caCreateAttachmentResponse = ewsServiceBinding.CreateAttachment(amAttachmentMessage);
if (caCreateAttachmentResponse.ResponseMessages.Items[0].ResponseClass == ResponseClassType.Error)
{
Console.WriteLine("Error Occured");
Console.WriteLine(caCreateAttachmentResponse.ResponseMessages.Items[0].MessageText);
}
else {
AttachmentInfoResponseMessageType amAttachmentResponseMessage = caCreateAttachmentResponse.ResponseMessages.Items[0] as AttachmentInfoResponseMessageType;
Console.WriteLine("Attachment was created");
Console.WriteLine("Change Key : " + amAttachmentResponseMessage.Attachments[0].AttachmentId.RootItemChangeKey.ToString());
iiAttachmentItemid.Id = amAttachmentResponseMessage.Attachments[0].AttachmentId.RootItemId.ToString();
iiAttachmentItemid.ChangeKey = amAttachmentResponseMessage.Attachments[0].AttachmentId.RootItemChangeKey.ToString();
}
return iiAttachmentItemid;
}
private void SendMessage(ExchangeServiceBinding ewsServiceBinding,ItemIdType iiCreateItemid) {
SendItemType siSendItem = new SendItemType();
siSendItem.ItemIds = new BaseItemIdType[1];
siSendItem.SavedItemFolderId = new TargetFolderIdType();
DistinguishedFolderIdType siSentItemsFolder = new DistinguishedFolderIdType();
siSentItemsFolder.Id = DistinguishedFolderIdNameType.sentitems;
siSendItem.SavedItemFolderId.Item = siSentItemsFolder;
siSendItem.SaveItemToFolder = true; ;
siSendItem.ItemIds[0] = (BaseItemIdType)iiCreateItemid;
SendItemResponseType srSendItemReponseMessage = ewsServiceBinding.SendItem(siSendItem);
if (srSendItemReponseMessage.ResponseMessages.Items[0].ResponseClass == ResponseClassType.Error)
{
Console.WriteLine("Error Occured");
Console.WriteLine(srSendItemReponseMessage.ResponseMessages.Items[0].MessageText);
}
else {
Console.WriteLine("Message Sent");
}

}
}
}

Friday, January 12, 2007

Audit and Export all emails in an Exchange Store Sent and Received to and from a certain domain

This is one that came up for me this week where for audit purposes all communication between a certain email domain needed to be found and exported in a Exchange Store. I all ready had the bones for this script in one of my other posts all I needed to do was adapt this by adding in some code to retrieve the fromemail and the to properties and then do a substring search of both of these properties. If a match is found the email is then exported by using WebDAV to get the stream of the item and then using an ADO stream to write the stream to a normal EML file that can then be opened in any mail client.

The script works using WebDAV and it connects to and scans every mailbox using the Admin virtual root which means the script can run with delegated Exchange admin rights. The first part of the script contains some ADSI queries to work out what the URL to the admin root is and then calls the RecurseFolder sub this sub is based on the code from the mailbox size KB . With a few exceptions the main one being is that it only checks normal mail folders this was to prevent an issue where searchfolders exist in a mailbox. The ResurseFolder sub job is basically to retrieve all the folder URL’s in the mailbox and then call the procfolder sub. The profolder sub processes all the email in a folder between the date-range specified. If a substring match is found on any of the from or to addresses the exportemail sub is called which contains some code to ensure that a unique filename is generated for each exported email.

By default the script exports email to a directory called exp on the c:\ to change this you need to modify the following line in the script.

fpath = "c:\exp\"

The front end of the script takes 4 commandline parameters the first is the servername of the server you want to run the script against the 2nd is the domain you want to scan and the 3rd and 4th is the date range you want to look at. EG so you can scan all correspondences for just a certain time period which will cut down on the time it takes for the script to run.The start and enddate needs to be in ISO format (year-month-date) eg to scan for all message sent to or from the domain blahdoman.com between October and January this year the commandline to run the script would be.

cscript mbauditcdomv2.vbs servername @blahdoman.com 2006-10-01 2007-02-01

I've created two copies of the script the mbauditcdomv2000.vbs version use the mailnickname to connect to each mailbox and is designed for Exchange 2000 where you cant use the email address to connect to the mailbox in webdav.

I’ve put a downloadable copy of the script here the script itself looks like

on error resume Next
fpath = "c:\exp\"
Servername = wscript.arguments(0)
domaintosearch = wscript.arguments(1)
datefrom = wscript.arguments(2) & "T00:00:00Z"
dateto = wscript.arguments(3) & "T00:00:00Z"
set req = createobject("microsoft.xmlhttp")
set com = createobject("ADODB.Command")
set conn = createobject("ADODB.Connection")
Set iAdRootDSE = GetObject("LDAP://RootDSE")
strNameingContext = iAdRootDSE.Get("configurationNamingContext")
strDefaultNamingContext = iAdRootDSE.Get("defaultNamingContext")
Conn.Provider = "ADsDSOObject"
Conn.Open "ADs Provider"
polQuery = "<LDAP://" & strNameingContext & ">;(&(objectCategory=msExchRecipientPolicy)(cn=Default
Policy));distinguishedName,gatewayProxy;subtree"
svcQuery = "<LDAP://" & strNameingContext & ">;(&(objectCategory=msExchExchangeServer)(cn="
& Servername & "));cn,name,legacyExchangeDN;subtree"
Com.ActiveConnection = Conn
Com.CommandText = polQuery
Set plRs = Com.Execute
while not plRs.eof
for each adrobj in plrs.fields("gatewayProxy").value
if instr(adrobj,"SMTP:") then dpDefaultpolicy =
right(adrobj,(len(adrobj)-instr(adrobj,"@")))
next
plrs.movenext
wend
wscript.echo dpDefaultpolicy
Com.CommandText = svcQuery
Set Rs = Com.Execute
while not rs.eof
GALQueryFilter = "(&(&(&(& (mailnickname=*)(!msExchHideFromAddressLists=TRUE)(|
(&(objectCategory=person)(objectClass=user)(msExchHomeServerName=" &
rs.fields("legacyExchangeDN") & ")) )))))"
strQuery = "<LDAP://" & strDefaultNamingContext & ">;" & GALQueryFilter & ";displayname,mail,distinguishedName,mailnickname,proxyaddresses;subtree"
com.Properties("Page Size") = 100
Com.CommandText = strQuery
Set Rs1 = Com.Execute
while not Rs1.eof
falias = "http://" & servername & "/exadmin/admin/" & dpDefaultpolicy & "/mbx/"
for each paddress in rs1.fields("proxyaddresses").value
if instr(paddress,"SMTP:") then
falias = falias & replace(paddress,"SMTP:","")
cusername = replace(paddress,"SMTP:","")
End if
next
ReDim tresarray(1,6)
wscript.echo falias
call RecurseFolder(falias)
rs1.movenext
wend
rs.movenext
wend
rs.close
set conn = nothing
set com = nothing
set wfile = nothing
set fso = nothing

Public Sub RecurseFolder(sUrl)

req.open "SEARCH", sUrl, False, "", ""
sQuery = "<?xml version=""1.0""?>"
sQuery = sQuery & "<g:searchrequest xmlns:g=""DAV:"">"
sQuery = sQuery & "<g:sql>SELECT ""http://schemas.microsoft.com/"
sQuery = sQuery & "mapi/proptag/x0e080003"", ""DAV:hassubs"" FROM SCOPE "
sQuery = sQuery & "('SHALLOW TRAVERSAL OF """ & sUrl & """') "
sQuery = sQuery & "WHERE ""DAV:isfolder"" = true and ""DAV:ishidden"" = false
and ""http://schemas.microsoft.com/mapi/proptag/x36010003"" = 1"
sQuery = sQuery & "</g:sql>"
sQuery = sQuery & "</g:searchrequest>"
req.setRequestHeader "Content-Type", "text/xml"
req.setRequestHeader "Translate", "f"
req.setRequestHeader "Depth", "0"
req.setRequestHeader "Content-Length", "" & Len(sQuery)
req.send sQuery
Set oXMLDoc = req.responseXML
Set oXMLSizeNodes = oXMLDoc.getElementsByTagName("d:x0e080003")
Set oXMLHREFNodes = oXMLDoc.getElementsByTagName("a:href")
Set oXMLHasSubsNodes = oXMLDoc.getElementsByTagName("a:hassubs")
For i = 0 to oXMLSizeNodes.length - 1
call procfolder(oXMLHREFNodes.Item(i).nodeTypedValue,sUrl)
wscript.echo oXMLHREFNodes.Item(i).nodeTypedValue
If oXMLHasSubsNodes.Item(i).nodeTypedValue = True Then
call RecurseFolder(oXMLHREFNodes.Item(i).nodeTypedValue)
End If
Next
End Sub

sub procfolder(strURL,pfname)
wscript.echo strURL
ReDim resarray(1,6)
strQuery = "<?xml version=""1.0""?><D:searchrequest xmlns:D = ""DAV:""
xmlns:b=""urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/"">"
strQuery = strQuery & "<D:sql>SELECT ""DAV:displayname"",
""urn:schemas:httpmail:subject"", "
strQuery = strQuery & """DAV:creationdate"", ""DAV:getcontentlength"", "
strQuery = strQuery & """urn:schemas:httpmail:fromemail"",
""urn:schemas:httpmail:to"""
strQuery = strQuery & " FROM scope('shallow traversal of """
strQuery = strQuery & strURL & """') Where ""DAV:ishidden"" = False AND
""DAV:isfolder"" = False AND "
strQuery = strQuery & """urn:schemas:httpmail:datereceived"" &lt; CAST(""" &
dateto & """ as 'dateTime') AND "
strQuery = strQuery & """urn:schemas:httpmail:datereceived"" &gt; CAST(""" &
datefrom & """ as 'dateTime')</D:sql></D:searchrequest>"
req.open "SEARCH", strURL, false
req.setrequestheader "Content-Type", "text/xml"
req.setRequestHeader "Translate","f"
req.send strQuery
If req.status >= 500 Then
ElseIf req.status = 207 Then
set oResponseDoc = req.responseXML
set oNodeList = oResponseDoc.getElementsByTagName("a:displayname")
set oNodeList1 = oResponseDoc.getElementsByTagName("a:href")
set oSize = oResponseDoc.getElementsByTagName("a:getcontentlength")
set odatereceived = oResponseDoc.getElementsByTagName("a:creationdate")
set fEmail = oResponseDoc.getElementsByTagName("d:fromemail")
set TEmail = oResponseDoc.getElementsByTagName("d:to")
For i = 0 To (oNodeList.length -1)
set oNode = oNodeList.nextNode
set oNode1 = oNodeList1.nextNode
set oNode2 = oSize.nextNode
set oNode3 = odatereceived.nextNode
set oNode4 = fEmail.nextNode
set oNode5 = TEmail.nextNode
wscript.echo oNode3.text
export = 0
If InStr(LCase(oNode4.text),LCase(domaintosearch))Then
export = 1
End If
if InStr(LCase(oNode5.text),LCase(domaintosearch))Then
export = 1
End If
If export = 1 Then
Call exportemail(oNode1.text,oNode.text)
wscript.echo "Exporting : " & oNode4.text
End if
Next
Else
End If

end sub

sub exportemail(exporthref,subject)
req.open "GET", exporthref, false
req.setRequestHeader "Translate","f"
req.send
fname = replace(replace(replace(replace(replace((cusername & "-" &
subject),":","-"),"\",""),"/",""),"?",""),chr(34),"")
fname =
replace(replace(replace(replace(replace(replace(fname,"<",""),">",""),chr(11),""),"*",""),"|",""),"(","")
fname = replace(replace(replace(fname,")",""),chr(12),""),chr(15),"")
Randomize ' Initialize random-number generator.
rndval = Int((20000000000 * Rnd) + 1)
fname = fpath & replace(lcase(fname),".eml",rndval & ".eml")
wscript.echo fname
set stm = createobject("ADODB.Stream")
stm.open
msgstring = req.responsetext
stm.type = 2
stm.Charset = "x-ansi"
stm.writetext msgstring,0
stm.Position = 0
stm.type = 1
stm.savetofile fname
set stm = nothing

End sub

Thursday, January 11, 2007

Exchange 2007 diagnostic logging Powershell quick set winform script

One of the things that has been dropped from the GUI (Exchange Management Console) in Exchange 2007 is the ability to set the diagnostic level of different Exchange components. You now have to use the Exchange Management Shell cmdlets get-eventloglevel and set-eventloglevel. While functional if you have to enable a number of different categories if you are trying to diagnose certain problems it can be a little cumbersome to try and come up with a different command-line for each one of the possible 150 components you might want to change. Because I do tend to enable and disable these things a fair bit when trying to fix a problems or diagnose why a piece a code or script might not be working I thought I’d come up with a little Powershell winform that would give me a GUI to do this with and allow me to quickly set the diagnostic logging level on a number of components with a few clicks of the mouse.

The script first starts by presenting a Winform to the user with one combo box which is populated with all the names of the Exchange servers in the Org populated from the Get-Exchangeserver cmdlet. When a servername is selected the getdiagvalues function runs which uses the get-eventloglevel cmdlet to fill a datagrid with all the current log levels for all components on that server. To set a new log level just select the row (or rows if you want to do multiple) and then the new log level from the combo box at the top of the form and press the apply button. When the apply button is pressed the UpdateLogLevel function is run this function first checks to see if multiple row in the grid where selected or just one row and then builds and executes the set-eventloglevel commandlet. After this it reruns the getdiagvalues function which repopulates the datagrid with the updated values.

The script is designed to run from within the Exchange Management shell it should work for remote servers okay but I haven’t really tested this and the set-eventloglevel doesn’t take a server parameter (where get-eventloglevel does).

I’ve put a download copy of the script here the script itself looks like


[System.Reflection.Assembly]::LoadWithPartialName("System.Drawing")
[System.Reflection.Assembly]::LoadWithPartialName("System.windows.forms")
Function getdiagvalues{
$daTable.clear()
$dgDataGrid.DataSource = $daTable
get-eventloglevel -Server $snServerNameDrop.SelectedItem.ToString() | ForEach-Object{
$daTable.Rows.Add($_.Identity,$_.EventLevel)
}
$dgDataGrid.DataSource = $daTable
}

Function UpdateLogLevel{
if ($dgDataGrid.SelectedRows.Count -eq 0){
$idIdtoSet = $dgDataGrid.Rows[$dgDataGrid.CurrentCell.RowIndex].Cells[0].Value
set-eventloglevel -Identity $idIdtoSet -Level $llLoglevelDrop.Text
}
else{
$msgbox = new-object -comobject wscript.shell
$lcLoopCount = 0
while ($lcLoopCount -le ($dgDataGrid.SelectedRows.Count-1)) {
# [void]$msgbox.popup($dgDataGrid.SelectedRows[$lcLoopCount].Cells[0].Value,0,"Cant Do MX lookup",1)
set-eventloglevel -Identity $dgDataGrid.SelectedRows[$lcLoopCount].Cells[0].Value -Level $llLoglevelDrop.SelectedItem
$lcLoopCount += 1}
}
getdiagvalues
}

$form = new-object System.Windows.Forms.form

$llLableloc = 50
$VlLoc = 50

$Dataset = New-Object System.Data.DataSet
$daTable = New-Object System.Data.DataTable
$daTable.TableName = "Diag"
$daTable.Columns.Add("Identity")
$daTable.Columns.Add("Current Setting")


# Add Server DropLable
$snServerNamelableBox = new-object System.Windows.Forms.Label
$snServerNamelableBox.Location = new-object System.Drawing.Size(10,20)
$snServerNamelableBox.size = new-object System.Drawing.Size(100,20)
$snServerNamelableBox.Text = "ServerName"
$form.Controls.Add($snServerNamelableBox)

# Add Server Drop Down
$snServerNameDrop = new-object System.Windows.Forms.ComboBox
$snServerNameDrop.Location = new-object System.Drawing.Size(130,20)
$snServerNameDrop.Size = new-object System.Drawing.Size(130,30)
get-exchangeserver | ForEach-Object{$snServerNameDrop.Items.Add($_.Name)}
$snServerNameDrop.Add_SelectedValueChanged({getdiagvalues})
$form.Controls.Add($snServerNameDrop)

# Add New Log Level Drop Down
$llLoglevelDrop = new-object System.Windows.Forms.ComboBox
$llLoglevelDrop.Location = new-object System.Drawing.Size(350,20)
$llLoglevelDrop.Size = new-object System.Drawing.Size(70,30)
$llLoglevelDrop.Items.Add("Lowest")
$llLoglevelDrop.Items.Add("Low")
$llLoglevelDrop.Items.Add("Medium")
$llLoglevelDrop.Items.Add("High")
$llLoglevelDrop.Items.Add("Expert")
$form.Controls.Add($llLoglevelDrop)

# Add Apply Button

$exButton = new-object System.Windows.Forms.Button
$exButton.Location = new-object System.Drawing.Size(430,20)
$exButton.Size = new-object System.Drawing.Size(60,20)
$exButton.Text = "Apply"
$exButton.Add_Click({UpdateLogLevel})
$form.Controls.Add($exButton)

# New setting Group Box

$OfGbox = new-object System.Windows.Forms.GroupBox
$OfGbox.Location = new-object System.Drawing.Size(300,0)
$OfGbox.Size = new-object System.Drawing.Size(200,50)
$OfGbox.Text = "New Log Level Settings"
$form.Controls.Add($OfGbox)

# Add DataGrid View

$dgDataGrid = new-object System.windows.forms.DataGridView
$dgDataGrid.Location = new-object System.Drawing.Size(10,80)
$dgDataGrid.size = new-object System.Drawing.Size(500,500)
$dgDataGrid.AutoSizeColumnsMode = "AllCells"
$dgDataGrid.SelectionMode = "FullRowSelect"
$form.Controls.Add($dgDataGrid)


$form.Text = "Exchange 2007 Diagnostic Logging Form"
$form.size = new-object System.Drawing.Size(600,600)
$form.autoscroll = $true
$form.topmost = $true

$form.ShowDialog()