Skip to main content

Exporting a Mailbox larger then 2 GB and spanning it across multiple PST files with a script

*update* I suggest checking out Michael Smith's RDO port of this script which is a much more developed examples http://theessentialexchange.com/blogs/michael/archive/2009/10/16/exporting-mailboxes-larger-than-2-gb-on-an-exchange-server.aspx . *update*

I have a small customer with an old server that recently had a staff member leave who had a very large mailbox over 5 GB. Now these guys are pretty tight still using Exchange 2000 and Office 2000 and don’t want to upgrade software or hardware so getting rid of this mailbox out the Exchange store while still giving people occasionally access to this mailbox was desirable and also challenging. Because all the clients are using Outlook 2000 I’m stuck with the 2 GB PST non-Unicode file limit which Exmerge is similarly afflicted with. Using Exmerge with date filters was one possible solution but I decided I’d rather just do a liner export where mail was exported one item at a time and would just span to the next PST when the space was exhausted in one. This did prove a little changeling at first but I did manage to come up with a script that worked using a combination of CDO 1.2 and RDO (Redemption Data Objects) which is part of Dmitry Streblechenko excellent Outlook Redemption library. RDO provides the ability to create PST files on the fly and also provides the same functionality as CDO 1.2. I did have some problems copying different types of objects with both libraries so I found using a combination of both allowed me to work around the problems I did have and successfully copy most objects from a mailbox.

How does it work?

The script is broken up into multiple subs and functions that all perform different tasks the first task that needs to be done is to logon to the mailbox and create the first PST file. Once that PST file is created the delete-items folder is located and a mapping is created using a dictionary object that maps the entryID of the mailboxe’s deleted items to the EntryID of the deleted items in the PST file. The script then loops through every folder in the Mailbox and then recreates this hierarchy in the PST file. This will happen every time the createpst function is called and ensures that the same folder structure is in each of the spanned PST files. EntryID mappings are created in the dictionary object for each folder as is used later on in the script to process each the contents of each folder and map the items that are being copied in to the right folder in the PST file. The CreatePST function uses the enumfolder function to enumerate any subfolders and the ProcessFolderRoot and ProcessFolderSub to create the folders in the PST and update the dictionary object.

Once the PST is setup the script then performs another enumeration on the folders in the mailbox and processes the items in each of the folder collections in the processitems sub. This sub loops though each item in a folder one at a time and calculates what the size of the PST file will be after copying the current item into the file based on the size of the item and the current file size of the PST file returned form the File System Object. If the new file size will be over a configure threshold value a new PST file is created by calling the CreatePST function. The destination folder object is reset and the script then resumes using the CDO 1.2 message object’s copyto function. If the cdo copyto function fails the RDO copyto function is tried as a fallback (this is a work around for some issues I had with both libraries). If the processitems sub detects the current folder is a contacts folder RDO is used to do the copy to stop the script from hitting the CDO security prompt issues. To track the progress of the script debugging information is written to the command prompt and also to a file in the same directory as the PST file. After each folder is processed the number of items in both the source and destination are written to the log file so this can be checked for debugging purposes.

How to use the script

The script requires CDO 1.2 (from Exchange, Outlook or the Standalone version) and Redemption. The script has a number of parameters you need to configure first


tnThreshold = 1800

This is the threshold value for the size of the spanned PST’s I found using 1800 to be the most effective threshold and made sure that the pst files where always under 2 GB. If you want to make the PST smaller eg maybe you want to fit the spanned PST's onto multiple CD’s you could reduce this value it doesn’t work out to be an exact size so you need to make sure you give yourself a buffer.

servername = "servername"

mbMailbox = "mailbox"


These variables should be obvious

bfBaseFilename = "expMailbox"

pfFilePath = "c:\temp\"


The base file name is what the exported PST will be called there will also be have a number appended to the file name to specify the number in the span set.


Note: This script is defiantly not a replacement for Exmerge or any other backup or export methods you have. (eg if I had Outlook 2003 at this client I would have used this instead). I can’t verify the accuracy or consistency of this script when exporting a mailbox so it’s use at your own risk and do your own testing. (Any always make sure you have backed up what ever you’re exporting on a different media). For me it seemed to work fine exporting my 5 GB mailbox to 3 pst file which can be accessed from the old outlook 2000 clients okay when ever needed.

I’ve put a downloadable copy of this script here the script itself looks like

Set doDictionaryObject = CreateObject("Scripting.Dictionary")
Set fso = CreateObject("Scripting.FileSystemObject")
Set fso = CreateObject("Scripting.FileSystemObject")
set RDOSession = CreateObject("Redemption.RDOSession")
tsize = 10
tnThreshold = 1800
servername = "ServerName"
mbMailbox = "Mailbox"
bfBaseFilename = "expMailbox"
pfFilePath = "c:\temp\"
fnFileName = ""
PST = ""
pstroot = ""
IPMRoot = ""
pfPstFile = ""
fNumber = 0
set wfile = fso.opentextfile(pfFilePath & bfBaseFilename & ".txt",2,true)
RDOSession.LogonExchangeMailbox mbMailbox,servername
Set dfDeletedItemsFolder = RDOSession.GetDefaultFolder(3)
CreateNewPst()

wscript.echo fnFileName
wscript.echo "Enumerate Messages"
for miLoop = 1 to IPMRoot.Folders.count
ProcessItems(IPMRoot.Folders(miLoop))
if IPMRoot.Folders(miLoop).Folders.count <> 0 then
call Enumfolders(IPMRoot.Folders(miLoop),PstRootFolder,2)
end if
next


function Enumfolders(FLDS,RootFolder,ltype)
for fl = 1 to FLDS.Folders.count
if ltype = 1 then
call ProcessFolderSub(FLDS.folders(fl),RootFolder)
else
ProcessItems(FLDS.folders(fl))
end if
wscript.echo FLDS.folders(fl).Name
if FLDS.folders(fl).Folders.count <> 0 then
if ltype = 1 then
call Enumfolders(FLDS.folders(fl),FLDS.folders(fl).EntryID,1)
else
call Enumfolders(FLDS.folders(fl),FLDS.folders(fl).EntryID,2)
end if
end if
next
End function

Function CreateNewPst()

doDictionaryObject.RemoveAll
fNumber = fNumber + 1
fnFileName = pfFilePath & bfBaseFilename & "-" & fNumber & ".pst"
set PST = RDOSession.Stores.AddPSTStore(fnFileName, 1, "Exported MailBox-" & now())
set pstroot = RDOSession.GetFolderFromID(PST.IPMRootFolder.EntryID,PST.EntryID)
For Each pstfld In PstRoot.folders
If pstfld.Name = "Deleted Items" Then
If fNumber = 1 Then
doDictionaryObject.add dfDeletedItemsFolder.EntryID, pstfld.EntryID
wscript.echo "Added Deleted Items Folder"
End if
End if
next
set IPMRoot = RDOSession.Stores.DefaultStore.IPMRootFolder
for fiLoop = 1 to IPMRoot.Folders.count
if IPMRoot.Folders(fiLoop).Name <> "Deleted Items" then
PstRootFolder = ProcessFolderRoot(IPMRoot.Folders(fiLoop),PST.IPMRootFolder.EntryID)
if IPMRoot.Folders(fiLoop).Folders.count <> 0 then
call Enumfolders(IPMRoot.Folders(fiLoop),IPMRoot.Folders(fiLoop).EntryID,1)
end If
Else
if IPMRoot.Folders(fiLoop).Folders.count <> 0 then
call Enumfolders(IPMRoot.Folders(fiLoop),IPMRoot.Folders(fiLoop).EntryID,1)
end if
end if
next
Set pfPstFile = fso.GetFile(fnFileName)

end function

function ProcessFolderRoot(Fld,parentfld)

set CDOPstfld = RDOSession.GetFolderFromID(parentfld,PST.EntryID)
wscript.echo fld.Name
Set newFolder = CDOPstfld.Folders.ADD(Fld.Name)
ProcessFolder = newfolder.EntryID
newfolder.fields(&H3613001E) = Fld.fields(&H3613001E)

doDictionaryObject.add Fld.EntryID,newfolder.EntryID
end Function

function ProcessFolderSub(Fld,parentfld)

set CDOPstfld = RDOSession.GetFolderFromID(doDictionaryObject.item(parentfld),PST.EntryID)
wscript.echo fld.Name
Set newFolder = CDOPstfld.Folders.ADD(Fld.Name)
ProcessFolder = newfolder.EntryID
newfolder.fields(&H3613001E) = Fld.fields(&H3613001E)

doDictionaryObject.add Fld.EntryID,newfolder.EntryID
end function

Sub ProcessItems(Fld)

If Fld.fields(&H3613001E) = "IPF.Contact" Then
set dfDestinationFolder = RDOSession.GetFolderFromID(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
wscript.echo dfDestinationFolder.Name
wfile.writeLine("Processing Folder : ") & dfDestinationFolder.Name
for fiItemloop = 1 to Fld.items.count
on error resume next
pfPredictednewSize = formatnumber((pfPstFile.size + Fld.items(fiItemloop).size)/1048576,2,0,0,0)
if err.number <> 0 Then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo Fld.items(fiItemloop).EntryID
wscript.echo "Subect of Item:"
wscript.echo Fld.items(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(Fld.items(fiItemloop).EntryID )
Wfile.writeline("Subect of Item:")
Wfile.writeline(Fld.items(fiItemloop).Subject)
err.clear
end if
If Int(pfPredictednewSize) >= Int(tsize) Then
Wscript.echo "10 MB Exported"
tsize = tsize + 10
End if
If Int(pfPredictednewSize) >= Int(tnThreshold) Then
wfile.writeLine("New PST about to be created - Destination - Number of Items : " & dfDestinationFolder.messages.count)
CreateNewPst()
set dfDestinationFolder = RDOSession.GetFolderFromID(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
call Fld.items(fiItemloop).copyto(dfDestinationFolder)
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo Fld.items(fiItemloop).EntryID
wscript.echo "Subect of Item:"
wscript.echo Fld.items(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(Fld.items(fiItemloop).EntryID )
Wfile.writeline("Subect of Item:")
Wfile.writeline(Fld.items(fiItemloop).Subject)
err.clear
end if
else
call Fld.items(fiItemloop).copyto(dfDestinationFolder)
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo Fld.items(fiItemloop).EntryID
wscript.echo "Subect of Item:"
wscript.echo Fld.items(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(Fld.items(fiItemloop).EntryID )
Wfile.writeline("Subect of Item:")
Wfile.writeline(Fld.items(fiItemloop).Subject)
err.clear
end if
End if
on error goto 0
Next
wfile.writeLine("Source - Number of Items : " & Fld.fields(&h36020003) & " Destination - Number of Items : " & dfDestinationFolder.items.count)
else
set CDOSession = CreateObject("MAPI.Session")
CDOSession.MAPIOBJECT = RDOSession.MAPIOBJECT
Set objInfoStore = CDOSession.GetInfoStore(PST.EntryID)
set srcFld = CDOSession.GetFolder(Fld.EntryID)
wfile.writeLine("Processing Folder : ") & srcFld.Name
set dfDestinationFolder = CDOSession.GetFolder(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
wscript.echo dfDestinationFolder.Name
for fiItemloop = 1 to srcFld.messages.count
on error resume next
pfPredictednewSize = formatnumber((pfPstFile.size + srcFld.messages(fiItemloop).size)/1048576,2,0,0,0)
if err.number <> 0 Then
Wscript.echo "Error Processing Item in " & srcFld.messages(fiItemloop).Name
wscript.echo "EntryID of Item:"
wscript.echo srcFld.messages(fiItemloop).id
wscript.echo "Subect of Item:"
wscript.echo srcFld.messages(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & srcFld.messages(fiItemloop).Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).id)
Wfile.writeline("Subect of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).Subject)
err.clear
rem Try to Copy with RDO
Set rdosrc = RDOSession.GetMessageFromID(srcFld.messages(fiItemloop).Id)
rdosrc.copyto(dfDestinationFolder)
if err.number <> 0 Then
Wscript.echo "Also Failed RDO Copy"
wfile.writeline("Also Failed RDO Copy")
Else
Wscript.echo "Copied with RDO Okay"
wfile.writeline("Copied with RDO Okay")
End if
err.clear
end if
If Int(pfPredictednewSize) >= Int(tsize) Then
Wscript.echo "10 MB Exported"
tsize = tsize + 10
End if
If Int(pfPredictednewSize) >= Int(tnThreshold) Then
wfile.writeLine("New PST about to be created - Destination - Number of Items : " & dfDestinationFolder.messages.count)
CreateNewPst()
set CDOSession = CreateObject("MAPI.Session")
CDOSession.MAPIOBJECT = RDOSession.MAPIOBJECT
Set objInfoStore = CDOSession.GetInfoStore(PST.EntryID)
set dfDestinationFolder = CDOSession.GetFolder(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
Set cpymsg = srcFld.messages(fiItemloop).copyto(dfDestinationFolder.ID)
cpymsg.update
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo srcFld.messages(fiItemloop).Id
wscript.echo "Subect of Item:"
wscript.echo srcFld.messages(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).id)
Wfile.writeline("Subect of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).Subject)
err.clear
rem Try to Copy with RDO
Set rdosrc = RDOSession.GetMessageFromID(srcFld.messages(fiItemloop).Id)
rdosrc.copyto(dfDestinationFolder)
if err.number <> 0 Then
Wscript.echo "Also Failed RDO Copy"
wfile.writeline("Also Failed RDO Copy")
Else
Wscript.echo "Copied with RDO Okay"
wfile.writeline("Copied with RDO Okay")
End if
err.clear
end if
Else
Set cpymsg = srcFld.messages(fiItemloop).copyto(dfDestinationFolder.ID)
cpymsg.update
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo srcFld.messages(fiItemloop).id
wscript.echo "Subect of Item:"
wscript.echo srcFld.messages(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).id)
Wfile.writeline("Subect of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).Subject)
err.clear
rem Try to Copy with RDO
Set rdosrc = RDOSession.GetMessageFromID(srcFld.messages(fiItemloop).Id)
rdosrc.copyto(dfDestinationFolder)
if err.number <> 0 Then
Wscript.echo "Also Failed RDO Copy"
wfile.writeline("Also Failed RDO Copy")
Else
Wscript.echo "Copied with RDO Okay"
wfile.writeline("Copied with RDO Okay")
End if
err.clear
end if
End if
on error goto 0
Next
wfile.writeLine("Source - Number of Items : " & srcFld.fields(&h36020003) & " Destination - Number of Items : " & dfDestinationFolder.messages.count)
End if
end sub

Popular posts from this blog

Export calendar Items to a CSV file using EWS and Powershell

Somebody asked about this last week and while I have a lot of EWS scripts that do access the Calendar I didn't have a simple example that just exported a list of the Calendar events with relevant information to a CSV file so here it is.

I've talked on this one before in this howto  but when you query the calendar folder using EWS you need to use a CalendarView which will expand any recurring appointments in a calendar. There are some limits when you use a calendarview in that you can only return a maximum of 2 years of appointments at a time and paging will limit the max number of items to 1000 per call. So if you have a calendar with a very large number of appointments you need to break your query into small date time blocks. In this example script I'm just grabbing the next 7 days of appointments if you want to query a longer period you need to adjust the following lines (keeping in mind what I just mentioned)

#Define Date to Query
$StartDate = (Get-Date)
$EndDate = (Ge…

EWS Managed API and Powershell How-To series Part 1

I thought I'd start the year with a series of posts that goes back over the basics of using the EWS Managed API from Powershell and provides a modular remarked example that you can easily cut and paste to build your own scripts. Along the way in this series I'll show a whole bunch of examples around specific things.

As a starting point for versions this will be Powershell Version 2.0  and the EWS Managed API 1.1 (which will soon change to 1.2 once released) http://www.microsoft.com/download/en/details.aspx?id=13480.

The starting point for any EWS script your going to write is connecting to Exchange for which there are three important pieces of information you will need. Firstly you need to know the version of Exchange your running in this script its going to be held in the following variable

$ExchangeVersion = [Microsoft.Exchange.WebServices.Data.ExchangeVersion]::Exchange2010_SP1

Other valid values for Exchange 2007 would be

$ExchangeVersion = [Microsoft.Exchange.WebServices.…

Writing a simple scripted process to download attachmentts in Exchange 2007/ 2010 using the EWS Managed API

Every complicated thing in life is made up of smaller simpler building blocks, when it comes to writing a script (or any code really) the more of these little building blocks you have to figure out the more the process of solving a problem can become bewildering. The Internet generally provides you with lots of half eaten sandwiches of information something someone else has taken a bite out but a lot of the time half done, and as with any code its usefulness declines over time as new and better API's and methods are derived. In this post I'm going to go through a simple scripted process that hopefully covers a few more of these smaller building blocks that you might face when asked to come up with a simple costless solution to perform an automated business function with a script.

So the process im going to look at is one that comes up a lot and that is you have an Email that comes into to certain mailbox every day with a certain subject in my case "Daily Export" this …
All sample scripts and source code is provided by for illustrative purposes only. All examples are untested in different environments and therefore, I cannot guarantee or imply reliability, serviceability, or function of these programs.

All code contained herein is provided to you "AS IS" without any warranties of any kind. The implied warranties of non-infringement, merchantability and fitness for a particular purpose are expressly disclaimed.