Skip to main content

Exporting a Mailbox larger then 2 GB and spanning it across multiple PST files with a script

*update* I suggest checking out Michael Smith's RDO port of this script which is a much more developed examples http://theessentialexchange.com/blogs/michael/archive/2009/10/16/exporting-mailboxes-larger-than-2-gb-on-an-exchange-server.aspx . *update*

I have a small customer with an old server that recently had a staff member leave who had a very large mailbox over 5 GB. Now these guys are pretty tight still using Exchange 2000 and Office 2000 and don’t want to upgrade software or hardware so getting rid of this mailbox out the Exchange store while still giving people occasionally access to this mailbox was desirable and also challenging. Because all the clients are using Outlook 2000 I’m stuck with the 2 GB PST non-Unicode file limit which Exmerge is similarly afflicted with. Using Exmerge with date filters was one possible solution but I decided I’d rather just do a liner export where mail was exported one item at a time and would just span to the next PST when the space was exhausted in one. This did prove a little changeling at first but I did manage to come up with a script that worked using a combination of CDO 1.2 and RDO (Redemption Data Objects) which is part of Dmitry Streblechenko excellent Outlook Redemption library. RDO provides the ability to create PST files on the fly and also provides the same functionality as CDO 1.2. I did have some problems copying different types of objects with both libraries so I found using a combination of both allowed me to work around the problems I did have and successfully copy most objects from a mailbox.

How does it work?

The script is broken up into multiple subs and functions that all perform different tasks the first task that needs to be done is to logon to the mailbox and create the first PST file. Once that PST file is created the delete-items folder is located and a mapping is created using a dictionary object that maps the entryID of the mailboxe’s deleted items to the EntryID of the deleted items in the PST file. The script then loops through every folder in the Mailbox and then recreates this hierarchy in the PST file. This will happen every time the createpst function is called and ensures that the same folder structure is in each of the spanned PST files. EntryID mappings are created in the dictionary object for each folder as is used later on in the script to process each the contents of each folder and map the items that are being copied in to the right folder in the PST file. The CreatePST function uses the enumfolder function to enumerate any subfolders and the ProcessFolderRoot and ProcessFolderSub to create the folders in the PST and update the dictionary object.

Once the PST is setup the script then performs another enumeration on the folders in the mailbox and processes the items in each of the folder collections in the processitems sub. This sub loops though each item in a folder one at a time and calculates what the size of the PST file will be after copying the current item into the file based on the size of the item and the current file size of the PST file returned form the File System Object. If the new file size will be over a configure threshold value a new PST file is created by calling the CreatePST function. The destination folder object is reset and the script then resumes using the CDO 1.2 message object’s copyto function. If the cdo copyto function fails the RDO copyto function is tried as a fallback (this is a work around for some issues I had with both libraries). If the processitems sub detects the current folder is a contacts folder RDO is used to do the copy to stop the script from hitting the CDO security prompt issues. To track the progress of the script debugging information is written to the command prompt and also to a file in the same directory as the PST file. After each folder is processed the number of items in both the source and destination are written to the log file so this can be checked for debugging purposes.

How to use the script

The script requires CDO 1.2 (from Exchange, Outlook or the Standalone version) and Redemption. The script has a number of parameters you need to configure first


tnThreshold = 1800

This is the threshold value for the size of the spanned PST’s I found using 1800 to be the most effective threshold and made sure that the pst files where always under 2 GB. If you want to make the PST smaller eg maybe you want to fit the spanned PST's onto multiple CD’s you could reduce this value it doesn’t work out to be an exact size so you need to make sure you give yourself a buffer.

servername = "servername"

mbMailbox = "mailbox"


These variables should be obvious

bfBaseFilename = "expMailbox"

pfFilePath = "c:\temp\"


The base file name is what the exported PST will be called there will also be have a number appended to the file name to specify the number in the span set.


Note: This script is defiantly not a replacement for Exmerge or any other backup or export methods you have. (eg if I had Outlook 2003 at this client I would have used this instead). I can’t verify the accuracy or consistency of this script when exporting a mailbox so it’s use at your own risk and do your own testing. (Any always make sure you have backed up what ever you’re exporting on a different media). For me it seemed to work fine exporting my 5 GB mailbox to 3 pst file which can be accessed from the old outlook 2000 clients okay when ever needed.

I’ve put a downloadable copy of this script here the script itself looks like

Set doDictionaryObject = CreateObject("Scripting.Dictionary")
Set fso = CreateObject("Scripting.FileSystemObject")
Set fso = CreateObject("Scripting.FileSystemObject")
set RDOSession = CreateObject("Redemption.RDOSession")
tsize = 10
tnThreshold = 1800
servername = "ServerName"
mbMailbox = "Mailbox"
bfBaseFilename = "expMailbox"
pfFilePath = "c:\temp\"
fnFileName = ""
PST = ""
pstroot = ""
IPMRoot = ""
pfPstFile = ""
fNumber = 0
set wfile = fso.opentextfile(pfFilePath & bfBaseFilename & ".txt",2,true)
RDOSession.LogonExchangeMailbox mbMailbox,servername
Set dfDeletedItemsFolder = RDOSession.GetDefaultFolder(3)
CreateNewPst()

wscript.echo fnFileName
wscript.echo "Enumerate Messages"
for miLoop = 1 to IPMRoot.Folders.count
ProcessItems(IPMRoot.Folders(miLoop))
if IPMRoot.Folders(miLoop).Folders.count <> 0 then
call Enumfolders(IPMRoot.Folders(miLoop),PstRootFolder,2)
end if
next


function Enumfolders(FLDS,RootFolder,ltype)
for fl = 1 to FLDS.Folders.count
if ltype = 1 then
call ProcessFolderSub(FLDS.folders(fl),RootFolder)
else
ProcessItems(FLDS.folders(fl))
end if
wscript.echo FLDS.folders(fl).Name
if FLDS.folders(fl).Folders.count <> 0 then
if ltype = 1 then
call Enumfolders(FLDS.folders(fl),FLDS.folders(fl).EntryID,1)
else
call Enumfolders(FLDS.folders(fl),FLDS.folders(fl).EntryID,2)
end if
end if
next
End function

Function CreateNewPst()

doDictionaryObject.RemoveAll
fNumber = fNumber + 1
fnFileName = pfFilePath & bfBaseFilename & "-" & fNumber & ".pst"
set PST = RDOSession.Stores.AddPSTStore(fnFileName, 1, "Exported MailBox-" & now())
set pstroot = RDOSession.GetFolderFromID(PST.IPMRootFolder.EntryID,PST.EntryID)
For Each pstfld In PstRoot.folders
If pstfld.Name = "Deleted Items" Then
If fNumber = 1 Then
doDictionaryObject.add dfDeletedItemsFolder.EntryID, pstfld.EntryID
wscript.echo "Added Deleted Items Folder"
End if
End if
next
set IPMRoot = RDOSession.Stores.DefaultStore.IPMRootFolder
for fiLoop = 1 to IPMRoot.Folders.count
if IPMRoot.Folders(fiLoop).Name <> "Deleted Items" then
PstRootFolder = ProcessFolderRoot(IPMRoot.Folders(fiLoop),PST.IPMRootFolder.EntryID)
if IPMRoot.Folders(fiLoop).Folders.count <> 0 then
call Enumfolders(IPMRoot.Folders(fiLoop),IPMRoot.Folders(fiLoop).EntryID,1)
end If
Else
if IPMRoot.Folders(fiLoop).Folders.count <> 0 then
call Enumfolders(IPMRoot.Folders(fiLoop),IPMRoot.Folders(fiLoop).EntryID,1)
end if
end if
next
Set pfPstFile = fso.GetFile(fnFileName)

end function

function ProcessFolderRoot(Fld,parentfld)

set CDOPstfld = RDOSession.GetFolderFromID(parentfld,PST.EntryID)
wscript.echo fld.Name
Set newFolder = CDOPstfld.Folders.ADD(Fld.Name)
ProcessFolder = newfolder.EntryID
newfolder.fields(&H3613001E) = Fld.fields(&H3613001E)

doDictionaryObject.add Fld.EntryID,newfolder.EntryID
end Function

function ProcessFolderSub(Fld,parentfld)

set CDOPstfld = RDOSession.GetFolderFromID(doDictionaryObject.item(parentfld),PST.EntryID)
wscript.echo fld.Name
Set newFolder = CDOPstfld.Folders.ADD(Fld.Name)
ProcessFolder = newfolder.EntryID
newfolder.fields(&H3613001E) = Fld.fields(&H3613001E)

doDictionaryObject.add Fld.EntryID,newfolder.EntryID
end function

Sub ProcessItems(Fld)

If Fld.fields(&H3613001E) = "IPF.Contact" Then
set dfDestinationFolder = RDOSession.GetFolderFromID(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
wscript.echo dfDestinationFolder.Name
wfile.writeLine("Processing Folder : ") & dfDestinationFolder.Name
for fiItemloop = 1 to Fld.items.count
on error resume next
pfPredictednewSize = formatnumber((pfPstFile.size + Fld.items(fiItemloop).size)/1048576,2,0,0,0)
if err.number <> 0 Then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo Fld.items(fiItemloop).EntryID
wscript.echo "Subect of Item:"
wscript.echo Fld.items(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(Fld.items(fiItemloop).EntryID )
Wfile.writeline("Subect of Item:")
Wfile.writeline(Fld.items(fiItemloop).Subject)
err.clear
end if
If Int(pfPredictednewSize) >= Int(tsize) Then
Wscript.echo "10 MB Exported"
tsize = tsize + 10
End if
If Int(pfPredictednewSize) >= Int(tnThreshold) Then
wfile.writeLine("New PST about to be created - Destination - Number of Items : " & dfDestinationFolder.messages.count)
CreateNewPst()
set dfDestinationFolder = RDOSession.GetFolderFromID(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
call Fld.items(fiItemloop).copyto(dfDestinationFolder)
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo Fld.items(fiItemloop).EntryID
wscript.echo "Subect of Item:"
wscript.echo Fld.items(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(Fld.items(fiItemloop).EntryID )
Wfile.writeline("Subect of Item:")
Wfile.writeline(Fld.items(fiItemloop).Subject)
err.clear
end if
else
call Fld.items(fiItemloop).copyto(dfDestinationFolder)
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo Fld.items(fiItemloop).EntryID
wscript.echo "Subect of Item:"
wscript.echo Fld.items(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(Fld.items(fiItemloop).EntryID )
Wfile.writeline("Subect of Item:")
Wfile.writeline(Fld.items(fiItemloop).Subject)
err.clear
end if
End if
on error goto 0
Next
wfile.writeLine("Source - Number of Items : " & Fld.fields(&h36020003) & " Destination - Number of Items : " & dfDestinationFolder.items.count)
else
set CDOSession = CreateObject("MAPI.Session")
CDOSession.MAPIOBJECT = RDOSession.MAPIOBJECT
Set objInfoStore = CDOSession.GetInfoStore(PST.EntryID)
set srcFld = CDOSession.GetFolder(Fld.EntryID)
wfile.writeLine("Processing Folder : ") & srcFld.Name
set dfDestinationFolder = CDOSession.GetFolder(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
wscript.echo dfDestinationFolder.Name
for fiItemloop = 1 to srcFld.messages.count
on error resume next
pfPredictednewSize = formatnumber((pfPstFile.size + srcFld.messages(fiItemloop).size)/1048576,2,0,0,0)
if err.number <> 0 Then
Wscript.echo "Error Processing Item in " & srcFld.messages(fiItemloop).Name
wscript.echo "EntryID of Item:"
wscript.echo srcFld.messages(fiItemloop).id
wscript.echo "Subect of Item:"
wscript.echo srcFld.messages(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & srcFld.messages(fiItemloop).Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).id)
Wfile.writeline("Subect of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).Subject)
err.clear
rem Try to Copy with RDO
Set rdosrc = RDOSession.GetMessageFromID(srcFld.messages(fiItemloop).Id)
rdosrc.copyto(dfDestinationFolder)
if err.number <> 0 Then
Wscript.echo "Also Failed RDO Copy"
wfile.writeline("Also Failed RDO Copy")
Else
Wscript.echo "Copied with RDO Okay"
wfile.writeline("Copied with RDO Okay")
End if
err.clear
end if
If Int(pfPredictednewSize) >= Int(tsize) Then
Wscript.echo "10 MB Exported"
tsize = tsize + 10
End if
If Int(pfPredictednewSize) >= Int(tnThreshold) Then
wfile.writeLine("New PST about to be created - Destination - Number of Items : " & dfDestinationFolder.messages.count)
CreateNewPst()
set CDOSession = CreateObject("MAPI.Session")
CDOSession.MAPIOBJECT = RDOSession.MAPIOBJECT
Set objInfoStore = CDOSession.GetInfoStore(PST.EntryID)
set dfDestinationFolder = CDOSession.GetFolder(doDictionaryObject.item(Fld.EntryID),PST.EntryID)
Set cpymsg = srcFld.messages(fiItemloop).copyto(dfDestinationFolder.ID)
cpymsg.update
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo srcFld.messages(fiItemloop).Id
wscript.echo "Subect of Item:"
wscript.echo srcFld.messages(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).id)
Wfile.writeline("Subect of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).Subject)
err.clear
rem Try to Copy with RDO
Set rdosrc = RDOSession.GetMessageFromID(srcFld.messages(fiItemloop).Id)
rdosrc.copyto(dfDestinationFolder)
if err.number <> 0 Then
Wscript.echo "Also Failed RDO Copy"
wfile.writeline("Also Failed RDO Copy")
Else
Wscript.echo "Copied with RDO Okay"
wfile.writeline("Copied with RDO Okay")
End if
err.clear
end if
Else
Set cpymsg = srcFld.messages(fiItemloop).copyto(dfDestinationFolder.ID)
cpymsg.update
if err.number <> 0 then
Wscript.echo "Error Processing Item in " & Fld.Name
wscript.echo "EntryID of Item:"
wscript.echo srcFld.messages(fiItemloop).id
wscript.echo "Subect of Item:"
wscript.echo srcFld.messages(fiItemloop).Subject
Wfile.writeline("Error Processing Item in " & Fld.Name)
Wfile.writeline("EntryID of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).id)
Wfile.writeline("Subect of Item:")
Wfile.writeline(srcFld.messages(fiItemloop).Subject)
err.clear
rem Try to Copy with RDO
Set rdosrc = RDOSession.GetMessageFromID(srcFld.messages(fiItemloop).Id)
rdosrc.copyto(dfDestinationFolder)
if err.number <> 0 Then
Wscript.echo "Also Failed RDO Copy"
wfile.writeline("Also Failed RDO Copy")
Else
Wscript.echo "Copied with RDO Okay"
wfile.writeline("Copied with RDO Okay")
End if
err.clear
end if
End if
on error goto 0
Next
wfile.writeLine("Source - Number of Items : " & srcFld.fields(&h36020003) & " Destination - Number of Items : " & dfDestinationFolder.messages.count)
End if
end sub
Labels:

Popular posts from this blog

The MailboxConcurrency limit and using Batching in the Microsoft Graph API

If your getting an error such as Application is over its MailboxConcurrency limit while using the Microsoft Graph API this post may help you understand why. Background   The Mailbox  concurrency limit when your using the Graph API is 4 as per https://docs.microsoft.com/en-us/graph/throttling#outlook-service-limits . This is evaluated for each app ID and mailbox combination so this means you can have different apps running under the same credentials and the poor behavior of one won't cause the other to be throttled. If you compared that to EWS you could have up to 27 concurrent connections but they are shared across all apps on a first come first served basis. Batching Batching in the Graph API is a way of combining multiple requests into a single HTTP request. Batching in the Exchange Mail API's EWS and MAPI has been around for a long time and its common, for email Apps to process large numbers of smaller items for a variety of reasons.  Batching in the Graph is limited to a m
Read more

Sending a Message in Exchange Online via REST from an Arduino MKR1000

This is part 2 of my MKR1000 article, in this previous post  I looked at sending a Message via EWS using Basic Authentication.  In this Post I'll look at using the new Outlook REST API  which requires using OAuth authentication to get an Access Token. The prerequisites for this sketch are the same as in the other post with the addition of the ArduinoJson library  https://github.com/bblanchon/ArduinoJson  which is used to parse the Authentication Results to extract the Access Token. Also the SSL certificates for the login.windows.net  and outlook.office365.com need to be uploaded to the devices using the wifi101 Firmware updater. To use Token Authentication you need to register an Application in Azure https://msdn.microsoft.com/en-us/office/office365/howto/add-common-consent-manually  with the Mail.Send permission. The application should be a Native Client app that use the Out of Band Callback urn:ietf:wg:oauth:2.0:oob. You need to authorize it in you tenant (eg build a small ap
Read more

How to test SMTP using Opportunistic TLS with Powershell and grab the public certificate a SMTP server is using

Most email services these day employ Opportunistic TLS when trying to send Messages which means that wherever possible the Messages will be encrypted rather then the plain text legacy of SMTP.  This method was defined in RFC 3207 "SMTP Service Extension for Secure SMTP over Transport Layer Security" and  there's a quite a good explanation of Opportunistic TLS on Wikipedia  https://en.wikipedia.org/wiki/Opportunistic_TLS .  This is used for both Server to Server (eg MTA to MTA) and Client to server (Eg a Message client like Outlook which acts as a MSA) the later being generally Authenticated. Basically it allows you to have a normal plain text SMTP conversation that is then upgraded to TLS using the STARTTLS verb. Not all servers will support this verb so if its not supported then a message is just sent as Plain text. TLS relies on PKI certificates and the administrative issue s that come around certificate management like expired certificates which is why I wrote th
Read more
All sample scripts and source code is provided by for illustrative purposes only. All examples are untested in different environments and therefore, I cannot guarantee or imply reliability, serviceability, or function of these programs.

All code contained herein is provided to you "AS IS" without any warranties of any kind. The implied warranties of non-infringement, merchantability and fitness for a particular purpose are expressly disclaimed.