Tuesday, March 21, 2006

Replicating an Exchange Folder using WebDAV Replication and CDO 1.2

One of the lesser know and cooler (well I think) features of WebDAV with Exchange is the ability to perform replication of a collection (or a folder). The Exchange SDK has some documentation on how this works and the general plumbing of how you structure requests and responses. Previously I had a script that would copy contacts between a public folder and mailbox this was a pretty simple script with no intelligence so it would produce duplicates if it was run twice etc. I wanted something that would go a little further then this script and allow full one way sync of a private contacts folder with a public contacts folder so basically a script that could be run at intervals and any updates to the private contacts folder such as changes of email addresses, phone number, addition of contacts or deletion of contacts would be reflected in the public contacts folder. For this combining the original CDO 1.2 script to copy contacts (the reason that CDO is used to do the copy instead of WebDAV is the ability to retain the MAPI properties on the item being copied) with WebDAV replication gave the desired functionality.

How it works.

This script is split into many functions and subroutines all which perform different functions here’s a basic run though of how the script works.

The front end of the script sets up variables for the folders that you wish to copy I used this script to copy contacts but it will work fine to replicate any other folder (I haven’t tested calendars). I’ve created two copies of the script the first is a non FBA version and the second is a FBA version if your running forms based authentication. In the FBA version some code is included to perform a synthetic form logon using the username and password and then retrieve the cookies for further use in the script for authentication.

The next part of the script performs a Mapi logon to the source mailbox the session is require to perform the copying of the contacts.

The getpfid function returns the MAPI EntryID for the destination folder by using WebDAV to return the entryID from the destination folder URL. This EntryID is required to perform the copy via CDO 1.2.

To use WebDAV replication you need to use a collblob which “is the opaque binary stream generated by the server that represents the state of the contents of a collection. The collblob contains information about changes in the contents of the collection on the server and the query specified in the request for the Manifest of a Collection. The collblob tracks only resources that match the search criteria specified in the SEARCH Method query in the manifest request. “. ref In WebDAV this collblob property itself is returned as a BASE 64 encoded string. You need to store this property and include it in any queries that you make regarding the folder you are replicating. With this script what I’ve done is when a query is made to ascertain the replication status of the folder the result for the collblob is stored in a custom property on the destination folder. So on the destination folder there is a property that is based on the name of the source folder that holds the collblob from the replication query of the source folder. This is what the next function does Collabblobget retrieves the previous collblob from the destination folder to be used in the query of the source folder.

The QueryMailbox sub executes the WebDAV query on the source folder using the previous collblob if one exists. The result of this query are then parsed the collblob that is returned is first stored in a custom property on the source folder as previously explained. The result of this query will contain all the resources within the collection that have been added, changed or deleted since the last query with that collblob. A case statement is used to create a logic tree on the changetype property depending on what type of change it is different functions and sub’s are called. If this is a new resource then the CopyContact sub is called and the EntryID and repl-UID is passed in. The CopyContact sub uses CDO 1.2 to copy the item between the mailbox and public folder and also it creates a custom property on the item to store the repl-UID of the original items this is important when it comes to making changes or deletion in the future. If the changetype is modify then instead of finding out what property has been changed I decided to simplify things by just deleting the old item and then re-copying the source item. If the changetype is delete then the DeleteContact sub is called which finds the contact in the public folder by searching on the Repl-UID stored in the custom property in the destination folder and then do a WebDAV delete on the resource.

Running the script

Before you run the script you need to customise different variables with the script for the
snServername = "servername"
mnMailboxname = "mailbox"
SourceURL = "http://" & snServername & "/exchange/" & mnMailboxname & "/contacts/"
DestinURL = "http://" & snServername & "/public/foldername/"

For the FBA script you also need the password and domain of the account you are going to use in the synthetic logon.

To run the script itself just use cscript replfld.vbs

This is script is not really designed to sync multiple folders at a time eg if you wanted to sync two users contacts folder with on public folder this type of script will still produce duplicates at the destination folder. I’m working on another script to do this.

I’ve put a downloadable copy of this script here the script itself looks like.

snServername = "servername"
mnMailboxname = "mailbox"
domain = "domain"
strpassword = "password"
strusername = domain & "\" & mnMailboxname
SourceURL = "https://" & snServername & "/exchange/" & mnMailboxname &
"/contacts/"
DestinURL = "https://" & snServername & "/public/foldername/"

szXml = "destination=https://" & snServername & "/exchange/&flags=0&username=" &
strusername
szXml = szXml & "&password=" & strpassword & "&SubmitCreds=Log On&forcedownlevel=0&trusted=0"
set req = createobject("microsoft.xmlhttp")
req.Open "post", "https://" & snServername & "/exchweb/bin/auth/owaauth.dll",
False
req.send szXml
reqhedrarry = split(req.GetAllResponseHeaders(), vbCrLf,-1,1)
for c = lbound(reqhedrarry) to ubound(reqhedrarry)
if instr(lcase(reqhedrarry(c)),"set-cookie: sessionid=") then reqsessionID =
right(reqhedrarry(c),len(reqhedrarry(c))-12)
if instr(lcase(reqhedrarry(c)),"set-cookie: cadata=") then reqcadata=
right(reqhedrarry(c),len(reqhedrarry(c))-12)
next
set objSession = CreateObject("MAPI.Session")
strProfile = snServername & vbLf & mnMailboxname
objSession.Logon "",,, False,, True, strProfile
Set objInfoStore = objSession.GetInfoStore(objSession.Inbox.StoreID)
Set objpubstore = objSession.InfoStores("Public Folders")
pfPublicFolderID = getpfid()
colbblob = Collabblobget()
QueryMailbox(colbblob)

wscript.echo "Done"

sub QueryMailbox(colbblob)

strQuery = "<?xml version=""1.0""?><D:searchrequest xmlns:D = ""DAV:"" xmlns:R=""http://schemas.microsoft.com/repl/""><R:repl><R:collblob>"
& colbblob & "</R:collblob></R:repl>"
strQuery = strQuery & "<D:sql>SELECT ""DAV:href"", ""urn:schemas:httpmail:subject"",
""http://schemas.microsoft.com/mapi/proptag/x0fff0102"",""
http://schemas.microsoft.com/repl/repl-uid""
"
strQuery = strQuery & " FROM scope('shallow traversal of """
strQuery = strQuery & SourceURL & """') Where ""DAV:ishidden"" = False AND ""DAV:isfolder""
= False "
strQuery = strQuery & "</D:sql></D:searchrequest>"
req.open "SEARCH", SourceURL, false
req.setrequestheader "Content-Type", "text/xml"
req.SetRequestHeader "cookie", reqsessionID
req.SetRequestHeader "cookie", reqCadata
req.setRequestHeader "Translate","f"
req.send strQuery
If req.status >= 500 Then
wscript.echo "Status: " & req.status
wscript.echo "Status text: An error occurred on the server."
ElseIf req.status = 207 Then
wscript.echo "Status: " & req.status
wscript.echo "Status text: " & req.statustext
set oResponseDoc = req.responseXML
set oNodeList = oResponseDoc.getElementsByTagName("d:collblob")
For i = 0 To (oNodeList.length -1)
set oNode = oNodeList.nextNode
colblob = oNode.Text
Collabblobset(colblob)
Next
set idNodeList = oResponseDoc.getElementsByTagName("f:x0fff0102")
set replidNodeList = oResponseDoc.getElementsByTagName("d:repl-uid")
set replchangeType = oResponseDoc.getElementsByTagName("d:changetype")
for id = 0 To (idNodeList.length -1)
set oNode1 = idNodeList.nextNode
set oNode2 = replidNodeList.nextNode
set oNode3 = replchangeType.nextNode
select case oNode3.text
case "new" call CopyContact(Octenttohex(oNode1.nodeTypedValue),oNode2.text)
case "delete" wscript.echo oNode3.text
wscript.echo oNode2.text
DeleteContact(oNode2.text)
case "change" Wscript.echo "Change"
call DeleteContact(oNode2.text)
call CopyContact(Octenttohex(oNode1.nodeTypedValue),oNode2.text)
end select
next
Else
wscript.echo "Status: " & req.status
wscript.echo "Status text: " & req.statustext
wscript.echo "Response text: " & req.responsetext
End If

End Sub

function Collabblobget()

xmlreqtxt = "<?xml version='1.0'?><a:propfind xmlns:a='DAV:' xmlns:cp='" &
SourceURL & "'><a:prop><cp:collblob/></a:prop></a:propfind>"
req.open "PROPFIND", DestinURL, false, "", ""
req.setRequestHeader "Content-Type", "text/xml; charset=""UTF-8"""
req.setRequestHeader "Depth", "0"
req.SetRequestHeader "cookie", reqsessionID
req.SetRequestHeader "cookie", reqCadata
req.setRequestHeader "Translate", "f"
req.send xmlreqtxt
set oResponseDoc = req.responseXML
set oCobNode = oResponseDoc.getElementsByTagName("d:collblob")
For i1 = 0 To (oCobNode.length -1)
set oNode = oCobNode.nextNode
Collabblobget = oNode.Text
Next

End function

Sub Collabblobset(colblob)
xmlstr = "<?xml version=""1.0""?>" _
& "<g:propertyupdate " _
& " xmlns:g=""DAV:"" xmlns:e=""http://schemas.microsoft.com/exchange/""" _
& " xmlns:dt=""urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/"" " _
& " xmlns:cp=""" & SourceURL & """ " _
& " xmlns:header=""urn:schemas:mailheader:"" " _
& " xmlns:mail=""urn:schemas:httpmail:""> " _
& " <g:set> " _
& " <g:prop> " _
& " <cp:collblob>" & colblob & "</cp:collblob> " _
& " </g:prop> " _
& " </g:set> " _
& "</g:propertyupdate>"

req.open "PROPPATCH", DestinURL, False
req.setRequestHeader "Content-Type", "text/xml;"
req.SetRequestHeader "cookie", reqsessionID
req.SetRequestHeader "cookie", reqCadata
req.setRequestHeader "Translate", "f"
req.setRequestHeader "Content-Length:", Len(xmlstr)
req.send(xmlstr)
wscript.echo req.responsetext

end sub

Sub CopyContact(messageEntryID,ReplID)
set objcontact = objSession.getmessage(messageEntryID)
set objCopyContact = objcontact.copyto(pfPublicFolderID,objpubstore.ID)
objCopyContact.Unread = false
objCopyContact.Fields.Add "0x8542", vbString, ReplID,"0820060000000000C000000000000046"
objCopyContact.Update
Set objCopyContact = Nothing
wscript.echo objcontact.subject

end Sub

Sub DeleteContact(replUID)

strQuery = "<?xml version=""1.0""?><D:searchrequest xmlns:D = ""DAV:"">"
strQuery = strQuery & "<D:sql>SELECT ""DAV:Displayname"""
strQuery = strQuery & " FROM scope('shallow traversal of """
strQuery = strQuery & DestinURL & """') Where ""http://schemas.microsoft.com/mapi/id/{00062008-0000-0000-C000-000000000046}/0x8542""
= '" & replUID & "' AND ""DAV:isfolder"" = False "
strQuery = strQuery & "</D:sql></D:searchrequest>"
req.open "SEARCH", DestinURL, false
req.setrequestheader "Content-Type", "text/xml"
req.SetRequestHeader "cookie", reqsessionID
req.SetRequestHeader "cookie", reqCadata
req.setRequestHeader "Translate","f"
req.send strQuery
wscript.echo req.responsetext
If req.status >= 500 Then
wscript.echo "Status: " & req.status
wscript.echo "Status text: An error occurred on the server."
ElseIf req.status = 207 Then
wscript.echo "Status: " & req.status
wscript.echo "Status text: " & req.statustext
set oResponseDoc = req.responseXML
set oNodeList = oResponseDoc.getElementsByTagName("a:href")
For i = 0 To (oNodeList.length -1)
set oNode = oNodeList.nextNode
wscript.echo oNode.text
req.open "DELETE", oNode.text, false
req.SetRequestHeader "cookie", reqsessionID
req.SetRequestHeader "cookie", reqCadata
req.send
wscript.echo "Status: " & req.status
Next
Else
wscript.echo "Status: " & req.status
wscript.echo "Status text: " & req.statustext
wscript.echo "Response text: " & req.responsetext
End If

end Sub

function getpfid()

xmlreqtxt = "<?xml version='1.0'?><a:propfind xmlns:a='DAV:' xmlns:e='http://schemas.microsoft.com/mapi/proptag/'><a:prop><e:x0FFF0102

/></a:prop></a:propfind>"
req.open "PROPFIND", DestinURL, false, "", ""
req.setRequestHeader "Content-Type", "text/xml; charset=""UTF-8"""
req.setRequestHeader "Depth", "0"
req.SetRequestHeader "cookie", reqsessionID
req.SetRequestHeader "cookie", reqCadata
req.setRequestHeader "Translate", "f"
req.send xmlreqtxt
set oResponseDoc = req.responseXML
set oNodeList = oResponseDoc.getElementsByTagName("d:x0FFF0102")
For i = 0 To (oNodeList.length -1)
set oNode = oNodeList.nextNode
getpfid = Octenttohex(oNode.nodeTypedValue)
Next

end function

Function Octenttohex(OctenArry)
ReDim aOut(UBound(OctenArry))
For i = 1 to UBound(OctenArry) + 1
if len(hex(ascb(midb(OctenArry,i,1)))) = 1 then
aOut(i-1) = "0" & hex(ascb(midb(OctenArry,i,1)))
else
aOut(i-1) = hex(ascb(midb(OctenArry,i,1)))
end if
Next
Octenttohex = join(aOUt,"")
End Function

16 comments:

Sudarshan Gaikaiwari said...

Is there a way this sample can be modified to copy the item to a mailbox on another server. Let us assume that the principal perfroming this operation has read and write access on all mailboxes on both the servers.

Glen said...

You cant do a direct copy from one server to other because there is no provider Mapi or WebDAV that allows you to have a simultaneous connection between two Exchange servers. What you need to use is a Intermediary such as a PST file or even export to disk then reimport to the other server (You need to look at RDO which has this function). However I would suggest you look at Exmerge which is already made for this job of copying mailbox;s and mail between two different servers.

Daniel said...

Hi Glen (wonderful blog, I keep reminding myself to visit it).

Say, could I use this script to replicate the contents of X resource calendars to different mailboxes? I need to be able to, for example, take 20 resource mailboxes, get their calendars, and replicate them to 20 different test mailboxes.

Could this be done? Can you point me to the right direction?

Thanks again, Daniel

Glen said...

Hi Daniel,

Copying between mailboxes is a little tricker your better of using RDO with CDO to get the ability to access another mailbox within the profile. http://www.dimastr.com/redemption/rdo/. I gave this script a few tweaks and added some RDO and i was able to copy and replicate a mailbox calendar okay haven't really done a lot of testing. I've post this version up here if you want to look http://msgdev.mvps.org/exdevblog/replfldfbardov1.zip

Jeff Popio said...

What would change in your replfldfbardov script to replicate the contacts on one mailbox to another? I need to synchronize two users contacts so that they replicate automatically. I'm thinking of using your script and and a scheduled task to run every 10 minutes.

Glen said...

Hi Jeff

If you are taking about the script i linked in the comment above your http://msgdev.mvps.org/exdevblog/replfldfbardov1.zip then all you should have to do is change the path so instead of pointing to the calender point it to contact. I haven't really tested it on contact but it worked okay on calenders for user that where on the same Exchange server.

Cheers
Glen

RobG said...

How would I modify this to replicate the other way ie PublicFolder -> Mailbox or database --> PublicFolder.

Does the colbblob exist for a PublicFolder to track changes ? Would this be better than setting an event sink on the folder to track changes ?

Glen said...

The colbblo can be used on any folder mailbox or public folder. To replicate the other way you'll need to pull apart the script and rewrite it in a different order changing around the the source and destinations

Kevin W. said...

Glen, I just came across your blog and used the FBA version of the script to replicate a Contact folder to a PF on an SBS2003 server. Worked like a charm. Thanks!

Have you written a similar script for the reverse: replicate a PF back to a personal folder?

Glen said...

The problem with PST's are that you can only use MAPI to access them part of this script would work okay but a lot of of it would need to be rewritten to work against a PST because it use WebDAV. You would never be able to replicate from a PST as a source.

Cheers
Glen

kdc said...

Killer little script! Is it possible to use 1 set of credentials to copy from different accounts? I tried to no avail.

Thx - K

Glen said...

Sure have a look at http://www.petri.co.il/grant_full_mailbox_rights_on_exchange_2000_2003.htm

An account with this type of permissions should work okay.

Cheers
Glen

Olliman said...

trying to run this script on a E2K7 Server (x64) I get a VBscript runtime error: ActiveX component can't create object: 'MAPI.Session'

Am I missing something?

Glen said...

This script was written for Exchange 2003 so im not sure if it will work on 2007. I would strongly advise against running this on directly on a server especially a 64 bit one. Try it from a client that has Outlook installed. By default Exchange 2007 doesn't have a Mapi client install which is why you getting these error. http://www.microsoft.com/downloads/details.aspx?FamilyID=e17e7f31-079a-43a9-bff2-0a110307611e&displaylang=en

I recommend that if you want to do something like this that you use Exchange Web Services which provides all this functionality in one interface and will be reliable and scalable way to do this.

Cheers
Glen

Trevor said...

I realize this script has not been tested on Exchange 2007, but unfortunately that's what we have in our environment - and of course testing would need to be done.

Could this script be used to move an ENTIRE mailbox to a specified destination public folder? Do you have any examples of using EWS to perform the same function?

Glen said...

This is a script for replicating the contents of a mailbox not really the best thing to use for heavy lifting you would be better using exmerge or just doing it manually with Outlook. On Exchange 2007 you have a lot more options in regards to doing this there are two sync operations which work very similar to webdav replication and also you have the ability to move item across stores using Moveitems. The Exchange 2007 SDK is probably your best bet in regards to coming up with a solution using EWS to do this.

Cheers
Glen