Friday, January 12, 2007

Audit and Export all emails in an Exchange Store Sent and Received to and from a certain domain

This is one that came up for me this week where for audit purposes all communication between a certain email domain needed to be found and exported in a Exchange Store. I all ready had the bones for this script in one of my other posts all I needed to do was adapt this by adding in some code to retrieve the fromemail and the to properties and then do a substring search of both of these properties. If a match is found the email is then exported by using WebDAV to get the stream of the item and then using an ADO stream to write the stream to a normal EML file that can then be opened in any mail client.

The script works using WebDAV and it connects to and scans every mailbox using the Admin virtual root which means the script can run with delegated Exchange admin rights. The first part of the script contains some ADSI queries to work out what the URL to the admin root is and then calls the RecurseFolder sub this sub is based on the code from the mailbox size KB . With a few exceptions the main one being is that it only checks normal mail folders this was to prevent an issue where searchfolders exist in a mailbox. The ResurseFolder sub job is basically to retrieve all the folder URL’s in the mailbox and then call the procfolder sub. The profolder sub processes all the email in a folder between the date-range specified. If a substring match is found on any of the from or to addresses the exportemail sub is called which contains some code to ensure that a unique filename is generated for each exported email.

By default the script exports email to a directory called exp on the c:\ to change this you need to modify the following line in the script.

fpath = "c:\exp\"

The front end of the script takes 4 commandline parameters the first is the servername of the server you want to run the script against the 2nd is the domain you want to scan and the 3rd and 4th is the date range you want to look at. EG so you can scan all correspondences for just a certain time period which will cut down on the time it takes for the script to run.The start and enddate needs to be in ISO format (year-month-date) eg to scan for all message sent to or from the domain blahdoman.com between October and January this year the commandline to run the script would be.

cscript mbauditcdomv2.vbs servername @blahdoman.com 2006-10-01 2007-02-01

I've created two copies of the script the mbauditcdomv2000.vbs version use the mailnickname to connect to each mailbox and is designed for Exchange 2000 where you cant use the email address to connect to the mailbox in webdav.

I’ve put a downloadable copy of the script here the script itself looks like

on error resume Next
fpath = "c:\exp\"
Servername = wscript.arguments(0)
domaintosearch = wscript.arguments(1)
datefrom = wscript.arguments(2) & "T00:00:00Z"
dateto = wscript.arguments(3) & "T00:00:00Z"
set req = createobject("microsoft.xmlhttp")
set com = createobject("ADODB.Command")
set conn = createobject("ADODB.Connection")
Set iAdRootDSE = GetObject("LDAP://RootDSE")
strNameingContext = iAdRootDSE.Get("configurationNamingContext")
strDefaultNamingContext = iAdRootDSE.Get("defaultNamingContext")
Conn.Provider = "ADsDSOObject"
Conn.Open "ADs Provider"
polQuery = "<LDAP://" & strNameingContext & ">;(&(objectCategory=msExchRecipientPolicy)(cn=Default
Policy));distinguishedName,gatewayProxy;subtree"
svcQuery = "<LDAP://" & strNameingContext & ">;(&(objectCategory=msExchExchangeServer)(cn="
& Servername & "));cn,name,legacyExchangeDN;subtree"
Com.ActiveConnection = Conn
Com.CommandText = polQuery
Set plRs = Com.Execute
while not plRs.eof
for each adrobj in plrs.fields("gatewayProxy").value
if instr(adrobj,"SMTP:") then dpDefaultpolicy =
right(adrobj,(len(adrobj)-instr(adrobj,"@")))
next
plrs.movenext
wend
wscript.echo dpDefaultpolicy
Com.CommandText = svcQuery
Set Rs = Com.Execute
while not rs.eof
GALQueryFilter = "(&(&(&(& (mailnickname=*)(!msExchHideFromAddressLists=TRUE)(|
(&(objectCategory=person)(objectClass=user)(msExchHomeServerName=" &
rs.fields("legacyExchangeDN") & ")) )))))"
strQuery = "<LDAP://" & strDefaultNamingContext & ">;" & GALQueryFilter & ";displayname,mail,distinguishedName,mailnickname,proxyaddresses;subtree"
com.Properties("Page Size") = 100
Com.CommandText = strQuery
Set Rs1 = Com.Execute
while not Rs1.eof
falias = "http://" & servername & "/exadmin/admin/" & dpDefaultpolicy & "/mbx/"
for each paddress in rs1.fields("proxyaddresses").value
if instr(paddress,"SMTP:") then
falias = falias & replace(paddress,"SMTP:","")
cusername = replace(paddress,"SMTP:","")
End if
next
ReDim tresarray(1,6)
wscript.echo falias
call RecurseFolder(falias)
rs1.movenext
wend
rs.movenext
wend
rs.close
set conn = nothing
set com = nothing
set wfile = nothing
set fso = nothing

Public Sub RecurseFolder(sUrl)

req.open "SEARCH", sUrl, False, "", ""
sQuery = "<?xml version=""1.0""?>"
sQuery = sQuery & "<g:searchrequest xmlns:g=""DAV:"">"
sQuery = sQuery & "<g:sql>SELECT ""http://schemas.microsoft.com/"
sQuery = sQuery & "mapi/proptag/x0e080003"", ""DAV:hassubs"" FROM SCOPE "
sQuery = sQuery & "('SHALLOW TRAVERSAL OF """ & sUrl & """') "
sQuery = sQuery & "WHERE ""DAV:isfolder"" = true and ""DAV:ishidden"" = false
and ""http://schemas.microsoft.com/mapi/proptag/x36010003"" = 1"
sQuery = sQuery & "</g:sql>"
sQuery = sQuery & "</g:searchrequest>"
req.setRequestHeader "Content-Type", "text/xml"
req.setRequestHeader "Translate", "f"
req.setRequestHeader "Depth", "0"
req.setRequestHeader "Content-Length", "" & Len(sQuery)
req.send sQuery
Set oXMLDoc = req.responseXML
Set oXMLSizeNodes = oXMLDoc.getElementsByTagName("d:x0e080003")
Set oXMLHREFNodes = oXMLDoc.getElementsByTagName("a:href")
Set oXMLHasSubsNodes = oXMLDoc.getElementsByTagName("a:hassubs")
For i = 0 to oXMLSizeNodes.length - 1
call procfolder(oXMLHREFNodes.Item(i).nodeTypedValue,sUrl)
wscript.echo oXMLHREFNodes.Item(i).nodeTypedValue
If oXMLHasSubsNodes.Item(i).nodeTypedValue = True Then
call RecurseFolder(oXMLHREFNodes.Item(i).nodeTypedValue)
End If
Next
End Sub

sub procfolder(strURL,pfname)
wscript.echo strURL
ReDim resarray(1,6)
strQuery = "<?xml version=""1.0""?><D:searchrequest xmlns:D = ""DAV:""
xmlns:b=""urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/"">"
strQuery = strQuery & "<D:sql>SELECT ""DAV:displayname"",
""urn:schemas:httpmail:subject"", "
strQuery = strQuery & """DAV:creationdate"", ""DAV:getcontentlength"", "
strQuery = strQuery & """urn:schemas:httpmail:fromemail"",
""urn:schemas:httpmail:to"""
strQuery = strQuery & " FROM scope('shallow traversal of """
strQuery = strQuery & strURL & """') Where ""DAV:ishidden"" = False AND
""DAV:isfolder"" = False AND "
strQuery = strQuery & """urn:schemas:httpmail:datereceived"" &lt; CAST(""" &
dateto & """ as 'dateTime') AND "
strQuery = strQuery & """urn:schemas:httpmail:datereceived"" &gt; CAST(""" &
datefrom & """ as 'dateTime')</D:sql></D:searchrequest>"
req.open "SEARCH", strURL, false
req.setrequestheader "Content-Type", "text/xml"
req.setRequestHeader "Translate","f"
req.send strQuery
If req.status >= 500 Then
ElseIf req.status = 207 Then
set oResponseDoc = req.responseXML
set oNodeList = oResponseDoc.getElementsByTagName("a:displayname")
set oNodeList1 = oResponseDoc.getElementsByTagName("a:href")
set oSize = oResponseDoc.getElementsByTagName("a:getcontentlength")
set odatereceived = oResponseDoc.getElementsByTagName("a:creationdate")
set fEmail = oResponseDoc.getElementsByTagName("d:fromemail")
set TEmail = oResponseDoc.getElementsByTagName("d:to")
For i = 0 To (oNodeList.length -1)
set oNode = oNodeList.nextNode
set oNode1 = oNodeList1.nextNode
set oNode2 = oSize.nextNode
set oNode3 = odatereceived.nextNode
set oNode4 = fEmail.nextNode
set oNode5 = TEmail.nextNode
wscript.echo oNode3.text
export = 0
If InStr(LCase(oNode4.text),LCase(domaintosearch))Then
export = 1
End If
if InStr(LCase(oNode5.text),LCase(domaintosearch))Then
export = 1
End If
If export = 1 Then
Call exportemail(oNode1.text,oNode.text)
wscript.echo "Exporting : " & oNode4.text
End if
Next
Else
End If

end sub

sub exportemail(exporthref,subject)
req.open "GET", exporthref, false
req.setRequestHeader "Translate","f"
req.send
fname = replace(replace(replace(replace(replace((cusername & "-" &
subject),":","-"),"\",""),"/",""),"?",""),chr(34),"")
fname =
replace(replace(replace(replace(replace(replace(fname,"<",""),">",""),chr(11),""),"*",""),"|",""),"(","")
fname = replace(replace(replace(fname,")",""),chr(12),""),chr(15),"")
Randomize ' Initialize random-number generator.
rndval = Int((20000000000 * Rnd) + 1)
fname = fpath & replace(lcase(fname),".eml",rndval & ".eml")
wscript.echo fname
set stm = createobject("ADODB.Stream")
stm.open
msgstring = req.responsetext
stm.type = 2
stm.Charset = "x-ansi"
stm.writetext msgstring,0
stm.Position = 0
stm.type = 1
stm.savetofile fname
set stm = nothing

End sub

9 comments:

Jason said...

Would it be possible to also search for a string in the subject line.

What I'm looking to do is to find all outbound messages to a list of domains with a certain phrase in the subject.

Thanks,
Jason

Glen said...

Yes all you need to do is change the query that is being used and add an additional clause to the where statement

Shawn said...

I'm running this on a 2003 backend. It spits out a list of http://servername... user@mydomain.com to the console but no files. The user I'm using to run the script has access to all the mailboxes. What might I be doing wrong or have configured wrong on the exchange side?

Glen said...

It must be erroring out somewhere try to rem the first line eg

rem on error resume next

Then when you run the script you should see any errors its generating

Cheers
Glen

Champs said...

I'm seeing the same thing, but nailed down the problem to an empty XML response in RecurseFolders -- Len(oXMLDoc.xml) is always 0.

Champs said...

And by that I mean I could use some help getting beyond that step if you have any ideas. I sure don't, and I assume that a permissions issue (tried as an Admin and backup operator) would either not get me this far or at least crank out an error of some sort in the XML response.

Glen said...

Make sure the account you using has delegated Exchange admin rights which is what it will need to accsss the Admin virtual root. Also you might be running https only on the virtual admin root which would cause things to fail. Also if you running Exchange 2007 this script wont work.

Magnus said...

Just came across this as it's just what I need but the last line about it not working on 2007 was very disappointing! Great work though.

Anonymous said...

This has helped a treat. Thank you. I know this has been on here a while however is it possible to "tweak" the script so it only extracts the most recent email from a conversation chain for the domain in question? If so, what do I need to change?

Thanks again
Arif