Tuesday, October 27, 2015

Using eDiscovery to do a Multi Mailbox search for Mailbox Item Statistics in Exchange

eDiscovery in Exchange 2013 and above has a multitude of uses when it comes to both data discovery and also reporting. One thing you can do with eDiscovery is run a single Query across multiple mailboxes using one request. A couple of month ago I posted this eDiscovery Powershell module on GitHub . This module has a number of cmdlets that does single mailbox queries using eDiscovery so I've created a new cmdlet Search-MultiMailboxesItemStats  for doing Multi Mailbox queries. This allows you to pass in an array of mailboxes you want processed and it will return statistics about how many Items and the Size of those items in bytes based on that query. For example to Query the number of Email received in the last month across a number of mailboxes use

Search-MultiMailboxesItemStats -Mailboxes @('mailbox@domain.com','mailbox2@domain.com')  -QueryString ('Received>' + (Get-Date).AddDays(-31).ToString("yyyy-MM-dd"))

And it will output something like

 
Or if you just want to look at the email that came from a specific domain you could use

Search-MultiMailboxesItemStats -Mailboxes @('mailbox@domain.com','mailbox2@domain.com')  -QueryString ('Received>' + (Get-Date).AddDays(-31).ToString("yyyy-MM-dd") + " AND From:yahoo.com")

another example eg search for the number of contacts in each Mailbox

Search-MultiMailboxesItemStats -Mailboxes @('mailbox@domain.com','mailbox2@domain.com')  -QueryString 'kind:contacts'


The latest version of the module is posted up on GitHub here or you can download a copy from here

Tuesday, October 20, 2015

Introducing the Data Rumble Message Tracking Outlook AddIn for Office 365

With great pride and a little trepidation I'd like to introduce my new venture Data Rumble and our first software release Message Tracking Outllook Addin for Office 365. As a company we are looking to focus on email data so looking at the Metadata an Email message picks up and leaves behind while its in transit to and from a mailbox is the logical first step for us. Launching something new is a lot of work so this first release is a bit of dry run to get all the underlying infrastructure and procedures in place.

At release the main feature of this AddIn is that it allows you to perform a Message Trace on a Message from within Outlook itself using the Office365 Reporting Web Service. This is a REST endpoint that allows you to perform a number of different Office365 Administrative reporting tasks it has been around for a couple of years now and pre dates some of the oauth features that the newer mailbox REST services have so I expect the Endpoint will change soon  (it also has the option to use PowerShell as an alternative to cater for instances where REST doesn't work). . But from this AddIn's perspective it makes use of this endpoint to perform a Message Trace (that you would normally do in the EAC or PowerShell eg https://technet.microsoft.com/EN-US/library/jj200741(v=exchg.150).aspx ) from within Outlook eg. this a screenshot


The Addin post processes the data returned by the Reporting service as well as data extracted from the messages Transport Headers and a few other Message properties and combines those together to provide more information around the message such as the Message header properties, Exchange online protection actions etc (although there is already a good tool for this https://testconnectivity.microsoft.com/ and a corresponding AddIn but this is just another view of the data),Sender and Recipient information. All Message Tracking is done based on the MessageId's extracted from the message in Outlook, it also extracts any associated MessageID's eg if this message is a reply, forward or part of a thread. So you can then query the logs for any associated Messages as well as query for other messages that have been sent or received to and from any of the sender, recipients or envelope recipients found in the log. (envelope recipients would could be BCC's, alternate recipients, forwards or recipients added by a Transport Agent that are available in the Tracing logs). I've created a short video below to showcase the features or you can check and download the software from the product page here.

Tuesday, October 06, 2015

Unread email ews Powershell Module with reply and forward counting

I've done a few of these Unread / Unused mailbox scripts over the years but this one has a bit of a difference. As well as counting the Total number of unread email in the Inbox over a certain period of time it uses the PidTagLastVerbExecuted property to count how many email messages over that period of time had the client action ReplytoSender, ReplyAll or forwarded and also the number of email in the SentItems folder. This property is set on messages in the Inbox message when one of those actions is taken by the client so it is useful for tracking the use of Mailboxes and gathering statistics around how they are being used. eg here are a few samples of running this module



The code uses both EWS and the Exchange Management Shell to get information about the Mailbox so you need to run it from within the EMS or a Remote PowerShell session (see this if your running it from Office365). I've put the script up on GitHub or you can download it from here. I've also created a Search Filter version of the code, this would work on Exchange 2007 and also if you have an issue where you only see a maximum of 250 items (which is an AQS bug in some version of Exchange) this will address this issue this is also on gihub here


The code itself looks like

function Connect-Exchange{ 
    param( 
     [Parameter(Position=0, Mandatory=$true)] [string]$MailboxName,
  [Parameter(Position=1, Mandatory=$true)] [System.Management.Automation.PSCredential]$Credentials,
  [Parameter(Position=2, Mandatory=$false)] [string]$url
    )  
  Begin
   {
  Load-EWSManagedAPI
  
  ## Set Exchange Version  
  $ExchangeVersion = [Microsoft.Exchange.WebServices.Data.ExchangeVersion]::Exchange2013_SP1
    
  ## Create Exchange Service Object  
  $service = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService($ExchangeVersion)  
    
  ## Set Credentials to use two options are availible Option1 to use explict credentials or Option 2 use the Default (logged On) credentials  
    
  #Credentials Option 1 using UPN for the windows Account  
  #$psCred = Get-Credential  
  $creds = New-Object System.Net.NetworkCredential($Credentials.UserName.ToString(),$Credentials.GetNetworkCredential().password.ToString())  
  $service.Credentials = $creds      
  #Credentials Option 2  
  #service.UseDefaultCredentials = $true  
   #$service.TraceEnabled = $true
  ## Choose to ignore any SSL Warning issues caused by Self Signed Certificates  
    
  Handle-SSL 
    
  ## Set the URL of the CAS (Client Access Server) to use two options are availbe to use Autodiscover to find the CAS URL or Hardcode the CAS to use  
    
  #CAS URL Option 1 Autodiscover  
  if($url){
   $uri=[system.URI] $url
   $service.Url = $uri    
  }
  else{
   $service.AutodiscoverUrl($MailboxName,{$true})  
  }
  Write-host ("Using CAS Server : " + $Service.url)   
     
  #CAS URL Option 2 Hardcoded  
    
  #$uri=[system.URI] "https://casservername/ews/exchange.asmx"  
  #$service.Url = $uri    
    
  ## Optional section for Exchange Impersonation  
    
  #$service.ImpersonatedUserId = new-object Microsoft.Exchange.WebServices.Data.ImpersonatedUserId([Microsoft.Exchange.WebServices.Data.ConnectingIdType]::SmtpAddress, $MailboxName) 
  if(!$service.URL){
   throw "Error connecting to EWS"
  }
  else
  {  
   return $service
  }
 }
}

function Load-EWSManagedAPI{
    param( 
    )  
  Begin
 {
  ## Load Managed API dll  
  ###CHECK FOR EWS MANAGED API, IF PRESENT IMPORT THE HIGHEST VERSION EWS DLL, ELSE EXIT
  $EWSDLL = (($(Get-ItemProperty -ErrorAction SilentlyContinue -Path Registry::$(Get-ChildItem -ErrorAction SilentlyContinue -Path 'Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Exchange\Web Services'|Sort-Object Name -Descending| Select-Object -First 1 -ExpandProperty Name)).'Install Directory') + "Microsoft.Exchange.WebServices.dll")
  if (Test-Path $EWSDLL)
      {
      Import-Module $EWSDLL
      }
  else
      {
      "$(get-date -format yyyyMMddHHmmss):"
      "This script requires the EWS Managed API 1.2 or later."
      "Please download and install the current version of the EWS Managed API from"
      "http://go.microsoft.com/fwlink/?LinkId=255472"
      ""
      "Exiting Script."
      exit
      } 
   }
}

function Handle-SSL{
    param( 
    )  
  Begin
 {
  ## Code From http://poshcode.org/624
  ## Create a compilation environment
  $Provider=New-Object Microsoft.CSharp.CSharpCodeProvider
  $Compiler=$Provider.CreateCompiler()
  $Params=New-Object System.CodeDom.Compiler.CompilerParameters
  $Params.GenerateExecutable=$False
  $Params.GenerateInMemory=$True
  $Params.IncludeDebugInformation=$False
  $Params.ReferencedAssemblies.Add("System.DLL") | Out-Null

$TASource=@'
  namespace Local.ToolkitExtensions.Net.CertificatePolicy{
    public class TrustAll : System.Net.ICertificatePolicy {
      public TrustAll() { 
      }
      public bool CheckValidationResult(System.Net.ServicePoint sp,
        System.Security.Cryptography.X509Certificates.X509Certificate cert, 
        System.Net.WebRequest req, int problem) {
        return true;
      }
    }
  }
'@ 
  $TAResults=$Provider.CompileAssemblyFromSource($Params,$TASource)
  $TAAssembly=$TAResults.CompiledAssembly

  ## We now create an instance of the TrustAll and attach it to the ServicePointManager
  $TrustAll=$TAAssembly.CreateInstance("Local.ToolkitExtensions.Net.CertificatePolicy.TrustAll")
  [System.Net.ServicePointManager]::CertificatePolicy=$TrustAll

  ## end code from http://poshcode.org/624

 }
}

function CovertBitValue($String){  
    $numItempattern = '(?=\().*(?=bytes)'  
    $matchedItemsNumber = [regex]::matches($String, $numItempattern)   
    $Mb = [INT64]$matchedItemsNumber[0].Value.Replace("(","").Replace(",","")  
    return [math]::round($Mb/1048576,0)  
}  

function Get-UnReadMessageCount{
    param( 
     [Parameter(Position=0, Mandatory=$true)] [string]$MailboxName,
  [Parameter(Position=1, Mandatory=$true)] [System.Management.Automation.PSCredential]$Credentials,
  [Parameter(Position=2, Mandatory=$false)] [switch]$useImpersonation,
  [Parameter(Position=3, Mandatory=$false)] [string]$url,
  [Parameter(Position=4, Mandatory=$true)] [Int32]$Months
    )  
  Begin
 {
  $eval1 = "Last" + $Months + "MonthsTotal"
  $eval2 = "Last" + $Months + "MonthsUnread"
  $eval3 = "Last" + $Months + "MonthsSent"
  $eval4 = "Last" + $Months + "MonthsReplyToSender"
  $eval5 = "Last" + $Months + "MonthsReplyToAll"
  $eval6 = "Last" + $Months + "MonthForward"
  $reply = 0;
  $replyall = 0
  $forward = 0
  $rptObj = "" | select  MailboxName,Mailboxsize,LastLogon,LastLogonAccount,$eval1,$eval2,$eval4,$eval5,$eval6,LastMailRecieved,$eval3,LastMailSent  
  $rptObj.MailboxName = $MailboxName  
  if($url){
   $service = Connect-Exchange -MailboxName $MailboxName -Credentials $Credentials -url $url 
  }
  else{
   $service = Connect-Exchange -MailboxName $MailboxName -Credentials $Credentials
  }
  if($useImpersonation.IsPresent){
   $service.ImpersonatedUserId = new-object Microsoft.Exchange.WebServices.Data.ImpersonatedUserId([Microsoft.Exchange.WebServices.Data.ConnectingIdType]::SmtpAddress, $MailboxName) 
  }
  $AQSString1 = "System.Message.DateReceived:>" + [system.DateTime]::Now.AddMonths(-$Months).ToString("yyyy-MM-dd")   
    $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox,$MailboxName)     
  $Inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)  
    $folderid= new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::SentItems,$MailboxName)     
  $SentItems = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$folderid)    
  $ivItemView = New-Object Microsoft.Exchange.WebServices.Data.ItemView(1000)  
  $psPropset= new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::IdOnly)  
  $psPropset.Add([Microsoft.Exchange.WebServices.Data.ItemSchema]::DateTimeReceived)
  $psPropset.Add([Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::IsRead)
  $PidTagLastVerbExecuted = new-object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition(0x1081,[Microsoft.Exchange.WebServices.Data.MapiPropertyType]::Integer); 
  $psPropset.Add($PidTagLastVerbExecuted)
  $ivItemView.PropertySet = $psPropset
    $MailboxStats = Get-MailboxStatistics $MailboxName  
  $ts = CovertBitValue($MailboxStats.TotalItemSize.ToString())  
  write-host ("Total Size : " + $MailboxStats.TotalItemSize) 
     $rptObj.MailboxSize = $ts  
  write-host ("Last Logon Time : " + $MailboxStats.LastLogonTime) 
  $rptObj.LastLogon = $MailboxStats.LastLogonTime  
  write-host ("Last Logon Account : " + $MailboxStats.LastLoggedOnUserAccount ) 
  $rptObj.LastLogonAccount = $MailboxStats.LastLoggedOnUserAccount  
  $fiItems = $null
  $unreadCount = 0
  $settc = $true
     do{ 
   $fiItems = $Inbox.findItems($AQSString1,$ivItemView)  
   if($settc){
    $rptObj.$eval1 = $fiItems.TotalCount  
    write-host ("Last " + $Months + " Months : " + $fiItems.TotalCount)
    if($fiItems.TotalCount -gt 0){  
        write-host ("Last Mail Recieved : " + $fiItems.Items[0].DateTimeReceived ) 
        $rptObj.LastMailRecieved = $fiItems.Items[0].DateTimeReceived  
    }      
    $settc = $false
   }
       foreach($Item in $fiItems.Items){
     $unReadVal = $null
     if($Item.TryGetProperty([Microsoft.Exchange.WebServices.Data.EmailMessageSchema]::IsRead,[ref]$unReadVal)){
      if(!$unReadVal){
       $unreadCount++
      }
     } 
       $lastVerb = $null
     if($Item.TryGetProperty($PidTagLastVerbExecuted,[ref]$lastVerb)){
      switch($lastVerb){
       102 { $reply++ }
       103 { $replyall++}
       104 { $forward++}
      }
     } 
       }    
       $ivItemView.Offset += $fiItems.Items.Count    
   }while($fiItems.MoreAvailable -eq $true) 

  write-host ("Last " + $Months + " Months Unread : " + $unreadCount ) 
  $rptObj.$eval2 = $unreadCount  
  $rptObj.$eval4 = $reply
  $rptObj.$eval5 = $replyall
  $rptObj.$eval6 = $forward
  $ivItemView = New-Object Microsoft.Exchange.WebServices.Data.ItemView(1)  
  $fiResults = $SentItems.findItems($AQSString1,$ivItemView)  
  write-host ("Last " + $Months + " Months Sent : " + $fiResults.TotalCount  )
  $rptObj.$eval3 = $fiResults.TotalCount  
  if($fiResults.TotalCount -gt 0){  
      write-host ("Last Mail Sent Date : " + $fiResults.Items[0].DateTimeSent  )
      $rptObj.LastMailSent = $fiResults.Items[0].DateTimeSent  
  }  
  Write-Output $rptObj  
 }
}