Saturday, June 22, 2013

Exchange 2010 - Recoverable Items Folder - Dumpster 2.0 - Single Item Recovery

In some discussions on Microsoft Technet (Exchange sections), there seemed to be some confusion about what elements are included in the size of a mailbox. Messages in the deleted items folder? What about messages in the "dumpster"? Is the dumpster part of the user's mailbox? Do messages in the dumpster count against the user's quota?

I wanted to clarify this for myself.

First, I want to review the structure of the various deleted items containers in an Exchange mailbox. This has become somewhat more complicated with the Recoverable Items folder, otherwise known as Dumpster 2.0, in Exchange 2010, and its subfolders, some of which are used only if certain features are enabled.

What subfolders? What features?

Let's take a look.


Dumpster 1.0 (Exchange 2007)


What I'll call the "deletion process" in Exchange 2007 was rather simple. This is how it worked:
  1. User deletes a message from the "Inbox" or "Sent Items" folder (etc.). The message goes to the "Deleted Items" folder. This is still part of the user's mailbox and counts against their quota.
  2. User empties the "Deleted Items" folder. The message goes to the "Dumpster". Items in this location apparently do not count against the mailbox quota (more on this below).
  3. The item remains in the Dumpster  for the duration of the retention policy configured for the mailbox database. By default, it is 14 days.

That's how it works in appearence...
 
In fact, the deleted items remain in the folders from which they were (in appearence) deleted. Exchange 2007 simply stamps them with the "ptagDeletedOnFlag" marker. This flag conceals the item so it is invisible in common user interfaces such as Outlook and OWA.
 
Therefore, one could argue that in Exchange 2007, the dumpster does not really exist! It is a sort of illusion, a virtual view of deleted messages that remain, though hidden, in the exact same folders where they were deleted.
 
 
Several comments:
 
  • The Exchange 2007 dumpster is not "searchable" which is a significant disadavantage for mailbox discovery and compliance.
  • Users can permanently purge items in the dumpster (if they know where to look). That too represents a disadvantage with respect to mailbox searches and compliance.
 
 
 
Soft Deletes and Hard Deletes

In the following sections of this post, I'll refer to different types of deletions. For clarity, let's review the difference between a "soft" delete and a "hard" delete. 

Soft delete
 
This is the most common type of deletion. The user selects the item and either clicks on the "black X" icon or right-clicks on the item and selects "Delete" in the menu options. They can also press the Delete key on their keyboard. No lack of choices here.
 
The deleted item goes to the "Deleted Items" folder. Most users can easily recover an item from this location by dragging it back to the original location or right-clicking on the item and selecting the "Move to Folder" option. Here again, they would move the item to the previous location (Inbox, Sent Items, etc.).
 
Hard Delete
 
In this case, the user selects the item, holds down the "Shift" key and then presses the "Delete" key. The item in question goes directly to the dumpster. It does NOT go to the Deleted Items folder first.



Dumpster 2.0 (Exchange 2010)

In Exchange 2010, the dumpster exists as a "non-IPM" folder, called "Recoverable Items". Each user's mailbox contains one of these "Recoverable Items" folders.

But just what is a "non-IPM" folder?

A non-IPM folder does not appear in common user interfaces like Outlook and OWA. 

It is invisible to the eyes of the end-user.
 
 
However, the "Recoverable Items" folder is a "real" folder that even includes 3 subfolders:

  • Deletions
  • Purges
  • Versions

What are these subfolders? What do they do? What must be done, if anything, to enable them?


Deletions

This subfolder contains items that were deleted from the "Deleted Items" folder or hard-deleted from other folders. These items are retained, by default, for 14 days.
 
Users can purge items in the Deletions subfolder, just like they could for the dumpster (version 1.0) in Exchange 2007, unless... a feature called "Single Item Recovery" is enabled. In that case, users cannot purge the Deletions subfolder (or only purge it in appearence). More on that in the next section.


Purges
 
The "Purges" subfolder is inactive - unless "Single Item Recovery" is enabled.

If "SIR" is not enabled, deleted items simply go to the Deletions subfolder as described above and stay there until removed at the end of the retention period (14 days by default).
 
If "SIR" is enabled, messages removed from the Deletions subfolder (before the expiration of the retention period) are placed in the Purges subfolder until the retention period does expire.
 
 
So... what's the difference? The item in question just sits in the Purges subfolder instead of the Deletions subfolder before being eliminated anyway by the mailbox cleanup process, right?
 
Here's the difference. End users cannot access the Purges subfolder. Even if they empty the Deletions folder, the deleted items remain in the Purges subfolder.
 
But only for the duration of the (14 day by default) retention period, right?
 
Yes, but if legal considerations require that a company retain mail for a longer period of time, a feature called "Litigation Hold" can be enabled. When this feature is enabled, items in the Purges folder will NOT be removed at the end of the 14 day retention period and users cannot permanently delete them either, since they cannot access this "Non-IPM" subfolder.

So there's no way to right-click and delete that incriminating message!
 
That's the primary difference with the Exchange 2007 "Dumpster 1.0".
 
 
Versions
 
The Versions subfolder is used when "Single Item Recovery" or "Litigation Hold" is enabled. The deletion of a legally significant item is not the only problem an organization might face. Users could potentially modify an item without deleting it altogether.
 
When "Litigation Hold" is enabled, a copy of any modified item is sent to the Versions subfolder.
 
Note: with or without Single Item Recovery, calendar items are retained in the "Recoverable Items" subfolders for 120 days - unless... "Litigation Hold" is enabled, in which case they would be retained as long as necessary.




Questions and answers and more questions.

So the following types of questions have come up in Exchange Technet forum discussions:

  • Is the dumpster part of the mailbox?
  • Do the items in the dumpster count against the users mailbox quota?
  • What is the relationship between the cmdlet "Get-MailboxStatistics", the "TotalItemsize" parameter and the "TotalDeletedItemSize" parameter.
 
First of all, Microsoft documentation states that the dumpster is part of the mailbox:

"Dumpster in Exchange 2010 is implemented as a folder called the Recoverable Items and is located within the Non-IPM subtree of the user's mailbox."

Indeed, corresponding illustrations show the dumpster as part of the mailbox



Reference:

Single Item Recovery in Exchange Server 2010

This was also the case with Exchange 2007, though in a different form.
 
On the other hand, it appears that items in the dumpster (version 1.0 or version 2.0) do not count against the user's mailbox quota.

 
What follows is the result of my own "experiments" on the subject. I've tested the above assumption in both Exchange 2007 (SP3) and Exchange 2010 (SP3). 



Exchange 2007

Here's the plan.

I'm going to:

  1. Delete items
  2. then delete them from the Deleted Items folder
  3. and then the dumpster...
Observing how these operations affect the size of the mailbox.


I started with this command but the output is not very readable:


[PS] C:\>Get-MailboxStatistics jsmith@mydomain.com | fl *size*

TotalDeletedItemSize : 3 310 038B

TotalItemSize : 115 069 436B

(Yes, I added those spaces).



So let's try this instead:


(Get-MailboxStatistics jsmith@mydomain.com).totalitemsize.value.ToMB()

109

(Get-MailboxStatistics jsmith@mydomain.com).totaldeleteditemsize.value.ToMB()

3



Let's also consider the size of the mailbox as shown in Outlook:




Note: for some reason, there is a slight difference between 109 MB and 112 MB. However, the resolution of this enigma is outside the scope of this blog post.

 
1. Delete Items


Now... John Smith deletes 20 MB + of messages in Outlook. What happens to the statistics?

(Get-MailboxStatistics jsmith@mydomain.com).totalitemsize.value.ToMB()

109

(Get-MailboxStatistics jsmith@mydomain.com).totaldeleteditemsize.value.ToMB()

3


Nothing... no change.
 
Conclusion: the deleted items folder is part of the mailbox AND the items in the deleted items folder are included in the calcuation of "total item size" or total mailbox size. That was expected.




2. Empty Deleted Items folder ("trash")


Now John Smith empties the "Deleted Items" folder. What happens?


(Get-MailboxStatistics jsmith@mydomain.com).totalitemsize.value.ToMB()

87

(Get-MailboxStatistics jsmith@mydomain.com).totaldeleteditemsize.value.ToMB()

25
 


The size of the mailbox decreases while the size of the dumpster increases.


Note: "totaldeleteditemsize" measures the size of the dumpster. I'll make some clarifications on this point in my conclusion below.


When I delete items from the Deleted Items folder, the value of "TIS" decreases (87 MB versus 109 MB).

On the other hand, "TDIS" increases (25 MB versus 3 MB).

If "TDIS" was part of "TIS", those numbers would not change, just as they do not change when items are moved from the Inbox or Sent Items to the Deleted Items folder.
 
This seems to confirm that "totaldeleteditemsize" is not included in "totalitemsize".



So... it does appear that although the dumpster is part of the mailbox (as a virtual view of deleted items), the contents of the dumpster do NOT count against the user mailbox quota.

This is what we see in Outlook:




Total size has decreased from 112 MB (in the previous screenshot) to 89 MB.



3. Purge the Dumpster


Now what happens if we empty the dumpster?


John Smith goes to Recover Deleted Items but instead of recovering them, he highlights them all and click on the black X (Purge selected items):





 

What happens to the mailbox statistics?


(Get-MailboxStatistics jsmith@mydomain.com).totalitemsize.value.ToMB()

87


(Get-MailboxStatistics jsmith@mydomain.com).totaldeleteditemsize.value.ToMB()

1



So, purging the dumpster in Exchange 2007 affects the TDIS value but not the TIS value.

We can then conclude that "TIS" measures the size of the mailbox, all folders except the dumpster (as it exists in E2K7) and "TDIS" measures the size of the dumpster and the dumpster alone.




Exchange 2010

In the following lines, I'll reproduce the experiments above but this time with a pair of users on an Exchange 2010 server.
 

This is the size of the respective mailboxes before any deletions have been made:


 


Get-MailboxStatistics aisha.bhari@mydomain.net | fl *size*

TotalDeletedItemSize : 1.406 KB (1,440 bytes)
TotalItemSize        : 7.24 MB (7,591,386 bytes)

Get-MailboxStatistics alannah.shaw@mydomain.net | fl *size*

TotalDeletedItemSize : 0 B (0 bytes)
TotalItemSize        : 6.951 MB (7,288,535 bytes)
 

Note: Outlook shows 7211 KB for Aisha and 7035 KB for Alannah, not quite exactly the same as the Get-MailboxStatistics output.
 
 
 
1. Simple deletion of items from Inbox
 
 
Now Aisha and Alannah will delete about 6 MB of messages.
 
What changes?
 
The "Total Item Size"? Mailbox minus the dumpster?
 
Or...
 
The "Total Deleted Item Size"? Just the "Recoverable Items" folder or dumpster?
 

Note: remember, we are testing the assumption that the dumpster, while residing inside the user mailbox, does not count against the quota.
 

Get-MailboxStatistics aisha.bhari@mydomain.net | fl *size*

TotalDeletedItemSize : 1.406 KB (1,440 bytes)
TotalItemSize        : 7.24 MB (7,591,526 bytes)

Get-MailboxStatistics alannah.shaw@mydomain.net | fl *size*

TotalDeletedItemSize : 0 B (0 bytes)
TotalItemSize        : 6.951 MB (7,288,535 bytes)
 

OK... nothing changed.

So deleting an item and (essentially) moving it to the deleted items folder has no effect whatsoever on the two values.
 
 
 
2. Empty Deleted Items folder ("trash")
 

Now... Aisha and Alannah will empty their Deleted Items folder.

What happens?


Get-MailboxStatistics aisha.bhari@mydomain.net | fl *size*

TotalDeletedItemSize : 6.677 MB (7,001,758 bytes)
TotalItemSize        : 579.8 KB (593,765 bytes)
 

As we can see, the TIS value decreases from 7.24 MB to around 580 KB.
 
On the other hand, the TDIS value increases from 1.4 KB to around 6.7 MB.
 
 
 
What about Alannah? Does the same thing happen to her?
 

Get-MailboxStatistics alannah.shaw@mydomain.net | fl *size*

TotalDeletedItemSize : 6.627 MB (6,949,415 bytes)
TotalItemSize        : 333.7 KB (341,691 bytes)

 
Sure enough!

Here are the Outlook results for mailbox size (after the above operations):

Aisha: 929 KB
Alannah: 685 KB


Once again, as we saw for Exchange 2007, there is an apparent discrepancy in the value displayed by Outlook and the value produced by the Get-MailboxStatistics cmdlet.
 
This might be an interesting subject for another blog post but, for the time being, our test demonstrates that the items in the Recoverable Items folder do not count against the user's mailbox quota.
 
 
 
3. Purge Recoverable Items folder
 

What happens when Aisha and Alannah purge their Recoverable Items folder?
In this scenario, neither Single Item Recovery or Litigation Hold have been enabled.
 

Get-MailboxStatistics aisha.bhari@mydomain.net | fl *size*

TotalDeletedItemSize : 5.536 KB (5,669 bytes)
TotalItemSize        : 579.8 KB (593,765 bytes)

Get-MailboxStatistics alannah.shaw@mydomain.net | fl *size*

TotalDeletedItemSize : 4.155 KB (4,255 bytes)
TotalItemSize        : 333.7 KB (341,691 bytes)
 

There's nothing but some residue left - roughly 5 KB of something, perhaps metadata (there are no items displayed in any case). Reminder: "TDIS" value was previously between 6 and 7 MB.
 
 
 
Conclusions


  1. Items in the dumpster (Exchange 2007 or Exchange 2010) do NOT count against the mailbox quota.
  2. The "Get-MailboxStatistics" "TotalItemsize" parameter counts all items in the mailbox, deleted items included, but does NOT count items in the dumpster (Recoverable Items folder in Exchange 2010).
  3. The "TotalDeletedItemSize" counts all the items in the dumpster - and only those items.




General Reference:

Jagott, S. & Stidely, J., et. al. (2010). Microsoft Exchange Server 2010 - Best Practices. Remond, WA: Microsoft Press








10 comments:

  1. This comment has been removed by a blog administrator.

    ReplyDelete
  2. This comment has been removed by a blog administrator.

    ReplyDelete
  3. This comment has been removed by a blog administrator.

    ReplyDelete
  4. This comment has been removed by a blog administrator.

    ReplyDelete
  5. This comment has been removed by a blog administrator.

    ReplyDelete
  6. This comment has been removed by a blog administrator.

    ReplyDelete
  7. Thanks,
    Question : SIR is enable on my user and the RetainDeletedItemsFor is set by default to 14 days. It's the same time for Deletions and Purge folder ?
    If I let Exchange works alone, he keep the message during 28 days (14 days in Deletion folder and 14 days in Purge folder) or only 14 days for purge and deletion folders ?

    ReplyDelete
  8. Ivan, sorry I did not see your question right away. Unfortunately, most of the comments are spam...

    I would say 28 days but I would test this to make sure.

    ReplyDelete
  9. Awesome explanation. Thank you very much
    Rugved

    ReplyDelete
  10. Thank You for this explanation, all is more clear in my mind :)

    ReplyDelete