Saturday, January 4, 2014

Windows Server 2012 - Active Directory - NTDSUTIL, part 2

In this second post about NTDSUTIL, I'm going to examine three other tasks we can perform with this tool:

1. Resetting the Directory Services Restore Mode (DSRM) password.
2. Finding duplicate SIDs.
3. Defragmenting the ntds.dit file (the Active Directory database).

 Resetting the DSRM password

When we promote a server to domain controller we are prompted to enter a password for the "Directory Services Restore Mode". We will need this password if we ever need to boot into "DSRM". True, with restartable directory services, we can perform many operations without having to boot into DSRM, offline defragmentation for example. But there is at least one case where we still have to boot into DSRM: restoring all or part of the Active Directory database. So what if we do not remember the DSRM password? What if we start to manage an Active Directory environment where documentation is limited or even non-existent? In this situation, we can reset the DSRM password in ntdsutil as follows (the words in bold are the commands that we must enter):

PS C:\> ntdsutil
C:\Windows\system32\ntdsutil.exe: set dsrm password
Reset DSRM Administrator Password: reset password on server DC5
Please type password for DS Restore Mode Administrator Account: **********
Please confirm new password: **********
Password has been set successfully.
Reset DSRM Administrator Password: quit
C:\Windows\system32\ntdsutil.exe: quit

Note: in this case, we do not have to "activate" an "instance" of ntds as we do for other ntdsutil commands.

Finding duplicate SIDs

Users (and other objects) in Active Directory have what is known as a security identifier or "SID". This is a property of the object that represents them in Active Directory and regulates access to resources in the network. When a user account is authenticated, it is granted a "token" that contains the SID not only for the user account itself but also the SIDs of the groups of which the user is a member. This token - and more precisely the SIDs that it contains - are compared to the Access Control Entries (ACE) of the Access Control Lists (ACL) to determine if the user should have access to the resource and if so, what level of access.

It is important, therefore, that Active Directory does not have duplicate SIDs, that is, objects with the same security identifier. If that were the case, it would be difficult to regulate access to resources if two users had the same SID. Windows security only examines the SID and is not programmed to consider other elements to distinguish between two objects with the same identifier.

In normal circumstances, duplicate SIDs should not exist. The RID master (one of the FSMO roles - or operations masters) distributes pools of "relative IDs" to the other domain controllers, each of which should thus have a distinct range of IDs to use when creating SIDs for Active Directory objects such as users, groups and computers. If one does not mismanage the RID Master, by restoring a failed domain controller after seizing the role on another, for example, there should not be duplicates.

We can check for duplicate SIDs all the same with the following set of commands:

PS C:\> ntdsutil
C:\Windows\system32\ntdsutil.exe: security account management
Security Account Maintenance: connect to server DC5
Security Account Maintenance: check duplicate sid
Duplicate SID check completed successfully. Check dupsid.log for any duplicates
Security Account Maintenance: quit
C:\Windows\system32\ntdsutil.exe: quit
PS C:\>
PS C:\> gci

    Directory: C:\

Mode          LastWriteTime     Length  Name
----                -------------           ------     ----
d----         7/26/2012   3:44 AM            PerfLogs
d-r--        12/30/2013  11:12 PM           Program Files
d----         7/26/2012   4:04 AM            Program Files (x86)
d-r--        12/31/2013   8:40 PM            Users
d----          1/1/2014   9:56 PM             Windows
-a---          1/3/2014   8:51 PM             0 dupsid.log

PS C:\>

In my case, I opened the dupsid.log (which is created at the root of the C: drive) and discovered no duplicate SIDs. But what if I had?

There is another command that will remove the duplicate SIDs:

cleanup duplicate sid

Which... as the Microsoft documentation explains "deletes all objects that have duplicate SIDs and logs these entries into the log file."

But what if these objects are users, groups or computers?

It looks like those objects would have to be recreated - unless the organization was willing to accept the existence of duplicate SIDs, which would be a fundamental security flaw.

Offline defragmentation of ntds.dit

The ntds.dit is defragmented online twice a day. Online defragmenation can create usable free space within the ntds.dit after deletions are made. New objects can be created in this free space without increasing the size of the ntds.dit file itself - until all the free space (sometimes called "white space") is used. At this point, the size of the ntds.dit file will increase again.
It is possible that the free space is so significant that it is wasting disk space on the volume where the ntds.dit file resides. In this case, we can opt to recover this free space with offline defragmentation.
But how can we tell how much free space is available? How can we tell if an offline defragmentation of the database would even be beneficial?
We can adjust a value in the registry that will cause Active Directory to log " Event ID 1646" when online defragmentation is complete. We set value 6 (Garbage Collection) to 1 at the following location in the registry:
The EventID 1646 entry will indicate, for example:

Free hard disk space (megabytes)
Total allocated hard disk space (megabytes):

If an offline defragmentation appears worthwhile, we can proceed as outlined in the following lines.

First, we want to determine the size of the existing ntds.dit database. By default, this file is located here:


We can observe the size of the file as shown in Windows Explorer or the command line output shown later.

Second, it is recommended to perform a backup of the ntds.dit database. With Windows Server 2012, either a Full Server Backup (-allcritical backup at the command line) or System State backup will achieve this.

Third, we must stop the NTDS service before the offline defragmentation can begin. I'll use the new Powershell cmdlets to do this:

PS C:\> stop-service ntds -force
Fourth, we need to make a folder where ntdsutil can place the defragmented database. In fact, a brand new database will be created. We will have to copy this new database to the location of the original ntds.dit database, moving, deleting or overwriting the older copy as preferred. Besides a Full Server Backup, I opted to copy the old database to another location before overwriting it with the defragmented copy (not shown here).
We perform the defragmentation itself with the following commands (in bold):


PS C:\> ntdsutil
C:\Windows\system32\ntdsutil.exe: activate instance ntds
Active instance set to "ntds".
C:\Windows\system32\ntdsutil.exe: files
file maintenance: info

Drive Information:
        C:\ NTFS (Fixed Drive  ) free(38.5 Gb) total(49.6 Gb)
DS Path Information:
        Database   : C:\Windows\NTDS\ntds.dit - 40.1 Mb
        Backup dir : C:\Windows\NTDS\dsadata.bak
        Working dir: C:\Windows\NTDS
        Log dir    : C:\Windows\NTDS - 40.0 Mb total
                        edbres00002.jrs - 10.0 Mb
                        edbres00001.jrs - 10.0 Mb
                        edb00006.log - 10.0 Mb
                        edb.log - 10.0 Mb
file maintenance:
file maintenance: compact to c:\windows\ntds\ntds-defrag

Initiating DEFRAGMENTATION mode...
     Source Database: C:\Windows\NTDS\ntds.dit
     Target Database: c:\windows\ntds\ntds-defrag\ntds.dit

                  Defragmentation  Status (% complete)
          0    10   20   30   40   50   60   70   80   90  100

It is recommended that you immediately perform a full backup
of this database. If you restore a backup made before the
defragmentation, the database will be rolled back to the state
it was in at the time of that backup.

Compaction is successful. You need to:
   copy "c:\windows\ntds\ntds-defrag\ntds.dit" "C:\Windows\NTDS\ntds.dit"
and delete the old log files:
   del C:\Windows\NTDS\*.log

file maintenance: info
Drive Information:
        C:\ NTFS (Fixed Drive  ) free(38.4 Gb) total(49.6 Gb)
DS Path Information:
        Database   : C:\Windows\NTDS\ntds.dit - 50.1 Mb
        Backup dir : C:\Windows\NTDS\dsadata.bak
        Working dir: C:\Windows\NTDS
        Log dir    : C:\Windows\NTDS - 40.0 Mb total
                        edbres00002.jrs - 10.0 Mb
                        edbres00001.jrs - 10.0 Mb
                        edb00006.log - 10.0 Mb
                        edb.log - 10.0 Mb
file maintenance: quit
C:\Windows\system32\ntdsutil.exe: quit
PS C:\>


At this point, as directed in the red print above, we copy the defragmented ntds.dit file to the original location and thus replace the old fragmented ntds.dit file.
The reader may have noticed that the size of the ntds.dit database has increased by 10 MB when we expected it to either decrease or remain the same (because of the very small number of objects in what is, essentially, a test database with some test users). This is also evident in a screenshot of the database as shown in Windows Explorer:
I'm not sure why the file size increased. If anyone knows, there is a comment section at the end of this post. Please feel free to contribute!
Otherwise, we should now backup the System State of this domain controller once again so if we need to restore the database in the near future, we will be able to restore the defragmented copy of the file.


  1. I would like to share with you my experience, I went to to buy a key, to my surprise, their attitude is very good, but the key work is normal, there is very little money I spent, very happy the first purchase and recommend it to you.