Migrating SQL Server Databases that use Database Master Keys

There’s a lot of things to consider when migrating databases between SQL Server instances. I want to provide some tips for dealing with Database Master Keys, and in particular the case when those keys are encrypted by the Service Master Key.

Before getting into the details, let’s take a quick overview of Service Master Keys, Database Master Keys and how they interact.

Service Master Key: At the top of the key hierarchy is the Service Master Key. There is one per SQL Server instance, it is a symmetric key, and it is stored in the master database. Used to encrypt Database Master Keys, Linked Server passwords and Credentials it is generated at first SQL Server startup.

There are no user configurable passwords associated with this key – it is encrypted by the SQL Server service account and the local machine key. On startup SQL Server can open the Service Master Key with either of these decryptions. If one of them fails – SQL Server will use the other one and ‘fix’ the failed decryption (if both fail – SQL Server will error). This is to account for situations like clusters where the local machine key will be different after a failover. This is also one reason why service accounts should be changed using SQL Server Configuration Manager – because then the Service Master Key encryption is regenerated correctly.

Service Master Keys can be manually regenerated using this statement:

alter service master key regenerate

This will decrypt and re-encrypt all secrets encrypted with the key.

The Service Master Key is backed up with the master database, but can be backed up and restored independently:

backup service master key to file = N'<filepath and filename>'
encryption by password = '<password>'
go

restore service master key from file = N'<filepath and filename>'
decryption by password = '<password>'
go

The Service Master Key properties are available by running the following query in the master database:

use master
go
select * from sys.symmetric_keys

Database Master Key: This is a database scoped symmetric key that is encrypted by the Service Master Key and stored in the database. It can be used to encrypt certificates and/or asymmetric keys within the database. Unlike the Service Master Key which is generated automatically, a Database Master Key must be created with DDL:

use <database>
go
create master key encryption by password = '<password>'

The Database Master Key can also be examined in the system catalog:

use <database>
go
select * from sys.symmetric_keys

Database Master Keys are backed up and restored with the database, but can also be backed up and restored independently.

use <database>
go
backup master key to file = N'<filepath and filename>'
	encryption by password = '<file password>'

restore master key from file = N'<filepath and filename>'
	decryption by password = '<file password>'
	encryption by password = '<new encryption password>'

There is one Database Master Key per database and it must be protected by a password. The Database Master Key can be encrypted by multiple passwords and any of these can be used to de-crypt the key. By default the Database Master Key is also encrypted by the Service Master Key, this can be switched off if needed.

Migrating Databases: If a database makes use of a Database Master Key, and that key is encrypted by the Service Master Key, then this encryption will need to be regenerated on the destination instance after any migration. Here are the basic steps.

  1. Firstly verify whether there are any Database Master Keys encrypted by the Service Master Key:
    select name
    from sys.databases
    where is_master_key_encrypted_by_server = 1
  2. (Optional) If you don’t know a valid password for the Database Master Key you can create a new one. (Remember that multiple passwords can encrypt the DMK)
    use <database>
    go
    alter master key 
    add encryption by password = 'migration_password'
    go
  3. Drop the encryption by the Service Master Key:
    use <database>
    go
    alter master key drop encryption by service master key
    go
  4. Migrate the database using either backup and restore, or detach and attach.
  5. Open the Database Master Key with a password (this could be the password created at step 2) and re-activate the encryption by Service Master Key – this will be mapped to the SMK on the new SQL instance:
    use <database>
    go
    open master key decryption by password = '<Password>'
    alter master key add encryption by service master key
    go
  6. (Optional) If you created a password specifically for the migration in step 2, then you should drop it:
    use <database>
    go
    alter master key 
    drop encryption by password = 'migration_password'
    go

Conclusion: The encryption, by default, of the Database Master Key by the Service Master Key means that it can easily go unnoticed by a DBA with sys admin privileges. My intention with this post was to provide a brief overview of the two keys, provide some statements used to modify and examine the keys, and outline the steps involved in migrating a database with a Database Master Key encrypted by the Service Master Key.

This entry was posted in Database options, Disaster Recovery, Security, SQL Server, T-SQL and tagged , , . Bookmark the permalink.

34 Responses to Migrating SQL Server Databases that use Database Master Keys

  1. cinofilo says:

    This is the best post I ever read about backup/restore and migration of a sql server db.
    Thank you so much

  2. testt2Bright says:

    Excellent article, complex encryption topic explained very clearly. Bravo, and keep up the good work!

  3. Federico says:

    Hi: this is definitely extremely useful. I am wondering why nothing is said about backing up and restoring the Symmetric Key. I have to pull 5 symmetric keys, certificates, database master key and a service master key, from a production environment an install/restore to a similar test environment; I haven’t come across any post that indicates the backing up of the Symmetric Key and its restore process… The point is that when I create the symmetric key over from scratch, it is not the same key as was used originally to encrypt the fields I need to decrypt and it doesn’t work.

  4. David Mayorga says:

    Thank you for the article! Just one thing that’s bugging me, why did you have to drop the Service Master Key? What happens if I omit that step? If I wanted to duplicate my database to another server for testing purposes I would not want to drop the Service Key on the production server.

  5. matt.bowler says:

    Hi David,

    I don’t drop the service master key, I drop the encryption by the service master key. I do this because I’m moving the database to a new server which will have a new service master key, and I want the database to be encrypted by that key. I’ve seen the results of not doing this – and you get errors when trying to read the encrypted data because the decryption fails because the SMK is different. (If this wasn’t the case – breaking encryption would be as simple as restoring a database backup onto a SQL instance that we had sa privileges on!)

    These steps are for a database migration – when dealing with situations like mirroring or log-shipping things are a bit more complicated. In that case one option is to leave the encryption by service master key in place and restore that key to the destination server. This works but comes with its own set of risks/considerations…

    • So if you don’t drop the encryption by SMK, can you still open the key with the password on the new server? If so then can you drop it and re-add it on the new server so you don’t have to remove it on the old server before doing a backup?

  6. David Mayorga says:

    Sorry, my bad, I did mean to say that you dropped the encryption by SMK.
    So, my guess is that when you backup your production database there must be something in there that indicates whether or not the Database Master Key is encrypted by the SMK?. If there was an encryption by SMK and I restore the database to another server then obviously that server’s SMK would not be able to open the DMK, all this avoided by dropping encryption by SMK before backing up.
    But would it work just as well if on the server were I’m restoring I dropped the encryption by SMK and then re-added it? For this I’d open the DMK with the password.

  7. matt.bowler says:

    You are right – dropping and re-adding the encryption by SMK after the restore will have the same net effect. BUT – you must remember to at least add a password encryption before you take the backup or you just won’t be able to decrypt the database at all after the restore. IMHO its just a bit tidier at this point to drop the SMK encryption as well.

  8. David Mayorga says:

    Indeed, specially if you don’t know the password. Thank you!

  9. ET says:

    Best step by step guide i have read anywhere!!

  10. John says:

    In step 3, we will drop the encryption by the Service Master Key on the source server. Do we need to add it back? If yes, then when do we add it back? and how to add it back on the source server? Sorry I am new to all these encryption/decryption + migration things on different servers . Thanks a lot really for your helps.

    • matt.bowler says:

      The idea here is to sever the dependency between the database and the server, so that the database we backup (or detach and attach) is a ‘self contained’ entity. Once the database is attached to a new server then we can re-encrypt with the service master key. If you are keeping the source database in place then you can re-encrypt after you’ve taken a backup.

  11. Pingback: Шифрование резервных копий в SQL Server 2014 |

  12. Extremely well written guide on a complex subject. Thank you!

  13. Brenda says:

    What does the regenerate do? When would you use it?

    • matt.bowler says:

      Regenerate generates a new service master key. I would assume that the most likely use case for this would be if security had become compromised. And it is used behind the scenes when a service account is changed.

  14. Jisha Mathew says:

    Very detailed and clear instructions on the subject. Thank you very much for this helpful article.

    • Akhilesh says:

      What needs to be done for decrypting the data after restoring the database from production to development? Can we achieve this by restoring the smk to development from production?

  15. leu says:

    Hi
    Need help … There is dev server with sql server 2005 and two instances matching the prod. There is LINKSERVE used to connect from one instance to the other instance in the same server. Recently, we have changed the ‘service’ account and since then the LINKSERVE error out with ‘An error occurred during decryption :15466). There is no database master key used; however, when trying to generate the key with the ‘alter … key regenerate’ command, it has failed. Would you point me what to do in this situation?

  16. lolo says:

    Hi, I have been asked by the security service to create a database master key for each applicative database on sql instances but when I backuped the database and restored to a different server ,I was still able to access the data without encrypting or decrypting anything. I thought that the database master key prevent from access the data when restoring to a different server. If not, can you tell me what is the use of the database master key ?

    • matt.bowler says:

      Creating a database master key does not automatically encrypt anything. You need to then use that key to explicitly encypt data and or other security objects.

      • Akhilesh narayanan says:

        Dear, What are the optons to follow if we dont know the password of DMK and need to decrypt the data on a UAT instance after restoring the backup?
        Will it work if we backup and restore SMK to UAT instance?

  17. Pingback: Database Master Keys can have more than one password. | SQL Studies

  18. Arshak says:

    Thank you for the great article indeed. I have one question. Once I have created database and encrypt some columns with the symmetric key. Then toke a back up of this database and restore it to another instance of sql server 2014. What was surprised me that without any database master key password I could encrypt my data(with the same certificate). I could not understand… If I can encrypt my data just with certificate then why do I need master key? Could you please help?

  19. Arshak says:

    Thank you for the great article indeed. I have one question. Once I have created database and encrypt some columns with the symmetric key. Then toke a back up of this database and restore it to another instance of sql server 2014. What was surprised me that without any database master key password I could encrypt my data(with the same certificate). I could not understand… If I can encrypt my data just with certificate then why do I need master key? Could you please help?

  20. Wells says:

    Thank you so much for the article. This topic can be difficult to grasp. My question is how is encryption handled with log shipped databases? Both primary and secondary have their own service master key right? So, I need to back up the service master key on both primary and secondary? Does a database master key need to be only be created on the primary databases and do those keys get shipped to the secondary? I also read somewhere that system databases cannot be encrypted so I assume only user databases can have their own database master key. Is that correct? Sorry for so many questions. Just trying to grasp how encryption works with log shipping and/or mirroring.

  21. akhilesh narayanan says:

    Hi, What will be impact of option 3 execution in production instance as per below?
    alter master key drop encryption by service master key
    also would like to know why we are not able to decrpt the data after restoring the service master key from production to development?
    — akhilesh

  22. AM says:

    First of all, Thank you for the post. It is very useful. I am migrating a database with DMK to AlwaysOn Availability group and different service accounts run SQL services on both primary and secondary. I am not sure there is any issue if I restore SMK from primary to secondary where SQL server run by different service accounts? As you said, we cannot omit “alter master key drop encryption by service master key”. Since I am working with PROD db, how can I roll back after I drop encryption? Should I run “ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY;” ? Thank you!
    -AM

  23. kashif says:

    If i dont want encryption anymore on the database in the destination sever, can i simply follow step 3 on source server (drop encryption) and do nothing on the destination.

  24. babu says:

    Hi All,
    I have migrated one database that used database master key. I would like to know the what step i need to follow and i don’t want disturb the source server. Thanks in advance

    • akhilesh narayanan says:

      Backup SMK from source server and restore on target server and do the following if you know the password of DMK.
      use
      go
      open master key decryption by password = ”
      alter master key add encryption by service master key
      go

  25. Akhilesh Narayanan says:

    What is the difference of backing up DMK from production and restoring on development instance & Adding new DMK password on production and restoring production database on development?

  26. Akhilesh Narayanan says:

    If we know the DMK password without restoring the SMK backup to development box can we access the encrypted data?

    Yes, The following will unlink the encryption from SMK, the above question is without these steps.

    use [user_database]
    go
    alter master key drop encryption by service master key
    go

Leave a comment