Overview

The checkpoint process writes dirty pages from the buffer pool area of memory to physical disk. It does not remove the pages from the buffer pool instead it marks them as clean again.Its purpose is to reduce the active portion of the log necessary to preform recovery so the frequency at which the checkpoint process is run determines the length of time it will take to recover the database when it is started up.  This needs to be balanced against the overhead required to run the checkpoint process. Similarly to the lazy writer process, checkpoint runs as a background task. We can query the sysprocesses system table to view its status:

SELECT cmd, spid, status, dbid, login_time, waittime, lastwaittype,
loginame,cpu, memusage, physical_io
FROM sysprocesses
WHERE  cmd IN ('LAZY WRITER', 'CHECKPOINT')

By default SQL Server determines the interval at which the checkpoint process runs. This is usually every 1 minute. This behaviour can be changed by setting the recovery interval which is an advanced configuration option.

Generally I don’t change the default value of 0 which is to allow SQL Server control the checkpoint process.

Checkpoint and the buffer pool

I want to see exactly what the checkpoint process does and how this impacts on the buffer pool. I have setup a test database called DBA (DB_ID = 7)with a table called tblTest. This is a small table with a couple of rows. I have already determined that it is using page 222. The first thing I want to do is clear the page from cache. TO do this I am going to call the checkpoint statement to clean the page and then I am going to call DBCC DROPCLEANBUFFERS  to remove all clean pages form the buffer pool.  Since I have set the recovery interval to 1000 minutes the automatic checkpoint process should not kick in. Just to be sure I am also going to turn on Trace flag  3502. This will log all checkpoint activity in the SQL Server Log.

 1: DBCC TRACEON (3502, -1);

 2:

 3: GO

 4:

 5: CHECKPOINT;

 6:

 7: GO

 8:

 9: DBCC DROPCLEANBUFFERS;

I can verify that the page is not in the buffer pool by using the dm_os_buffer_descriptors  dmv.

 1: select * from sys.dm_os_buffer_descriptors

 2: where page_id = 222 and database_id = 7

Next I am going to bring the tblTest into the buffer pool and dirty it., I am going to do this by updating all the existing rows in the table.

I can verify that the page is now in the buffer pool and that the page has been modified (dirty). The is_modiifed column in sys.dm_os_buffer_discriptors indicates the status of the page (clean / dirty as 1/0)

 1: select * from sys.dm_os_buffer_descriptors

 2: where page_id = 222 and database_id = 7

When we run a checkpoint again the page should remain in the buffer pool but it should now be clean

 1: CHECPOINT;

 2:

 3: GO

 4:

 5: select * from sys.dm_os_buffer_descriptors

 6: where page_id = 222 and database_id = 7

This shows that the checkpoint process cleaned the page and left it in the buffer pool

Checkpoint and the transaction log

I am not going to get into Transaction Log management. For the purposes of the demo I setup the following

• The database is in the simple recovery mode
• There were no outstanding active transactions
• No other transaction log dependant features such as replication or DB mirroring were setup on the box.

I have a database setup and I have run a number of transactions against it. The recovery interval has been set to 1000 minutes. I am going to use DBCC LOGINFO and the undocumented

 1:

 2:

 3: DBCC LOGINFO;

 4:

 5:

 6: SELECT [Current LSN],[previous LSN], [Operation], context,

 7: [Checkpoint Begin], [checkpoint end], [page id], [minimum lsn],

 8: Dirty Pages] spid, [Begin Time], [Transaction Name],

 9: [End Time], [Lock Information],Description

 10: FROM ::fn_dblog(NULL, NULL) ;

 11:

 1: checkpoint;

 2:

 3: dbcc loginfo;

 4:

 5: SELECT [Current LSN],[previous LSN], Operation, context,

 6: [Checkpoint Begin], [checkpoint end], [page id], [minimum lsn],

 7: [Dirty Pages] spid, [Begin Time], [Transaction Name], [End Time],

 8: [Lock Information], Description

 9: FROM ::fn_dblog(NULL, NULL)

So when does checkpoint run

When the database is shutdown a checkpoint is run against each database. The only exception to this is when shutdown is called with the no wait option or the server crashes.

The checkpoint process is run before each backup

The transaction exceeds maximum’s for the active portion

exec xp_fixeddrives

To setup the mirroring status check, setup a WMI Alert that queries DATABASE_MIRRORING_STATE_CHANGE.

Add a new alert and select WMI Alert from the Type box. The namespace should automatically be populated with the correct setting. The following example should be pasted into the Query box. It checks the status of three mirrored databases. Change the database names accordingly:-

SELECT *
FROM DATABASE_MIRRORING_STATE_CHANGE
WHERE ((State = 7 or State = 8) AND DatabaseName = 'Database1') OR ((State = 7 or State = 8) AND DatabaseName = 'Database2')
OR ((State = 7 or State = 8) AND DatabaseName = 'Database3')

State 7 indicates a Manual Failover whilst state 8 indicates an Automatic Failover. If any database now fails over then the alert should fire.

Obviously we need to setup an appropriate response for the alert. Create an Agent job containing the following SQL task:-

if exists (select database_id
from sys.database_mirroring
where db_name(database_id) = 'Database1'
and mirroring_role_desc = 'principal')
ALTER DATABASE Database1 SET PARTNER FAILOVER
GO

if exists (select database_id
from sys.database_mirroring
where db_name(database_id) = 'Database2'
and mirroring_role_desc = 'principal')
ALTER DATABASE Database2 SET PARTNER FAILOVER
GO

if exists (select database_id
from sys.database_mirroring
where db_name(database_id) = 'Database3'
and mirroring_role_desc = 'principal')
ALTER DATABASE Database3 SET PARTNER FAILOVER
GO

The above code checks to see whether any mirrored databases are still set as principal and automatically fails them over. Open up the Alert in SQL Agent and select this newly created job to execute if the alert is fired. Now, if one or more of the databases fails over, the remaining databases will also do so.

The issue I have with the aforementioned solution is that it relies on continually calling a job to check the mirroring status of that database. Therefore, wherever possible, I add the following to each job step:-

If Exists (select * from sys.database_mirroring
dm join sys.databases d on (dm.database_id=d.database_id)
where (d.name=N'Your_Mirrored_Database')and mirroring_role_desc <> 'Principal') PRINT 'Mirrored Version of Database. No Action Taken.'
ELSE
BEGIN
    EXEC MyBackupProc
END

The mirroring status (mirroring_role_desc) of the database is tested before calling the applicable stored procedure. If the status is found to not be ‘principal’, then the procedure is not called and a simple text message is printed. With the ‘Backup’ Agent job being present on both servers, only one job will fire the procedure; the other will simply print out the message which can then be viewed within the job history. No more errors!

The following example demonstrates the process. An Audit table is created to store the results. An UPDATE is then issued against a number of columns. OUTPUT writes the old and new versions of the data (along with the id of the row changed and the date and time) to the audit table. Adding the output clause to stored procedures will allow you to audit all data changes in your database, although there will obviously be some overhead due to additional writes.

CREATE TABLE Audit_Table (CustomerID varchar(12),
                         OldAddress1 varchar(50),
                         OldAddress2 varchar(50),
                         OldAddress3 varchar(50),
                         OldAddress4 varchar(50),
                         OldAddress5 varchar(50),
                         OldPostCode varchar(12),
                         NewAddress1 varchar(50),
                         NewAddress2 varchar(50),
                         NewAddress3 varchar(50),
                         NewAddress4 varchar(50),
                         NewAddress5 varchar(50),
                         NewPostCode varchar(12),
                         DateChanged datetime);

UPDATE CustomerTable
SET Address1 = '99 My Road',
      Address2 = 'My Area',
      Address3 = 'My City',
      Address4 = 'My County',
      Address5 = 'My Country',
      PostCode = 'AA1 1AA'
OUTPUT INSERTED.CustomerID,
	    DELETED.Address1 AS 'OldAddress1',
	    INSERTED.Address1 AS 'NewAddress1',
	    DELETED.Address2 AS 'OldAddress2',
	    INSERTED.Address2 AS 'NewAddress2',
	    DELETED.Address3 AS 'OldAddress3',
	    INSERTED.Address3 AS 'NewAddress3',
	    DELETED.Address4 AS 'OldAddress4',
	    INSERTED.Address4 AS 'NewAddress4',
	    DELETED.Address5 AS 'OldAddress5',
	    INSERTED.Address5 AS 'NewAddress5',
	    DELETED.stu_capc AS 'OldPostCode',
	    INSERTED.stu_capc AS 'NewPostCode',
	    GETDATE() AS 'DateChanged'
INTO AuditTable
WHERE CustomerTable.CustomerID = 'AB123456';

Find the path of any existing traces by executing the following:-

select path 
from sys.traces

Once the path and file name are known, run the following SQL:-

-- Load contents of trace into a SQL temporary table
-- Do not specify log numbers to load all log files for a trace
SELECT * INTO ##temp_trc
FROM fn_trace_gettable('C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\LOG\log.trc', default);

SELECT * FROM ##temp_trc

The above example loads all trace files like log%.trc [log1.trc, log2.trc etc] into a temporary table. You can of course load into a physically stored table by removing the hashes. If you want to load data from just one trace file from a group of rolled over trace files then specify the number on the end [e.g. C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\LOG\log7.trc].

SELECT SUSER_NAME() AS 'Login', USER_NAME() AS 'User'

Or try specifying an ID. ID of 1 should return SA.

SELECT SUSER_NAME(1) AS 'Login'

Severity Level 19: SQL Server Error in Resource

These messages indicate that some nonconfigurable internal limit has been exceeded and the current batch process is terminated. Severity level 19 errors occur rarely; however, they must be corrected by the system administrator or your primary support provider. The administrator should be informed every time a severity level 19 message occurs.

Severity Levels 20 through 25

Severity levels from 20 through 25 indicate system problems. These are fatal errors, which means that the process (the program code that accomplishes the task specified in your statement) is no longer running. The process freezes before it stops, records information about what occurred, and then terminates. The client connection to SQL Server closes, and depending on the problem, the client might not be able to reconnect.

Error messages with a severity level of 19 or higher stop the current batch. Errors messages with a severity level of 20 or higher are considered fatal errors and terminate the client connection. Errors messages in this range may affect all of the processes in the database, and may indicate that a database or object is damaged. Error messages with a severity level from 19 through 25 are written to the error log.

Severity Level 20: SQL Server Fatal Error in Current Process

These messages indicate that a statement has encountered a problem. Because the problem has affected only the current process, it is unlikely that the database itself has been damaged.

Severity Level 21: SQL Server Fatal Error in Database (dbid) Processes

These messages indicate that you have encountered a problem that affects all processes in the current database; however, it is unlikely that the database itself has been damaged.

Severity Level 22: SQL Server Fatal Error Table Integrity Suspect

These messages indicate that the table or index specified in the message has been damaged by a software or hardware problem.

Severity level 22 errors occur rarely; however, if you should encounter one, run DBCC CHECKDB to determine if other objects in the database are also damaged. It is possible that the problem is in the cache only and not on the disk itself. If so, restarting SQL Server corrects the problem. To continue working, you must reconnect to SQL Server. Otherwise, use DBCC to repair the problem. In some cases, it may be necessary to restore the database.

If restarting does not help, the problem is on the disk. Sometimes destroying the object specified in the error message can solve the problem. For example, if the message tells you that SQL Server has found a row with a length of 0 in a nonclustered index, delete the index and rebuild it.

Severity Level 23: SQL Server Fatal Error: Database Integrity Suspect

These messages indicate that the integrity of the entire database is in question because of a hardware or software problem.

Severity level 23 errors occur rarely; however, if you should encounter one, run DBCC CHECKDB to determine the extent of the damage. It is possible that the problem is in the cache only and not on the disk itself. If so, restarting SQL Server corrects the problem. To continue working, you must reconnect to SQL Server. Otherwise, use DBCC to repair the problem. In some cases, it may be necessary to restore the database.

Severity Level 24: Hardware Error

These messages indicate some type of media failure. The system administrator might have to reload the database. It might also be necessary to call your hardware vendor.

Setting up Alerts

You should set up alerts in SQL Server Agent for any errors that have a severity level of 19 or above. Use the following script to generate the alerts, ensuring you replace “Operator Name” with the appropriate name of the operator to notify by email:-

USE [msdb]
GO

EXEC msdb.dbo.sp_add_alert @name=N'019 - Fatal Error In Resource',
		@message_id=0,
		@severity=19,
		@enabled=1,
		@delay_between_responses=1800,
		@include_event_description_in=1,
		@category_name=N'[Uncategorized]',
		@job_id=N'00000000-0000-0000-0000-000000000000'
GO

EXEC msdb.dbo.sp_add_alert @name=N'020 - Fatal Error in Current Process',
		@message_id=0,
		@severity=20,
		@enabled=1,
		@delay_between_responses=1800,
		@include_event_description_in=1,
		@category_name=N'[Uncategorized]',
		@job_id=N'00000000-0000-0000-0000-000000000000'
GO

EXEC msdb.dbo.sp_add_alert @name=N'021 - Fatal Error in Database Processes',
		@message_id=0,
		@severity=21,
		@enabled=1,
		@delay_between_responses=1800,
		@include_event_description_in=1,
		@category_name=N'[Uncategorized]',
		@job_id=N'00000000-0000-0000-0000-000000000000'
GO

EXEC msdb.dbo.sp_add_alert @name=N'022 - Fatal Error: Table Integrity Suspect',
		@message_id=0,
		@severity=22,
		@enabled=1,
		@delay_between_responses=1800,
		@include_event_description_in=1,
		@category_name=N'[Uncategorized]',
		@job_id=N'00000000-0000-0000-0000-000000000000'
GO

EXEC msdb.dbo.sp_add_alert @name=N'023 - Fatal Error: Database Integrity Suspect',
		@message_id=0,
		@severity=23,
		@enabled=1,
		@delay_between_responses=1800,
		@include_event_description_in=1,
		@category_name=N'[Uncategorized]',
		@job_id=N'00000000-0000-0000-0000-000000000000'
GO

EXEC msdb.dbo.sp_add_alert @name=N'024 - Fatal Error: Hardware Error',
		@message_id=0,
		@severity=24,
		@enabled=1,
		@delay_between_responses=1800,
		@include_event_description_in=1,
		@category_name=N'[Uncategorized]',
		@job_id=N'00000000-0000-0000-0000-000000000000'
GO

EXEC msdb.dbo.sp_add_alert @name=N'025 - Fatal Error',
		@message_id=0,
		@severity=25,
		@enabled=1,
		@delay_between_responses=1800,
		@include_event_description_in=1,
		@category_name=N'[Uncategorized]',
		@job_id=N'00000000-0000-0000-0000-000000000000'
GO

EXEC msdb.dbo.sp_add_notification @alert_name=N'019 - Fatal Error In Resource', @operator_name=N'Operator Name', @notification_method = 1
GO

EXEC msdb.dbo.sp_add_notification @alert_name=N'020 - Fatal Error in Current Process', @operator_name=N'Operator Name', @notification_method = 1
GO

EXEC msdb.dbo.sp_add_notification @alert_name=N'021 - Fatal Error in Database Processes', @operator_name=N'Operator Name', @notification_method = 1
GO

EXEC msdb.dbo.sp_add_notification @alert_name=N'022 - Fatal Error: Table Integrity Suspect', @operator_name=N'Operator Name', @notification_method = 1
GO

EXEC msdb.dbo.sp_add_notification @alert_name=N'023 - Fatal Error: Database Integrity Suspect', @operator_name=N'Operator Name', @notification_method = 1
GO

EXEC msdb.dbo.sp_add_notification @alert_name=N'024 - Fatal Error: Hardware Error', @operator_name=N'Operator Name', @notification_method = 1
GO

EXEC msdb.dbo.sp_add_notification @alert_name=N'025 - Fatal Error', @operator_name=N'Operator Name', @notification_method = 1
GO

To check to see whether a default trace is indeed currently running, execute the following:-

SELECT * 
FROM sys.configurations 
WHERE configuration_id = 1568

If the value is ’1′ then the trace is running. Once you have established that the trace is enabled, you need to run a query to determine the most recent trace file. Trace files automatically rollover, so you need to know which one to query.

SELECT traceid, value 
FROM [fn_trace_getinfo](default) 
WHERE [property] = 2;

The full path and file name will be returned. You can then run the following query and include the correct path to the most recent file. You could of course setup a variable to automatically populate the path.

SELECT *  
FROM [fn_trace_gettable]('C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\LOG\log_49.trc', DEFAULT) 
ORDER BY StartTime;

You can disable the trace by issuing the following commands. However, don’t expect a dramatic increase in server performance as the trace uses only minimal resources.

EXEC master.dbo.sp_configure 'allow updates', 1;
GO 
EXEC master.dbo.sp_configure 'show advanced options', 1;
GO 
EXEC master.dbo.sp_configure 'default trace enabled', 0;
GO
RECONFIGURE WITH OVERRIDE;
GO
EXEC master.dbo.sp_configure 'show advanced options', 0;
GO
EXEC master.dbo.sp_configure 'allow updates', 0;
GO

/*Perform a 'USE <database name>' to select the database in which to run the script.*/
-- Declare variables
SET NOCOUNT ON;
DECLARE @tablename varchar(255);
DECLARE @execstr   varchar(400);
DECLARE @objectid  int;
DECLARE @indexid   int;
DECLARE @frag      decimal;
DECLARE @maxfrag   decimal;

-- Decide on the maximum fragmentation to allow for.
SELECT @maxfrag = 30.0;

-- Declare a cursor.
DECLARE tables CURSOR FOR
   SELECT TABLE_SCHEMA + '.' + TABLE_NAME
   FROM INFORMATION_SCHEMA.TABLES
   WHERE TABLE_TYPE = 'BASE TABLE';

-- Create the table.
CREATE TABLE #fraglist (
   ObjectName char(255),
   ObjectId int,
   IndexName char(255),
   IndexId int,
   Lvl int,
   CountPages int,
   CountRows int,
   MinRecSize int,
   MaxRecSize int,
   AvgRecSize int,
   ForRecCount int,
   Extents int,
   ExtentSwitches int,
   AvgFreeBytes int,
   AvgPageDensity int,
   ScanDensity decimal,
   BestCount int,
   ActualCount int,
   LogicalFrag decimal,
   ExtentFrag decimal);

-- Open the cursor.
OPEN tables;

-- Loop through all the tables in the database.
FETCH NEXT
   FROM tables
   INTO @tablename;

WHILE @@FETCH_STATUS = 0
BEGIN
-- Do the showcontig of all indexes of the table
   INSERT INTO #fraglist 
   EXEC ('DBCC SHOWCONTIG (''' + @tablename + ''') 
      WITH FAST, TABLERESULTS, ALL_INDEXES, NO_INFOMSGS');
   FETCH NEXT
      FROM tables
      INTO @tablename;
END;

-- Close and deallocate the cursor.
CLOSE tables;
DEALLOCATE tables;

-- Declare the cursor for the list of indexes to be defragged.
DECLARE indexes CURSOR FOR
   SELECT ObjectName, ObjectId, IndexId, LogicalFrag
   FROM #fraglist
   WHERE LogicalFrag >= @maxfrag
      AND INDEXPROPERTY (ObjectId, IndexName, 'IndexDepth') > 0;

-- Open the cursor.
OPEN indexes;

-- Loop through the indexes.
FETCH NEXT
   FROM indexes
   INTO @tablename, @objectid, @indexid, @frag;

WHILE @@FETCH_STATUS = 0
BEGIN
   PRINT 'Executing DBCC INDEXDEFRAG (0, ' + RTRIM(@tablename) + ',
      ' + RTRIM(@indexid) + ') - fragmentation currently '
       + RTRIM(CONVERT(varchar(15),@frag)) + '%';
   SELECT @execstr = 'DBCC INDEXDEFRAG (0, ' + RTRIM(@objectid) + ',
       ' + RTRIM(@indexid) + ')';
   EXEC (@execstr);

   FETCH NEXT
      FROM indexes
      INTO @tablename, @objectid, @indexid, @frag;
END;

-- Close and deallocate the cursor.
CLOSE indexes;
DEALLOCATE indexes;

-- Delete the temporary table.
DROP TABLE #fraglist;
GO

SELECT [name] AS 'Table', 
[create_date] AS 'Creation Date', 
[modify_date] AS 'Modification Date' 
FROM sys.objects
WHERE type_desc = 'USER_TABLE' 
ORDER BY [modify_date] DESC

Firstly make a backup of the existing database and restore it on the destination Server.

Next create a project in Integration Services (IS) and use the ‘transfer logins task’ to move the logins, to complete this the following properties need populating :

1. Setup source connection
2. Setup destination connection
3. for LoginsToTransfer Specify SelectedLogins
4. Choose the logins to move
5. Save and Execute the task

Ensure  all the logins on the source server  have all their attributes populated. When I initially ran the the IS package I got the error message “This property may not exist for this object, or may not be retrievable due to insufficient access rights” I traced the cause to   not having a default database associated with the login, this error message appeared for all logins that were missing the attribute, even if they weren’t been moved.

Once the IS task completes successfully;

On the destination server, for each of the SQL Server logins Execute the following TSQL code. I did not have to do this for the NT logins.

USE <Database Name>
GO
EXEC sp_change_users_login ‘UPDATE_ONE’, ‘<username>‘, ‘<login name>‘
GO

References

http://sqlblog.com/blogs/eric_johnson/archive/2008/10/17/fixing-orphaned-users.aspx