DBA Scripts: Run a Full SQL Server Health Check

🔧Part of the DBA-Tools Project, copy/paste SQL Server scripts and health checks.In: Maintenance & Automation › Collectors & Baselines

Inherited a server, or sure something is wrong but not what? One command runs 45 read-only checks against the instance. A second turns the results into a ranked CRITICAL / WARNING / INFO list. An optional third hands the lot to an AI to find the root causes.


Step 1: Collect

git clone https://github.com/peterwhyte-lgtm/dba-tools
cd dba-tools
.\Initialize-Environment.ps1

.\powershell\reporting\Invoke-HealthCheckCollection.ps1 -ServerInstance . -Quiet

This runs 45 scripts against the instance, one per area: inventory, backups, waits, security, TempDB, indexes, Agent jobs, and High Availability (Always On replica state, failover readiness and latency, mirroring endpoints, replication, FCI node blips). Each result is saved as a CSV in a timestamped folder under output-files\healthcheck\.

All 45 scripts carry a SAFE:ReadOnly header, so nothing in the collection changes the instance. Leave off -Quiet to see each script’s path and connection details as it runs. Run it as sysadmin.

PowerShell window running Invoke-HealthCheckCollection against the local instance without -Quiet. After the banner with server, output folder and time, each script prints its path, server, Windows authentication, database, output format and CSV path, then an OK line: server-info first, then os-hardware.
Permissionssysadmin: all 45 run.
VIEW SERVER STATE, VIEW ANY DATABASE and VIEW ANY DEFINITION: 8 fail. 3 read the Agent job tables in msdb, 3 call xp_readerrorlog, plus autogrowth history (the default trace) and database sizes (a database the login cannot open).
CONNECT SQL only: 23 fail.
A failed script shows [FAILED] with the error under it, and the run carries on.
Invoke-HealthCheckCollection on the lab instanceexample output, not something to copy
============================================ DBA Health Check Collection ============================================ Server : . Output : C:\Users\Peter\my-data\dba-tools\output-files\healthcheck\.-yyyyMMdd-174817 Time : 17:48:17 -------------------------------------------- [OK] server-info [OK] os-hardware [OK] database-health [OK] database-sizes [OK] database-files [OK] backup-times [OK] backup-coverage [OK] tlog-usage [OK] memory-config [OK] wait-stats [OK] active-sessions [OK] tempdb-usage [OK] job-failures [OK] recent-errors [OK] dbcc-checkdb [OK] suspect-pages [OK] io-usage [OK] disk-space [OK] growth-risk [OK] security-surface-area [OK] weak-logins [OK] missing-indexes [OK] tempdb-config [OK] plan-cache [OK] linked-server-security [OK] vlf-count [OK] maintenance-jobs [OK] failed-logins [OK] query-store-status [OK] extended-events [OK] cdc-and-ct [OK] service-broker [OK] sysadmin-members [OK] instance-config [OK] top-cpu-queries [OK] agent-jobs-overview [OK] orphaned-users [OK] certificate-expiry [OK] autogrowth-history [OK] ag-replica-state [OK] ag-failover-readiness [OK] ag-latency [OK] mirroring-endpoint-health [OK] replication-status [OK] last-node-blip -------------------------------------------- OK: 45 | Failed: 0 | Skipped: 0

All 45 ran clean. 3 came back empty: suspect pages, CDC and Change Tracking, and orphaned users. That is not a failure. The check ran, found nothing and wrote an empty CSV.

The High Availability scripts write a status row instead, such as “Always On Availability Groups is not enabled or no groups are configured on this instance.” That way the assessment reports those areas as checked, not skipped.


Step 2: Review

.\powershell\reporting\Review-HealthCheckOutput.ps1

This reads every CSV from the most recent collection, applies fixed threshold rules and gives you a ranked findings list: CRITICAL, WARNING, INFO. 35 of the 45 CSVs have rules. They look for:

  • Backups: no full backup, or one older than 24 hours (CRITICAL). In FULL or BULK_LOGGED recovery, no log backup or one older than 4 hours.
  • Integrity: DBCC CHECKDB older than 7 days, or older than 14 days or never run (CRITICAL). Any suspect pages (CRITICAL).
  • Database settings: databases not ONLINE (CRITICAL), auto-shrink, auto-close and percent-based autogrowth.
  • Space: a log over 80% used, data files under 10% free, databases at or near their growth limit, volumes under 10% free (under 5% is CRITICAL), and VLF counts over 50, 200 and 1000 (INFO, WARNING, CRITICAL).
  • Performance: read or write latency over 50 ms, PAGEIOLATCH, WRITELOG, RESOURCE_SEMAPHORE, CXPACKET, CXCONSUMER, LCK_M_X or ASYNC_NETWORK_IO over 10% of total wait time, max server memory left at the default, TempDB file count, sizing and autogrowth, and a plan cache full of single-use plans. Blocking and open transactions are INFO.
  • Agent jobs: any job failure in the last 7 days, no DBA - Backup - FULL job, and maintenance jobs that failed or are disabled.
  • Security: sa enabled (CRITICAL) and other weak login settings, locked-out logins and repeated failed logins, xp_cmdshell enabled, risky linked server logins, orphaned users, and expired or expiring certificates.
  • Error log: corruption and crash patterns (823, 824, 825, 832, stack dumps) are CRITICAL. Memory pressure (paged out, error 802 or 701) is a WARNING.
  • High Availability and features: AG replicas not healthy or disconnected, failover readiness, endpoints not STARTED, inactive replication subscriptions, cluster failover entries in the last 7 days, Query Store switched to READ_ONLY, and CDC, Change Tracking or Service Broker warnings. User Extended Events sessions are INFO.

The other 10 never raise a finding on their own: server and OS inventory, backup coverage, TempDB usage, missing indexes, sysadmin members, instance configuration, top CPU queries, the Agent job overview and autogrowth history. They are context for step 3.

From the same collection, excerpted:

Review-HealthCheckOutput on the lab instanceexample output, not something to copy
============================================ DBA Health Check Review ============================================ Folder : output-files/healthcheck/.-yyyyMMdd-174817 Collected : 17:48:17 Reviewed : 17:49:40 -------------------------------------------- [CRITICAL] Backup DemoDatabase Full backup is 649 hours old (threshold: 24h) [CRITICAL] DBCC CHECKDB DBAMonitor No CHECKDB ever recorded - integrity of this database is unknown [CRITICAL] Security sa Login risk flag: SA_ENABLED ... [WARNING ] Error Log Memory pressure 5 entry/entries reporting working set paged out or insufficient buffer pool memory - check max server memory vs host RAM [WARNING ] SQL Agent DBA - Collect Deadlocks 100 failure(s) in the last 7 days [WARNING ] Transaction Log DBAMonitor Log is 95.4% used (984.2 MB of 1032.0 MB) - risk of log-full [WARNING ] Wait Statistics RESOURCE_SEMAPHORE 62% of total wait time - Memory grant pressure - queries queuing for execution memory ... [INFO ] VLF Count DemoDatabase VLF count 72 is elevated (>50). Monitor - resize on next maintenance window. ... -------------------------------------------- CRITICAL: 32 | WARNING: 80 | INFO: 4

In the console the same review is colour-coded by severity, CRITICAL in red, WARNING in yellow and INFO in cyan:

Two excerpts of Review-HealthCheckOutput in a PowerShell window. The first shows CRITICAL findings in red: databases with no full backup on record and full backups 817 to 1834 hours old against a 24 hour threshold. The second shows WARNING findings in yellow, Agent jobs failing up to 100 times in 7 days, a log 95.8% used and RESOURCE_SEMAPHORE at 95.6% of total wait time, then INFO findings in cyan for the error log, Extended Events sessions, a linked server and open transactions.

That is what a rules-based health check gives you: a real list, with findings.csv written next to the raw CSVs for tracking over time. But 116 flat findings still leave the real DBA work undone. Which of these are one problem wearing several hats, and which do you fix first?


Step 3: AI Assessment

This is the step that makes the other two worth automating. The same CSVs, plus findings.csv, go to an AI that follows a fixed rubric (powershell/reporting/ai-assessment-rubric.md). Its job is what the rules can’t do: connect signals across files, find root causes, judge severity in context, and write a prioritised report you can act on.

The AI step is optional. Steps 1 and 2 are plain PowerShell and T-SQL, need no API key and no internet access, and still give you the ranked findings list. In a locked-down environment, stop after step 2.

.\powershell\reporting\Invoke-AiAssessment.ps1

It needs ANTHROPIC_API_KEY set, and the report lands in output-files\assessments\. -DryRun writes the exact prompt to the same folder without calling the API. For the collection above that was 47 CSVs and roughly 47,600 tokens. Inside Claude Code, the assessment runs directly against the same CSVs and rubric, with no API call.

From an assessment of an earlier collection on the same lab instance:

Verdict“HPAI01 is a Developer Edition lab instance that is currently starved of memory to the point where the host is paging SQL Server out, the plan cache has been flushed empty, and the monitoring collectors themselves are failing with out-of-memory errors. Nothing here is production-facing, so nothing needs action today in an operational sense, but the single most important fix is the memory squeeze (1,600 MB cap on a 7.78 GB host that is overcommitted), because it is quietly breaking every other capability on the box: collectors, Agent jobs, and any meaningful performance testing. Second priority: SQL Server Agent is not running, so no scheduled collection or maintenance is happening at all.”

That’s the difference in one paragraph. The rules gave separate findings for 7 failing Agent jobs, a memory-pressure pattern in the error log, and a plan cache full of single-use plans. The assessment read the same evidence and found one problem.

The buffer pool is too small for the host, so Windows pages SQL Server out. The plan cache gets flushed, which is why top-cpu-queries.csv came back empty. Collector steps die with error 802, and the top waits are what a starved buffer pool produces.

Priority issue1. Memory starvation is degrading the whole instance
Evidence: max server memory 1,600 MB against 7.78 GB physical (memory-config.csv); SQL committed only ~227 MB with 68 MB in use; error log shows 11 entries in 24h of “significant part of SQL Server process memory has been paged out” … plus collector job steps failing with error 802 “insufficient memory available in the buffer pool”; dm_exec_query_stats is empty (0 rows, 6 cached plans), the plan cache is being flushed continuously; top waits after filtering are PAGEIOLATCH_SH 53.7%, RESOURCE_SEMAPHORE 9.8% (avg 9.2 s), THREADPOOL 9.1%.
Impact: collectors fail, query performance data is unusable (top-cpu-queries.csv is empty because nothing survives in cache), and any test run on this box measures paging, not SQL Server.
Fix: decide the box’s memory budget. If SQL matters here, raise max server memory to 3 to 4 GB and set min server memory to match (sp_configure), or accept the squeeze and stop the collectors.

The full report also split those 7 failing jobs. Most were dying from the memory pressure (4 logged error 802). The deadlock collector had never succeeded, for a different reason: Agent runs T-SQL job steps with QUOTED_IDENTIFIER OFF and the step uses XML methods, so it failed with error 1934 on every run. Same “Last run FAILED” line in the review, two unrelated root causes. A threshold rule cannot make that split.


Why the Steps Are Separate

  • Collection and review can run unattended, on a schedule. A daily collect and review means the findings are already waiting when an incident starts.
  • Review is deterministic. The same data gives the same findings every time, so you can track whether an issue count is going up or down over weeks.
  • Assessment is where judgement happens. It costs more to run and differs every time, so run it when someone is going to read it and act, not every 15 minutes alongside the collectors.

Reading the Assessment

  • Read the Verdict first. It is one paragraph that answers “do I need to act today”. Everything below it is supporting detail.
  • Priority Issues are already correlated. If several CRITICAL rows are grouped into one issue, that grouping is the finding. Don’t split them back into separate tickets without a reason.
  • Watch List items are not problems yet. Each one says what threshold would make it one. Look again on the next run.
  • Checked and Clean matters as much as the findings. It tells you an area was checked and found fine, not just that nothing came up.

Adjusting the Thresholds

The review thresholds assume daily backups and typical SLAs. To change them, edit powershell\reporting\Review-HealthCheckOutput.ps1 directly. Full-backup age, log-backup age, I/O latency and DBCC CHECKDB staleness are all plain conditionals in the script, not a config file to learn.


Running Against a Remote Server

.\powershell\reporting\Invoke-HealthCheckCollection.ps1 -ServerInstance PROD01\SQL2019
.\powershell\reporting\Review-HealthCheckOutput.ps1
.\powershell\reporting\Invoke-AiAssessment.ps1

Each server’s collection lands in its own timestamped folder (PROD01-SQL2019-<timestamp>). The review and the AI step read the most recent folder unless you pass -FolderPath, so run them straight after each collection or point them at the folder. For several instances at once, Invoke-MultiServerHealthCheck.ps1 -Servers "SVR01,SVR02,SVR03" runs the collection and the review against each one and writes a combined findings CSV.


This workflow has dba-tools gather the evidence and an AI correlate it. The sqldba MCP server is the same idea pointed the other way: the assistant asks this site, rather than answering from memory, whenever it needs an error, a wait type, a patch level or a script.


Frequently Asked Questions

What permissions does the health check need?

sysadmin. A monitoring login with VIEW SERVER STATE, VIEW ANY DATABASE and VIEW ANY DEFINITION still fails 8 of the 45 scripts: the Agent job tables in msdb need db_datareader there, xp_readerrorlog needs sysadmin in practice, the default trace needs ALTER TRACE, and database sizes opens every database. MS Docs lists the server permissions if you want to build a narrower login and accept gaps.

Does it change anything on the server?

No. All 45 collection scripts are marked SAFE:ReadOnly, and the only files written are the CSVs on the machine running the command. The review and the AI step read those CSVs and never connect to SQL Server.

Why are some of the CSV files empty?

The check ran and found nothing. On the lab run, suspect pages, CDC and Change Tracking, and orphaned users came back empty, which is the healthy answer for all 3. A script that errored shows [FAILED] in the console instead.

How do I run it on a schedule?

The repo ships no scheduler. Any job that can run PowerShell 7 can call pwsh -NoProfile -File .\powershell\reporting\Invoke-HealthCheckCollection.ps1 -ServerInstance PROD01 -Quiet followed by Review-HealthCheckOutput.ps1, which picks up the newest folder. Run the AI step by hand, when someone is going to read it.

Does the AI step send my data anywhere?

Through the API script, yes: CSV contents, including server, database and login names and file paths, go to the Anthropic API. Run -DryRun first and the exact prompt is written to output-files\assessments\ for review before anything leaves the machine. Collect and review never leave it.


Summary

Collection and review turn “is this server okay” into a ranked list, with no DMV queries typed by hand. The assessment turns that list into what a rules engine can’t produce: root causes across files, severity in context, and a next action for each priority issue. Run collection and review on a schedule. Run the assessment when someone is going to read it.


Related

This workflow’s collection touches nearly every pillar on the site; a few of the most directly related:

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *