Recently one of our customers asked us how to ensure that their SQL Servers were being monitored, including watcher nodes. The SQL Servers is easy (install and configure the SQL management packs) but finding the watcher nodes is a little trickier.

What Is a Watcher Node?

A watcher node is an agent which runs synthetic transactions to ensure proper operation of an application. In the case of SQL, the synthetic transaction would be connecting to a SQL Server, and optionally running a SQL query, and comparing the performance to expected results. The watcher node should be on a different server than the SQL Servers, to ensure that “outside” connections can be made and succeed. The same thing applies for SCOM Web Consoles, TCP port checking, IIS servers, etc.

Since the watcher node is not running on a SQL Server, it is important to ensure that the watcher node is also being monitored. Basic operating system monitoring is important, but the watcher should also have a management pack which runs the synthetic transactions, records the results, and compares to expected values.

And this is where our customer was stumped: how to find the systems acting as watcher nodes. Fortunately, SCOM and MP Studio make this easy.

Watcher Nodes in SCOM

Most (but not all) watcher nodes are implemented in SCOM as a class derived from Perspective. For instance, if we create a SQL watcher node in MP Studio (which uses the same SCOM Template as the SCOM Console), we find the following class hierarchy.

Notice that Perspective is one of the base classes in this hierarchy. Let’s see what other classes derive from Perspective.

Of course, with additional monitoring for other applications, there may be additional classes derived from some of the abstract classes above, or even from the existing non-abstract classes.

Now that we know what classes are implemented for watcher nodes, all we need to do is get a list of the class instances and we will have found all the watcher nodes. For instance, let’s look for Web Console Watcher class instances.

We see that there are 3 watcher nodes, one for each management server. The Path property above shows which servers they are running on, and the Web Console Url property show us the URI they will be using to test the Web Consoles with synthetic transactions. (In this case the watcher node is running on the server being monitored, but often that is not the case.)

When we look at the SilectStore Database Monitor class instances, we find the following:

Again, the Path property shows which servers are running this watcher node. The server DemoHost is not a SQL Server, so in this case the watcher node is monitoring a different server.

In the case of the SQL Template, the class does not give information about what is being monitored (connection string, SQL query, etc.) That information can be determined, if needed, but it’s outside of the scope of this article. Let us know if you would be interested in more information.

Finding Watcher Nodes using MP Studio

MP Studio introduced SCOM Explorer several releases ago. We will use that to find the information shown above.

Start MP Studio and make sure that you have registered a management server for each of the management groups you want to explore. Then use the right-click menu item Explore Management Group for the selected management group.

This will open the Explore – management group name window, after reading in all the information about the health model from the management group. Now you can select Classes from the Filter drop down and search for Perspective. Click on the Perspective row.

If you click on the Show Hierarchy button, you will get a list of all the classes derived from Perspective (as seen earlier in this article). Click on the Web Console Watcher link in the message box to select the Web Console Watcher class.

Click the Show Instances… button to see the list of class instances for the Web Console Watcher class, which will show you the systems where this watcher is running.

You can do the same with any other class derived from Perspective to find other watcher class instances. While not all watcher nodes are derived from Perspective, you can search for those other classes and look for instances as needed.

Once you know which systems are watcher nodes for your application, you can ensure that those systems are being monitored by SCOM, and that management packs for the watcher nodes have been installed. You can also use the MP Studio workflow report to check that the watcher node rules and monitors are running on the systems as expected.