RES Automation Manager : the Dispatcher discovery process

In one of my previous blog articles I promised to give you a closer look at the Dispatcher Discovery process. It took me a while but now I will fulfill that promise. Together with my colleague Bas van Goor I created an overview of the discovery process and in this article I will walk you through it.

Why do we actually need a discovery process?

The RES Automation Manager Agent doesn’t communicate directly with the database. It does so by means of one or more Dispatchers and for an uninterrupted communication process it needs to have a list of functional and reachable Dispatchers in its memory. Building an initial list of Dispatchers in memory is what we call the discovery process.

First let me show you the entire picture and then I will zoom in on the various steps.

Step 1:

When the service starts it obviously doesn’t have any Dispatcher list in memory yet so it will start building one, the discovery process begins. The last time the Agent received its settings from the database it saved them to one of the following registry keys:

 HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\RES\AutomationManager\Agent  (64 bit)
HKEY_LOCAL_MACHINE\SOFTWARE\RES\AutomationManager\Agent (32 bit)

So the first thing the Agent will do is check the registry for discovery settings.

Step 2:

The first value to look for is the DispatcherList value, which represents the’ Dispatcher discovery’ setting in your RES Automation Management Console. This setting can be set on Global, Team or Agent level. If a Dispatcher Address List is configured the DispatcherList value will contain names or IP addresses of Dispatchers. If the value is empty it means that Autodetect is configured as Discovery method and the Agent will try to find Dispatcher by sending a Multicast message to 224.1.1.50 on port 3163 (UDP).

All Dispatchers will respond to such a message with the identification of the environment they belong to. The Agent checks it against its own stored data and in case of a match the Agent will add the Dispatcher(s) to its Dispatcher list in memory. If the Agents gets no response at all the next action depends on the DispatcherRecovery value. I will get back to that later on.

Step 3:

The next registry value the Agent checks is TryAutoDiscoverFirst. This value corresponds with the ‘First try Autodetect’ tick box you see when you’re configuring a Dispatcher Address List in your Console. If that box is ticked the registry value will be 1 and the Agent will start a Multicast procedure.

Step 4:

If the TryAutoDiscoverFirst is 0 or if the Multicast procedure from step 3 didn’t result in response from any usable Dispatchers the Agent will then send unicasts to all the names and/or IP addresses configured in the DispatcherList value on port 3163 (TCP). Any Dispatcher from that list that responds (from the correct environment of course) will be added to the Dispatcher list in memory.

Step 5:

If the Agent still hasn’t found a Dispatcher to talk to the DispatcherRecovery value is checked. This value reflects the Dispatcher Recovery setting in your Console. If the value is 0 the whole discovery process will start again at step 1. If the value is 1 the Agent will revert to a list of ALL known Dispatchers, which is stored in the registry under CachedDispatchers. Later on in this article I will explain how this value is filled with data.

Step 6:

If any Dispatchers did respond with a valid environment identification, we finally have a list of responsive Dispatchers in memory and the Agent can register itself in the database through one of them.

Step 7:

The first thing an Agent does after successfully logging on to the database is get the full Dispatcher list from that database. This list contains all addresses the Dispatchers registered themselves with, FQDNs, NetBIOS names and IP Addresses alike.  This list is stored in the CachedDispatchers value mentioned in step 5

Step 8:

But that’s not all, there is still a final setting to check. With the ‘Dispatcher locations’ setting in the Management Console you can specify whether the Agent should use only the Dispatchers it was able to find in the discovery process or retrieve a full list of Dispatchers from the database. This is stored in the DoNotRetrieveCompleteList value in the registry. If it is set to 0 the list of all known Dispatcher addresses is added to the Dispatcher list in memory and the Agent will start using all of these addresses randomly. Otherwise it will maintain the list of discovered Dispatchers from step 6 and work with that.

Step 9:

Now that the Agent knows which Dispatchers it could use it starts getting all sorts of information from the database. First it tries to find out if any of the Dispatcher discovery settings (on Global, Team or agent level) have been changed. If so, it will get the new settings from the database and store it in the Registry. Now the entire process will start all over again with the new settings. So if you manually tweaked the registry values to get the Agent to connect these changes are now overwritten.

Step 10:

Now the Agent will regularly connect to a random Dispatcher from the list and check for changes. If it comes across a Dispatcher that doesn’t respond anymore it will remove that Dispatcher from the address list in its memory. Whenever there are no more Dispatchers left in that list it will start the Discovery process again. Next to that the discovery process restarts every 5 minutes to make sure that any newly added Dispatchers are included as quickly as possible.

So, when does the discovery process start?

-          When the RES Automation Manager  Agent Service starts
-          When the discovery process failed to deliver any responsive Dispatchers
-          When the Dispatcher list in memory is empty because no Dispatcher from that list responds anymore
-          Every 5 minutes

I hope that after reading this article you will be able to look at an RES Automation Manager Agent trace file and see the Woman In The Red Dress. I still have some tips and Best Practices regarding the discovery process but this article is long enough as it is. I will put it in a separate article soon.