********
Services
********


I. Dia Services Overview
========================


A. Introduction
---------------

An iDigi Dia service is a scheduled task or thread which does not
cleanly fit into the device or presentation module abstractions.

Services typically do not create new channels in the channel database
nor present information to a user or across a network.  Instead,
services provide additional system functionality such as performing
routine maintenance or health checks.

An example of a module that would be best expressed as a service would
be a software memory utilization watchdog.  This watchdog would
periodically run and evaluate the amount of free system memory remaining.
If the amount of free memory is less than a configured threshold the
system would be requested to reboot.


B. Service Implementation Requirements
--------------------------------------

All services in the Dia must meet the following requirements: 

   * They must inherit from :class:`~services.service_base.ServiceBase`. 
   * They must have zero or more settings. 
   * They must implement the following
     :class:`~services.service_base.ServiceBase` interface methods:

	* **init()**: called when the service is initialized. 
	* **apply_settings()**:  called when new settings are applied. 
	* **start()**:  called when the service is started. 
	* **stop()**:  called when the service is stopped. 

C. Example Services
-------------------

For examples of system service implementations please see the
"src/services" directory in the iDigi Dia distribution.  A simple
watchdog service has been implemented in the file "watchdog_service.py"


II. Getting Started with Services
=================================


A. Introduction
---------------

Developers who are interested in writing their own services should
first consult the examples in (src/services). It is important to
remember that services should typically provide behavior which lies 
outside the device or presentation abstractions. As mentioned before, 
routine maintenance and health checks are excellent examples.

The Null Service (src/services/null_service.py) is provided as a 
simple example of a service which meets all of the 
implementation requirements specified above. Although this service
has no actual functionality, it can still be used to test the
service manager. It also provides an easy starting place for users
who are unfamiliar with the service paradigm.


B. Implementation
-----------------

Fortunately, the three central requirements of a service (inheriting
from :class:`~services.service_base.ServiceBase`, having zero or more
settings, and implementing the necessary interface methods) are fairly
easy to satisfy. A service will generally consist of two sections: an
initial set of imports which satisfy the first requirement, and a
:class:`~services.service_base.ServiceBase` class which satisfies the
second and third.

The Null Service is the simplest example of this type of structure,
beginning with a pair of imports::

	from settings.settings_base import SettingsBase, Setting
	from services.service_base import ServiceBase

The second import, from :class:`~services.service_base.ServiceBase`,
is required.  The first import is not necessarily required by a
service with no settings (such as the Null Service).

In most cases, the settings and interface methods are established in a
class which inherits from :class:`~services.service_base.ServiceBase`,
as seen in null_service.py::

	class NullService(ServiceBase):
    		def __init__(self, name, core_services):
        		self.__name = name
        		self.__core = core_services

        		settings_list = [ ]

        		## Initialize settings:
        		ServiceBase.__init__(self, name=name, settings_list=settings_list)

   		def apply_settings(self):
        		SettingsBase.merge_settings(self)
        		accepted, rejected, not_found = SettingsBase.verify_settings(self)
        		if len(rejected) or len(not_found):
            		# there were problems with settings, terminate early:
            		print "NullService(%s): Settings rejected/not found: %s %s" % \
                			(self.__name, rejected, not_found)
            	return (accepted, rejected, not_found)
        		SettingsBase.commit_settings(self, accepted)

        		return (accepted, rejected, not_found)
    
    		def start(self):
        		"""Start the console instance.  Returns bool."""
        		print "NullService(%s): started." % (self.__name)
        
			return True

   		def stop(self):
        		"""Stop the console instance.  Returns bool."""
        		print "NullService(%s): stopped." % (self.__name)
        
			return True

To understand what this class accomplishes, remember that the Null
Service only needs to fulfill the four basic requirements of a
service- with a more complex example (such as the Watchdog 
Service), additional methods and definitions would probably
be seen. 

The __init__ method, accepting a name and a set of core services,
accomplishes three things. First, it sets the instance's own 'name'
and 'core_services' values. Second, it creates a list of settings (in
this case, an empty one). Finally, it initializes the settings through
:class:`~services.service_base.ServiceBase`. Like the service class
itself, the __init__ method may be expanded upon in more complex
implementations, but the core functionality (storing values, defining
settings, initializing settings) is usually the same.

The next method, 'apply_settings', transfers all 
pending settings into the running settings group, returning a tuple
of three dictionaries: a dictionary of settings which were accepted,
a dictionary of settings which were rejected, and a dictionary
of required settings which were not found. This method is often
similar (or identical) between services.

Finally, 'start' and 'stop' (as one would expect) control the
execution of the service, returning 'True' after running. The content 
of 'start' and 'stop' can vary from process scheduling or system output 
to simple 'pass' statements, depending on the requirements of the service.


C. Execution
------------

Using the Null Service for its primary task (testing the service
manager) is fairly straightforward. First, the project's dia.yml
file must be edited, which can be done in any text editor. To add and
enable the Null Service (or any service, for that matter), there 
must first be a services section in the config file. Adding this section,
if it does not already exist, is a simple task- the line 'services:' must 
simply be added to the file. Oftentimes, the configuration file
will already have a services section, in which case the header
must simply be uncommented.
If the header is commented or removed, services will be disabled by
default- adding the header, naturally, enables them.

Once services have been enabled, a configuration block must be added
for each service which will be used- in this case, the Null
Service. As seen elsewhere in the documentation, the Null 
Service configuration block is written as follows::

	- name: null
        driver: services.null_service:NullService

To add this service to our config file, it must simply be placed within
the services section and indented appropriately. Like the services header, 
service blocks which are already present must be uncommented to enable them. 
With these steps complete, the services section in the config file should 
resemble the following::

	services:

	# Create a new null service instance. This service will passively
	# test the service manager.
	  - name: null
	    driver: services.null_service:NullService

Finally, the project can be compiled and run, as seen
in the 'Getting Started' guide. Notice that the "NullService(None): started"
message originates from the "start" method inside the Null Service::

	iDigi Device Integration Application Version 1.4.1
	Using settings file: dia.yml
	Core: initial garbage collection of 0 objects.
	Core: post-settings garbage collection of 28 objects.
	Starting Scheduler...
	Starting Channel Manager...
	Starting Device Driver Manager...
	Starting Presentation Manager...
	Web Presentation (web0): using port 8001 and BaseHTTPServer
	XMLRPC(xmlrpc): starting server on port 8080
	Starting Services Manager...
	NullService(None): started.
	Core services started.

Now, with a basic understanding of how services are implemented, we
can examine a service which does something more interesting.

III. The Watchdog Service
=========================

A. Introduction
---------------

The Watchdog Service, as explained in the user's guide, "provides 
a means to monitor memory utilization of the system and reboot if 
the memory is getting too low." Like the Null Service, it follows
a predictable design pattern, but it also adds settings and defines 
new functions in a unique way. This makes it a more interesting 
example for developers who are planning to write their own services.

B. Imports
----------

One of the primary differences between the Watchdog service and the
Null Service is that the former requires a few additional imports to
function, outside of the expected
:class:`~services.service_base.ServiceBase` and
:class:`~settings.settings_base.SettingsBase` imports. See below::

	from services.service_base import ServiceBase
	from settings.settings_base import SettingsBase, Setting
	import digi_ElementTree as etree
	import time

	import string
	try:
    		from rci import process_request
	except:
    		print "DEBUG RCI IN PLACE, are you running on a PC?"
    		mem = 500000
    		# For debugging on a PC
    		def process_request(s):
        		global mem
        		print s
        		mem -= 100000
        		print mem
        		return '''\
	<rci_reply version="1.1">
    		<query_state>
        		<device_stats>
            		<cpu>3</cpu>
            		<uptime>3307</uptime>
            		<datetime>Mon Sep 28 10:59:38 2009</datetime>
            		<totalmem>16777216</totalmem>
            		<freemem>%d</freemem>
        		</device_stats>
    		</query_state>
	</rci_reply>''' % mem

	try:
    		import digiwdog
    		_watchdog_available = True
	except:
    		_watchdog_available = False

In this case, the imports are fairly diverse, ranging from
python time functions to Digi's own watchdog module. Although
developers are free to make whatever imports they require, 
they should make certain that the modules are available 
in their environment- particularly if the service will be used
in a device environment. For instance, many Digi devices have
limited support for pieces of the 'os' and 'socket' modules, so 
it would be unwise to write a device-specific service which
depends on them. As another, device-specific example, the
amount of available memory cannot currently be retrieved from 
the ConnectPort X3, so the Watchdog Service would not work properly
on that device.


C. Settings
-----------

The Null Service, despite the fact that it fulfills
all the requirements, doesn't serve as a very helpful 
example for developers who need to include settings in
their projects. Fortunately, the Watchdog Service has
a collection of settings (both required and optional)
which makes it somewhat easier to understand their use. 
These appear in the __init__ method, as follows::

        settings_list = [
            Setting(
                name='watchdog_interval', type=int, required=True, 
                default_value=120,
                verify_function=lambda x: x >= 60 and x <= 3600),
            Setting(
                name='low_memory_threshold', type=long, required=False,
                verify_function=lambda x: x >= 40960),
            Setting(
                name='auto_restart_interval', type=long, required=False, 
                verify_function=lambda x: x >= 600), 
        ]

Every setting must, at minimum, must be initialized with 
a 'name' and a 'type'. Additional parameters, such as 'required'
and 'verify_function', will revert to default values if they are
not specified. These default values are described below.

**Required Values:**

	* **name**: Name of the setting. No default value. 

	* **type**: A type constructor to create the item from a string. 
		No default value. 

**Optional Values:**

	* **parser**: A function to convert any given value to a value
      	  of the setting's type, or raise an exception on
      	  failure. Defaults to 'None'.

	* **required**: Boolean value specifying if this setting is required. 
	  Defaults to 'False'.

	* **default_value**: The default value for the setting. 
	  Defaults to 'None'.

	* **verify_function**: A function which validates the setting
	  after it has been created.  Defaults to a lambda function
	  which always evaluates as 'True'.

Because the Watchdog Service actually has settings (unlike the Null
Service), they must first be added to the configuration block in dia.yml
before the project is compiled. As an example, to set the 
watchdog_interval to 120 and the low_memory_threshold to 40960, the 
configuration block would be set up as follows::

	- name: watchdog_service0
	  driver: services.watchdog_service:WatchdogService
	  settings:
	      watchdog_interval: 120
	      low_memory_threshold: 40960

Recall that watchdog_interval is a required setting, whereas 
low_memory_threshold and auto_restart_interval are not. This
explains why only two of the three settings are listed.

Once a Service's settings are applied, their values can be retrieved
using the get_setting() method defined in
:class:`~settings.settings_base.SettingsBase`.  For instance, the
"auto_restart_interval" value could be retrieved within the Watchdog
Service as follows::

	auto_restart_interval = SettingsBase.get_setting(self,"auto_restart_interval")


D. Methods
----------

Useful services generally include their own methods and/or classes. 
These can combine information from a device, user configuration values, 
or any number of other elements to produce some desired behavior. 
Although many services run continuously, it is not required that they 
do so. For example, the included "Timing" and "Value" services 
simply rebuild the annotation system upon being initialized- 
their start/stop methods consist only of "pass" statements. 

The Watchdog service, in contrast, runs actively
once the service manager is booted up. Because it accomplishes
a fairly simple task (monitoring the available system memory), 
the method structure gives us an idea of how a typical service
might be implemented. An extremely in-depth explanation of all
the methods is beyond the scope of this document, but the purpose
of each is outlined in the following list.

	* **__init__:** Required interface method. Creates class-specific
	  variables and initializes the list of settings.

	* **apply_settings:** Required. Identical to the version of
  	  apply_settings seen in the Null Service.

	* **start:** Required. Retrieves the watchdog interval from
	  the running settings group, and uses the value to create and
	  schedule an instance of Digi's watchdog.

	* **schedule:** Uses the watchdog interval to establish when the
	  next 'check' should occur.

	* **stop:** Required. Opens the watchdog module and removes the next
	  scheduled memory check, effectively stopping the service.

	* **get_free_memory:** Uses an RCI request to return the amount of 
	  available memory as an integer.

	* **reset:** Resets the device via RCI.

	* **stroke:** "Strokes" the watchdog, resetting the timer and
	  preventing the device from rebooting.

	* **_do_auto_restart:** If the user requested an
	  auto_restart_interval, this method will check if the
	  interval has passed, and reboot the device if so.

	* **_do_low_memory:** If the user requested a
	  low_memory_threshold check, and the amount of free memory is
	  below this threshold, this method will reboot the device.

	* **check:** Performs the four watchdog service checks
	  mentioned above (_do_auto_restart,_do_low_memory,stroke,and
	  schedule).



IV. Conclusion: Creating New Services
=====================================

As mentioned earlier, developers who need to write their own services
should first review the provided examples. The Null and Watchdog
services serve as a good starting point, but they only touch on the
variety of tasks that services can perform. Generally, though, the
development process will follow a familiar arc.

First, a python file for the service should be added to the
'src/services' directory in the DIA folder. This file must, at a
minumum, import from :class:`~services.service_base.ServiceBase` and
:class:`~settings.settings_base.SettingsBase`, as demonstrated
earlier. Once the file exists, a class should be added which inherits
from :class:`~services.service_base.ServiceBase`. This class will
contain the required '__init__', 'apply_settings', 'start', and 'stop'
interface methods. The '__init__' method is of particular interest,
because it initializes the list of settings that can be configured by
the user. Once the required methods are in place, additional methods
(or even additional classes) can be added as needed.

Once the service has been written, a configuration block must be 
added to the 'cfg\dia.yml' file before it can be used. This block
should be added to the 'services:' section, and formatted as follows::

   - name: "service_instance_name"
     driver: dot.separated.path.to.service:ServiceClassName
     settings:
         setting_name0: setting_value0
         setting_name1: setting_value1
         ...
         setting_namen: setting_valuen

With the services section configured (and uncommented), the project
can be compiled and tested! Services are a powerful tool, but developers
should be certain that their desired functionality
fits into the service paradigm.

