- iDigi Dia Home
- User
  - Getting Started
  - Dia Concepts
  - Devices
  - Loggers
  - Presentations
  - Services
  - Tracing
- Developer
  - Overview
  - Drivers
  - Loggers
  - Presentations
  - Services
  - Integrating Application
  - Tracing Manager
  - Debugging
  - Specific Considerations
  - API Reference

- Email
- Phone
- Contact us:

### Navigation

- index
- next |
- previous |
- iDigi Dia |
- Developer Documentation »

# Dia Development Overview

## I. Introduction

The Digi Device Integration Application (“Dia”) is an application software platform for Digi’s family of gateway devices. It is intended to make the task of developing custom applications for remote monitoring and sensor networking applications easier. The Dia framework is written in the Python programming language and uses the Python interpreter on Digi’s gateway and embedded devices. It may also be executed on a PC for prototyping purposes when a suitable Python interpreter is available.

The Dia framework is targeted at applications which aim to solve the problem of gathering data from a set of devices–such as ZigBee sensors, wired industrial equipment, GPS devices, etc–by abstracting the data from these devices as sets of samples which flow into channels, logging these samples, and re-presenting this data to a next highest layer application over a wired or wireless IP connection. The framework offers a well thought out and tested architecture that is advantageous when compared to attempting to write an enterprise-grade application from scratch.

The framework is abstracted into three primary subsystems:

> - The device driver subsystem
> - The channel subsystem
> - The presentation subsystem

Device drivers are responsible for abstracting real-world devices into a set of quantities known as properties. A device driver instance will publish the values of these properties via a named channel known as a “channel property” in the form of a sample. Channels are accessible globally in the system via their unique name. A driver’s properties are named by the pattern:

    device_instance_name.property_name

For example, a GPS device might have two properties defined: latitude and longitude. A running instance of our GPS device might be called, “gps0”. As the driver runs, it will collect information from the physical GPS device and gather the most recent latitude and longitude values. The driver instance will create sample objects for each of these two values and publish the new samples to the channel. The values for latitude and longitude will then be globally available in the system as the channels “gps0.latitude” and “gps0.longitude” respectively.

The channel subsystem houses the `channel database` and data logger instances. The `channel database` manages the global namespace of all channel names and allows interested parties to be able to create new channels, remove existing channels, get samples, set samples, or subscribe to a channel to be notified upon sample updates.

The data logging system has priority access to changes in the channel database. As changes are made to the database they enter the domain of the data logging manager. Changes captured to a file by a data logging instance may be played back into a fresh instance of a `channel database` allowing for objects which know how to interact with the channel database API to also be able to present data from a log.

The presentation system is responsible for presenting samples to users or other machines running applications at the next highest layer. Presentations receive their samples from the `channel database` either by polling or asking to be notified on changes using the `channel publisher` object’s subscription interface.

Examples of presentations include a command-line interface, a web interface, XML-RPC, Modbus, or the iDigi Platform Web Services interface.

A diagram of the system looks like the following:

## II. File System Layout

The source code distribution of the iDigi Dia is organized into various directories to reflect the design of the system. There are two distinct locations for working with the system: the top-level directory which contains the system build tools and configuration files and the “src” directory.

The top level directory of the iDigi Dia source code distribution contains the following entities:

> - bin/
> - cfg/
> - clients/
> - demos/
> - dia.py
> - doc/
> - lib/
> - make.py
> - src/
> - tools/
> - xml_elements/

### 1. The “bin” Directory

The “bin” directory contains the output of the make.py script (see below). The files which are generated in this directory may be deployed to a Gateway for execution by the device’s Python interpreter.

### 2. The “cfg” Directory

The “cfg” directory contains the configuration files for the iDigi Dia. The default configuration file is called “dia.yml”

### 3. The “clients” Directory

The “clients” directory contains sample software clients which may be used to interface with the various presentation layer modules within the iDigi Dia. An example client is the XML-RPC client stored in the “xmlrpc” sub-directory.

The XML-RPC client interfaces with a running instance of the Dia to create a command-line interface by communicating with the Dia’s XML-RPC presentation interface.

### 4. The “demos” Directory

The “demos” directory contains several iDigi Dia framework configuration files (YML files) which can help the end user use Dia for the first time.

### 5. The “dia.py” File

The “dia.py” file is the main executable script which starts the Dia. The “dia.py” is able to be executed on both a Digi gateway device or on a PC environment. In order to run the script on the gateway device “dia.py”, along with the contents of the “bin” directory must be uploaded to the device beforehand.

The “dia.py” script may be given a single argument: the name of the configuration file to load. By default the name of this configuration file is “dia.yml”

### 6. The “doc” Directory

The “doc” directory contains documentation pertaining to the iDigi Dia system.

### 7. The “lib” Directory

The “lib” directory contains third party Python modules that are not part of the core system. Some of these modules have been created by Digi and some of them are available in the public domain.

Some of the modules, such as the YAML module, have been modified to better perform on the Digi gateway system. It is important not to replace these modules with newer versions without preserving these modifications.

### 8. The “make.py” File

The “make.py” file is the script used to generate a compiled, binary distribution suitable for loading on the Digi gateway device.

At the top of this file there are a number of hand editable parameters which may be changed in order to make the build script operate in a variety of environments.

### 9. The “src” Directory

The “src” directory contains the main set of source files which compose the iDigi Dia system. Its contents are explained in greater detail below.

### 10. The “tools” Directory

The “tools” directory contains a set of tools used by the “make.py” script in the generation of the binary distribution of the iDigi Dia application.

### 11. The “xml_elements” Directory

The “xml_elements” directory contains the files (in XML format) and necessary icons to represent in a graphic way all the Dia elements for the Digi ESP for Python IDE.

## III. The Contents of the “src” Directory

The “src” directory contains the majority of the implementation of the Digi Device Integration Application. It contains the following entities:

> - __init__.py
> - channels/
> - common/
> - core/
> - devices/
> - presentations/
> - samples/
> - settings/
> - services/

Each directory holds the files and sub-directories which constitute the implementation of a particular subsystem within the iDigi Dia. For example the “presentation” directory contains the presentation subsystem implementation: it holds the presentation manager, a presentation base class, and implementations of a presentation instances such as the command-line interface and XML-RPC implementations.

### 1. The “__init__.py” File

Each and every directory which will be accessible as a python module must contain an “__init__.py” file.

See: http://docs.python.org/tut/node8.html for more information.

### 2. The “channels” Directory

The “channels” directory contains the implementation of the channel and logging subsystems. These subsystems are responsible for organizing and logging samples in the Dia.

### 3. The “common” Directory

The “common” directory houses source files which are used by multiple systems in the Dia. These sources differ from the top-level “lib” directory in that the source has not been adapted from any third-party module but wholly invented for use within the framework.

An example module in this directory is the `common.classloader` module which provides dynamic loading capabilities for the entire system.

The “common” directory also contains a sub-directory called “types”. This directory holds various definitions for specialized types which are commonly used in the system.

One such type is a `Boolean` type. This `Boolean` type accepts a wider range of values in its constructor (such as the strings “On” or “Off” to map to True and False respectively) as well as options for choosing how the value is represented as a string.

### 4. The “core” Directory

The “core” directory contains the `Core Services manager` implementation. The `Core Services` manager starts at system start-up (“epoch”) time. Services that are started in the Core Service manager include:

Component Managers
  - :py:class:`The Channel Manager <channels.channel_manager.ChannelManager>`
  - :py:class:`The Device Driver Manager <devices.device_driver_manager.DeviceDriverManager>`
  - :py:class:`The Presentation Manager <presentations.presentation_manager.PresentationManager>`
  - :py:class:`The Service Manager <services.service_manager.ServiceManager>`
Core Functionality
  - :py:class:`The tracing manager <common.tracing_manager.TracingManager>`
  - :py:class:`The main scheduler <core.scheduler.Scheduler>`
  - :py:class:`Access to system settings <settings.settings_base.SettingsBase>`

Each of these top-level services is given a handle to the core. In turn, child services may make requests of the core for references to other services. For example, the :py:class:`The Device Driver Manager <devices.device_driver_manager.DeviceDriverManager>` may ask for a reference to the `Channel Manager` via the `Core Services Manager`.

### 5. The “devices” Directory

The “devices” directory contains the :py:class:`The Device Driver Manager <devices.device_driver_manager.DeviceDriverManager>` and all device implementations in the Dia. The :py:class:`The Device Driver Manager <devices.device_driver_manager.DeviceDriverManager>` is responsible for loading all devices in the system. Device implementation instances abstract data streams into channel properties.

In the “devices” directory there is a directory named “vendors”. If a new device driver is created by an organization, the organization should first create a sub-directory within the “vendors” directory and then deposit their device driver implementation source files within this new directory.

For example, if “Acme Inc.” creates a new sensor driver in a file named “acme_pressure_sensor.py” a directory named “acme_inc” should be created in “vendors” and then “acme_pressure_sensor.py” should be placed within it.

### 6. The “presentations” Directory

The “presentations” directory houses the :py:class:`Presentation Manager <presentations.presentation_manager.PresentationManager>` and all presentation implementations. The :py:class:`Presentation Manager <presentations.presentation_manager.PresentationManager>` is responsible for loading all presentations in the system. Presentation instances display or re-present samples from the `channel database` to the next highest layer, often another device attached to the gateway over a network.

Examples of presentations include the Command Line Interface, the XML-RPC service, and the iDigi DB XML database modules.

### 7. The “samples” Directory

The “samples” directory contains the sample object implementation. Sample objects are produced by device drivers, placed into the channel database–where they are possibly logged–and finally consumed by one or more presentations. Sample objects are generally the most numerous object instances in the system.

### 8. The “settings” Directory

The “settings” directory holds the settings sub-system for the iDigi Dia framework. Contained within this directory is the settings base class implementation and the settings serializer implementations.

In order for an object within the framework to be able to read, to write, and to persist their settings they must derive from the settings base class.

Settings in the system may be saved in any format for which there exists an appropriate serializer implementation. Current serializations are named YAML and PYR. The YAML (http://www.yaml.org) serializer is intended to be more user-friendly and the primary configuration file format exposed to users. During the execution of ‘make.py’, the .yml file provided is converted and serialized to the PYR format. This is a direct Python language represenatation of the settings. This conversion is performed as an optimization to reduce the libraries required on a running instance of the Dia on a gateway device.

The serialization system is abstracted in such a way that other formats–such as XML or JSON–could be supported without much additional effort.

### 9. The “services” Directory

The “services” directory contains the services sub-system of the iDigi Dia. Included within this directory is the service base class implementation and the services manager.

A service is defined as a re-entrant task or thread which does not cleanly fit into the device or presentation abstractions. Typically services do not produce new channels in the `channel database`–that would be a device–nor present information to a user or across a network–as that would be a presentation.

Instead, services provide additional functionality such as the implementation of a watchdog, or other system maintenance tasks.

## IV. A Description of Framework Initialization From Epoch Time

The start-up of the Digi Device Integration Application begins from the “dia.py” file. Execution passes through the following phases:

> 1. “dia.py” starts, parses command line parameters and creates a `Core Services` instance.
> 2. When the `Core Services` instance begins it attempts to load the systems settings. If successful, it starts the core services:

> > - :py:class:`The Tracing Manager <common.tracing_manager.TracingManager>`
> > - :py:class:`The Channel Manager <channels.channel_manager.ChannelManager>`
> > - :py:class:`The Device Driver Manager <devices.device_driver_manager.DeviceDriverManager>`
> > - :py:class:`The Presentation Manager <presentations.presentation_manager.PresentationManager>`
> > - :py:class:`The Service Manager <services.service_manager.ServiceManager>`
> > - :py:class:`The system scheduler <core.scheduler.Scheduler>`
> > - :py:class:`Access to system settings <settings.settings_base.SettingsBase>`

The :py:class:`tracing manager <common.tracing_manager.TracingManager>` will initialize all tracing filters and outputs.

The :py:class:`system scheduler <core.scheduler.Scheduler>` will start and be available for use by all other core services and their instances.

The :py:class:`Channel Manager <channels.channel_manager.ChannelManager>` will initialize the `channel database` for use by other system services; the Logging Manager start as a child instance of the :py:class:`Channel Manager <channels.channel_manager.ChannelManager>`.

The :py:class:`Device Driver Manager <devices.device_driver_manager.DeviceDriverManager>` will start, attempt to load and initialize all devices in the system–in the order they were defined in the settings database.

The :py:class:`Presentation Manager <presentations.presentation_manager.PresentationManager>` will start, load and initialize all presentation instances defined in the system.

The :py:class:`Service Manager <services.service_manager.ServiceManager>` will start, load and initialize all service instances defined in the system.

After all the systems have loaded and initialized the system is available for use.

### Navigation

- index
- next |
- previous |
- iDigi Dia |
- Developer Documentation »