Monday, May 26, 2014

Design Patterns for an Internet Of Things

Design Patterns for an Internet Of Things
A Design Pattern Framework for IoT Architecture

Design Patterns are reusable solutions to common problems


Design Patterns provide well known ways to solve design problems commonly encountered in a particular discipline or problem domain. As an example, three different design patterns to handle traffic flow at a road intersection are stop signs, traffic lights, and roundabout. Each have advantages and disadvantages specific to particular traffic patterns and other contextual factors.


IoT presents design problems in many areas and at many levels in the system. There are many diverse use cases, with different resource constraints, and with many standards, products, and technologies available. How do we determine which are appropriate, what the specific advantages and disadvantages are without a context from which to evaluate their use?


Design Patterns are building blocks of architecture


Design patterns provide a way to build an end to end solution in well specified ways and to provide an understanding of the use of different components of the system in a system context. A reference architecture can be constructed from a set of design patterns, and from this the behavior of the system may be modeled and understood.


The articles published here up to this point have focused on a set of design patterns around data models and information models for the Internet of Things, and most recently began investigating an open stack approach, where use-case appropriate end to end solutions can be built from various components and design patterns.


Next I would like to look at a set of system design patterns useful in the construction of IoT architecture solutions, with a focus on common patterns for interoperability. I have divided the design patterns somewhat arbitrarily into areas of information models, interaction models, application programming models, infrastructure models, and use case patterns.


The design pattern examples were drawn from www and internet, best practices and standards, those which appear to be successful and well known, and those which satisfy some need or solve some problem. These are only examples and not meant to be exhaustive. However, the intention is to enable covering most common IoT use cases using a set of design patterns which can use  well known standards and practices.


Each of these areas is further decomposed into a spectrum of higher level patterns built on lower level patterns. I present only a few examples of common design patterns; this is not meant to be an exhaustive treatment of the subject.


Design patterns for connected things represents the fundamental proposition of an Internet of Things; that is connecting things using networks and software. Here are some examples of fundamental design patterns for IoT:


  • Software connected to thing via a network: the fundamental proposition of the IoT is that connected software is of higher value than embedded software. Connected software is less resource constrained and can integrate more diverse data sources.

  • Virtualization: Software connecting to an abstract representation of the thing. Virtualization makes it easier to create reusable software and devices.

  • Virtualization through middleware: allows many (web) applications to interact with things. Middleware can cache the state of the thing and minimize network traffic and power drain on constrained devices, and can also serve as a persistent end point for things that aren’t reachable over the network due to power cycling, firewalls, etc.

  • Middleware Platform: Allows many-to-many interactions between applications and things. Enables realization of connected environments and network effects. Can provide standard interfaces for things and application software.
  • Thing-thing interaction: things that contain some application software interact directly with other things on local networks


Design patterns for IoT use cases describe the system level use case mapped onto high level design patterns like gateways and web services:


  • Devices talk to other devices peer-to-peer: local network connectivity enables proximal ad-hoc networking, service federation and chaining, media stream continuity.
  • Personal tracking device uses smartphone as gateway: common pattern for bluetooth and WiFi connectivity.
  • Smarthome local application controller and gateway: application gateway pattern.


  • Monitor a large number of devices over a large area: collect, filter, analyze pattern used in Smart City, surveillance systems, large scale resource monitoring.


  • People interacting with autonomic feedback loops: general purpose use case involving people and automation in defined roles. Autonomic control relieves people of the job of monitoring, but lets them be involved in high level control and exception handling.


There are many more design patterns at all levels of design. The following list contains some more common patterns based on modern web patterns and practices that are relevant to IoT architecture.


Design patterns for information models consist of lower layers of data models and representation, upon which are built higher level encapsulation and function. Some examples of information model design patterns:
  • Structured data: XML documents, JSON objects
  • Web Objects: multiple resources at a URI endpoint, object encapsulation
  • VIrtual Objects, Smart Objects: a set of resources that represent a physical thing or other data source
  • Composite Objects: Virtual Objects composed of resources from other objects
  • Hypermedia, HATEOAS: Applications use pointers to URI endpoints that externalize application state
  • Semantic Hyperlinks: Hyperlinks with embedded semantic tags, e.g. relation=value
  • Information model: Collection of semantic hyperlinks describing a resource
  • Context model: Information model layer describing context of a set of resources
  • Binding model: Information model describing dynamic binding of actions to resources
  • Resource Directory, Catalog: A collection of model instances describing sets of resources
  • Resource constructor: Information model that informs the construction of resource instance
  • Access control model: Information model that specifies access control policies and constraints for a set of resources


Design Patterns for Interaction describe how different parts of a system interact and communicate with each other, including communication protocols. Some examples:
  • REST: REpresentational State Transfer, design pattern allowing for externalization of application state in reusable, shareable resources
  • Asynchronous Events: State updates propagate through the system as they occur
  • Resource Binding: Associating a resource with an action, bridges REST to Asynchronous Events
  • Observer Pattern: A binding of resource updates to a protocol action or handler
  • Publish/Subscribe: A communication pattern where a client registers interest in a topic by subscribing, updates to a topic are published to all subscribers
  • Broker: A central service to connect publishers with subscribers
  • Proxy: A machine that provides an interface on behalf of another interface
  • Protocol Bridge: A bidirectional translator between two protocols
  • Resource Discovery: A process where resources are found by specifying attributes
  • Resource Registration: An endpoint informs a resource directory of it’s resources
  • Sleeping/Non-reachable Endpoint: An endpoint is not reachable and must participate in protocol by initiating all interactions with reachable or always-on endpoints


Design patterns for Application Programming describe ways that software and interfaces are created, managed, deployed, and used in IoT applications.
  • REST Objects: Mapping of REST API resources onto program objects in the application language, using libraries
  • Event handler, onEvent: application code that responds to asynchronous events
  • Event driven flow: a set of application handlers that operate in an event driven graph containing series cascade and parallel constructs
  • State Machine: a logic construct where a next state depends on a set of inputs and the current state, evaluated by a set of logic rules associated with each state
  • State Externalization: the ability to create stateless application software by mapping application state onto external resources
  • Rule oriented programming: using a set of rules or rule language to program state machine logic
  • Abstraction of applications: stateless application software uses application templates for reusability
  • Application templates: abstract application components with well defined interfaces
  • Modular applications: applications consisting of one or more reusable components
  • Applications run anywhere, location independent applications: Application components can run anywhere, in devices, on local network servers, in gateways, in edge servers, in cloud, on user devices.
  • Discovery and Linking: Integrates resources into applications by resolving resource links, sets attributes in application objects
  • Object Constructor: Creates application software objects from metadata models


Design patterns for infrastructure describe how different network and device technology is used to solve problems with the physical infrastructure of IoT. How do low power devices connect to wireless sensor networks and ultimately connect to services and applications:
  • 6LowPAN edge router: moves packets from 6LowPAN network to IPV6, does header compression
  • WSN access point: mediated access from WSN network to IP backbone using a joining protocol e.g. WiFi
  • Mesh routing: routing network protocol messages through other endpoint nodes in a network
  • Application gateway: device with both network connectivity and the ability to run application components locally
  • Behind-NAT connectivity: using reachable service or broker, applications and devices connect to each other from behind NAT firewalls
  • M2M WAN: Wireless Service Providers specialize in M2M and provide wireless WAN networks, e.g. SigFox


Design patterns for IoT security describe design patterns for IoT security problems.
  • Access control using data models: semantic hyperlinks control access to resources based on the embedded metadata
  • Social to physical graph relationship: well defined concepts of ownership and access delegation between people, entities, and things
  • PGP and asymmetric public-key cryptography on devices: ways of creating SSL sessions and signing data between devices and applications
  • DTLS over UDP: security for resource constrained devices
  • End-to-end encryption: transmitting and storing encrypted data independent of channel encryption                                                                                                         
  • Device Management: using device identity, registration, and secure key exchange


There is no one “reference” architecture for an Internet of Things. Design Patterns are a way to construct architecture solutions for specific use cases and use case classes.


The work to find system architecture solutions to Internet of Things problems has led to an obvious conclusion that there is no single architecture appropriate for most IoT use cases. The full spectrum of IoT presents a broad range of diverse use cases and resource constraints, and thus motivates a range of architecture solutions. Still, we would like to ground the discussion in a reference set of technical concepts, to help promote a unified understanding and break down the silos of thought around IoT architecture. We would also like to find opportunities for standardization and commonality.


Different architecture solutions are appropriate for different use case classes, and architecture is expected to be reusable within a particular class of use cases.Therefore it makes more sense to talk about IoT architecture as a set of Design Patterns, working together to achieve an end-to-end solution for some problem.

Saturday, October 19, 2013

A Modular Open Source Platform for Web Scale IoT Interoperability

A Modular Open Source Platform for Web Scale IoT Interoperability

Michael J Koster
October 19th, 2013

Interoperability for the Internet of Things

Previous articles in this series discussed architectures and approaches to provide meaningful interoperability. Protocol interoperability allows any application to interact with any connected thing using any M2M protocol.

Interoperability also provides for reuse of software components, allowing a common platform and tool set across diverse use cases. It should be easy to integrate data and things from diverse sources into a single application. Diverse UI platforms and new UI/UX models should be enabled to be easily integrated into systems.

Figure 1 shows how a platform for interoperability would allow applications to discover and connect to diverse things using a choice of M2M protocols.



Figure 1 - Interoperability


Data models enable machine understanding independent of M2M protocols

Data models describe connected things to applications, driving discovery and linking,
and allow abstraction of M2M protocols. The effect is similar to the separation of the data plane from the control plane in software defined networking.

In figure 2, a Semantic Proxy provides a common representation of data models that originate in different catalogs and repositories. Because each application or resource endpoint is using a common data model, any M2M protocol can be used to update the sensor or thing data. The common data model describes enough about the endpoints to allow abstraction of the transport mechanism.


Figure 2 - Data Models enable M2M abstractions enable M2M Interoperability

It should be noted that the semantic proxy consists of mainly agreed-upon bindings of relations (predicates) and attributes (objects) and representation formats that allow each application to conveniently use it’s favorite catalog and representation format while interoperating with others.

Open Source Software for IoT is becoming common

Recent developments in open source IoT components are starting to enable an ecosystem as well as a modular open source IoT software stack.

  • The Mosquitto MQTT broker and client are robust, full featured components.
  • Node-RED is a new visual programming tool for event-driven software composition.
  • CoAP stack software is available in several forms to integrate into different endpoints.
  • RDFLib provides a robust linked data endpoint.
  • Neo4J is a graph database that provides a full SPARQL endpoint with scalable discovery potential.
  • The Dojo UI toolkit is aligned with HTML5 and can provide robust multi-context multi-screen UI models.

IoT Toolkit can provide a common object model and data model framework to enable endpoint discovery and transport abstraction as outlined earlier.

There are now enough components to enable IoT Toolkit to be a component of a larger system based on an Open Source IoT Stack, and focused on Interoperability and Web Standards.  

Model-View-Controller Structure in IoT Applications

The Internet of Things includes systems of feedback control, where observations of key attributes provide information that eventually creates a change in the system through actuation.

This cycle of Observation => Information => Actuation creates a feedback control loop where the observed property is controlled in a system known as a closed loop.

Figure 3 shows the typical structure of some closed feedback loops in IoT systems. There are autonomic feedback loops which involve only software making decisions. An example of this is a motion sensor turning on a light.


Figure 3 - IoT Control Loops and the Model-View-Controller macro-pattern

There are also cybernetic feedback loops, which involve a person in the loop participating in making decisions. In this example, the person in the cybernetic loop is updating settings of an autonomic loop controller.

In the above example, the Model is a representation or an abstraction of the physical things and their attributes, which informs a Controller. The Controller is a piece of software which makes actuation decisions based on the information, and sends actuation commands to the thing using it’s modeled affordances. The software goal is to maintain a desired state of the thing through it’s model.

In the above example, people interact with the system through a View, which is constructed based on information from the model. By observing information presented in views, people make conclusions about the state of the things and consequently make control decisions from which they update the controller.

Some basic mapping of components to a M-V-C macro pattern are shown in figure 4.

IoT Toolkit provides an object encapsulation of the data models and event models in the Smart Object API, and enables connection to physical objects. IoT Toolkit provides scalable resource discovery, adapts to diverse data formats and protocols, provides storage and persistence of objects and data streams, and facilitates a model-driven binding of physical objects to URLs.

Node-RED allows resources from IoT Toolkit Smart Objects to be wired directly to software handlers and other resources, for example actuators, displays, and analyzers. The internal event model of Node-RED roughly corresponds to MQTT, and as such makes a natural connection to the MQTT Observer resource of the Smart Object API.



Figure 4 - Mapping of Software Components to the M-V-C Macro Pattern

The Node Builder is a new component that will discover Smart Object and other resources and data sources, and create Node instances from them that can be wired into Node-RED flow graphs. The Node Builder can use the Semantic Proxy to search and link information from multiple diverse catalogs and repositories into event driven applications. The Node Builder will also provide APIs to facilitate the automatic and run-time use of discovery and linkage to build ad-hoc graphs based on real-time context.

Dojo Toolkit is an open source UI and application system that builds Web scale and mobile application components based on a UI toolkit. The Dashboard builder is another new component that enables Widgets and controller components based on Dojo Toolkit to be wired into Node-RED flows and organized onto user screens. Information from the data models of things will provide units, scale and qualititative interpretation metadata for controlling the presentation on diverse devices and for different use cases.

Model-View-Controller Workflow in the creation of application graphs is depicted in Figure 5.

The Node Builder discovers resources and data sources from catalogs and existing data models according to the requirements of the application. For example, if the application is a Smart Thermostat, the data sources would include temperature in the room being controlled, a control for the heating device in the room, information about windows and solar gain, local weather and weather forecast, and current and predicted room occupancy.

The developer then discovers, selects, or creates application software components and UI components to build the application behavior and UI from.


Figure 5 - Application Development Tools and Workflow

The Application developer then creates a graph. The resources and application components can be either wired together manually by drawing arcs with the tool, or components can be automatically assembled by the Node Builder based on a database of models (ML system) or user templates, etc. Outputs are connected to actuators or UI components which can be selected from the Dashboard builder.

At the end of the process, an application graph is deployed by Node-RED which connects the resources, creates a UI instance, and begins operating on the datastreams and events.

The Run-Time Architecture of the deployed system is shown in figure 6.

Diverse protocols are used to apply the most suitable M2M system for a given connection, whether from cloud to cloud, sensor to gateway, sensor to cloud, gateway to cloud, or data models being accessed from a catalog.

Data models from various catalogs are referenced to create a running instance of an application, and are used occasionally by the running application to discover or adapt to new contextual information.

A Local Control Gateway provides an always-on node for low power sensors to connect, and runs local applications for observing and controlling water flow and air movement. Other sensors connect to both gateway and Personal cloud service, and some sensors connect directly to cloud services.


Figure 6 - Example of Run-Time Deployment

An instance of IoT Toolkit is shown deployed as “Gateway-as-a-Service” to connect a service provider, in this case ThingWorx, into the system to observe and expose it’s resources.

User Interface devices can connect to cloud services or directly to gateways, though there is likely more hardware service capability available in services than gateways at present.

IoT Toolkit instances in the cloud service and gateway can both run application software, providing a local backup capability if internet service is disrupted, or for better response time, or for availability of hardware e.g. GPUs for analytics. Many gateways and services can co-operate to provide layers of control and analysis.

Building an Open Stack for the Internet of Things

The components for creating a robust end-to-end Open Source IoT stack, from sensor to web application, are now becoming available. As more of the use cases become understood and addressed, this will build out in several key areas.

By adding a few new components to glue application graphs together from resources, we can create web scale interoperable applications that can be distributed across local application gateways and cloud services.

Existing components can be integrated at web scale, using URIs to bind connections, to create a macro instance of the well-known Model-View-Controller pattern, which allows for parallel autonomic feedback loops and cybernetic feedback loops.

The Model-VIew-Controller macro pattern provides a framework for the structured division of responsibility between people and software in IoT applications. It also provides a framework for high level interoperability between data sources, control elements, and UI elements.

Such a stack can form the basis of a new service infrastructure, which can provide Platform-as-a-Service functionality while avoiding the single points of failure and lock-in typically associated with PaaS. Open Source PaaS can provide the mechanism for fault-tolerant run-anywhere application platform functionality needed by IoT applications as we begin to depend on that functionality more and more in our daily lives.