Introducing the Service Ecosystem developed by SAEON’s uLwazi Node

By André Engelbrecht, with contributions by Amelia Hilgart, SAEON uLwazi Node

For the past year, SAEON uLwazi has been developing a ‘data services ecosystem’ and websites for climate change adaptation for the Department of Environmental Affairs (DEA).

Each service serves as an individual component for a different aspect of climate adaptation, including adaptation monitoring and evaluation, climate change response, and detailing disaster impact along with additional services which tie the system together.

Below, these services and websites are contextualised to explain their purpose and how they integrate with each other to form an ecosystem, including:

Vocabulary Mediation Service (VMS) – A mediation/transformation service for vocabulary data that standardises data output structure and semantics
Identity Service (IDS) – A single-sign-on for all the systems
National Climate Change Response Database (NCCRD) – A database that hosts a variety of climate change responses and their details
National Hazardous Events Database (NHE) – A database that describes hazardous events to better understand how people, infrastructure and different economic sectors are impacted by an event
National Climate Change Information System (NCCIS) – A monitoring and evaluation questionnaire and database designed to identify the strengths and weaknesses in government and industrial climate change adaptation strategies

Please note that at the time of writing all these services are still in the beta development phase and have not yet been released to the general public.

Vocabulary Mediation Service (VMS)

First on the list is SAEON’s Vocabulary Mediation Service. A vocabulary is a set of terms or keywords constituting the range of allowed values for some variable. A vocabulary may be a simple list of terms, or it may be a complex hierarchy of terms, as in an ontology of hazards or South Africa’s administrative regions.

Many vocabulary lists can be retrieved by querying the appropriate online services. However, almost every vocabulary lookup service on the internet has its own standards for data input and output structure. This is far from ideal for a systems developer wanting to query a range of services, because each interface requires unique or specialised code.

SAEON’s VMS aims to solve this problem by providing a standardised input (query) and output structure for vocabulary data. This service acts as a mediator between external services and end users or systems, transforming all output into a specified standard structure.

The output standard can be specified as follows:

StandardVocabOutput

Items (List of StandardVocabItem)

StandardVocabItem

Id (String, Unique, Required)
Value (String, Required)
Children (List of StandardVocabItem, Optional AdditionalData (List of Key(String)-Value(String) pairs, Optional) )

This service currently offers only a handful of in-house stored vocabularies, but in time could be expanded to provide standardised access to external services as well.

Currently hosted vocabularies:

Hazards – Official hierarchy of hazards recognised by the South African government
Regions – Official hierarchy of South African regions (provinces, district municipalities and local municipalities)
Government departments – Official hierarchy of South African government departments, agencies and state-owned enterprises
Sectors – Official hierarchy of business sectors recognised by the South African government

All these lists are available as hierarchical or flat lists and offer the capability to search by name/value and to view details of individual entries.

This service is used by the NCCRD, NHE and NCCIS for data lookups, thus standardising vocabulary implementation and values across SAEON’s websites and services.

These standard vocabularies will be extended in future to serve additional systems commissioned by DEA and will also form an input into standards such as the National Framework for Climate Services and the National Framework for Risk and Vulnerability Assessment.

Identity Service (IDS)

This service forms an integral part of SAEON’s and DEA’s service ecosystem. Most of our websites and back-end services require authentication for at least some actions. The IDS offers a Single-Sign-On (SSO) experience across all of SAEON’s websites and services.

What this means is should an account be needed for any of the websites and/or services, a single account can be generated once-off. This global account is then made available to all the other web services or applications via the IDS.

Internally we used the latest technologies to keep you safe, such as IdentityServer v4 and ASP.Net Core 2 Identity. Login/logout is managed by a single portal and your identity is shared across our services. The Identity Service website was built using ASP.Net Core MVC.

National Climate Change Response Database (NCCRD)

The NCCRD is a portal for capturing and reporting climate change projects and their details. Each project can have any number of adaptation or mitigation actions, which are the individual actions under the main project.

The NCCRD also offers modern reporting capabilities with maps, charts, summary views and more to document and describe the technical and financial feasibility of an action, the projected reduction of greenhouse gases for each action, actions to address specific climate change hazards, the amount of funding going into different action types, and how many actions different industrial sectors are contributing to the database.

Data and report views from the NCCRD also feed into other systems like the NCCIS (described below), where statistics generated with data from the NCCRD, as well as a ‘widget’ view of the NCCRD, provide quick views of the data and access into the NCCRD.

National Hazardous Events Database (NHE)

The National Hazardous Events database offers a platform to capture hazardous events occurring across South Africa and their impacts on South Africans. A questionnaire was designed using indicators from the UN Sendai Framework for Disaster Risk Reduction and the UN Sustainable Development Goal projects recontextualised for the South African context to ensure that impacts to human health and habitation, various sectors of the economy and infrastructure are documented in a quantifiable way.

The data can be viewed in a dashboard containing location maps, charts and other views which specify the areas impacted, the total number of disasters, the amount of money directed towards disaster response, a breakdown of the kind of hazards that occur and the total number of injuries and fatalities.

Currently, the database contains the ‘declared disasters’ list maintained by the National Disaster Management Centre, but in the next phase of development, events detected via remote sensing, social media, crowdsourced contributions and online news items, among others, will also be added to give users a more holistic view of the scope of disastrous events occurring in South Africa.

Just as with the NCCRD, data from the NHE feeds into other systems such as the NCCIS, where you’ll see stats and a widget view of the NHE for quick access/reference. The NHE currently does not require authentication but might do so in future versions, in which case authentication will be governed by the IDS.

National Climate Change Information System (NCCIS)

The NCCIS functions as a monitoring and evaluation system which incorporates inputs from the NCCRD and NHE (with others in the future, including systems developed by service providers on behalf of DEA). The idea is to provide a unified view of climate change adaptation and events that result from climate change.

“The monitoring and evaluation (M&E) of adaptation responses is composed of building blocks and key elements which have been unpacked further into desired adaptation outcomes to identify specific aspects of climate resilience for direct monitoring and evaluation.

“The desired adaptation outcomes describe, in a general sense, a desired state that will enhance South Africa’s transition towards climate resilience and fall into two distinct groups. Six of the nine desired adaptation outcomes (G1-G6) describe the ‘inputs’ (e.g. processes, resources and capacities) that need to be in place to enable effective climate change adaptation; and three desired adaptation outcomes (G7-G9) describe the key ‘impacts’ of adaptation interventions and associated measures (e.g. reductions in vulnerability of human and natural systems).”

The goals were designed based on input from various international and national policies and frameworks including the Paris Climate Accords, the Sendai Framework for Disaster Risk Reduction, the Sustainable Development Goals, the UN Convention to Combat Desertification, and the UN Habitat Focus Areas.The first of these goals ties directly into the NCCRD system in the background via unique identifiers shared between the systems.

Summary of the Ecosystem

The VMS and IDS allow interactions between the NCCIS systems (which include the NCCRD and NHE) as well as other systems currently under development by SAEON uLwazi. The IDS makes it possible for an individual user to be identifiable to the entire CCIS (NCCIS, the NCCRD or the NHE) such that any information entered by that individual can then be tied together.

The VMS allows that data to be compared so that one can query how much damage occurred in a particular economic sector due do a specific type of hazardous event, to if any adaptation plans have been submitted by that sector for that event type, to an evaluation of the effectiveness of those plans. This allows for reporting on various levels to create a unique overview of the state of climate change in South Africa.

Extending the Ecosystem through external integration

Apart from all these systems integrating with each other, all these systems can also be integrated into external or third-party systems in various ways. This means that if, for example, the provincial government of Gauteng wanted their own CCIS, they can subset the scope of the CCIS services for their regions as a website, or embed those services directly into their existing website and have their own system for data capture that reports back to the national system. This is mostly made possible because each of the individual platforms offer API data services, but also because all these systems are open-source projects hosted on GitHub as public projects.

The following levels of external integration are possible:

Data-level integration: Province/local authority develops its own system, but exchanges data with DEA via interoperable services and using specified vocabularies.
Code-level integration: Province/local authority develops its own fork of DEA open source components; by definition these exchange data via interoperability services and use the designated vocabularies.
Component-level integration: Province/local authority embeds components hosted by DEA into their own systems. Interoperability and vocabulary implementation are implicit.
Configured instances: A subset of data and resources is configured and branded for a province/local authority, with systems hosted by DEA.

Specifications

All the websites were built using modern and responsive technologies such as React.js and MDBootstrap, which look and feel like products built by companies such as Google and Facebook, making the use of the system more intuitive to most experienced internet users.

For interested parties who are a little more technologically savvy, apart from the website, the NCCIS, NCCRD and NHE also offer OData specification API services for back-end data access. All the data stored in the databases are accessible via these API services. Because of the OData specification implemented, any of the endpoints offered via these APIs can be queried using standard OData query syntax. The APIs were built using modern technologies such as ASP.Net Core Web API and Entity Framework Core.