Skip to content

User Interface: Dashboard

Important Links

Project: Dashboard

The term dashboard is used with various meanings, in the scope of Soilwise the following uses are relevant:

Other parts of the dashboard are:

SoilWise Dashboard is intended to support the implementation of User stories, deliver useful and usable apps for various stakeholders, provide interface for user testing and present data and knowledge in useable way.

Search interface on metadata

A typical example of a catalogue search interface is the current ESDAC catalogue.

Ranking, relations, full-text search, and filtering

Optimal search capabilities are provided by the catalogue backend, this component leverages these capabilities to provide users with an optimal user interface to effectively use those capabilities.

The EJPSoil prototype, foreseen to be re-used within SoilWise, uses a tailored frontend, focusing on:

  • Paginated search results, sorted alphabetically, by date
  • Minimalistic User Interface, to prevent a technical feel
  • Option to filter by facets
  • Option to provide feedback to publisher/author
  • Readable link in the browser bar, to facilitate link sharing
  • Preview of the dataset (if a OGC:Service is available), else display of its spatial extent

What can be improved:

  • Show relations to other records
  • Better distinguish link types; service/api, download, records, documentation, etc.
  • Indication of remote link availability/conformance
  • If a record originates from (multiple) catalogues, add a link to the origin
  • Ranking (backend)

Technology

  • Jinja2 templates (html, css) as a tailored skin on pycsw/pygeoapi, or dedicated frontend (vuejs, react)
  • Leaflet/OpenLayers/MapLibre

Chatbot

Large Language models (LLM) enriched with SoilWise content can offer an alternative interface to assist the user in finding and accessing the relevant knowledge or data source. Users interact with the chatbot interactively to define the relevant question and have it answered. The LLM will provide an answer but also provide references to sources on which the answer was based, in which the user can extend the search. The LLM can also support the user in gaining access to the source, using which software, for example.

Map viewer

A light-weight client map viewer component will be employed:

  • as a frontend of Map Server component to visualize provided WMS, WFS, WCS layers
  • as an integrated part of the Catalogue to visualize primarily the geographical extent of data described in the metadata record and a snapshot visualization of the data
  • to provide the full preview of data, when a link to web service or data browse graphics (preview image) is available

A dedicated mapviewer, such as TerriaJS, can support users in accessing relevant data which has been prepared for optimal consumption in a typical GIS Desktop environment. For example maps of spatial distribution of soil properties or health indicators over Europe. A typical example is Soilgrids.

An interesting aspect of a community like EUSO is the ability to prepare and share a map with stakeholders to trigger some discussion on phenomena at a location.

Examine the need for viewing novel formats such as Vector tiles, COG, GeoZarr, GeoParquet directly on a map background. The benefit of these formats is that no (OGC) service is required to facilitate data visualisation.

Technology

TerriaJS is an environment to share maps (react+leaflet+cesium), but also create maps and share them with stakeholders.

Overview of catalogue content

Traditional dashboards

The INSPIRE Geoportal increased its usage with their new dashboard-like interface for each EU Member State the number of published datasets per topic is upfront in the application. Dashboards on catalogue content provide mechanisms to generate overviews of that content to provide such insight.

Technology

The EJP Soil Catalogue Dashboard is based on Apache Superset, with direct access to the PostgreSQL database containing the catalogue records. GeoNode recently implemented Superset, enabling users to define their diagrams on relevant data from the catalogue (as an alternative to a map viewer).

Dashboarding functionality is available in Geonetwork, using the Elastic Search Kibana dashboarding. Similar functionality for the pycsw needs to be investigated, verified respectively.

The source data for the dashboards is very likely enriched with AI-generated indicators. LLMs also seem able to provide overviews of sets of content.

Manual data & metadata authoring

Important Links

Project: Metadata authoring

The SWR provides two major ways of data & metadata authoring

  1. in an automatized manner, as described in the Harvester component;
  2. in a manual mode, as described within this Manual data & metadata authoring component.

Note that option (1) is the preferred one from the SWR point of view as it allows to massively tackle metadata and knowledge of remotely available resources, including Catalogue servers of Mission Soil projects, Zenodo, Cordis, INSPIRE Geoportal and others.

A manual mode comprises four levels of data and metadata upload (note that the following points 1 and 3 work for both data and metadata, while points 2 and 4 work solely for metadata):

  1. Manual upload of one data file and/or metadata record as a file: a user selects a file from the local drive or types in a publicly available URL and defines the (meta)data structure (for more details, see Data model component) and optionally other relevant information like target user group, open/restrict publication of this new data/metadata record etc. The Manual data & metadata upload component imports the file, assigns a UUID if needed, and stores the data in Zenodo, while metadata are stored in the Storage component.
  2. Manual upload of one metadata record as a source code: the functionality of the Manual data & metadata upload component is almost identical to the previous option (file upload). The only difference is that a user is copying a source code rather than a file upload.
  3. Manual batch upload: this option allows a user to import a set of data files and/or metadata records in the form of e.g. XML or JSON files. The following parameters need to be defined:
  4. directory, full path on the server’s file system of the directory to scan and try to import all the available files;
  5. file type, e.g. XML, JSON (note that the full list of supported file types needs to be elaborated yet);
  6. Manual connection to a Web service and semi-automatic extraction of available metadata: a user types in a publicly available URL pointing to a service metadata document, e.g. GetCapabilities response of OGC Web service. The Manual data & metadata upload component extracts metadata that can be copied in line with the desired metadata structure, i.e. a metadata profile. The Manual data & metadata upload component assigns a UUID if needed, and stores the metadata in the Storage component. Most likely, the metadata extracted from a service metadata document (e.g. GetCapabilities) would not be sufficient to address all the mandatory metadata elements defined by the desired metadata structure, i.e. a metadata profile. In such a case, a manual fill in would be needed

The diagram below provides an overview of the workflow of metadata authoring

flowchart LR
    G[fa:fa-code-compare Git] -->|mcf| CI{{pygeometa}} 
    CI -->|iso19139| DB[(fa:fa-database Database)]
    DB --> C(Catalogue)
    C --> G

The authoring workflow uses a GIT backend, additions to the catalogue are entered by members of the GIT repository directly or via pull request (review). Records are stored in iso19139:2007 XML or MCF. MCF is a subset of iso19139:2007 in a YAML encoding, defined by the pygeometa community. This library is used to convert the MCF to any requested metadata format.

A webbased form is available for users uncomfortable with editing an MCF file directly.

With every change on the git repository, an export of the metadata is made to a Postgres Database (or the triple store).

Foreseen functionality

  • GUI and backend for online form
  • validation of inserted values
  • storing inserted metadata record

Technology

  • pycsw and GeoNetwork includes capabilities to harvest remote sources, it does not include a dashboard to create and monitor harvesters
  • A Git based participatory metadata workflow as introduced in EJP Soil and foreseen for SoilWise as a follow-up
    • Users should be endorsed to register their metadata via known repositories, such as Zenodo, CORDIS, INSPIRE, ... at most register the identifier (DOI, CSW) of the record at EUSO, metadata will be mirrored from those locations at intervals
    • Data can be maintained in a Git Repository, such as the EJP Soil repository, preferably using a readably serialisation, such as YML
    • In EJP Soil we experiment with the metadata control file format (MCF), a subset of iso19139
    • A web editor for MCF is available at osgeo.github.io
    • Users can also submit metadata using a CSV (excel) format, which is converted to MCF in a CI-CD
    • A web based frontend can be developed to guide the users in submitting records (generate a pull request in their name)
    • Validation of inserted values
    • A CI-CD script which runs as part of a pull request triggers a validation and rejects (or optimises) a record if it does not match our quality criteria

Integration opportunities

The Manual data & metadata upload component will show its full potential when being in the SWR tightly connected to (1) SWR Catalogue, (2) metadata validation, and (3) Requires authentication and authorisation.

Open issues

The Manual data & metadata upload component shall be technologically aligned with the SWR Catalogue and Harvester. Both considered software solutions, i.e. GeoNetwork and pycsw support the core desired functionality all these three SWR components.

The above-described mechanisms showed the “as is” manual metadata upload. Nevertheless, it is foreseen that the SWR will also support “on-the-fly transformation towards interoperability”. Such functionality is currently under discussion. The desired functionality aims to assist data producers and publishers with a pipeline that allows them to map their source data & metadata structures into target interoperable data & metadata structures. An example of this is an example of an upload of interpreted soil data and their on-the-fly transformation into a structure defined by the INSPIRE Directive, application schema from data specification on Soil respectively. A data & metadata upload pipeline supporting “on-the-fly transformation towards interoperability” will be described in greater detail later in line with SoilWise developments.

Data download & export

A UI component could be made available as part of the SWR Catalogue application which facilitates access to subsets of data from a data download or API. A user selects the relevant feature type/band, defines a spatial or attribute filter and selects an output format (or harmonised model). The component will process the data and notify the user when the result is available for download. The API-based data publication is described as part of APIs.