HgIS

Správa a analýza dat o životním prostředí
Environmental data management and analysis

User Tools

Site Tools


en:concept

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
en:concept [2019-04-21]
Kamil Nešetřil [Business Intelligence]
en:concept [2019-07-14] (current)
Line 1: Line 1:
 ====== System Architecture ====== ====== System Architecture ======
-This is the technical description of HgIS. It is not very useful for users not interested in information technology.+<WRAP box> ​This is the technical description of HgIS. It is not very useful for users not interested in information technology.
  
-The architecture of the system corresponds to a spatial business intelligence solution (GeoBI) – a combination of BI and GIS.+The architecture of the system corresponds to a spatial business intelligence solution (GeoBI) – a combination of BI and GIS. </​WRAP>​
 ===== Analysis ===== ===== Analysis =====
 The data needed to develop groundwater models are of different types. They are time-dependent and fully 3D. The source data exist in different formats as databases, data exchange formats (e.g. DBF, XML or flat files), archive data (e.g. MS Excel or MS Word), spatial data (e.g. ESRI SHP, KML or geodatabases). These data need to be retrieved into a single data structure to be used together. Highly structured data usually do not contain any interpretation or additional knowledge. Therefore it is necessary to adequately store and process all types of data. Some should be saved in a structured form so it can be further used (creating graphs, tables, maps, cross sections, etc.). Other data are used ad-hoc in the form it was obtained in so it is sufficient just to be stored and accessible – e.g. in the file system. The data and processes are depicted in table 1 that stands for the data flow diagram. The data needed to develop groundwater models are of different types. They are time-dependent and fully 3D. The source data exist in different formats as databases, data exchange formats (e.g. DBF, XML or flat files), archive data (e.g. MS Excel or MS Word), spatial data (e.g. ESRI SHP, KML or geodatabases). These data need to be retrieved into a single data structure to be used together. Highly structured data usually do not contain any interpretation or additional knowledge. Therefore it is necessary to adequately store and process all types of data. Some should be saved in a structured form so it can be further used (creating graphs, tables, maps, cross sections, etc.). Other data are used ad-hoc in the form it was obtained in so it is sufficient just to be stored and accessible – e.g. in the file system. The data and processes are depicted in table 1 that stands for the data flow diagram.
  
-Table 1. Data flows – data sorted from structured to unstructured:+//Table 1. Data flows – data sorted from structured to unstructured//
 ^  Data source ​                                                          ​^ ​ Storage ​               ^  Usage                                                                         ​^ ​ Content ​                     ^ ^  Data source ​                                                          ​^ ​ Storage ​               ^  Usage                                                                         ​^ ​ Content ​                     ^
 | Structured and semi-structured data – observations (databases, files) ​ | Data warehouse ​         | Reporting, visualization incl. geological profiles and cross sections, export ​ | Data                          | | Structured and semi-structured data – observations (databases, files) ​ | Data warehouse ​         | Reporting, visualization incl. geological profiles and cross sections, export ​ | Data                          |
Line 15: Line 15:
 ===== Business Intelligence ===== ===== Business Intelligence =====
 Groundwater information management can be described as loading of both archive and actual data (that are not modified anymore) from diverse (structured and semi-structured) sources; visualization of data in tables and graphs (downloadable in common formats as MS Word and MS Excel), data analysis, and model development. The same description corresponds to a completely different discipline – to Business Intelligence (<​nowiki>​BI</​nowiki>​),​ where <​nowiki>​BI</​nowiki>​ uses data about a company to support its manager’s decision-making. Groundwater information management can be described as loading of both archive and actual data (that are not modified anymore) from diverse (structured and semi-structured) sources; visualization of data in tables and graphs (downloadable in common formats as MS Word and MS Excel), data analysis, and model development. The same description corresponds to a completely different discipline – to Business Intelligence (<​nowiki>​BI</​nowiki>​),​ where <​nowiki>​BI</​nowiki>​ uses data about a company to support its manager’s decision-making.
-Therefore HgIS utilizes [[http://​community.pentaho.com|Pentaho]] – the <​nowiki>​BI</​nowiki>​ platform. It is a Java-based product of Hitachi Vantara with an open-source version. The Pentaho platform contains ETL – Pentaho Data Integration PDI. Reports designed by Pentaho Report Designer (<​nowiki>​PRD</​nowiki>​) can be run on a local computer or on the <​nowiki>​BI</​nowiki>​ application server Pentaho ​Business Analytics ​(PBA). <​nowiki>​PBA</​nowiki>​ facilitates users to design dashboards, analyze OLAP cubes etc. The Pentaho platform can be easily integrated or embedded into other applications.+Therefore HgIS utilizes [[http://​community.pentaho.com|Pentaho]] – the <​nowiki>​BI</​nowiki>​ platform. It is a Java-based product of Hitachi Vantara with an open-source version. The Pentaho platform contains ETL – Pentaho Data Integration PDI. Reports designed by Pentaho Report Designer (<​nowiki>​PRD</​nowiki>​) can be run on a local computer or on the <​nowiki>​BI</​nowiki>​ application server Pentaho ​Server ​(PBA). <​nowiki>​PBA</​nowiki>​ facilitates users to design dashboards, analyze OLAP cubes etc. The Pentaho platform can be easily integrated or embedded into other applications.
  
-===== Hydrogeological information system ​HgIS =====+===== HgIS =====
 HgIS is an information system developed at the Technical University of Liberec in the Czech Republic. Its purpose is to load data from the available data sources of any kind, to visualize and analyze data (to support the formulation of alternative conceptual models) and to implement simple models based on the data. Table 2 shows how specific kinds of data are managed in HgIS. Although it is focused on groundwater,​ it is also being used for the broader range of environmental data. HgIS is an information system developed at the Technical University of Liberec in the Czech Republic. Its purpose is to load data from the available data sources of any kind, to visualize and analyze data (to support the formulation of alternative conceptual models) and to implement simple models based on the data. Table 2 shows how specific kinds of data are managed in HgIS. Although it is focused on groundwater,​ it is also being used for the broader range of environmental data.
  
-Table 2. HgIS architecture+//Table 2. HgIS architecture//
 ^  Data source ​                                     ^  ->                              ^  Storage ​                                                        ​^ ​ ->                        ^  Usage                                     ^ ^  Data source ​                                     ^  ->                              ^  Storage ​                                                        ​^ ​ ->                        ^  Usage                                     ^
 | Observations (XML, MS Excel, flat files, SQL databases) ​ | ETL (PDI)                        | Data warehouse (PostgreSQL) ​                                       | BI Platform (Pentaho) ​    | Reporting, visualization,​ procedural models ​ | | Observations (XML, MS Excel, flat files, SQL databases) ​ | ETL (PDI)                        | Data warehouse (PostgreSQL) ​                                       | BI Platform (Pentaho) ​    | Reporting, visualization,​ procedural models ​ |
Line 29: Line 29:
  ​\\ ​  ​\\ ​
 {{:​en:​schemahgis_en.png?​direct|The architecture of HgIS – arrows denote data flow}} {{:​en:​schemahgis_en.png?​direct|The architecture of HgIS – arrows denote data flow}}
-Figure ​1. HgIS architecture+//FigureHgIS architecture//
  
 ===== Database – data warehouse ===== ===== Database – data warehouse =====
-It is reasonable to use an existing data model for the newly developed information system. We have reviewed available data exchange standards and data models as GWML, other application schemas of GML, INSPIRE, Hg2O, Arc Hydro Groundwater,​ Data Model of National Groundwater Information System, H+ and some others (see [[en:​competitors|]] and [[en:​standards|]]). None of those data models was used. Besides some other issues, some were not suitable for needs of groundwater practitioner;​ some were too concise or not sufficiently documented. All data models, data exchange formats and data models of EDMS were reviewed and used as an inspiration for the developed data model. <wrap hide>​Visualization of the hydrogeological data on a desktop computer can be easily performed with EnviroInsite from EI LLC (enviroinsite.com) ​– low priced software in .NET. It can be used to display maps (including localized tables and graphs), technical documentation of boreholes, geological cross-sections,​ 3D geological models and interpolation in 2D and 3D. The data model of HgIS is based on an existing data model of EnviroInsite. Therefore the database and the visualization software have consistent data structures that reduce the need for non-unique data transformation,​ and so it does not confuse users.+It is reasonable to use an existing data model for the newly developed information system. We have reviewed available data exchange standards and data models as GWML, other application schemas of GML, INSPIRE, Hg2O, Arc Hydro Groundwater,​ Data Model of National Groundwater Information System, H+ and some others (see [[en:​competitors|]] and [[en:​standards|]]). None of those data models was used. Besides some other issues, some were not suitable for needs of groundwater practitioner;​ some were too concise or not sufficiently documented. All data models, data exchange formats and data models of EDMS were reviewed and used as an inspiration for the developed data model. <wrap hide>​Visualization of the hydrogeological data on a desktop computer can be easily performed with EnviroInsite from EI LLC – low priced software in .NET. It can be used to display maps (including localized tables and graphs), technical documentation of boreholes, geological cross-sections,​ 3D geological models and interpolation in 2D and 3D. The data model of HgIS is based on an existing data model of EnviroInsite. Therefore the database and the visualization software have consistent data structures that reduce the need for non-unique data transformation,​ and so it does not confuse users.
 The original data model of EnviroInsite (9 tables) was extended to 36 tables because the EnviroInsite data model contains the data relevant for visualization only. The original tables were extended by additional fields and the model was further normalized.</​wrap>​ The data model contains data on:  The original data model of EnviroInsite (9 tables) was extended to 36 tables because the EnviroInsite data model contains the data relevant for visualization only. The original tables were extended by additional fields and the model was further normalized.</​wrap>​ The data model contains data on: 
   * observation objects*,   * observation objects*,
en/concept.txt · Last modified: 2019-07-14