Explanation below; Map Pictures - Shared Website Snapshots for colleagues operating at farms.
Farms have an international scope therefore their data design and analysis infrastructure needs to be able to span global view.
Elasticsearch / Kibana / Logstash (ELK Stack) purpose built for internet cyber security originally. The ELK stack is currently utilized across many domains as a data storage layer for general analysis purposes able to store petabytes level, if need be.
I have met a research group that operates on farms located in NWS Australia.
NSW Australia teams have developed regional soil and land use "WMS" - Web Mapping Systems can be linked into Elasticsearch. Data is owned by the government; there are processes in place for entities to access specific reports that are held in backend databases and PDF reports.
The ELK stack supports adding private data layers on top of public data sources that reference specific places. I have added a couple that reference a Orange Mountain Winery, and the Harvest Cafe adjacent to a school near the town of Borenore west of the city of Orange.
Map attributes can be displayed based illustrating user based points of interest.
Below are two fictitious private data events generated for this blog post referencing Dr. Abed Chaudhury. I often view LinkedIn pictures of Dr. Chaudhury out in the farm fields on bright sunny days. Based on a brief interaction with him in Eugene Oregon he might be drinking tea in the mornings and wine in the evenings.
Above comment about morning tea; below comment about wine.
The fictitious events I inserted for illustrative purposes that indicate there are methods to add private data overlays across public data maps.
Reference to the School below publicly available from Open Street Maps server with coordinates and user supplied data attributes.
OSM - Open Street Maps IOS/Android/ Windows and Linux clients and servers are open source. Reference points added into OSM are added as public data.
A common design pattern is to add private data attributes that can be correlated to users, coordinates and timestamps. The ELK data systems has many different avenues to create, append, translate, query, analyze and report consistently across teams. I have utilized ELK stack since 2015 to analyze Gigabytes of aircraft flight dynamics and stock market data sets. ELK is a platform to keep track of all the data across time, spatial and all the other data attributes that are collected.
The illustrations shown above are geospatially focused; many other visualizations are available to users easily built using Kibana, a server that provides UI's to build client user based dashboards by combining multiple visualizations.
In general many teams utilize simple tools like Excel or professional statistical analysis toolkits like MiniTab, JMP/SAS, SPSS along with R, Python and SQL. The central question remains - how to setup a collaborative environment to share, report and continually add new attributes across time.
Although not shown in the illustrations above ELK supports users, groups and dashboards that can be partitioned by project designs then shared to additional internal and external groups. Users can host their own ELK servers internally or utilize managed cloud solutions by numerous vendors.
General analysis concepts below:
X is a dimension, humans are commonly focused on time
For time Left of X is in the past
X is now
Right of X is in the future
Another common focus in our spatial locations.
Using time and space - X is where we are now in geospatial coordinates and time
Left of X is what the region was associated in the past
Right of X is what the region maybe like in the future
X can be arbitrary defined to be a parameter vector consisting of many attributes
As humans we often pose the question how will X change in the future given the past X.
At X we believe that we know what we need to define all the parameters
As we move toward the future we learn new parameters that are important.
Map makers have defined "layers" 20 years ago; computer science groups initially added databases.
Science disciplines always had their models based on systemic development of differential equations in biology, chemistry, botany, entomology, geology, genomics, hydrology, mycology, pedology, soils and all the functional specializations.
Each discipline will have different parameters, knowledge, processes, tools - common languages and models help to push it together.
For now I recommend that ELK infrastructure is a good toolset that meets the requirements. I have had a long relationship with Elastic.co as a user. I am not a reseller; I utilize their tool inside to convey my results to other colleagues.
Above and beyond that I specialize in multivariate data systems of systems analysis that helps discover nonlinear correlations enabling new insights across the domain of study.
Comments
Post a Comment