STANC: An open standard based spatial data Lakehouse Architecture for Archiving Heterogeneous Geo-IoT Data at Scale

A cloud-native lakehouse architecture for archiving heterogeneous Geo-IoT data at scale.

Objective

Develop a cloud-native lakehouse storage architecture using OGC SensorThingsAPI

Description

The OGC SensorThings API is an international standard that defines a comprehensive way to model, define and retrieve location enabled IoT sensor data. Although the standard serves as a unifying interface to interconnect and retrieve varying bulks of geospatial IoT data streams, it does not specify a way to store the ingested data. This serves as a great opportunity to use the standard for both analytical and operational use cases. The open-source implementations of the standard have mostly used relational database technologies that are best suited for operational use cases at small-medium volumes of data. It is well acknowledged that while RDBMS systems may be an excellent choice for managing small-medium scale geospatial relational data, they’re not the best solution when dealing with data at scale for analytical purposes. There is a need for an alternative storage layer for OGC SensorThings API. Hence, we present STANC (SensorThings Archival: Native to Cloud) which is a novel, cloud-native, spatial data lakehouse, based on OGC SensorThings API and Apache Iceberg.

Project Details

Collaborator(s): Humaid Kidwai

Highlights:

  • GeoParquet Community Day, San Francisco:
    • Travelled to San Francisco to participate in GeoParquet Community Day organized by the Cloud Native Geospatial Foundation to present a lightning talk on the initial design architecture of STANC
    • Talk: https://www.youtube.com/watch?v=i_GlnaOHEe8&t=1284s
  • Google Summer of Code:
    • Accepted to participate in Google Summer of Code 2024 as a contributor at 52° North Spatial Information Research GmbH
    • Designed & developed a prototype implementation of STANC using AWS services
    • Project: https://summerofcode.withgoogle.com/programs/2024/projects/fHFlLaVM

GitHub: https://github.com/humaidkidwai/sensorweb-server-sta

Date: 2023-2025