From GPS and virtual globes to spatial computing - 2020

Abstract

Spatial computing is a set of ideas, solutions, tools, technologies, and systems that transform our lives with a new prospect of understanding, navigating, visualizing and using locations. In this community whitepaper, we present a perspective on the changing world of spatial computing, research challenges and opportunities and geoprivacy issues for spatial computing. First, this paper provides an overview of the changing world of spatial computing. Next, promising technologies that resulted from the integration of spatial computing in the everyday lives of people is discussed. This integration results with promising technologies, research challenges and opportunities and geoprivacy issues that must be addressed to achieve the potential of spatial computing.

Appendices

Appendix A: Emerging applications

Spatial computing is paving the way for the realization of compelling visions in many domains. Below, we provide some examples, which include applications in national security, climate data analytics, and transportation, to name a few.

1.1 A.1 National security agency (NSA)

Of interest is knowing where more attention should be focused, knowledge discovery about entities, relationships, events, and questions, gleaning sufficient information to answer relevant questions and knowing what questions can be answered with current information. Challenges include big data, heterogeneous data (with differing resolution, confidence/trust/certainty, diagnosticity, and intentionality), data with spatial and temporal bias, and the ability to detect changes, trends, and anomalies.

1.2 A.2 National geospatial-intelligence agency (NGA) [22]

With the exponential growth in influx of images from a large variety of sensors (still through motion), high-end analytical processes that rely on accurate geospatial data as starting point are needed for registration, fusion, and activity-based intelligence or human geography analytics. Expectations of continuous improvements to geolocation accuracy continue to grow and rigorous Photogrammetry- based geo-positioning capabilities have become critical in developing a foundation for advanced GEOINT production [65] and exploitation [67].

1.3 A.3 National institute of environmental health sciences (NIEHS)

Spatial aspects, e.g., neighborhood context [88], are critical in understanding many contributors to disease including environmental toxicant exposure as well as human behavior and lifestyle choices. This exposome, a characterization of a persons lifetime exposures, is becoming an increasingly popular subject of research for public health [77].

1.4 A.4 National cancer institute (NCI)

Epidemiologists use spatial analysis techniques [12] to identify cancer clusters [75] (i.e., locations with unusually high densities) and track infectious disease such as SARS and bird flu.

1.5 A.5 National aeronautics and space administration (NASA)

Climate data is becoming more important to a wide range of applications. Spatial computing is important in the climate domain for climate data analytics which involves large, complex data sets. Server-side analytics and agile delivery of capabilities will be crucial for supporting spatio-temporal analytic code development and the technical capacity to build high-performance, parallel storage systems for spatio-temporal data collection (e.g., the idea of canned, canonical spatio-temporal ops is very appealing).

1.6 A.6 NSF earthcube

Both science and society are being transformed by data. Modern geo-science involves large heterogeneous datasets and computationally intensive, integrative, and multi-scale methods. Multidisciplinary collaborations across individuals, groups, teams, and communities are needed to address the complexity. The current sea of data from distributed sources, central repositories, sensors, etc., is ushering a new age of observation and analytics. Earthcube is trying to address these new realities by developing a distributed, community-guided cyber infrastructure to publish, discover, reuse, and integrate data across the geosciences.

1.7 A.7 NSF SEES

Support is needed for the constellation of problems in the geosciences the core evolving basic and applied sciences of understanding the entire Earth and its physics (e.g., ocean, atmosphere and land), biology (e.g., plants animals, ecology), sociology (e.g., sustainable economic development, human geography), etc. For example, there is a growing need for a cyber- infrastructure [11] to facilitate our understanding of the Earth as a complex system. Technological advances have greatly facilitated the collection of data (from the field or laboratory) and the simulation of Earth systems. This has resulted in exponential growth of geosciences data and the dramatic increase in our ability to accommodate complexity in models of Earth systems. These new data sources, referred to as Spatial Big Data, surpass the capability of current spatial computing systems to process efficiently. New research into massively scalable techniques for processing and mining Spatial Big Data via novel cyber-infrastructures will be key for Geo-Informatics.

1.8 A.8 U.S. department of transportation (DOT)

With the advances in spatial computing technologies (e.g., IntelliDrive, navigation, gps, etc.), novel transportation interactions are being sought such as vehicle to vehicle communications (speed, location, brake status, etc.) and vehicle to infrastructure communications (e.g., curve speed warning, red light violation warning, etc.) for improving situational awareness (where a vehicle can see? nearby vehicles and knows roadway conditions that remain unseen to the driver) and reducing or even eliminating crashes through driver advisories, warnings, or vehicle control augmentation. Spatial Computing will enable connections among moving objects such as cars, pedestrians, and bicycles, to help avoid collisions or coordinate movement using Dedicated Short Range Communications (DSRC). Transportation agencies and automotive manufacturers are pursuing this vision under the IntelliDrive initiative [34, 79]. For example, the U.S.DOT recently announced a challenge to explore the question: When vehicles talk to each other, what should they say?, aiming to make driving safer and more efficient [5].

1.9 A.9 U.S.DOJ/NIJ

Public safety professionals use spatial analysis to identify crime hotspots to select police patrol routes, social interventions, etc.

1.10 A.10 FAA

Current air-traffic control systems rely on radar. Due to the imprecision of this technology, large gaps between aircraft are required to ensure safety and avoid collisions. Consequently, the air space over America has become more and more congested, with the military needing to open up reserved air space over holiday weekends. If air traffic control systems were switched to a next-generation GPS-based system, the large gaps between aircraft would no longer be needed as the traffic controllers would have much more precise data. The Federal Aviation Administration (FAA) is actively exploring this vision to relieve congestion in many air corridors [34].

1.11 A.11 U.S.DOE

Interesting new opportunities exist for bio-fuels and eco-routing. For bio-fuels, there is the challenge of diminishing returns due to their relatively low energy content and the inherent trade-off between the energy required for processing and transportation versus the energy produced. Therefore, determining the location of bio-fuel processing plants is an important consideration. GPS navigation services [28] are just beginning to experiment with providing eco-routes which aim to reduce fuel consumption, as compared to reducing distance traveled, or time spent. These techniques along with smarter suggestions for ride sharing and public transportation will enable significant fuel conservation. The rise of Spatial Big Data may enable computers to suggest not only compatible ride- share partners, but they may lead to retooled bus routes based on the spatio-temporal movements of individuals. With these new data sources, can we develop efficient and privacy-preserving techniques to automatically suggest public transportation, compatible ride-share partners and smart driving routes [5, 79]?

1.12 A.12 DHS

The Department of Homeland Security provides the coordinated, comprehensive federal response in the event of a terrorist attack, natural disaster or other large-scale emergency while working with federal, state, local, and private sector partners to ensure a swift and effective recovery effort. They focus on three critical components of emergency management: incident management, resource management, and supply chain management. Overall, the efficacy and performance of emergency management depend not only on how well each individual component performs but, more important, on the performance of the overall integrated system.

1.13 A.13 FCC

The Federal Communications Commission is collaborating (with FEMA and the wireless industry) on the Commercial Mobile Alert System (CMAS) for geo-targeting emergency alerts to specific geographic areas through cell towers, which pushes the information to dedicated receivers in CMAS-enabled mobile devices. The potential of this system is already evident due to recent events when hurricane Sandy flooded the streets of Manhattan and many New Yorkers received text message alerts on their mobile phones that strongly urged them to seek shelter.

1.14 A.14 IBM smarter planet

The initiative seeks to highlight how forward-thinking leaders in business, government and civil society around the world are capturing the potential of smarter systems to achieve economic growth, near-term efficiency, sustainable development and societal progress [69, 98].

1.15 A.15 ESRI geo-design

Geodesign [10] provides a design framework and supporting technology for professionals to leverage geographic information, resulting in designs that more closely follow natural systems. These systems can be used for monitoring a variety of Earth resources (e.g., agriculture fields, fresh water lakes, etc.) and trends (e.g., deforestation, pollution, etc.) for timely detection and management of problems such as impending crop failures and crop-stress anywhere in the world.

1.16 A.16 Many more

In addition to these examples, numerous problems faced by many organizations are pushing the limits of spatial computing technology.

Appendix B: Representative spatial computer science questions

1.1 B.1 Collaborative systems

How can computation overcome geographic constraints such as transportation cost, language [2] and cultural variation across locations?

1.2 B.2 Theory and algorithm design

Can we design new algorithm paradigms for spatio-temporal problems, as these problems violate the dynamic programming assumptions of stationary ranking of candidates? How can one design robust representations and algorithms for spatio-temporal computation to control the approximation errors resulting fro discretization of continuous space and time? What are scalable and numerically robust methods for computing determinants of very large sparse (but not banded) matrices in context of maximum likelihood parameter estimation for spatial auto-regression mode?

1.3 B.3 Software

For the best balance between performance and flexibility, what it the appropriate allocation of spatial data-types and operations across hardware, assembly language, OS kernel, run-time systems, network stack, database management systems, geographic information systems and application programs?

1.4 B.4 Hardware

Which spatio-temporal computations are hard to speed up with GPUs? multi- core? map-reduce? Which benefit? How may one determine location of a person (or device) despite challenges of motion, GPS-signal jamming, GPS-signal unavailability indoor, etc.?

1.5 B.5 Security and privacy

How may one authenticate location of a person or device despite the challenges of motion, location-spoofing, physical trojan-horses, etc.? Does GPS-tracking violate privacy? What is the relationship between the resolution of spatio-temporal data and privacy? How do we quantify privacy of spatio-temporal data? What computational methods can enhance the privacy of spatio-temporal data?

1.6 B.6 Networks

How may one determine, authenticate and guarantee the location of an Internet entity (e.g., client, server, packet) despite autonomy, heterogeneity, transparency, etc?

1.7 B.7 Data - database

How may we reduce the semantic gap between spatio-temporal computations and primitives (e.g., ontology, taxonomies, abstract data-types) provided by current computing systems? How do we store, access, and transform spatio-temporal concepts, facilitating data sharing, data transfer, and data archiving, while ensuring minimum information loss? How do we fuse disparate spatial data sources to understand geographic phenomena or detect an event, when it is not possible via study of a single data source?

1.8 B.8 Data - data analytics

How may machine learning techniques be generalized to address spatio-temporal challenges of auto-correlation, non-stationarity, heterogeneity, multi-scale, etc.? How can we elevate data analytics above current engineering practices to incorporate scientific rigor (e.g., reproducibility, objectiveness)? How can spatio-temporal data be analyzed without compromising privacy? How can frequent spatio-temporal patterns [19] be mined despite transaction-induced distortions (e.g., either loss or double-counting of neighborhood relationships)? How can data analytic models be generalized for spatio-temporal network data (e.g., crime reports in cities) to identify patterns of urban life? What can be mined from geo-social media logs, e.g., check-ins, mobile device trajectories, etc.? How may one estimate evacuee population? Traffic speed and congestion? Urban patterns of life?

1.9 B.9 Visualization, graphics

How may one visualize spatio-temporal datasets with uncertainties in location, time and attributes? How can we automate map creation similar to attempts in the database field to automate database administration tasks (e.g., index building, etc)?

1.10 B.10 Artificial intelligence

What are components of spatial intelligence? Can computers have as much spatial intelligence as humans?

1.11 B.11 Spatial reasoning

How can computational agents reason about spatio-temporal concepts (e.g., constraints, relationships)?

1.12 B.12 Spatial cognition

How can spatial thinking enhance participation in STEM fields? How do humans represent and learn cognitive maps? What is impact of GPS devices on human learning? What is the spatial computing impact of changing to a mobile ego-centric frame of reference from an earth-centric frame such as latitude, longitude, and altitude?

1.13 B.13 Human computer interaction

How can user interfaces exploit the new generation of miniature depth cameras that will be integrated with mobile and wearable devices? What kinds of interaction tasks can be performed more efficiently and more accurately with these systems? How can ubiquitous interactive room-scale scanning and tracking systems change the way in which we interact with computers and each other? How can we create user interfaces that bridge the gap between spatial computing “in the small” (typically on indoor desktop systems with stereo displays and precise 3D tracking) and spatial computing “in the large” (typically outdoors using coarse GNSS on mobile/wearable devices)?

Appendix C: Platform trends

The main platform trends stem from Graphics and Vision, Interaction Devices, LiDAR, GPS Modernization, Cell Phones, Indoor Localization, Internet Localization, and Cloud Computing. These platform trends are summarized below.

1.1 C.1 Graphics and vision

Increases in the scale and detail of virtual models are driven by the desire for worlds that are more complete, detailed, varying, and realistic. Significant advances in graphics hardware will make it feasible to deal with much larger scales. For larger scale and more detailed models, representation, creation, and usage must be considered. Representation needs to be considered because all details cannot be stored for highly detailed models. Creation is important because precise manual descriptions of virtual models are not possible. Usage is critical because processing with new models is non-trivial and things are possible that were not possible before.

1.2 C.2 Interaction devices

The democratization of technology has lead to ubiquitous computation and sensing. Commonly available interaction devices include smartphones (with multi-core CPU, GPU, Wi-Fi, 4G, GNSS, accelerometers, gyros, compass, cameras), game controllers (with Accelerometers, gyros, compass, cameras, depth cameras, electromagnetic trackers), and desktop peripherals (e.g., cameras). New challenges arise in bridging the gap between geospatial and 3D user interfaces (e.g., large to small, outdoors to indoors, coarse to fine, position/ orientation to full body pose, Hz to kHz).

A key trend here is the proliferation of depth camera systems. These first entered consumer devices through game console peripherals designed to sense users a few meters away from the display (Kinect for Xbox). However, there is now a new generation of inexpensive camera-based depth tracking systems for desktop applications that work in the sub-meter and even sub-foot range: Microsoft Kinect for Windows, PrimeSense Carmine, PMD Technologies, SoftKinetic DepthSense, Creative Interactive Gesture Camera). These devices and their SDKs support interactive tracking of 3D full body pose (at a distance), head/hand/finger tracking (up close), and modeling of the environment when the device can be moved around (e.g., KinectFusion).

1.3 C.3 Localization

Next generation localization includes image-based, indoor-based, and internet-based techniques. Due to the prevalence of mobile/handheld devices [50] with numerous sensors (e.g., smart phones) and the recent advances in computer vision and recognition, image-based localization is an emerging trend for both indoor and outdoor localization. The idea is to take a query image with a mobile device equipped with sensors (e.g., gyros, GPS, accelerometers), build a geo-tagged image database (preferably 3D), retrieve the “best” match from the database, and recover the pose of the query image with respect to the retrieved image database. This has application in augmented reality and location-based advertising and services. For indoor localization, augmented reality has interesting challenges when dealing with a wide range of scales/resolutions and conditions. Examples of scale include finding a meeting room in a building, finding a paper in the room, finding an equation on the paper, determining which variable is the weighting in the equation, etc. Trends involve optimization for what matters and using all sources (e.g., large + detailed models, constraints, inferences, cloud, etc.). For internet-based localization, tremendous possibilities exist as we move to cm/dm real-time starting with networked differential GPS at sub-meter scales.

1.4 C.4 GPS modernization

With land area of approximately 1.5×108 km2, human population of about 7 billion people, number of cell phones at 5.6 billion (80 % of the world), and number of seconds per year at 3.14 x 107, map making at human scales, particularly in developing countries, is a challenge. Interesting opportunities have arisen in geodetic support for disaster relief amid very little data, validation, crowd sourcing, and crowd mapping.

1.5 C.5 Mobile devices

With the ubiquity of cellphones, interesting questions arise such as how may one overcome challenges of limited user attention, display, power, etc? How can one accurately determine location (and orientation) of mobile clients in GPS-denied spaces such as indoors and underground? What can be mined from geo-social media logs, e.g., check-ins, mobile device trajectories, etc [45]?

1.6 C.6 Cloud computing

The advent of big spatio-temporal data has raised interesting challenges such as which spatio- temporal computations are hard to speed up with cloud computing and which benefit. New challenges in spatio- temporal graphs, streaming spatial data, load balancing, distributed query processing [58] and data partitioning should be considered.