Super Spatial Search Options with Data Hub

Last year, we introduced a powerful new search feature to Data Hub (back then the product was still known as XYZ). Data Hub is our geospatial data storage system with powerful APIs and integration with HERE Studio. This new feature let you search against properties in your spaces and joined the already powerful spatial search features provided by the APIs. As I've not yet written about the ways you can perform spatial searches with your Data Hub features, I thought an overview would be useful. These search features would all be useful to help direct your users to the resources they need based on their location.

Before I begin, lets take a look at the data. I'm using data from the American National Parks Service. They provide a GeoJSON file (https://www.nps.gov/lib/npmap.js/4.0.0/examples/data/national-parks.geojson) that I've enhanced with calls to their API to add more data. I spoke more about this here: Using HERE Studio to Map National Parks The end result is a GeoJSON file with near 400 features covering America's national parks. Here's one as an example:

        {
    "type": "Feature",
    "id": 0,
    "properties": {
        "phoneNumber": "6175661689",
        "emailAddress": "frla_interpretation@nps.gov",
        "activities": "Guided Tours,Junior Ranger Program,Museum Exhibits,Self-Guided Tours - Walking",
        "directionsInfo": "Site is located on the southwest corner of Warren and Dudley Streets in Brookline, south of Route 9, near the Brookline Reservoir.\n\nSite is 0.7 miles from the Brookline Hills MBTA stop on the Green Line, D Branch.",
        "directionsUrl": "http://www.nps.gov/frla/planyourvisit/directions.htm",
        "url": "https://www.nps.gov/frla/index.htm",
        "weatherInfo": "Summer: Warm temperatures, average high temperature around 80 degrees Fahrenheit, often with humidity. July and August bring the hottest temperatures.\nFall: Cooler temperatures, mean temperatures between 45 and 65 degrees Fahrenheit, sometimes rainy. Peak fall foliage is in mid-October.\nWinter: Cold, with snow, average low temperature around 25 degrees Fahrenheit. \nSpring: Cold to cool temperatures, average mean temperatures between 40 and 60 degrees Fahrenheit.",
        "name": "Frederick Law Olmsted",
        "topics": "Civil War,Landscape Design,Schools and Education,Wars and Conflicts",
        "description": "Frederick Law Olmsted (1822-1903) is recognized as the founder of American landscape architecture and the nation's foremost parkmaker. Olmsted moved his home to suburban Boston in 1883 and established the world's first full-scale professional office for the practice of landscape design. During the next century, his sons and successors perpetuated Olmsted's design ideals, philosophy, and influence.",
        "image": "https://www.nps.gov/common/uploads/structured_data/3C853BE9-1DD8-B71B-0B625B6B8B89F1A0.jpg",
        "designation": "National Historic Site",
        "parkCode": "frla",
        "fullName": "Frederick Law Olmsted National Historic Site"
    },
    "geometry": {
        "type": "Point",
        "coordinates": [
            -71.13112956925647,
            42.32550867371509
        ]
    }
},

  

You can view the raw file on GitHub if you wish: https://github.com/cfjedimaster/heredemos/blob/master/misc_related/national-parks-studio.geojson. So now that you have an idea of the data, let's talk about the different ways you can search.

The API

All the examples will be making use of the Data Hub REST APIs. You can find the latest reference documentation at https://xyz.api.here.com/hub/static/swagger/. They will all use one particular space that contains the data described above. I'm also going to use a key generated by our token manager that provides read only access to my spaces (making it safe to share here).

Find Features in a Circle

The first spatial search we'll demonstrate is a simple "everything in a circle" filter. Given a center point, and a radius, what features exist within it? The endpoint for this search looks like this:

        https://xyz.api.here.com/hub/spaces/${SPACE_ID}/spatial?access_token=${ACCESS_TOKEN}&lat=${LAT}&lon=-${LON}&radius=${RADIUS}

  

You pass in a location (lat and lon) and a radius in meters. Here's a concrete example that searches for parks around a point centered in Lafayette, Louisiana:

        https://xyz.api.here.com/hub/spaces/${SPACE_ID}/spatial?access_token=${ACCESS_TOKEN}&lat=30.22478&lon=-92.02402&radius=300000

  

This should roughly correspond to the map below (generated using our awesome Map Image API):

The result is a GeoJSON object which means you can access any results in the features key:

If you modify the radius to make it larger or smaller then the number of results change accordingly. Here's a complete script written in Node.js.

        require('dotenv').config();

const fetch = require('node-fetch');
const ACCESS_TOKEN = process.env.ACCESS_TOKEN;

const SPACE_ID = 'ylfzY538';

fetch(`https://xyz.api.here.com/hub/spaces/${SPACE_ID}/spatial?access_token=${ACCESS_TOKEN}&lat=30.22478&lon=-92.02402&radius=300000`)
.then(res => {
    return res.json();
})
.then(res => {
    // show one sample
    if (res.features.length >= 1) console.log(JSON.stringify(res.features[0].properties, null, '\t'));
    console.log(res.features.length + ' results\n\n');
})
.catch(e => {
    console.error(e);    
});

  

To keep the output a bit simpler, the script only displays the first result and the total found.

Find Features in a Bounding Box

The next way to search for features is via a bounding box. As you can imagine, this is literally creating a box on a map and returning features inside that box. To use this feature you need four parameters:

• west - the longitude of the western (or left) side of the box
• north - the latitude of the northern (or top) side of the box
• east - the longitude of the eastern (or right) side of the box
• south - the latitude of the southern (or bottom) side of the box

In order to get some values for a test, I opened HERE WeGo and selected two locations, one to the northwest of Lafayette and one to the southwest. I then used the "See more info" box to grab the longitude and latitude of those selections.

This gave me two coordinates: 32.82697, -94.24928 and 29.9372, -90.11567. I can take these values and use them to perform my API call with the following URL:

        https://xyz.api.here.com/hub/spaces/${SPACE_ID}/bbox?access_token=${ACCESS_TOKEN}&west=-94.24928&north=32.82697&east=-90.11567&south=29.9372

  

The API supports more options (all my examples today are just that, examples of what can be done, so be sure to check the docs for full listings of features) but one in particular may interest you. Any feature that's "clipped" by the box, i.e. a park that may be part in and part out, will be included in the result. You can disable this by passing clip=false in your call.

Here's another Node.js script (and remember, it's a REST API so you can use any darn language you prefer!) that performs a bounding box search:

        require('dotenv').config();

const fetch = require('node-fetch');
const ACCESS_TOKEN = process.env.ACCESS_TOKEN;

const SPACE_ID = 'ylfzY538';

fetch(`https://xyz.api.here.com/hub/spaces/${SPACE_ID}/bbox?access_token=${ACCESS_TOKEN}&west=-94.24928&north=32.82697&east=-90.11567&south=29.9372`)
.then(res => {
    return res.json();
})
.then(res => {
    // show one sample
    if (res.features.length >= 1) console.log(JSON.stringify(res.features[0].properties, null, '\t'));
    console.log(res.features.length + ' results\n\n');
})
.catch(e => {
    console.error(e);    
});

  

For our space, this returns four features, and like before, if you expand or shrink the box your total number of results will change.

Find Features in Geometric Shapes

If a bounding box isn't quite specific enough, you can create a more detailed search area by defining multiple different geometric shapes. GeoJSON supports different ways of defining the location of an object, and each of those ways can also be used to find features in Data Hub. In case you aren't up to date with all the various capabilities of GeoJSON, here's what's supported, and what it means for searching.

• Point and MultiPoint: Lets you match against features that intersect with the point or points.
• Line and MultiLine: This lets you create a line and determine if items are found along that line. The radius value becomes important here as you can create "thicker" lines that let you search for features along a route. So imagine a line geometry that follows a road. By using a thick radius, you could find features that are near the road.
• Polygon and MultiPolygon: Given a polygon, find features inside it.

While it isn't necessarily light reading, I do encourage folks to read the GeoJSON specification, as it contains full details on how these geometries are formed.

In order to use this feature you'll use the spatial endpoint from the first example, but with POST instead of GET. You'll use the post body to pass in your geometry to search.

Let's consider an example. Using HERE WeGo again, I clicked 4 times around the state of Louisiana to create a basic (and crude) polygon for the state. Every time I clicked I made note of the latitude and longitude.

Next - I checked the GeoJSON spec for polygons and saw it had this basic form:

        {
    "type":"Polygon",
    "coordinates":[
        [ 
            [lon1, lat1], 
            [lon2, lat2],
            [lon3, lat3],
            [lon4, lat4]
            [lon1, lat1],
        ]
    ]
}

  

The coordinates aspect is a bit complex. First, it's an array of arrays where the first item in the array is the polygon and items 2 through N would be "exclusions" or holes in the polygon. Since my polygon doesn't have holes, I only have the first item. Next, notice that in GeoJSON, the first item in a coordinate is the longitude, not the latitude. Most people right locations in longitude,latitude format. Finally I had to be sure to "close" the polygon by including the first point as the last point.

In the script below, I've modified my previous search against the spatial endpoint to use POST, specify the right content header, and pass my data as the body of the call:

        require('dotenv').config();

const fetch = require('node-fetch');
const ACCESS_TOKEN = process.env.ACCESS_TOKEN;

const SPACE_ID = 'ylfzY538';

let body = {
    "type": "Polygon",
    "coordinates": [
        [
            [-94.0565, 33.0183],
            [-91.1669, 32.9510],
            [-89.0984,29.1543],
            [-93.8898, 29.7350],
            [-94.0565, 33.0183]
        ]
    ]
}

fetch(`https://xyz.api.here.com/hub/spaces/${SPACE_ID}/spatial?access_token=${ACCESS_TOKEN}`, {
    method:'POST',
    body:JSON.stringify(body),
    headers: {
        'Content-Type':'application/geo+json'
    }
})
.then(res => {
    return res.json();
})
.then(res => {
    if (res.features.length >= 1) {
        res.features.forEach(f => console.log(f.properties.fullName));
    }
    console.log(res.features.length + ' results\n\n');
})
.catch(e => {
    console.error(e);    
});

  

When run, I see parks that roughly match what I expected:

As you can imagine, it can be somewhat complex to get this exactly right. If you do screw up, the API generally responds well, as an example:

        {
  type: 'ErrorResponse',
  error: 'IllegalArgument',
  errorMessage: "Geometry isn't valid!",
  streamId: '50ab3d1f-7494-497a-b3c2-e5d2c3fab329'
}

  

Find Features based on Other Features

For the final form of search, we'll look at today, here's a fascinating one that may be of use to folks dealing with multiple spaces. Spatial search can be told to use the geometry of a feature from a completely separate space as a center point. This can be pretty powerful. If the other space has data that moves on a regular basis, code that performs searches based on it wouldn't have to be updated.

To test this, I created a new space with a grand total of one feature - a point in New Orleans. I then did a search against the space we've been using and passed both the ID of the new space and the ID of the feature. This is done using refSpaceId and refFeatureId values. Here's an updated Node script that shows this:

        require('dotenv').config();

const fetch = require('node-fetch');
const ACCESS_TOKEN = process.env.ACCESS_TOKEN;

const SPACE_ID = 'ylfzY538';

const NOLA_SPACE = '4N1J3TtN';
const NOLA_FEATURE = 'wSxVVnDD1YLcNCHa';

fetch(`https://xyz.api.here.com/hub/spaces/${SPACE_ID}/spatial?access_token=${ACCESS_TOKEN}&refSpaceId=${NOLA_SPACE}&refFeatureId=${NOLA_FEATURE}&radius=3000`)
.then(res => {
    return res.json();
})
.then(res => {
    // show one sample
    if (res.features.length >= 1) console.log(JSON.stringify(res.features[0].properties, null, '\t'));
    console.log(res.features.length + ' results\n\n');
})
.catch(e => {
    console.error(e);    
});

  

This correctly returns one park that's within the circle defined by the other's spaces geometry.

Wrap Up

I hope this tour of spatial search options for Data Hub was exciting for you as it was for me! Remember to consult the Swagger docs for full details on additional arguments and features not covered here and to check out HERE Studio for mapping your data.