Pssst! Our 'secret sauce' is LiDAR

Our maps have reached an unprecedented level of precision thanks to the trillions of 3D data points that serve as the unseen foundation of HERE products. This digital imprint of reality is captured by our TRUE cars using a remote sensing device called LiDAR, which stands for Light Detection And Ranging technology. You might call LiDAR our maps’ “secret sauce” – so what is it, anyway?

LiDAR first emerged in the 1960s and was founded on the same principle of the Doppler effect and radar: wavelengths are shot at an object, and the time it takes for those wavelengths to travel to that object, bounce off the object, and travel back to the sensor, informs the distance to that point.

In the case of LiDAR, lasers are used to calculate relative distance by scanning the earth from side to side. The laser scanners emit light energy, called a “pulse,” to the surrounding environment, which bounces off surfaces in the surrounding environment (i.e. curbs, trees and buildings), and that reflection back to the sensor is aptly called the “return.” The LiDAR system then measures the distance – or “range” – by calculating the time it takes for the pulse to return back. Since light travels at a constant speed of about 300,000 km per second (roughly 186,000 miles per second), that is paired with the time of flight for the light photon, revealing the distance. In other words, LiDAR is based on the following simple equation (divided by two to factor in the return trip):

Our LiDAR uses 32 lasers to send out pulses, hitting 700,000 points per second with a range of 70 meters (about 230 feet) and an average accuracy of within 2 centimetres. That is a remarkably detailed image, and created unbelievably quickly. In fact, the captures are so accurate that our cars can drive at normal speeds to collect street-level data. Each of these points has an X, Y and Z coordinate, so we can measure the exact distance between any two 3D points: this allows it to record the height of a curb, the width of a road and the depth of a sidewalk.

The position of the LiDAR is important as well. The small cylindrical hardware that resembles a soda can is tilted at an angle to capture a 360-degree imprint vertically. As the vessel carrying the LiDAR moves forward (such as a satellite, airplane or in our case, a TRUE car), the data imprint reflects the movement across a horizontal plane. The result is a galaxy of 3D data points overlaid to represent not just the 3D footprint of one block on one street, but entire cities.

However, the LiDAR data relies on two other instruments to give context to its location and factor in the position of the recording vessel, such as altitude. The first is an inertial measurement unit (IMU), which collects positions, heading and attitude data (including yaw, pitch and roll) using accelerometers and gyroscopes. LiDAR is also synchronized with a global positioning system (GPS), which tracks the coordinates of where the data is captured (longitude and latitude).

From the Moon to the Grammys

Over the years, LiDAR’s extreme precision has endeared it to a range of fields and applications. NASA’s 1971 Apollo 15 mission used LiDAR to measure planetary surface height and topography on the moon. Architects use LiDAR data to model the location of buildings, and forestry experts apply it to measure canopy heights, leaf area and determine biodiversity. Unmanned vehicles, such as autonomous cars, are leveraging LiDAR to detect obstacles and react to the countless changing factors in an environment.

LiDAR is also used in archaeology to assess the topography around sites and obstructing vegetation.
Radiohead fans may recall the Grammy-nominated video that was shot using LiDAR company Velodyne’s off-the-shelf product.

The singing heads in this video showcase what the 3D data looks like untouched; for our maps, we use our cameras’ street level imagery to add pigmentation to the points, making them look more dynamic and recognizable as the world we see with the naked eye.

Another fascinating and unexpected application for the data is cataloguing at-risk historic areas. This year, we announced our partnership with the nonprofit CyArk, which is dedicated to documenting and preserving sites of cultural significance. Should a natural disaster, human destruction or simply age cause these meaningful cultural spots to decay, HERE’s LiDAR data serves as a record of what once stood for future generations to explore and study. In a more day-to-day application, the data can also be used as a reference for minor repairs to ensure that the fix accurately reflects the original structure.

To kick off this partnership, we provided LiDAR data from the historic neighborhoods of Philadelphia and New Orleans, two important cities that meet the requirements for historic preservation.

For our automotive partners, LiDAR-based maps are important because unlike the conventional maps that humans read, the accuracy and detail of HD maps enables machines to calculate their position more precisely, change lanes, keep within speed limits and route to a destination. As we plan for a future of driverless cars, such infrastructure is of the utmost importance to giving the cars an understanding of where they’re going, and not just reacting to their environments in real-time.

A LiDAR image after it has been processed.

 These are just a few ways LiDAR is fluorescing the 3D world around us. What other fields do you think could benefit from the high accuracy documentation of LiDAR data?