Pokemon Go Players Unknowingly Built A Massive Image Database Now Used To Train Robots

In 2016, a mobile game transformed parks, streets, and ordinary local spots into a kind of playground for millions of people. Players would head outdoors with their phones, wandering through various locations in an effort to capture virtual creatures. In the process, they frequently stopped at monuments and public landmarks. At the time, many were unaware that these seemingly simple activities were also contributing to the creation of a massive collection of real-world imagery-data that is now being utilised to develop new technologies, reported Digit.in.

Around ten years ago, Niantic's mobile game, Pokmon Go, surged in popularity across numerous countries, including India. Utilising augmented reality technology, the game overlaid digital Pokmon onto real-world locations, encouraging players to explore their surroundings much like actual Pokemon Trainers. 

According to a report by NewsForce, the images and scans gathered through Pokemon Go and other augmented reality applications have culminated in the creation of a massive dataset comprising over 30 billion real-world images. This implies that whenever a player scanned a specific landmark within the game, they were-albeit unknowingly-contributing to the construction of this colossal database for Niantic.

For a considerable time, the game has incentivised players to visit specific real-world locations, such as gyms, monuments, and public spaces. On numerous occasions, players were prompted to use their smartphone cameras to scan these locations in order to enhance the game's accuracy, however, these scanning sessions were simultaneously capturing additional layers of information.

With every scan, in addition to the visual imagery, critical metadata associated with the device was also recorded. This included location coordinates, camera orientation, data regarding device movement, and various other sensor readings. Viewed in isolation, these scans appeared to be nothing more than standard gameplay interactions, yet, when aggregated with data from millions of players, they coalesced to form an extraordinarily detailed visual map of the real world.

Niantic states that the data collected through its augmented reality games now comprises nearly 30 billion images, captured from various angles, under different lighting conditions, and at different times of the day. Many of these images depict the surroundings of over a million popular real-world locations-places that players frequently visit while playing the games. Since every image is linked to precise spatial information, this dataset constructs a multi-angle 3D representation of streets, buildings, and public spaces. This vast collection of real-world imagery has become the foundation for new technology developed by Niantic's AI spinout company, Niantic Spatial.

How The Data Is Being Used

Niantic Spatial is utilising this massive dataset to create a "Visual Positioning System." Rather than relying solely on GPS signals, this system determines a device's location by matching the visual scene captured by its camera against the company's global image database.

This approach is particularly useful in densely populated urban areas, where GPS signals often bounce off buildings, leading to inaccuracies. In such scenarios, location estimates can vary by several meters-a discrepancy that could cause a delivery robot to end up on the wrong street or at the wrong building entrance.

This technology is already being utilised by Niantic's partner company, Coco Robotics. A last-mile delivery startup, Coco Robotics operates a fleet of nearly a thousand sidewalk robots across several cities in the US and Europe. These robots travel at speeds of approximately five miles per hour, transporting items such as groceries or large food orders. To ensure accurate delivery, they must navigate not merely to a general vicinity, but directly to the correct doorstep.

Each robot employs multiple cameras to perceive its surrounding environment, matching those visual inputs against Niantic Spatial's world model. By combining camera-based localisation with GPS, these robots are able to pinpoint their position with significantly greater accuracy. Simply put, the very spatial data that once helped players find virtual Pokemon is now guiding real machines through the streets of busy cities.