It's always helpful to have text or verbal claims about where a video was taken, but when it comes to terrorists or other criminals, they might not be telling the truth. 

Researchers from Ramón Llull University in Spain have created a system capable of geolocating videos by comparing audiovisual content with a worldwide multimedia database and it is able to locate where the videos were taken with no indication of where they were produced.

All of the data contained in the videois merged together and grouped in clusters so that, using algorithms developed by the researchers, they can be compared with those of a large collection of recorded videos already geolocated around the world. Almost 10,000 sequences from the MediaEval Placing task audiovisual database, a benchmarking initiative or assessment of algorithms for multimedia content, were used as reference for the proof-of-concept.

 "The acoustic information can be as valid as the visual and, on occasions, even more so when it comes to geolocating a video," says co-author Xavier Sevillano. "In this field we use some physics and mathematical vectors taken from the field of recognition of acoustic sources, because they have already demonstrated positive results. The videos which are most similar in audiovisual terms to what we want to find are searched for in the database, to detect the most probable geographical coordinates." 

The scientist points out that the proposed system "despite having a limited database in terms of size and geographical coverage, is capable of geolocating videos with more accuracy than its competitors". More specifically, it is capable of locating 3% of videos within a ten-kilometer radius of their actual geographical location, and in 1% of cases it is accurate to one kilometer. The percentages are still modest, although they are four times more precise than the accuracy reached up until now using the same database.

The researchers recognize that their method will require a much greater audiovisual base to apply it to the millions of videos that circulate on the Internet, but they highlight its usefulness in locating those which do not have textual metadata and the potential possibilities that it offers.

"This method could help rescue teams to track down where a person or group disappeared in a remote place, detecting the locations shown in the videos which could have been uploaded to a social network before losing contact," says Sevillano.

In the future, security forces could also use it, even to recognize locations of hostage executions and operations of terrorist groups such as Al Qaeda or ISIS. "Our system does not make any assumptions regarding the location of the videos, but in these cases we are given very valuable additional information to limit the searches, as we already know that we are dealing with the area of Iraq or Syria, and therefore, we would only use reference videos from there," explains the researcher.

Another much more immediate application is to facilitate geographical browsing in video libraries, such as YouTube, which just celebrated its 10th anniversary. "For example, if I want to go on holiday to New York and I feel like watching videos of Manhattan, when I type in a search on YouTube I get videos coming up recorded on the island, but also the performance of the seventies group The Manhattans and the trailer of the Woody Allen film Manhattan, which are not relevant to my search, and in these cases, the new technology can also help."

Citation: Xavier Sevillano, Xavier Valero, Francesc Alías. "Look, listen and find: A purely audiovisual approach to online videos geotagging". Information Sciences 295: 558-572, 2015.