MapQaTor: A System for Efficient Annotation of Map Query Datasets
Mapping and navigation services like Google Maps, Apple Maps, Openstreet
Maps, are essential for accessing various location-based data, yet they often
struggle to handle natural language geospatial queries. Recent advancements in
Large Language Models (LLMs) show promise in question answering (QA), but
creating reliable geospatial QA datasets from map services remains challenging.
We introduce MapQaTor, a web application that streamlines the creation of
reproducible, traceable map-based QA datasets. With its plug-and-play
architecture, MapQaTor enables seamless integration with any maps API, allowing
users to gather and visualize data from diverse sources with minimal setup. By
caching API responses, the platform ensures consistent ground truth, enhancing
the reliability of the data even as real-world information evolves. MapQaTor
centralizes data retrieval, annotation, and visualization within a single
platform, offering a unique opportunity to evaluate the current state of
LLM-based geospatial reasoning while advancing their capabilities for improved
geospatial understanding. Evaluation metrics show that, MapQaTor speeds up the
annotation process by at least 30 times compared to manual methods,
underscoring its potential for developing geospatial resources, such as complex
map reasoning datasets. The website is live at: https://mapqator.github.io/ and
a demo video is available at: https://youtu.be/7_aV9Wmhs6Q.
67740b5f846a267749caed44
Mahir Labib Dihan
claimed_verified
2025-01-01T20:12:31.356Z
6565a60e0ff3292512bae26d
https://cdn-avatars.huggingface.co/v1/production/uploads/6565a60e0ff3292512bae26d/5P6wgEhCI0ndFVgX9KX8k.jpeg
Mahir Labib Dihan
user
mahirlabibdihan
67740b5f846a267749caed45
Mohammed Eunus Ali
67740b5f846a267749caed46
Md Rizwan Parvez
MapQaTor: A System for Efficient Annotation of Map Query Datasets
Mapping and navigation services like Google Maps, Apple Maps, Openstreet
Maps, are essential for accessing various location-based data, yet they often
struggle to handle natural language geospatial queries. Recent advancements in
Large Language Models (LLMs) show promise in question answering (QA), but
creating reliable geospatial QA datasets from map services remains challenging.
We introduce MapQaTor, a web application that streamlines the creation of
reproducible, traceable map-based QA datasets. With its plug-and-play
architecture, MapQaTor enables seamless integration with any maps API, allowing
users to gather and visualize data from diverse sources with minimal setup. By
caching API responses, the platform ensures consistent ground truth, enhancing
the reliability of the data even as real-world information evolves. MapQaTor
centralizes data retrieval, annotation, and visualization within a single
platform, offering a unique opportunity to evaluate the current state of
LLM-based geospatial reasoning while advancing their capabilities for improved
geospatial understanding. Evaluation metrics show that, MapQaTor speeds up the
annotation process by at least 30 times compared to manual methods,
underscoring its potential for developing geospatial resources, such as complex
map reasoning datasets. The website is live at: https://mapqator.github.io/ and
a demo video is available at: https://youtu.be/7_aV9Wmhs6Q.
MapQaTor is a web application that facilitates the creation of map-based QA datasets using diverse maps APIs, improving the efficiency and reliability of geospatial data annotation and evaluation.
Large Language Models LLMs question answering QA geospatial queries maps API ground truth data retrieval annotation visualization geospatial reasoning complex map reasoning datasets