LUMIN: Language-based Understanding and Multimodal Interface for NASA PDS

Document Type

Article

Publication Date

2025

Abstract

Accessing and querying NASA's Planetary Data System (PDS) can present a steep learning curve due to the complex metadata structures and large volume of filterable parameters. To address this, we developed an LLM-powered assistant to help users generate accurate filter selections for planetary datasets, with an initial focus on Mars Reconnaissance Orbiter (MRO) data. Our approach leverages Retrieval-Augmented Generation (RAG) with hybrid methods, including BM25 and dense embeddings, to interpret natural language queries and recommend relevant filters. We also introduce a similarity search function that enables users to retrieve scientifically relevant datasets based on semantically related queries or previous selections, improving the refinement and contextualization of results. In addition, we developed an image analysis module that allows users to input planetary images and returns visually and contextually similar images by using a vision encoder and a pre-indexed embedding database. This functionality is currently limited to the Mars Science Laboratory (MSL) mission, specifically imagery related to the Curiosity rover. This module also extracts available details about the input image, enhancing user understanding and supporting visual-based data discovery. For PDS3 datasets (legacy format), we implemented a script that achieves over 80% accuracy and precision in predicting appropriate filters. For PDS-4 (the modern metadata structure), we are testing a new AI assistant that leverages the PDS API client to guide users through the well-organized dropdown filters, potentially removing the need for direct URL prediction due to the improved schema. A unified GUI is also in development to integrate all assistant modules and further enhance accessibility. This system enhances usability across both metadata structures, significantly lowering the barrier for scientific data discovery and increasing the efficiency of user interaction with planetary archives. © 2025 IEEE.

Link to Full Text

Share

COinS