VESA: A Visualization-Enabled Search Application for Exploratory Dataset Discovery

1.1 Contribution

This paper makes the following contributions:

1. It introduces VESA that supports exploratory dataset discovery by representing search results across multiple coordinated visual perspectives. The system enables exploration of metadata along thematic, spatial, temporal, and relational dimensions, providing both dataset-level discovery and repository-level overview. This directly addresses key limitations of current data search systems, including reliance on keyword-based search, lack of overview, and limited support for representing relationships among metadata entities.

2. It proposes a backend-agnostic architecture that integrates heterogeneous repositories through an adapter-based approach. Repository-specific adapters map metadata with different schemas to the common visual dimensions, enabling consistent exploration without dependence on a specific data model or infrastructure.

3. We provide an open-source implementation of VESA, including adapters for two repositories, and demonstrate the feasibility of the approach through Earth System Science (ESS) deployment and evaluation by domain experts. The repository is available as open-source software at https://github.com/DLR-SC/VESA2 and archived via Zenodo (Malikathazham and Betz, 2026).

4. Based on literature and our own system design, we derive a set of guidelines and requirements for building effective data search systems tailored to scientific datasets.

Currently, VESA’s capabilities are demonstrated through provided adapters for two repositories, which users can directly explore. Alternatively, users can create adapters for their own repositories using the provided documentation and adapter template. Furthermore, VESA is deployed for an ESS use case, where a knowledge graph backend is used to harvest and harmonize metadata from two repositories. This prototype version is available to explore at https://vesa.webapps.nfdi4earth.de/.

The remainder of this paper is structured as follows. First, in section 2, we reviewed the related work from different perspectives and examined the challenges identified in the empirical studies. Then, we introduce VESA, along with its theoretical grounding and architecture in section 3. In section 4, we provide details about the system implementation, frontend design, and usage scenarios, and in section 5, we present the use case of VESA for the ESS domain. Then in section 6, we present the results of our evaluations done with the ESS domain scientist. In section 7, we present the guidelines for the construction of an effective data search application. Finally, discussion and conclusion in section 8.

2.1 Challenges in data search services

Scientific data discovery has received increasing attention from both academic and infrastructure communities. This is reflected in the expansion of the coordinated efforts of organizations like the Research Data Alliance (RDA) (https://www.rd-alliance.org/), Go FAIR (https://www.go-fair.org/), and OPEN AIR (https://www.openaire.eu/) initiatives. All of which promote data sharing, interoperability, and the development of discovery-focused infrastructures aligned with FAIR principles. Despite that, most current systems continue to apply search paradigms originally designed for document retrieval, without accounting for the fundamental differences between documents and datasets. Several studies (Kern and Mathiak, 2015; Krämer et al., 2021) have pointed out that dataset discovery involves unique challenges—such as interpreting structured metadata, handling incomplete or ambiguous coverage descriptions, and supporting exploration across spatial, temporal, and relational dimensions. Unlike documents, which can often be interpreted through title and full-text analysis, datasets require contextual understanding of their attributes, provenance, and use cases. Nevertheless, the dominant approach in existing systems remains keyword-based querying, resulting in limited relevance, insufficient overview, and a frustrating user experience when large numbers of results are returned (Dörk et al., 2008). Khalsa, Peter and Mingfang (2018), in their survey of 98 repositories, found that the majority of data portals still rely on basic keyword search interfaces, which present results in long, paginated lists. These interfaces provide little insight into how datasets are distributed or related, and users are often left to click through each entry to assess relevance. Additionally, studies from Wu et al. (2019) and Borst and Limani (2020) emphasize the growing demand for domain-agnostic solutions that can bridge disparate repositories and facilitate cross-domain data discovery.

2.2 User behavior and sensemaking in data discovery

Understanding user behavior in data discovery is critical for designing effective data search systems. Unlike traditional information retrieval, dataset search is a more cognitively demanding process—as users often lack precise knowledge of available data and must iteratively refine their queries. In a study conducted by Krämer et al. (2021), researchers are shown to engage in iterative, exploratory search processes, often refining queries multiple times and frequently using filters to locate relevant datasets. Koesten et al. (2021) further emphasize the sensemaking challenges users face, noting that dataset discovery involves evaluating the context, provenance, and relationships of data attributes and entities. This highlights the need for tools that support users not only in finding datasets but also in understanding them. The importance of navigation and sensemaking is echoed in earlier work by Fox et al. (1993) and Pérez-Montoro and Nualart (2015), who demonstrate how visual navigation aids comprehension when users must move through large information spaces. In this context, studies in human–computer interaction (HCI) and information visualization suggest that coordinated multiple views (CMVs)—where interacting with one visual component updates others—can enhance exploratory tasks and promote deeper engagement (Heer and Shneiderman, 2012; Shneiderman, 2003).

2.3 Visual analytics for data discovery

Visualization systems have proven to be of significant value in information retrieval. Particularly by enabling users to identify previously unknown patterns and insights (Saraiya, North and Duca, 2005) within complex data collections. Despite its potential, the adoption of visual analytics tools in scientific data discovery remains limited, especially when compared to their widespread use for data analysis, result presentation, and data exploration in scientific domains (Kaur, Klan and König-Ries, 2018). There are some notable efforts, such as GFBIO’s VAT (Beilschmidt et al., 2017), which provides a powerful environment for search of similar datasets in the biodiversity domain. However, it is constrained to their own data repositories and specific schema, limiting the broader applicability and scalability. Similarly, domain-specific systems such as eBird (Auer et al., 2024), Ocean Data View at https://odv.awi.de/, or Vertnet at https://vertnet.org/ provide detailed visual querying and mapping capabilities, but only within specific data domains. Other platforms, including ARIADNE portal at https://portal.ariadne-infrastructure.eu/ for archaeological datasets and Germany’s national Geoportal at https://www.geoportal.de/, provide interactive map views, timelines, or thematic filters to support data discovery. While these systems allow users to explore datasets through different visual perspectives, the initial search process typically remains text- and keyword-driven. Users are therefore required to formulate explicit queries before any meaningful visual context is presented, limiting opportunities for exploratory discovery. Moreover, the visual views provided in these systems are often disjointed—meaning the different components (e.g., map, list, metadata details) operate in parallel and lack strong interactivity or coordination. Actions taken in one view (e.g., selecting a geographic region) may not dynamically update the others, reducing the effectiveness of the interface for sensemaking or multidimensional exploration.

Consequently, there remains a lack of visualization-enabled search systems that support exploratory dataset discovery through tightly integrated, coordinated visual views across multiple metadata dimensions.

2.4 Comparison among other data search repositories

To position VESA with respect to existing visual data discovery environments, we inspected five established platforms: Geoportal (https://www.geoportal.de/), NASA Earthdata Search (https://search.earthdata.nasa.gov/), the ARIADNE portal (https://portal.ariadne-infrastructure.eu/search), BCO-DMO (https://www.bco-dmo.org/search/dataset), and the GFBio VAT (https://vat.gfbio.org/#/). The aim of this comparison was not to evaluate these platforms as competing systems, but to clarify how their discovery workflows differ from the metadata-level visual search approach implemented in VESA.

Geoportal provides a strong geospatial catalog and map-layer exploration functionality. Users can search for geodata and geoservices, select thematic map layers, and interact with spatial visualizations. However, its workflow is mainly organized around exploring a dataset’s content rather than searching for a dataset. This differs from VESA, which shows a coordinated overview of repository metadata across several discovery dimensions. Geoportal users typically begin by choosing a theme or searching for a known geospatial resource. In VESA, users can first inspect the thematic, spatial, temporal, and authorship-related structure of repository content before formulating a precise query.

The ARIADNE portal is closer to a research data search platform and provides dedicated support for archaeological data discovery. It offers spatial and temporal search together with additional facets and catalog-based refinement. Most metadata dimensions beyond space and time are primarily exposed through facets, filters, or catalog structures rather than through coordinated visual views. The temporal and spatial components support search refinement, but they are not tightly coupled visual components in which interaction within one view immediately reveals changes across other views. VESA therefore differs by treating metadata entities as coordinated discovery dimensions within one visual search environment.

NASA Earthdata Search is a mature discovery environment for Earth observation and environmental data. It supports keyword search, browsing by broad Earth science topics, and refinement through spatial and temporal facets. It also allows users to inspect individual data collections and their spatial or temporal characteristics via map and temporal bars. Its visual assistance is mainly connected to map-based filtering and data content inspection. Users who are unfamiliar with the repository still need to select a broad topic, use predefined filters, or formulate an initial search term. VESA addresses this cold-start problem by presenting prominent keywords and visual summaries of repository content as its default screen.

BCO-DMO provides rich domain-specific metadata and supports discovery of oceanographic datasets through keyword search, facets, metadata descriptions, and spatial distribution information. This is highly valuable for users searching within the marine and oceanographic domain. However, the discovery process remains largely catalog-, keyword-, and facet-driven, with a spatial map that provides useful geographic orientation. VESA differs by showing each dimension visually and connecting them as linked components of the search process.

The GFBio Visual Analysis Tool provides stronger visual analytics support than many conventional repository search interfaces. It allows users to select biodiversity data and then visualize, transform, and analyze these data in a geospatial environment. Its visual functionality is mainly applied after an initial data selection has been made. At the discovery level, the search process still begins with keyword search or map-based selection. VESA places visualization earlier in the discovery workflow by using coordinated visual components to support repository-level exploration before and during search refinement.

2.5 Summary

In summary, while prior work advanced dataset indexing, metadata standardization, and visual exploration techniques, these efforts remain largely disconnected in practice. Existing data search systems predominantly rely on keyword-based, facet-based interfaces and list-based result presentations, offering limited support for exploratory discovery. At the same time, visual analytics approaches have demonstrated the effectiveness of coordinated, multi-perspective exploration, but the few systems available in scientific repository settings are typically tightly coupled to specific domains and repository infrastructures, which limits their reuse and broader applicability. Moreover, as also discussed in the previous section, their visualization abilities are primarily for raw data exploration rather than for data search. VESA differs by focusing on coordinated visual search at the metadata level. It provides users with an initial overview of repository content through multiple discovery dimensions and allows interactions in one dimension to update the others. This approach is especially useful for exploratory search situations where users do not yet know the terminology, scope, or structure of a repository.

3.1 Architecture

In Figure 2, we show the domain-agnostic architecture of VESA. To enable VESA to work on their repository, it is a pre-requisite that a data manager creates a proxy API (Application Programming Interface) or repository adapter (User Provided Repositories Endpoint), which serves as a schema translator or a mapper. The reasoning behind it and more detail about this adapter is given in section 4. The link to the adapter endpoint is provided through Source Configuration, which is a frontend component. As shown in Figure 3, through source configuration page, user provides endpoint and other configurable parameters about the repository. Using this information, the system makes connection with the source repositories and fetch the metadata records. During this process, Synchronization and Ingestion Services helps VESA to make a persistent connection with the Sources and retrieve the records based on the standard defined in the user-created repository adapter. Thus, the core VESA remains independent of heterogeneous schemas, data formats, and repository-specific constraints, and it only interacts with the clean, translated output. The records are then stored in VESA’s Database Storage. This database is implemented as an Arango-based graph database with three document collections (dataset, author, and keyword) and two edge collections: hasAuthor, linking datasets to authors, and hasKeyword, linking datasets to keywords. Application Logic connects the backend database layer with the visualization transformers. It executes queries (e.g., using Arango Query Language) to retrieve relevant data and prepares it for downstream transformation. The extracted information is then forwarded to the Visualization Transformers in appropriate formats. Visualization transformers consist of algorithms that define the data processing templates required for each visualization, because each visualization expects data in a specific structure. For example, a word cloud requires keyword frequency, co-occurrence relationships, and associated dataset identifiers, while a map requires spatial coordinates and dataset density per region. In the same way, a Map needs information about the datasets and its spatial coordinates or regions and frequency of datasets in the specific region. The Data Access Interface acts as a unified point between the backend system and the frontend application. They expose processed and transformed data generated by the Application Logic and Visualization Transformers. Each endpoint is designed to serve data tailored to a specific visualization, such as spatial distributions, temporal trends, or keyword relationships. By abstracting the backend complexity, API endpoints ensure that the frontend remains decoupled from the underlying data storage and processing mechanisms. Interaction handling manages user-driven actions within the frontend, such as filtering, selecting, or exploring datasets across different visual components. It interprets user interactions and translates them into corresponding queries or filtering operations. These interactions trigger updates in the underlying data requests via Data Access Interface and ensure that all visual components remain synchronized. This mechanism enables CMVs, allowing users to iteratively explore and refine their search in an interactive manner. Visualization Views represent the visual interface through which users explore and interpret the data. These include multiple coordinated visual components such as maps, timelines, chord diagrams, and word clouds. Each view presents a specific perspective of the dataset (e.g., spatial, temporal, or relational) and remains dynamically linked to each other. Updates triggered by interaction handling are reflected across all views, enabling seamless exploratory analysis and supporting sensemaking through visual analytics techniques.

Figure 2

Figure 3

4.1 VESA frontend

Current software only shows four visualizations and a Table view. For the future, the plan is to include a visualization library so that users can include custom visualizations based on their own repository needs.

Word Cloud (Figure 1a) is linked to the keywords in the metadata. The size of the word shows the frequency of that term referred to in different datasets. Thus, the higher the frequency, the more prominent that term is in the collection. Selecting one keyword further shows the related keywords in the Word Cloud and consequently filters the other related attributes and datasets in the connected visualizations. For the construction of this Word Cloud, the keywords were extracted, normalized, and validated to include only those that are in textual form. Numbers and special characters are rejected in this process. As we were not using any semantic checks via ontologies or strict vocabularies of this domain, this step was important to keep the list clean for further processing. To show only relevant and important keywords, their weights were derived via Term Frequency and Inverse Document Frequency (TF/IDF) metric, where the number of documents are the number of metadata files in which the keyword is available. Based on the weights of TF/IDF, keywords with the top 30% of high scores were then filtered and displayed in the visualization. The threshold was selected based on tuning and testing mechanism, and 30% was the best selection for us. For future versions of this software, user can decide this threshold as per their choice. So, as not all keywords from the data store were shown directly on the word cloud, a provision is made to search for all of them in our data store via running a search from the Autocomplete Search Bar (Figure 1b) component.

Map (Figure 1d) enables filtering based on the spatial coverage of the datasets. Every repository label is assigned a color during the Software Configuration phase (see Figure 3), which shows up as different colors of dots in the Map. Hovering over the dot also shows its coordinates on the top bar of the map canvas. The selected dot gets highlighted in red. User can select more than one dataset by directly clicking on them. The dots or points on the Map that get clustered when a user zooms out of the Map or when the Map is at its default position. Then, on the clustered ball, user can see the number of datasets related to the clustered region. At the left corner, different icons in blue colors are placed. “L” is to enable and disable Legend, Home icon brings the map back to its default position, “+” is for zooming in and “-” is for zooming out of the map. When a user is in the zoom-in position, she can also select a CLEAR button at the top to come back to the default position.

Line Chart (Figure 1e) shows the temporal coverage of the datasets. The default X axis shows the temporal range, which is defined by the maximum and minimum date range of all the datasets available in our data store. The Y axis shows the number of datasets. The Line Chart below helps in zooming in and filtering through the years and then the result is shown on the Line Chart above. From here, one can filter datasets based on years, months, and dates. Hovering over the line on this chart shows the number of datasets that belong to the specific date. Brushing over the Line chart zooms in to the brushed time periods and shows the related frequencies.

Chord Diagram (Figure 1f) illustrates co-authorship frequency in the filtered datasets, with thicker chords indicating more frequent collaboration between pairs of authors.

List View (Figure 1c) in the middle presents the filtered results. It shows the title and the authors of the filtered datasets. Clicking on the title will take user to their actual DOI or source page. This view also shows the count of the retrieved datasets, based on user-activated visual filters.

4.2 Usage scenarios

In the following, we illustrate the relevance and applicability of the proposed system through two use cases from two different users of a data search system.

4.2.1 User searching for a dataset

Consider a scientist named Dr. Smith researching climate change impacts. He might start by clicking on the keyword “temperature” in the Word Cloud visualization. This action would show other related keywords such as “climate” and “precipitation,” and filter the datasets accordingly across the other visualizations (see Figure 4-1). Selecting a keyword “precipitation” (Figure 4-2) would further refine the results to show datasets related to precipitation measurements on the Map, the relevant time periods in the Line Charts, and the authors involved in these studies in the Chord Diagram (see Figure 4-3,4) respectively. By this iterative process, a user can efficiently refine their search and can quickly identify the most relevant datasets without the need for multiple separate queries. This streamlined approach saved Dr. Smith time, eliminating days of manual searches and enabling him to compile a comprehensive and relevant dataset for his research.

Figure 4

6.1 Results

All eight participants who took part in this survey were within the age groups varying primarily from 25 to 64 years. Data visualization interaction frequency ranges from “Rarely” to “More than once a week,” suggesting different familiarity levels with data visualization tools. As shown in Figure 5, users in age groups under 55 years use visualizations more frequently for their work (more than once a week and month).

Figure 5

6.2 Participants performance with the experimental tasks

Due to the inherent nature of the survey, no prior training or walkthrough was possible. Therefore, it was essential to assess the correctness of the responses to determine whether participants were able to independently understand and interpret the visualizations, regardless of their prior experience with such charts. If participants could correctly read the information presented in each visual component, they would also be able to use that component to support metadata exploration in VESA, which is the overall objective of this work. The evaluation focused on four visual components: the word cloud, list view, line chart, and map view. Results showed high levels of accuracy across all views: 100% correct responses for the word cloud, list view, and line chart, and 87.5% for the map view. In total, only one error was observed across 32 individual responses, indicating a strong overall understanding. Thus, the participant performance in this task was approximately 95.83%, demonstrating that the visual components of the interface were largely correctly interpreted without requiring prior training.

6.3 Comparison with other tools

Our participants used a variety of data search applications, with the majority using PANGAEA and Google Dataset Search, while others used BASE, DataCite Search, WDCC, ECMWF, R, and interfaces provided by their own repositories. When asked to rank our search application in comparison to their usual one, 50% consider it better, 38% consider it the same, and 12% consider it worse. Further, for the questions on “If the overview of the datasets from the two repositories is clear to understand and intrigued any new insight or further search queries?”, the reaction was mixed. Some appreciated the visual exploration and noted interesting trends (e.g., dataset growth over time) and the relationships between authors and publications. Others found the interface initially overwhelming and requested more onboarding or guided navigation.

6.4 Usability evaluation

Furthermore, we summarized the results of the Likert-scale questions related to the expressiveness and usability of the user interface (see Figure 6). The majority of participants consider this tool as aesthetically designed, which needs no to limited training, is good in capturing the interest and is easy to interact with. On the topic of application being cluttered or not, participants had mixed reactions. Figure 6 shows that around 50% answered it could be false; however, comments pointed word placements in word cloud to be cluttered and could be fixed. In word cloud, algorithm places the terms randomly based on the frequency of its occurrence. On the other hand, some considered this visualization intriguing and somewhat intuitive. When we tried to understand the reason, we realized that many participants had major issues with no description, tutorial or help available for the tool. As one said “as first time user, I dont get sense of the lower two frames. I would need more labels and explanations.” Moreover, as this tool was based on limited data sources and datasets, many were not able to find their datasets, leading to further more doubts.

Figure 6

6.5 Qualitative analysis of the user responses

Overall, it was a positive response. As one mentioned “it is a modern approach and enable a play factor, which keeps interest.” Others said “It certainly looks intriguing. The keywords are nicely displayed. The interaction of the authors shown.” They liked to drill down the search through different aspects and can visually see the result. Also liked the overview of available datasets at a single glance. However, throughout their comments, it was found that some level of training is required either in the form of manual or a tutorial. Furthermore, we were able to track down some areas of improvement, which are: adding more datasets and data sources, options to rearrange different frames, and a color palette. Some issues were related to the missing data and metadata quality, as one said “I get a landsat result with a location shown in the map - landsat data should be over a region.”

6.6 Summarizing results

In summary, the survey results indicate a generally positive reception of our system among participants from ESS and related domains. Most users found the system to be aesthetically designed, easy to interact with, and capable of capturing interest. Despite no prior training, participants performed remarkably well in understanding the visualizations, with a 95.8% overall correctness rate across four visual components: word cloud, list view, line chart, and map. Qualitative feedback highlighted appreciation for the visual exploration features and interface design, while also pointing out areas for improvement such as clearer labeling, help documentation, and reduced complexity for first-time users. In comparison to their usual search tools, the majority rated the application as either equal to or better, suggesting strong potential for adoption, particularly with minor usability enhancements. These findings support the system’s effectiveness as a visual search interface and provide a strong foundation for future iterations. However, the findings are based on a limited sample size and limited datasets (333 datasets), which may constrain the generalization and statistical significance of the results. While the insights provide encouraging early evidence of the system’s usability and effectiveness, a large-scale user study would be necessary to draw more definitive conclusions.