MatSci-YAMZ: Integrating Human and Artificial Intelligence Vocabulary

Related work

Various tools and infrastructures have been developed to support community-driven vocabulary development and semantic alignment in the materials research domain. Formal ontologies often rely on smaller groups of domain experts working with ontology engineers to construct highly structured knowledge organization systems (KOS). The European Materials Modelling Council’s Elementary Multiperspective Materials Ontology (EMMO) provides a high-level framework that can integrate additional subject-specific ontologies into it (Goldbeck et al., 2019). Another effort is the PMD core ontology (PMDco), which utilizes an “ontology playground,” which supports community-driven sessions whereby materials scientists can suggest new terms, report issues, and learn about ontology development (Bayerlein et al., 2024).

Alongside formal ontology efforts, other communal approaches to developing materials science vocabulary have emerged. The National Institute of Standards and Technology (NIST) has focused on building shared materials terminology through both top-down and community contributions to its Materials Resource Registry (NMRR) vocabulary (Medina-Smith et al., 2016; Plante et al., 2021). Similarly, the Novel Materials Database (NOMAD) takes a more communal approach to metadata and vocabulary developments for its data dictionary, soliciting terms from researchers and standardizing them for use in the NOMAD repository (Draxl and Scheffler, 2018). Another recent collaborative vocabulary development environment is MetaCook, specifically the module VocPopuli, which supports user-generated terminology and definitions through a standalone software environment that facilitates the development of ontologies from them (Bagov et al., 2022).

MatSci-YAMZ does not claim direct technical interoperability with EMMO, PMDco, NMRR, or NOMAD. Similar to these other systems, MatSci-YAMZ supports community-driven terminology development and the overriding aim of generating shared, consistent, and reusable vocabulary. Crowdsourced definitions produced in MatSci-YAMZ are stored in a PostgreSQL database with provenance records, and a future aim is to assign Archival Resource Key (ARK) persistent identifiers. The ARKs could potentially be mapped to equivalent concepts in EMMO, PMDco, or the NMRR, depending on the scope of these communities. The MatSci-YAMZ community validation process (human commenting, voting, and LLM-assisted refinement) is similar to the community curation models supporting both the PMDco’s ontology playground and NOMAD’s data dictionary. These similar frameworks may potentially provide future infrastructure for term alignment. Technical integration with these other initiatives would require significant commitments and sustainability plans.

Overall, while these developments have advanced approaches to collaborative vocabulary development and consensus building, integration of LLMs into existing tools and their workflows is unknown within the scope of their efforts to develop and curate terminology. There is therefore a need to explore AI in this context given the growing sophistication and integration of LLMs into many data-driven workflows. Researchers have shown interest in AI approaches to describing research results, and LLMs may enhance both the efficiency and quality of vocabulary development. This need has motivated the development of MatSci-YAMZ and the work presented in this paper.

Overview

The primary motivation for developing the MatSci-YAMZ application was to explore the role LLMs might play in collaborative vocabulary development within the materials science community. Materials science subdomains draw from other disciplines, e.g., physics, chemistry, engineering, and generally focus on the process-structure-property-performance relationships of their results, which may be either computational or experimental in nature (Olson, 1997). A system such as MatSci-YAMZ might improve vocabulary usage among material scientists by providing more robust alternative definitions or greater disambiguation of terminology. Implementation of these features occurred in three phases: requirements and model selection, implementation and coding, and fine-tuning (Figure 1). MatSci-YAMZ also introduces provenance tracking of changes made to LLM-generated definitions, which allows us to develop insights into the lifecycle of terminology.

Figure 1

Phase 1: Requirements and model selection

The initial phase of development focused on defining specific features and requirements for the project. We chose the existing application YAMZ based upon its prior applicability to studying materials research vocabulary (McClellan et al., 2024). The MatSci-YAMZ application is coded in JavaScript and uses PostgreSQL for database management, both of which permit flexibility to upgrade or alter backend functionality. JavaScript allowed for the faster integration of the LLM into the workflow using Ollama, an open-source tool that enabled us to run LLMs directly on our local hardware. PostgreSQL also supports provenance tracking, an additional feature that allows us to record information regarding changes to all definitions.

MatSci-YAMZ incorporates and extends features of the original YAMZ application. The application uses Google authentication to provide users access to enter definitions, comment, and vote. Users provide a term, definition, and example, which are all required fields, and they can also electively comment or vote on either their own or other users’ definitions. The user-generated MatSci-YAMZ definition view for the term ‘material’ can be seen in Figure 2a. The system automatically populates an LLM-generated definition based upon the first user-generated term’s example, Figure 2b. There is only one LLM definition for a given term to reduce the quantity of terms users have to engage with.

Figure 2a

Figure 2b

To locate a suitable LLM, we examined several models and evaluated their performance based upon several characteristics, including:

• Definition length in words

• Example length in words

• Time for LLM to produce definition

• Overlap of LLM definition with a pre-existing reference definition from (Callister and Rethwisch, 2020)

We used Ollama to download and execute four models to test performance: Gemma3:27b, deepseek-r1:70b, gpt-oss, and qwen3:32b (DeepSeek-AI et al., 2025; Merhej et al., 2025; Open-AI et al., 2025; Yang et al., 2025). We then examined a few commonly used materials science terms to compare model outputs with standard textbook definitions. These included ‘Anisotropy’, ‘Dislocation’, ‘Grain Boundary’, and ‘Creep’. Tables 1a and 1b summarize LLM definitions for ‘Anisotropy’.

Each model was loaded, and the system message was set to the following: ‘You are to define materials science terms. Keep definitions concise and don’t be conversational, just respond with a definition and an example using the term with the given definition’. Each test term was then submitted to a given model to produce a definition and example of the term being used. This evaluation served primarily as a calibration exercise to ensure models could generate materially relevant definitions and to identify basic performance characteristics. When applicable, models were tested with ‘thinking mode’ both enabled and disabled. ‘Thinking mode’ is a feature that allows a model to generate and use intermediate reasoning steps before producing its final answer (Wu et al., 2024).

A set of criteria was established to guide LLM model selection for MatSci-YAMZ. Criteria selection was guided by preliminary testing, computational constraints, and a balance between semantic quality, accessibility, and computational efficiency. Larger models demonstrated the capacity for more detailed definitions, although they generally require significantly longer processing times and higher computational costs. We established basic selection criteria that included 1) alignment with established definitions from key materials science resources, 2) conciseness and clarity of the AI-generated definitions, and 3) response time in relation to the previous two metrics. As can be seen in Table 1a, the models produced definitions that generally aligned with the reference definition for ‘anisotropy’ as defined by Callister and Rethwisch (2020). With the definitions produced, to establish model efficiency, the number of tokens for each definition and example pair was added together and divided by the duration it took the model to produce them, and this yielded the token per second (TPS) metric. As can be seen in Table 1b, definitions produced with ‘thinking mode’ enabled yielded longer definitions and examples, but the qwen3:32b and deepseek-r1 took much longer to produce, while the gpt-oss model, which only operates in ‘thinking mode’, functioned much faster at 26.25s, yielding 5.37 TPS. The non-thinking mode models generally produced shorter definition example pairs, ranging from 40–52 tokens, while simultaneously taking less time to produce definitions. The gemma3:27b model, which has no ‘thinking mode’, yielded 5.24 TPS, outperforming the qwen3:32b and deepseek r-1 models. Our ultimate choice came down to gpt-oss and gemma3:27b, and the latter was chosen for its speed, producing definition and example pairs in 8.56s compared to 26.25s for gpt-oss. Speed was valued here over the computational cost.

Phase 2: Development

The development phase took place in two distinct stages. The first stage involved tailoring MatSci-YAMZ to the requirements in Phase 1. The application was written in JavaScript using the NextJS React framework as the front end, as this combination allowed us to more efficiently coordinate development of both. For our database, we utilized PostgreSQL, which was also used in the original YAMZ application, and the schema currently employed is available in Figure 3. Apart from the LLM integration, MatSci-YAMZ includes version control or tracking to note historical changes to definitions within the PostgreSQL database.

Figure 3

The second stage of coding focused on LLM inclusion in the preexisting YAMZ application. To practically implement the LLM into the MatSci-YAMZ workflow, we chose Ollama to run the model for its ease of use. The first benefit it provided was its simplicity to set up. Ollama, combined with Docker operating system virtualization, allowed us to quickly download and run models with multiple billions of parameters by automatically setting up GPU acceleration, increasing the performance of all models. When it came to integrating Ollama into the application, we took advantage of its REST API, which permitted us to run the LLM on one server and the web application on another. Not only did this ultimately increase the reliability of the entire application, but it also ensured that users were less likely to experience delays while the computationally expensive models were running in the background. While the Ollama API is extensive, only one endpoint was needed. This endpoint allowed us to run the model, set the system prompt, and choose how long it would stay in memory.

The coding and implementation stage was also where we developed a framework that would allow materials researchers to leverage aspects of human definitions within the context of the LLM. In keeping with our LLM requirements, we retained the system prompts to produce the initial definition and added a prompt to execute changes based upon user comments to the LLM-generated definitions. This latter prompt structure is designed to activate meaningful changes to the LLM-generated definitions based upon user information. An important and novel change from the original YAMZ application was the inclusion of version tracking, which allows administrators to see how the LLM responds to user inputs. Versions of LLM-generated definitions are stored in a PostgreSQL table, allowing researchers to run standard queries to study and compare changes across multiple definitions if necessary. The version tracking allows administrators to see a record of human and LLM interactions that produce user-visible LLM-generated definitions.

Phase 3: Fine-tuning

This phase focused on fine-tuning the features developed in phase 2. We tested the application with a small group of materials scientists to ensure it functioned according to the requirements described in Phase 1 (Greenberg et al., 2025). To accomplish this, we engaged several materials researchers who entered definitions and offered informal feedback. The purpose of these tests was to determine if the application functioned properly and that the LLM returned possible definitions related to materials science. The original workflow of MatSci-YAMZ required an administrator to manually run the LLM process. Based on feedback from testers, it was updated to automatically produce definitions. Further testing with additional users will be necessary to assess questions about the quality and relevance of definitions or user experience issues. In addition to user comments, we also wanted to ensure that the workflow functioned properly and that versions of definitions were captured in the PostgreSQL database. We detected no anomalies in small-scale use, and the Ollama interface functioned properly.