Zero Trust Frameworks for Misinformation Detection in Industry 5.0

The models are trained using two datasets of fabricated news: GossipCop and PolitiFact. These datasets are sourced from FakeNewsNet [19–21].

In this manuscript, we introduce The GossipCop dataset consists of news articles centered around celebrity gossip. It comprises two files: gossipcopreal.csv and gossipcopfake.csv. The first file, gossipcopreal.csv, contains over 11,357 entries with accurate information, while the gossipcopfake.csv file includes over 10,783 entries with false information. These two files are merged and labeled as 1 for real news and 0 for fake news. The dataset includes columns such as id, URL, title, tweet-id, and true. For our purposes, we utilize the title column as the textual data[19–21].

Also, The PolitiFact dataset consists of news articles and tweets related to political news in the United States. It consists of two files: politifactreal.csv, which contains 432 tweets with genuine news, and politifactfake.csv, which includes 618 tweets with fabricated news. Similar to the GossipCop dataset, these files are merged and labeled as 1 for real news and 0 for fake news. The dataset contains columns such as id, URL, title, tweet id, and true[19–21].

80% of the data is allocated for training purposes, while the remaining 20% is reserved for testing. The training data is utilized to train the machine learning model by providing input and output information. Once the model is trained, the training data becomes unseen data and is utilized to evaluate the model’s performance.

The data preprocessing in this study encompasses text lowercasing, tokenization, removal of punctuation, and elimination of stopwords. Additionally, stemming and lemmatization are applied to reduce words to their base forms. In GPT finetuning, labels are transformed into text format to match the text-to-text translation nature of the task. This alignment ensures smooth handling of textual input and output, thereby improving the model’s ability to produce accurate and contextually appropriate translations.

It is important to note that the above preprocessing steps (lowercasing, stopword removal, stemming, and lemmatization) were applied exclusively to the traditional machine learning baselines (GB, XGBoost, LR, DT, SVC), which require explicit feature engineering through TF-IDF vectorization. For all transformer-based models (BERT, DistilBERT, DeBERTa) and GPT-3 variants, the raw text was used with each model’s native tokenizer (WordPiece for BERT variants; BPE for GPT-3). No stopword removal, stemming, or lemmatization was applied to transformer inputs, as these operations would disrupt the subword tokenization patterns learned during pre-training.

During the detailed finetuning process of the GPT-3 model, an important step is optimizing the input for better performance. This is done by appending a specific suffix, \n\nIntent:\n\n, to the input text. This additional sequence acts as a clear marker, indicating the end of the input section and ensuring the model processes the data correctly. Similarly, for the output generated by the finetuned GPT-3 model, it is crucial to indicate the endpoint clearly. This is achieved by appending the output with the terminating tag END. This marker guides the model during the generation process, helping it produce output that meets the specified requirements and maintains coherence.

By using these optimization techniques during finetuning, the GPT-3 model can effectively utilize the \n\nIntent:\n\n suffix for input delimitation and the END tag for precise output termination. These careful measures enhance the model’s fluency and accuracy, enabling it to generate contextually relevant and coherent outputs. Without these suffixes, the input and output tokens would not be distinguishable, leading to merged tokens, noise, and repetitive sentences in the model’s output.

In the experimental evaluation for fake news detection, several models have been utilized as baselines, each representing a distinct category of machine learning and deep learning approaches. These baseline models provide a comparative benchmark against which the performance of the GPT model variants can be assessed.

The baselines are opted based on the recent improvements and well-known algorithms for classification and NLP tasks. The machine learning models are foundational models, deep learning models and transformer models are considered to be state-of-the-art models in the NLP domain. Additionally, they are employed in the recent literature[14,15,22]. The first category includes traditional machine learning algorithms such as Gradient Boosting (GB)[23], XGBoost[24], Logistic Regression (LR)[25], Decision Trees (DT)[26], and Support Vector Classifier (SVC)[27], known for their interpretability, ease of implementation, and strong performance in various tasks. The second category comprises deep learning models, specifically Long Short-Term Memory (LSTM)[28], Bidirectional LSTM (Bì-lStM)[29], and Convolutional Neural Network LSTM (CNN Bi-LSTM)[30], which excel at capturing complex patterns in sequential data, making them particularly effective for text-related tasks like fake news detection. The third category encompasses transformer-based models, including Bidirectional Encoder Representations from Transformers (BERT)[31], DistilBERT[32], and DeBERTa[33], which utilize self-attention mechanisms to understand word context and handle long-term dependencies, achieving state-of-the-art performance in numerous natural language processing tasks.

The Generative Pretrained Transformer 3 (GPT-3)[34] model is characterized by four distinct variants, namely Ada, Babbage, Curie, and Davinci. Ada, with the fewest parameters, represents the lower end of the complexity spectrum, while Davinci, embodying the most parameters, signifies the upper bound. Fine-tuning GPT-3, inherently a text-to-text transformer, for binary classification mandates a transformation of numeric labels into text-based equivalents. Traditional baseline models employ a numerical system with 0 denoting fake news and 1 signifying genuine news. For the fine-tuning process with GPT-3, these labels are transcribed into textual form: genuine news is labelled as “This is true news” and fake news is labelled as “This is fake news”.

Upon the completion of labelling, a specific suffix, denoted by \n\nIntent:\n\n, is affixed to the input, and another suffix, END, is affixed to signify the output’s end. The \n\nIntent:\n\n suffix was chosen following OpenAI’s fine-tuning best practices documentation, which recommends a consistent, distinctive separator that does not commonly appear in natural text. The double newline characters ensure clear visual and tokenization-level separation between the input prompt and the expected classification output. Alternative delimiters such as \n\nClassification:\n\n, ###, and ---> were considered during preliminary experiments; however, the \n\nIntent:\n\n format was selected for consistency with OpenAI’s recommended fine-tuning templates and because it yielded the most stable convergence during early training trials These suffixes are of paramount importance, serving as clear delimiters for the input and output boundaries as shown in Figure 1.

Figure 1.

GPT-3 Finetuning

Thereafter, this data undergoes a transformation into a JSON format with prompt acting as the input and completion embodying the output. The culmination of this process involves fine-tuning the model on OpenAI servers, a process initiated by the execution of the command openai api fine_tunes.create -t <TRAIN_FILE_ID_OR_PATH> -m <BASE_MODEL>. The hyperparameters of the GPT-3 models include number of epochs, the maximum number of tokens, learning rate, batch size and frequency penalty. The number of epochs is 4, maximum combined prompt-plus-completion token length 2,049 and all other parameters are set to default.

Given that the input texts, in our datasets are typically short (10–30 tokens) and the classification completions are brief (4–5 tokens), the effective token usage per sample is well within the 2,049-token combined limit. The max_tokens parameter for completion generation was set to 10 to restrict output length and prevent extraneous generation.

The OpenAI Completions API returns log-probabilities (logprobs) for each generated token during inference. When the fine-tuned GPT-3 model generates a classification completion, the confidence score is computed by exponentiating the mean log-probability across all completion tokens: Confidence = exp(1/N ∑^N_i logprob_i), where N is the number of tokens in the completion. This yields a continuous value between 0 and 1. Additionally, by setting the logprobs parameter to return probabilities for alternative tokens, the system compares the likelihood assigned to the first divergent token between the “true” and “fake” completions to derive a normalized binary confidence score. This confidence score is used by the PEP decision thresholds described in Section 2.5.

To bridge the gap between the GPT-3 classification model and a formal Zero Trust security framework, we describe the architectural integration of the misinformation detection module within a Zero Trust Network Access (ZTNA) framework. In our proposed architecture, the GPT-3 fine-tuned classifier operates as a Policy Decision Point (PDP). Every response generated by a GenAI system (e.g., a chatbot, voice assistant, or automated report generator) is treated as an untrusted entity following the “never trust, always verify” principle.

The data flow proceeds as follows: (1) A GenAI system generates a response. (2) The response is intercepted by the Policy Enforcement Point (PEP) before delivery to the end user. (3) The PEP forwards the response text to the GPT-3 classifier (PDP) for misinformation assessment. (4) The GPT-3 classifier returns a classification label (real or fake) along with a confidence score. (5) Based on the classification output, the PEP enforces one of three actions: Allow (classification: real, confidence > 0.9), Flag for Human Review (confidence between 0.7 and 0.9), or Block (classification: fake, confidence > 0.9). (6) All decisions are logged in a centralized audit system for continuous monitoring and compliance.

This continuous verification loop ensures that no GenAI output is implicitly trusted, aligning with Zero Trust principles. The PEP can be deployed at the API gateway level for cloud-based GenAI services, or as an on-device middleware component for edge deployments in Industry 5.0 environments. Figure 2 illustrates this integration architecture.

Figure 2.

Zero Trust Security Architecture integration to Zero Trust Network Access (ZTNA) Framework

Table 2 illustrates the experimental results on Gossipcop test data. Among the traditional machine learning models, the Logistic Regression (LR) and Support Vector Classifier (SVC) models distinguished themselves by outperforming their counterparts. With an impressive accuracy rate of 84.40%, the LR model demonstrated not only proficiency in accurately classifying inputs but also exhibited remarkable precision (84.57%), recall (84.46%), and F1-Score (83.24%). This suggests the model is effective in both identifying relevant instances and reducing false alarms. Trailing slightly behind, the SVC model proved its mettle with robust scores of 84.35% in precision, 85.63% in recall, 84.67% in the F1-score, and an overall accuracy of 84.98%.

In the realm of deep learning models, the Convolutional Neural Network augmented with a Bidirectional Long Short-Term Memory (CNN Bi-LSTM) demonstrated superior performance. It garnered commendable scores in precision (81.34%), recall (81.54%), and the F1-score (81.67%), while maintaining an overall accuracy of 81.32%. The standalone LSTM and Bi-LSTM models, although not leading the pack, still demonstrated noteworthy competitiveness.

Moving on to transformer-based models, it is clear that the variants of the GPT-3 model are in a league of their own, outclassing all other models in the experiment. In particular, GPT-3 Ada showcased exemplary performance with stellar precision (96.48%), recall (97.86%), F1-Score (97.18%), and accuracy scores (95.69%). The remaining GPT-3 variants, including Baggage, Curie, and Davinci, were not far behind, all achieving mid to high 90s in their performance metrics, which underscores the profound efficacy of these models.

Figure 3 [B] shows the ROC curves of GPT models simulated on the GossipCop dataset and Figure 3 [C] shows the PRC curves of the GPT-3 models on the GossipCop dataset. The ROC and PRC curves show that GPT Ada is performing the best, as it has the highest Area Under the Receiver Operating Characteristic (AUROC) and highest Area Under the Precision-Recall Curve(AUPRC).

Figure 3.

GossipCop: (A). Confusion Matrix, (B). ROC Curves, (C). PRC Curves of GPT-3 Ada

From the confusion matrix Figure 3 [A], we can see that the GPT-3 Ada model has performed well in correctly classifying instances, both in positive and negative classes. The true positives and true negatives are relatively high (3286 and 951, respectively), indicating that the model has made accurate predictions most of the time. On the other hand, the numbers of false positives and false negatives are relatively low (120 and 71, respectively), suggesting the model’s robustness in avoiding misclassifications. However, while the model seems to have been more proficient at correctly identifying positive instances (as indicated by the higher TP), there appears to be a slightly higher tendency for the model to misclassify negative instances as positive, which could lead to more false alarms. Despite this, the overall performance of the GPT-3 Ada model, as reflected by this confusion matrix, is impressive.

Table 3 represents the experimental results on Politifact test data. Among traditional machine learning models, the Support Vector Classifier (SVC) and Logistic Regression (LR) models demonstrated exceptional performance. The SVC model exhibited outstanding efficacy, with a precision of 83.26%, recall of 83.41%, an F1-Score of 83.36%, and an accuracy of 83.45%. The LR model, while not leading the pack, presented commendable scores with 81.24% precision, 81.35% recall, an F1-score of 81.40%, and an overall accuracy of 81.46%.

In the category of deep learning models, the Long Short-Term Memory (LSTM) model emerged as the most proficient, with 84.24% precision, 84.65% recall, an F1-Score of 84.43%, and an accuracy of 84.32%. While the Bi-LSTM and CNN Bi-LSTM models fell slightly short of these scores, their performance remained noteworthy. Delving into the realm of transformer-based models, the BERT model reigned superior among its counterparts, DistilBERT and DeBERTa. It achieved an impressive F1-Score of 88.07% and an accuracy of 88.81%, bolstered by high precision and recall scores of 87.05% and 87.19% respectively.

However, it was the GPT-3 variants that unequivocally eclipsed all other models. The GPT-3 Ada model, in particular, stood out as the most exceptional performer, boasting a remarkable accuracy of 91.98% and an F1-Score of 93.68%, accompanied by high precision (91.30%) and recall (96.18%) scores. The remaining GPT-3 variants, Baggage, Curie, and Davinci, also showcased formidable performance, with metrics lingering in the upper 90s, thereby underlining their robust capabilities. Figure 4 [B] and Figure 4 [C] show the ROC and PRC curves for the PolitiFact dataset. The comparison confirms that GPT-3 Ada achieves the highest AUROC and AUPRC among all variants. The confusion matrix in Figure 4 [A] shows TP=126, TN=69, FP=12, and FN=5, yielding 212 total test samples consistent with the 20% split of the 1,050-sample PolitiFact dataset.

Figure 4.

PolitiFact: (A). Confusion Matrix, (B). ROC Curves, (C). PRC Curves of GPT-3 Ada

In both the datasets GPT-3 Ada performed the best. The LLMs are trained on much more data compared to the baseline models. As we know chatgpt is very good at understanding natural language, it is expected that the variants of the GPT are expected to perform the best. The GPT-3 Davinci is expected to provide the best results as it has the highest number of parameters and known to be the most powerful. That is true for text generation, this experiment focuses on binary classification. Although, all the GPT-3 variants performed equally well.

Notably, GPT-3 Ada outperforms Davinci in several metrics despite being the smallest variant. This counterintuitive result can be attributed to several factors specific to binary classification tasks. First, for simple binary classification, smaller models like Ada can converge more efficiently during fine-tuning because fewer parameters need to be optimized, reducing the risk of overfitting on relatively small datasets (~10K samples). Second, Davinci’s larger capacity (175B parameters in its full form) is architecturally optimized for complex generative tasks such as text completion and reasoning; for straightforward binary decisions, this extra capacity may introduce noise rather than improved discriminative ability. Third, the fine-tuning data volume in our experiments may be insufficient to fully leverage Davinci’s representational capacity, causing a form of underfitting relative to its model size. These findings suggest that for resource-constrained Industry 5.0 deployments, the smaller Ada variant offers comparable or marginally better classification performance alongside significantly lower computational overhead. However, it should be noted that the narrow performance margins between GPT-3 variants (e.g., 0.13% between Ada and Davinci on GossipCop) are based on single-run evaluations and should not be interpreted as statistically significant differences without multi-seed validation.

The GPT-3 models performed better in classifying this might be because the GPT-3 is pre-trained on huge amounts of data that is publicly available. Such pretraining helps the models to be better at understanding context and natural language processing. Additionally, GPT-3 excels in understanding context in natural language. Fake news often involves subtle nuances and context-dependent clues. The ability to understand the context in which a statement is made can be crucial for accurate classification. Moreover, it is one of the largest language models, with 175 billion parameters. This scale allows it to capture complex patterns and relationships in language, potentially making it more effective at classification of fake news.

Table 4 illustrates comparative performance analysis, quantifying the success of various state-of-the-art fake news detection models in terms of their accuracy scores. It specifically uses two reputable datasets in the field Politifact and Gossipcop for a robust comparison across different methodologies.

Starting with the methodology put forth by Dong et al., the accuracy scores reveal a significant proficiency of this model in identifying fake news. The model showcases a robust performance, particularly when tested against the Gossipcop dataset, where it achieves an impressive accuracy of 94.86%. Meanwhile, on the Politifact dataset, its performance, while lower, is still substantial at 82.39%.

Cui et al.’s approach yields slightly lower accuracies, achieving 77.24% on the Politifact dataset and 80.42% on the Gossipcop dataset, yet still demonstrating reasonable competence in identifying fraudulent news content. Furukawa et al.’s methodology proves more effective, with accuracies of 85.1% on Politifact and 90% on Gossipcop, highlighting its strong capability in distinguishing between real and fake news. Salama et al.’s method shows remarkable consistency, with accuracies of 83.93% on Politifact and 83.82% on Gossipcop, testifying to its reliability and stability in diverse scenarios. Shu et al.’s strategy exhibits variability, achieving a high 90.4% accuracy on Politifact but a lower 80.8% on Gossipcop, indicating sensitivity to specific dataset features. Among these, the proposed method using GPT-3 models excels, achieving the highest accuracies of 91.98% on Politifact and 95.69% on Gossipcop, validating its efficacy in discerning fake news and reinforcing its suitability for zero-trust security in Industry 5.0 environments.

GPT-3 can process up to 2,049 tokens whereas other PLMs like BERT can only process 500 to 1000. The proposed approach can process longer texts and documents. The Industry 5.0 involves complex text to process. Large Language Models like GPT-3 can handle better than the existing pre-trained language models.

The integration of GPT-3 stands out for its simplicity and reliability when compared to other algorithms. Accessing and fine-tuning GPT-3 is accomplished seamlessly through API calls, reducing both the costs and expertise needed for managing infrastructure to ensure a robust pipeline. This streamlined process contrasts with the potential months required for setting up other systems. Moreover, this accessibility facilitates smooth integration with edge devices.

The GPT-3 models can be adopted to the evolving patterns of the misinformation by continuously updating through re-finetuning on the dynamic data by transfer learning. Other pre-trained models like BERT and DistilBERT requires the need of maintaining infrastructure and establishing proper machine learning workflow, which consumes time and cost. In GPT-3, as finetuning is via API calls it becomes cost effective and reduce time consumption.

Industry 5.0 environments often require real-time or near-real-time processing, particularly for edge devices and time-sensitive human-machine interactions. To evaluate the practical deployability of the GPT-3 variants, we present an analysis of inference latency based on OpenAI API benchmarks and our experimental observations. Table 5 summarizes the approximate inference latency, cost per 1,000 tokens, and suitability for edge deployment across all four GPT-3 variants.

As shown in Table 5, GPT-3 Ada offers the lowest inference latency (50–100ms per request) and the lowest cost, while simultaneously achieving the highest average accuracy across both datasets. This makes Ada the optimal choice for real-time misinformation detection in Industry 5.0 edge environments where latency and cost are critical constraints. Davinci, while comparable in accuracy, incurs 3–5x higher latency and 50x higher cost, making it unsuitable for time-sensitive industrial applications. These results further justify the selection of Ada as the recommended variant for Zero Trust deployment in Industry 5.0.