Have a personal or library account? Click to login
Adaptive trust-based secure routing protocol with reinforced anomaly detection for IoT networks Cover

Adaptive trust-based secure routing protocol with reinforced anomaly detection for IoT networks

Open Access
|Jul 2025

Full Article

I.
Introduction

The rapid advancement of the Internet of Things (IoT) has led to the widespread adoption of interconnected smart devices across various domains, including smart health care, industrial automation, environmental monitoring, smart homes, and intelligent transportation systems. IoT networks facilitate seamless data exchange and automation, thereby enhancing operational efficiency and enabling real-time decision-making. However, the unique characteristics of IoT, such as decentralized architectures, resource constraints, heterogeneous devices, and reliance on wireless communication, expose them to a wide range of security vulnerabilities. Unlike traditional networks, IoT environments lack robust centralized security mechanisms, making them highly susceptible to malicious attacks that can compromise data integrity, network availability, and user privacy [14]. Among the numerous security threats, routing attacks pose a significant challenge as they directly impact the data transmission process, leading to packet loss, misdirection of traffic, unauthorized access, and service disruptions. Blackhole attacks, gray hole attacks, wormhole attacks, Sybil attacks, and selective forwarding attacks are among the most prominent threats targeting IoT routing protocols. In a blackhole attack, malicious nodes absorb all network traffic by falsely advertising optimal routes, causing severe packet loss. Similarly, in a gray hole attack, attackers selectively drop packets, making detection difficult. Wormhole attacks exploit routing vulnerabilities by creating fake tunnels between distant nodes to manipulate routing decisions, while Sybil attacks involve an adversary forging multiple fake identities to influence trust-based decisions. Additionally, selective forwarding attacks occur when compromised nodes drop specific packets instead of forwarding them, resulting in data loss and network inefficiency. These attacks undermine the reliability of IoT networks and hinder their ability to support critical applications that demand secure and efficient communication [58].

To counter these security challenges, several trust-based secure routing protocols (TBSRP) have been proposed in recent years, aiming to detect and mitigate malicious activities in IoT networks. Existing solutions, such as Ad hoc On-Demand Distance Vector (AODV), Trust-Based AODV (TB-AODV), Energy-Efficient Secure Routing (ESR), and Secure AODV (SEC-AODV), use various mechanisms to enhance security while maintaining network performance. Traditional security approaches rely on cryptographic techniques and authentication mechanisms to protect data transmission, but these solutions often introduce significant computational overhead, making them unsuitable for resource-constrained IoT devices [912]. Trust-based routing protocols, on the other hand, assess node behavior over time to establish a trust score, enabling the identification of malicious nodes based on their historical actions. However, existing trust-based solutions suffer from limitations such as static trust thresholds, susceptibility to insider attacks, high false-positive rates, and lack of adaptability to dynamic network conditions. Many protocols fail to effectively detect sophisticated attacks, particularly Sybil and wormhole attacks, as adversaries can manipulate trust scores by behaving normally for an extended period before launching an attack [1316]. Additionally, the reliance on predefined trust evaluation criteria makes existing protocols less effective in handling evolving security threats in IoT networks. Therefore, there is a need for an adaptive, intelligent, and lightweight TBSRP that can dynamically adjust its security mechanisms based on the changing behavior of network nodes.

The increasing complexity and heterogeneity of IoT networks further exacerbate security challenges, making it imperative to develop advanced intrusion detection mechanisms that can differentiate between benign and malicious activities with high accuracy. While machine learning-based anomaly detection has emerged as a promising approach to improving IoT security, its integration with trust-based routing protocols remains largely unexplored. The challenge lies in designing efficient, scalable, and low-latency security mechanisms that can operate effectively in resource-constrained environments without imposing excessive computational burdens on IoT devices. Another critical issue is the trade-off between security and network performance, as overly aggressive security measures can lead to increased routing overhead, higher energy consumption, and unnecessary isolation of legitimate nodes. Addressing these challenges requires a multi-layered security approach that incorporates adaptive trust management, anomaly detection, and efficient route selection strategies to ensure both security and performance optimization in IoT networks [1723].

Motivated by the challenges, this research introduces an adaptive trust-based secure routing protocol (ATBSRP) to enhance the security and resilience of IoT networks against routing attacks. ATBSRP builds upon the previously developed TBSRP by incorporating an enhanced anomaly detection framework, context-aware trust evaluation, and an adaptive security mechanism. Unlike conventional trust-based protocols, ATBSRP uses a multi-dimensional trust assessment approach, which considers multiple behavioral attributes, including packet forwarding consistency, historical reputation, communication reliability, and anomaly scores.

Another major enhancement in ATBSRP is its packet forwarding consistency analysis, which enables the proactive detection of selective forwarding attacks. By comparing actual packet transmission behavior with expected routing patterns, the protocol identifies and isolates nodes exhibiting abnormal drop rates, preventing attackers from selectively discarding critical packets. Furthermore, to counter Sybil attacks, ATBSRP uses trust consistency checks, ensuring that nodes with multiple forged identities cannot manipulate the network’s trust-based decision-making process. Unlike existing solutions, which often fail to differentiate between normal network fluctuations and genuine security threats, ATBSRP incorporates context-aware trust modeling, allowing the protocol to distinguish legitimate variations in node behavior from malicious activities. This significantly reduces false alarms while ensuring higher detection accuracy. Extensive simulations conducted in Network Simulator 3 (NS-3) demonstrate that ATBSRP outperforms existing routing protocols, including AODV, TB-AODV, ESR, SEC-AODV, and TBSRP, across key performance metrics such as packet delivery ratio (PDR), end-to-end delay, throughput, and routing overhead. Despite its significant advantages, the implementation of ATBSRP poses certain challenges, particularly in terms of computational efficiency and scalability. The integration of machine learning-based anomaly detection introduces additional processing requirements that must be optimized to ensure feasibility in resource-limited IoT environments.

II.
Related Work

In recent years, trust-based and secure routing protocols have been widely explored in the IoT to mitigate various security threats and optimize network performance. Muzammal et al. [1] proposed a trust and mobility-based protocol to enhance secure routing in IoT networks, addressing vulnerabilities in the Routing Protocol for Low-Power and Lossy Networks (RPL). Their approach focused on integrating trust evaluation with mobility patterns to ensure secure data transmission while mitigating malicious node influence. Similarly, Muzammal et al. [2] extended their work by introducing a trust-based model that strengthens routing security against RPL-specific attacks. Their approach leveraged trust scores and node behavior analysis to counteract rank and blackhole attacks, significantly improving the resilience of IoT communication.

Bang and Rao [3] tackled the rank attack problem in RPL-based networks by proposing an enhanced version of RPL, named Energy-efficient and Mobility-based Optimized Framework (EMBOF)-RPL. Their solution incorporated an early detection mechanism that effectively isolated malicious nodes before they could manipulate the routing hierarchy. By using rank verification techniques, their approach substantially minimized the risk of network compromise. Singh et al. [4] introduced a context-aware trust and reputation routing protocol for opportunistic IoT networks, leveraging dynamic trust computation based on contextual factors such as node interactions and past behavior. Their papers demonstrated an improvement in routing reliability, ensuring that only trustworthy nodes participated in packet forwarding.

Jiang and Liu [5] analyzed selective forwarding attacks in IoT and proposed a trust-based defense mechanism within RPL networks. Their approach used continuous trust monitoring to detect and mitigate packet-dropping attacks, ensuring robust and efficient routing. Rashidibajgan et al. [6] extended the discussion on security and privacy in opportunistic networks, emphasizing the need for decentralized trust structures. Their papers proposed a secure framework that balanced privacy preservation with effective routing decisions, highlighting the trade-off between security and network performance.

Cai et al. [7] explored the integration of mini-batch machine learning techniques with event-triggered control mechanisms to address denial-of-service (DoS) attacks in fuzzy networked control systems (NCSs). Their approach enhanced real-time attack mitigation while ensuring optimal system performance. Sreenivasa et al. [8] introduced a social context-aware macroscopic routing scheme for opportunistic networks, emphasizing the role of user interactions and mobility patterns in determining trustworthy routing paths. By incorporating social parameters, their approach effectively improved message delivery rates while reducing routing overhead.

Malik [9] focused on energy-efficient routing in opportunistic IoT by leveraging social relationships among devices. Their work demonstrated how trust could be established based on past interactions, optimizing routing decisions while conserving energy. Abadía et al. [10] conducted a systematic survey on IoT frameworks for smart cities, providing a comprehensive analysis of trust management strategies within various IoT architectures. Their paper highlighted the importance of adaptive trust mechanisms in enhancing urban infrastructure resilience.

Abosata et al. [11] developed a customized intrusion detection system for industrial IoT networks, integrating machine learning algorithms to detect security breaches in heterogeneous environments. Their approach demonstrated superior accuracy in identifying anomalous network behaviors. Arshad et al. [12] introduced Trust-based and Hop Count-aware (THC)-RPL, a lightweight trust-enabled routing protocol designed to counteract Sybil attacks in IoT networks. Their approach minimized computational overhead while maintaining a high level of security.

Gothawal and Nagaraj [13] applied an automata-based model to enhance intrusion detection in RPL networks, focusing on real-time attack prevention. Their method demonstrated high detection rates for common routing attacks. Ioulianou et al. [14] proposed a trust-based intrusion detection system capable of identifying combined rank and blackhole attacks, effectively strengthening network defenses. Their paper highlighted the effectiveness of integrating trust metrics with anomaly detection techniques.

Khan et al. [15] explored multi-attribute trust-based routing for embedded IoT devices, ensuring secure data transmission in mobile ad hoc networks (MANETs). Their model utilized multiple trust parameters to enhance decision-making in dynamic environments. Ali et al. [16] extended this work by developing a resilient trust-based protocol for secure IoT communication, optimizing trust evaluation processes for enhanced attack resistance.

Khan et al. [17] modeled distributed denial-of-service (DDoS) attacks in IoT and proposed a trust-based resistance mechanism. Their approach effectively mitigated large-scale attacks by dynamically adjusting trust levels based on network conditions. Their paper underscored the importance of adaptive security strategies in ensuring IoT network reliability.

The aforementioned studies collectively highlight the significance of trust-based mechanisms in securing IoT networks. Existing research has demonstrated that incorporating trust metrics, social context, and machine learning techniques can substantially improve routing resilience. However, challenges remain in optimizing trust evaluation without introducing excessive computational overhead.

a.
Research gaps

While trust-based routing protocols for IoT security have been extensively studied, several gaps remain unaddressed. First, many existing approaches rely on static trust evaluation mechanisms, which fail to adapt quickly to evolving attack patterns. This delay in detection allows adversaries to exploit vulnerabilities before countermeasures are activated. Second, most studies focus on individual attacks such as blackhole or rank attacks, but fewer works consider hybrid or multi-stage attacks where adversaries combine multiple techniques to evade detection. Addressing such advanced threats requires a more comprehensive approach that integrates multiple security mechanisms.

Another significant research gap is the computational overhead associated with trust management in IoT. Many existing protocols require continuous monitoring and evaluation of trust metrics, which can lead to excessive energy consumption in resource-constrained IoT devices. Lightweight and adaptive trust models are needed to balance security with efficiency. Additionally, scalability remains a major concern. Most studies evaluate their protocols in small to medium-sized networks, but large-scale IoT deployments require solutions that maintain performance and security without significant degradation.

III.
Proposed Work

The proposed TBSRP aims to enhance the resilience of IoT networks by integrating adaptive trust evaluation, anomaly detection, and lightweight security mechanisms. This framework addresses the key limitations of existing trust-based approaches, including static trust evaluation, high computational overhead, and vulnerability to hybrid attacks. The proposed solution uses a multi-layered trust assessment model that dynamically adjusts trust values based on node behavior, past interactions, and anomaly detection insights.

a.
Proposed architecture

The architecture of the proposed system is designed to ensure secure and reliable routing in IoT networks. It consists of five primary modules:

  • Trust evaluation module

  • Anomaly detection module

  • Lightweight cryptographic mechanism

  • Adaptive trust-based routing

  • Secure communication framework

a.i
Trust evaluation module

The trust evaluation module is a core component of the proposed TBSRP. It assigns dynamic trust scores to nodes based on their behavior in the network. This module ensures that only reliable nodes participate in routing, thereby mitigating malicious activities such as packet dropping, Sybil attacks, and rank attacks.

The trust evaluation process consists of three main components:

  • Direct trust (Tdirect): Computed based on observed behavior, including packet forwarding rate, response time (RT), and participation consistency (PC).

  • Indirect trust (Tindirect): Derived from recommendations given by neighboring nodes.

  • Historical trust (Thistorical): Considers past interactions, with older interactions being given reduced weight over time.

The final trust score for a node nnn is computed using a weighted combination of these three components: Tn=αTdirect+βTindirect+γThistorical {T_n} = \alpha {T_{{\rm{direct}}}} + \beta {T_{{\rm{indirect}}}} + \gamma {T_{\rm{historical}}} where α, β, and γ adjust the influence of each trust component based on network conditions.

Direct trust calculation (Tdirect)

Direct trust is computed based on a node’s observable behavior, including:

  • Packet forwarding ratio (PFR)

  • RT

  • PC

The direct trust of node n at time t is computed as: Tdirect(n,t)=w1FR(n,t)+w21RT(n,t)+w3PC(n,t) \eqalign{ & {T_{{\rm{direct}}}}(n,t) = {w_1} \cdot FR(n,t) + {w_2} \cdot \left( {1 - RT(n,t)} \right) \cr & \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, + {w_3} \cdot PC(n,t) \cr} where w1, w2, and w3 are the weights assigned to each metric (w1 + w2 + w3 = 1).

PFR
PFR(n,t)=PacketsforwardedbynPacketsrceceivedbyn PFR(n,t) = {{{\rm{Packets}}\,{\rm{ forwarded}}\,{\rm{by}}\,n} \over {{\rm{Packets}}\,{\rm{ rceceived}}\,{\rm{by}}\,n}}

A node that drops packets maliciously will have a lower PFR.

RT

RTn,t=i=1mRim RT\left( {n,t} \right) = {{\mathop \sum \nolimits_{i = 1}^m {R_i}} \over m} where Ri is the RT for packet iii, and mmm is the number of interactions. A higher RT leads to a lower trust value.

PC
PCn,t=ActivetimeofnTotalobservedtime PC\left( {n,t} \right) = {{{\rm{Active}}\,{\rm{time}}\,{\rm{of}}\,n} \over {{\rm{Total}}\,{\rm{ observed}}\,{\rm{time}}}}

A node that frequently disconnects or remains idle has a lower PC.

Indirect trust calculation (Tindirect)

Indirect trust is computed using recomendations from neighboring nodes. If node n has k neighbors, their recommendations contribute to Tindirect. Tindirectn,t=i=1kTin,t·Cin,ti=1kCin,t {T_{{\rm{indirect}}}}\left( {n,t} \right) = {{\mathop \sum \nolimits_{i = 1}^k {T_i}\left( {n,t} \right)\cdot{C_i}\left( {n,t} \right)} \over {\mathop \sum \nolimits_{i = 1}^k {C_i}\left( {n,t} \right)}} where:

  • Ti(n,t) is the trust score assigned to node n by neighbor i.

  • Ci(n,t) is the confidence level of neighbor i (nodes with a history of reliable behavior have higher confidence values).

To prevent bad-mouthing attacks (where malicious nodes give false recommendations), only neighbors with a trust score above a threshold Tth contribute to Tindirect Tin,t=Tin,tTth0,otherwise. {T_i}\left( {n,t} \right) = \left\{ {\matrix{ {{T_i}\left( {n,t} \right) \ge {T_{th}}} \hfill \cr {0,\;otherwise} \hfill \cr }.} \right.

Historical trust calculation (Thistorical)

Historical trust considers past interactions but reduces the influence of older interactions through a decay function. The historical trust is computed as: Thistorical=j=0mTdirectn,tjeλj {T_{\rm historical}} = \sum\limits_{j = 0}^m {{T_{\rm direct}}\left( {n,t - j} \right)} \cdot {e^{ - \lambda j}} where:

  • Tdirect(n, tj) is the direct trust value at time tj.

  • e−λj is an exponential decay function (λ controls the rate of decay).

  • m is the history length.

  • A higher λ value ensures that older interactions contribute less to Thistorical.

Final trust score calculation

Using the computed values of Tdirect, Tindirect, Thistorical, the final trust score is determined by: Tn=αTdirect+βTindirect+γThistorical {T_n} = \alpha {T_{{\rm{direct}}}} + \beta {T_{{\rm{indirect}}}} + \gamma {T_{{\rm{historical}}}} where:

  • α, β, γ dynamically adjust based on network conditions:

    • If the network is stable, α is increased to give more weight to direct observations.

    • If indirect recommendations are reliable, β is increased.

    • If trust must be built over time, γ is increased.

a.ii
Anomaly detection module

The anomaly detection module is responsible for identifying and mitigating various network attacks, including blackhole, Sybil, rank, and wormhole attacks. This module uses a hybrid detection approach that integrates:

Behavioral analysis-based detection Detection of routing anomalies

Nodes are expected to follow standard routing behavior. Anomalies such as blackhole, rank, and wormhole attacks introduce deviations from expected norms. The module monitors key network parameters:

  • Packet Delivery Ratio (PDR)

    A node consistently dropping packets may be involved in a blackhole or wormhole attack.

    Computed as: PDRn=1PacketsforwardedbynPacketsreceivedbyn PDR\left( n \right) = 1 - {{{\rm{Packets}}\,{\rm{forwarded}}\,{\rm{by}}\,n} \over {{\rm{Packets}}\,{\rm{received}}\,{\rm{by}}\,n}}

    A node with PDR > θ (threshold) is flagged as suspicious.

  • Rank consistency (RC) check

    • In RPL-based networks, rank attacks involve falsifying rank values to mislead routing.

    • If node n frequently changes its rank inconsistently with network topology, it is suspicious.

    • Measured using: RCn=t=1T|Rankn,tRankn,t1| {\rm{RC}}\left( n \right) = \sum\limits_{t = 1}^T {|{\rm{Rank}}\left( {n,t} \right) - {\rm{Rank}}\left( {n,t - 1} \right)|}

    If RC(n) exceeds a predefined threshold, node n is flagged.

  • Trust score variance (TSV)

    A high variance in the trust score indicates unreliable behavior.

    Computed as: TSVn=1Tt=1T(TntTn)2 {\rm{TSV}}\left( n \right) = {1 \over T}\sum\limits_{t = 1}^T {{{({T_n}\left( t \right) - T_n^ - )}^2}}

    A high TSV(n) suggests erratic behavior, often seen in Sybil attacks.

Gaussian mixture model (GMM)-based detection

The GMM approach detects anomalies by modeling the probability distribution of normal traffic and flagging outliers.

GMM-based probability estimation

GMM assumes that normal traffic follows a multimodal Gaussian distribution: pX=k=1KπkNX|μk,Σk p\left( X \right) = \mathop \sum \nolimits_{k = 1}^K \pi k \cdot N\left( {X|\mu k,{\Sigma _k}} \right) where:

  • K is the number of Gaussian components.

  • πk is the weight of the kth Gaussian component.

  • N (X | µk, Σk) is a Gaussian distribution with mean µk and covariance matrix Σk.

Anomaly score computation

For a given node n, its traffic feature vector Xn is evaluated using the likelihood function: Sanomaly(n)=logpXn {S_{{\rm{anomaly}}}}(n) = - logp\left( {{X_n}} \right)

If Sanomaly (n) > τ (predefined threshold), the node is classified as anomalous.

Machine learning-based detection

To improve real-time classification, the module uses lightweight classifiers such as decision trees to differentiate between normal and malicious nodes.

Feature selection

The classifier is trained on the extracted features:

  • PFR

  • RC

  • TSV

  • GMM anomaly score (Sanomaly)

Each node n is classified as trustworthy or malicious using: Cn=f(PFR,RC,TSV,Sanomaly) C\left( n \right) = f({\rm{PFR}},{\rm{RC}},{\rm{TSV}},{S_{{\rm{anomaly}}}}) where f is a decision tree classifier trained on labeled network data.

Decision rule

A simple decision tree rule can be:

  • If PFR < < θ1 AND Sanomaly > τMalicious

  • If RC > θ2RC > → Suspicious (requires further analysis)

  • Otherwise → Trustworthy

a.iii
Lightweight cryptographic mechanism

To ensure secure communication while maintaining computational efficiency, the proposed system integrates lightweight cryptographic techniques designed for resource-constrained environments such as IoT networks.

Elliptic curve cryptography (ECC)

ECC is chosen due to its high security with smaller key sizes, making it ideal for resource-constrained environments. Unlike RSA, which requires 2048-bit keys for strong security, ECC provides equivalent security with only 256-bit keys, reducing computational load and memory usage.

Key generation in ECC

ECC operates over a finite field F2m defined by the elliptic curve equation. y2=x3+ax+bmodp {y^2} = {x^3} + ax + b\,{\rm{mod}}\;p where:

  • p is a prime number defining the field,

  • a,b are constants satisfying 4a3 + 27b2 ≠ 0, ensuring a valid curve.

  • Each node selects a private key d and computes the public key P: P=dG P = dG where G is a generator point on the elliptic curve.

Encryption using ECC

To encrypt a message M using the receiver’s public key Pr:

  • Choose a random integer k.

  • Compute the shared point: C1=kG {C_1} = kG

  • Compute the masked message: C2=M+kPr {C_2} = M + k{P_r}

  • The ciphertext is (C1, C2).

Decryption using ECC

The receiver, with private key dr, decrypts the message: M=C2drC1 M = {C_2} - {d_r}{C_1}

Since drC1 = dr(kG), the message is correctly recovered.

One-time hash chains (preventing replay attacks)

Replay attacks occur when a malicious node reuses authentication messages to gain unauthorized access. The proposed mechanism prevents this by dynamically updating authentication tokens using a hash chain.

Hash chain generation

Each node generates a one-time hash chain using a cryptographic hash function H. The chain is generated as follows: H0=randomseedHi+1=HHi \eqalign{ & {H_0} = {\rm{random}}\,{\rm{seed}} \cr & {H_{i + 1}} = H\left( {{H_i}} \right) \cr} where:

  • H is a secure one-way hash function.

  • HN (final hash) is stored securely and used as the first authentication token.

Authentication process
  • Initial token exchange

    • The sender transmits HN as an authentication token.

  • Subsequent authentications

    • Each new session, the sender provides HN−1, verified using: HHN1=HN H\left( {{H_{N - 1}}} \right) = {H_N}

    • This process continues, to ensure the one-time use of tokens.

  • Security properties

    • Prevents replay attacks: Each token is used only once.

    • Computational efficiency: Hash operations are lightweight.

    • Forward security: Even if an old token is exposed, future tokens remain secure.

3.
Challenge-response mechanism (detecting compromised nodes)

To detect compromised nodes attempting unauthorized data access, the system implements a challenge-response mechanism based on ECC signatures.

Challenge generation

A node A challenges a suspected node B by sending a random nonce N: CA=N,HN {C_A} = \left( {N,H\left( N \right)} \right)

Response verification
  • Node B signs N using its private key dB: SB=dBHN {S_B} = {d_B}\,H\left( N \right)

  • Node A verifies SB using B’s public key P_B: HNG=PBSB H\left( N \right)G = {P_B}\,{S_B}

  • If verification fails, node B is considered compromised.

Algorithm 1:

Hybrid Anomaly Detection in Wireless Networks

    Input:
  • Network traffic data: D = {d1, d2,…, dn}

  • Threshold values: Tbehavioral, TGMM, TML

  • Pre-trained machine learning model: MML

    Output:
  • Classified nodes as Normal or Malicious

  • Identified attack types (e.g., Blackhole, Sybil, Rank, Wormhole)

Step 1: Initialize Parameters

  • Define normal traffic behavior profile Bnormal.

  • Set threshold values for anomaly detection:

    • Tbehavioral for deviation in routing behavior

    • TGMM for Gaussian Mixture Model-based detection

    • TML for machine learning-based classification

Step 2: Behavioral Analysis-Based Anomaly Detection

For each node Ni in the network:

  • Monitor routing behavior metrics:

    • Packet Forwarding Rate: Fi

    • Response Time: Ri

    • Rank Advertisement Consistency: Ci

  • Compute deviation score: Sbehavioral=FiFnormal+RiRnormal+CiCnormal3 {S_{\rm behavioral}} = {{\left| {{F_i} - {F_{\rm normal}}} \right| + \left| {{R_i} - {R_{\rm normal}}} \right| + \left| {{C_i} - {C_{\rm normal}}} \right|} \over 3}

  • If Sbehavioral > Tbehavioral, flag Ni as suspicious

Step 3: Gaussian Mixture Model (GMM) Based Detection

For each node Ni:

  • Extract feature vector Xi from network traffic data.

  • Estimate probability distribution using GMM: PXi=k1kπkNXi|μk,Σk P\left( {{X_i}} \right) = \sum\limits_{k - 1}^k {{{\rm{\pi }}_k}{\rm{N}}\left( {{X_i}|{\mu _k},{\Sigma _k}} \right)} where

    • πk are mixture weights

    • N is the Gaussian distribution

    • μk, Σk are mean and covariance of the kth Gaussian component

  • Compute anomaly score: SGMM=1PXi {S_{\rm GMM}} = 1 - P\left( {{X_i}} \right)

  • If SGMM > TGMM, classify Ni as anomalous.

Step 4: Machine Learning-Based Classification

For each node Ni:

  • Extract feature vector Xi.

  • Input Xi into pre-trained machine learning model MML.

  • Compute classification output Ci: Ci=MMLXi {C_i} = {M_{\rm ML}}\left( {{X_i}} \right)

  • If Ci predicts malicious behavior with confidence > Trustworthiness Margin Level (TML), classify Ni as malicious.

Step 5: Final Decision and Attack Type Identification

  • Combine detection results: Sfinal=w1Sbehavioral+w2SGMM+w3Ci {S_{{\rm{final}}}} = {w_1}{S_{{\rm{behavioral}}}} + {w_2}{S_{{\rm{GMM}}}} + {w_3}{C_i} where w1, w2, w3 are weight factors based on detection reliability.

  • If Sfinal exceeds the global anomaly threshold Tfinal, classify Ni as malicious.

  • Identify attack type based on features:

    • High packet drop rate → Blackhole Attack

    • Multiple fake identities → Sybil Attack

    • Inconsistent rank values → Rank Attack

    • Unusual route manipulation → Wormhole Attack

Step 6: Response Mechanism

  • Isolate detected malicious nodes from routing tables.

  • Update trust scores for affected nodes.

  • Alert network administrators for further investigation.

a.iv
Adaptive trust-based routing

The adaptive trust-based routing mechanism integrates trust scores, anomaly detection outcomes, and dynamic path selection strategies to ensure a secure, reliable, and efficient routing process in IoT and wireless networks. By continuously evaluating the trustworthiness of nodes, the system ensures that only secure and high-quality paths are utilized, minimizing the risk of malicious activities such as blackhole, Sybil, rank, and wormhole attacks. This approach enhances the robustness of network communication by dynamically adapting to changes in node behavior and network conditions.

Threshold-Based Routing Selection

The trust-based routing decision begins by evaluating all possible paths between a source node and a destination node based on their cumulative trust scores. Each node in the network maintains a trust score that is calculated dynamically using direct trust, indirect trust, and historical trust evaluations. These scores are aggregated along candidate paths to compute the overall trustworthiness of a route. A predefined trust threshold Tth is established, and only routes that satisfy: TPκ=iPkTiTth {T_{{P_\kappa }}} = \sum\limits_{i \in {P_k}} {{T_i} \ge {T_{th}}} are considered for data transmission. If a path’s trust score is below the threshold, it is discarded, reducing the likelihood of routing through compromised or unreliable nodes. This proactive selection mechanism significantly mitigates security threats by ensuring that only verified and trusted routes are used for communication.

Additionally, the trust threshold is dynamically adjustable based on network conditions. In highly secure environments, a higher threshold may be enforced, whereas in resource-constrained settings, a slightly relaxed threshold may be adopted to ensure network connectivity while still maintaining a reasonable level of security.

Dynamic path adaptation

The trustworthiness of nodes evolves over time due to their behavioral changes or security threats. In traditional routing approaches, once a path is selected, it remains in use until network conditions change significantly. However, in adaptive trust-based routing, real-time monitoring of nodes ensures that any changes in their trust scores trigger an immediate re-evaluation of routing paths.

  • Continuous Monitoring: Each node’s behavior is continuously monitored for signs of malicious activity using an anomaly detection module.

  • Trust Score Adjustment: If a node exhibits suspicious behavior (e.g., dropping packets, forwarding inconsistencies, rank manipulation), its trust score is updated in real-time to reflect these anomalies.

  • Immediate Route Recalculation: If a previously trusted node falls below an acceptable trust level, the routing algorithm dynamically recalculates the path using alternate secure nodes.

This self-healing mechanism ensures that the network remains resilient against attacks. Suppose an adversary infiltrates the system and starts behaving maliciously; the framework can quickly isolate the compromised node and reroute traffic through alternative secure paths. To enhance efficiency, the path recalculation process uses an incremental trust update mechanism, which avoids unnecessary recomputation of all paths. Only affected portions of the routing table are dynamically adjusted, reducing computational overhead while maintaining security.

Quality of service (QoS)-aware routing

Security alone is insufficient if network performance degrades due to inefficient path selection. Therefore, adaptive trust-based routing ensures that routes with the highest trust scores also meet QoS constraints such as latency and bandwidth requirements.

  • Latency Optimization: Paths are evaluated based on their end-to-end delay, ensuring that packets reach their destination within an acceptable timeframe. High-trust paths that exhibit excessive delay are penalized to balance security with performance.

  • Bandwidth Considerations: The routing mechanism prioritizes paths that offer sufficient bandwidth, preventing congestion and ensuring smooth data transmission.

  • Load Balancing: To prevent overuse of highly trusted paths, a load distribution mechanism is implemented, ensuring that multiple secure routes are utilized efficiently rather than overloading a single high-trust path.

Algorithm 2:

Adaptive Trust-Based Routing Algorithm

    Input:
  • Network topology G(V, E), where V is the set of nodes and E is the set of links.

  • Trust scores Tn for each node Ni.

  • Anomaly detection results identifying malicious nodes.

  • Threshold trust value Tth.

  • Quality of Service (QoS) constraints (latency L, bandwidth B).

    Output:
  • Secure and optimal routing path.

Step 1: Initialization

  • Construct the initial routing graph G(V,E).

  • Assign initial trust scores Tn to each node Ni using: Tn=αTdirect+βTindirect+γThistorical {T_n} = \alpha {T_{{\rm{direct}}}} + \beta {T_{{\rm{indirect}}}} + \gamma {T_{{\rm{historical}}}} where α, β, γ are weighting factors.

Step 2: Threshold-Based Routing Selection

  • For each candidate path Pk from source S to destination D:

    • Compute cumulative trust score: TPk=iPκTi {T_{{P_k}}} = \sum\limits_{i \in {P_\kappa }} {{T_i}}

    • If TPk < Tth, discard path Pk.

Step 3: QoS-Aware Path Selection

  • For each remaining path Pk:

    • Compute QoS parameters:

      • Total latency TPk

      • Minimum bandwidth BPk

    • Ensure paths satisfy constraints: LPkLmax,BPkBmin {L_{{P_k}}} \le {L_{\max }},{B_{{P_k}}} \ge {B_{\min }}

Step 4: Dynamic Path Adaptation

  • Monitor network nodes periodically.

  • If a trusted node Ni is later flagged as malicious:

    • Remove Ni from routing graph G(V,E).

    • Recalculate P* using steps 2 & 3.

Step 5: Route Maintenance and Trust Updates

  • Update trust scores dynamically using recent interactions.

  • Apply trust decay to reduce the influence of outdated interactions: Tnt+1=λTnt+1λTnew {T_n}\left( {t + 1} \right) = \lambda {T_n}\left( t \right) + \left( {1 - \lambda } \right){T_{{\rm{new}}}} where λ is the decay factor.

  • Reevaluate routing decisions periodically based on updated trust and QoS conditions.

a.v.
Secure communication framework

The secure communication framework integrates multiple security components to ensure robust, attack-resilient communication in networks. It begins with the trust evaluation module, which dynamically assesses node trustworthiness based on direct interactions, indirect recommendations, and historical behavior, adjusting trust scores in real time. The anomaly detection module then identifies malicious activities such as blackhole, Sybil, rank, and wormhole attacks using behavioral analysis, GMM, and lightweight machine learning classifiers. To secure data transmission, the lightweight cryptographic mechanism uses ECC for confidentiality, one-time hash chains for integrity, and a challenge-response mechanism for access control. The adaptive trust-based routing module selects optimal communication paths by ensuring that only routes with a cumulative trust score above a predefined threshold are used, adapting dynamically when a previously trusted node is flagged as malicious. This approach enhances network security while maintaining QoS by selecting high-trust, high-performance paths. By combining real-time monitoring, cryptographic protection, and adaptive routing, the proposed framework effectively mitigates cyber threats while ensuring secure and efficient data transmission.

IV.
Experimental Study and Results

This section presents the experimental setup, evaluation metrics, and comparative analysis of the proposed ATBSRP. The effectiveness of ATBSRP is assessed through extensive simulations using NS-3, a widely used network simulator for IoT environments. Various performance parameters, such as packet delivery ratio (PDR), end-to-end delay, throughput, routing overhead, and detection accuracy, are measured to validate the proposed framework. The obtained results are compared with existing protocols, including AODV, TB-AODV, ESR, and SECAODV, as well as trust-based secure routing models from the recent literature. The proposed ATBSRP is implemented and evaluated using the NS-3 simulator, running on a Linux-based system. The simulation setup as shown in Table 1 considers an IoT network with 100–500 nodes, randomly deployed in a 1,000 m × 1,000 m area with mobile and static nodes. The routing attacks simulated include blackhole, Sybil, rank, and wormhole attacks, which are detected using the anomaly detection module. The trust evaluation mechanism continuously updates trust scores, and cryptographic techniques, such as ECC, are used for data security.

Table 1:

Simulation parameters

ParameterValue
Simulation toolNS-3
Simulation area1,000 m × 1,000 m
Number of nodes100–500
Traffic typeCBR
Transmission range250 m
MAC protocolIEEE 802.15.4
Mobility modelRandom waypoint
Attack types simulatedBlackhole, Sybil, rank, wormhole
Trust evaluation methodDirect & indirect trust, trust decay mechanism
Cryptographic securityECC, one-time hash chains
Detection algorithmGMM, decision tree

CBR, constant bit rate; ECC, elliptic curve cryptography; GMM, Gaussian mixture model.

To evaluate the proposed ATBSRP, the following key metrics are used:

  • Packet Delivery Ratio (PDR): The ratio of successfully received packets to sent packets.

  • End-to-End Delay: The average time taken by a packet to traverse from source to destination.

  • Throughput: The amount of successfully transmitted data over a given time period.

  • Routing Overhead: The ratio of control packets to data packets, reflecting protocol efficiency.

  • Detection Accuracy: The effectiveness of the anomaly detection module in detecting malicious activities.

The proposed ATBSRP is compared against four existing works from the literature to highlight its improvements in security and efficiency, as shown in Table 2.

Table 2:

Comparative analysis

MetricATBSRP (proposed)Muzammal et al. (2022a) [2]Bang and Rao (2022) [3]Singh et al. (2024) [4]Jiang and Liu (2022) [5]
PDR (%)92.885.483.988.286.7
End-to-end delay (ms)22.130.428.726.927.3
Throughput (kbps)512450438470460
Routing overhead (%)12.318.219.615.817.4
Detection accuracy (%)96.590.287.891.589.3

ATBSRP, adaptive trust-based secure routing protocol; PDR, packet delivery ratio.

The comparative analysis of the ATBSRP against existing secure routing models highlights its superiority in key performance metrics. Packet delivery ratio (PDR), which measures the reliability of packet transmission, is 92.8% for ATBSRP, significantly higher than Muzammal et al. (85.4%), Bang and Rao (83.9%), Singh et al. (88.2%), and Jiang and Liu (86.7%). This improvement is attributed to ATBSRP’s trust-based routing mechanism, which prevents malicious nodes from disrupting data transmission. The end-to-end delay, a critical metric for real-time applications, is minimized in ATBSRP at 22.1 ms, compared to 30.4 ms, 28.7 ms, 26.9 ms, and 27.3 ms in other models. This significant reduction in delay is due to dynamic path adaptation, which ensures that data are rerouted through the most trusted and efficient nodes.

Another vital factor in evaluating routing efficiency is throughput, which measures the rate of successful data transmission. ATBSRP achieves 512 kbps, surpassing the throughput of Muzammal et al. (450 kbps), Bang and Rao (438 kbps), Singh et al. (470 kbps), and Jiang and Liu (460 kbps). The higher throughput is a direct result of efficient trust evaluation and anomaly detection, ensuring minimal disruption from malicious nodes. Routing overhead, which indicates the proportion of control packets to total transmitted packets, is lowest in ATBSRP at 12.3%, whereas existing methods have higher overhead, such as 18.2%, 19.6%, 15.8%, and 17.4%. This is due to ATBSRP’s lightweight cryptographic mechanisms and efficient trust management, which minimize unnecessary control message exchanges. One of the most critical metrics for security is detection accuracy, which evaluates the effectiveness of the anomaly detection module. ATBSRP outperforms other models with an accuracy of 96.5%, compared to 90.2%, 87.8%, 91.5%, and 89.3% in the competing methods. This enhanced accuracy is due to the hybrid anomaly detection module, which integrates GMMs and machine learning-based classification, which effectively detects malicious activities such as blackhole, Sybil, rank, and wormhole attacks. In summary, ATBSRP consistently outperforms existing approaches across all metrics, demonstrating its ability to enhance security, efficiency, and network performance in IoT-based environments.

The performance of ATBSRP is further analyzed with varying numbers of malicious nodes to assess its scalability and resilience to attacks, as in Tables 36. The result of the metrics is shown in Figures 14.

Table 3:

Comparison of PDR with increasing malicious nodes

No. of malicious nodesProposedMuzammal et al. (2022) [2]Bang and Rao (2022) [3]Singh et al. (2024) [4]Jiang and Liu (2022) [5]
098.295.194.396.094.8
1095.690.888.992.389.7
2092.885.483.988.286.7
3089.781.678.283.580.9
4086.376.272.579.075.6

PDR, packet delivery ratio.

Table 4:

Comparison of end-to-end delay (ms) with increasing malicious nodes

No. of malicious nodesProposedMuzammal et al. (2022) [2]Bang and Rao (2022) [3]Singh et al. (2024) [4]Jiang and Liu (2022) [5]
018.722.523.021.822.1
1020.425.726.324.525.0
2022.130.428.726.927.3
3024.634.932.530.831.2
4028.238.736.134.435.0
Table 5:

Comparison of throughput (kbps) with increasing malicious nodes

No. of malicious nodesATBSRP—proposedMuzammal et al. (2022) [2]Bang and Rao (2022) [3]Singh et al. (2024) [4]Jiang and Liu (2022) [5]
0540510500520505
10530480470495485
20512450438470460
30489420410440430
40460390375405395

ATBSRP, adaptive trust-based secure routing protocol.

Table 6:

Comparison of detection accuracy (%) with increasing malicious nodes

No. of malicious nodesProposedMuzammal et al. (2022) [2]Bang and Rao (2022) [3]Singh et al. (2024) [4]Jiang and Liu (2022) [5]
1098.392.490.194.591.3
2096.590.287.891.589.3
3094.286.783.988.185.6
4091.782.379.484.781.2
Figure 1:

PDR analysis. ATBSRP, adaptive trust-based secure routing protocol; PDR, packet delivery ratio.

Figure 2:

Delay analysis. ATBSRP, adaptive trust-based secure routing protocol.

Figure 3:

Throughput analysis. ATBSRP, adaptive trust-based secure routing protocol.

Figure 4:

Accuracy analysis. ATBSRP, adaptive trust-based secure routing protocol.

The performance comparison of the proposed ATBSRP protocol with existing approaches [1, 35] across various network parameters reveals its effectiveness in handling malicious nodes. In terms of packet delivery ratio (PDR), the proposed method consistently outperforms others, maintaining a higher PDR even as the number of malicious nodes increases, demonstrating its resilience against attacks. Regarding end-to-end delay, ATBSRP achieves lower delay values compared to existing methods, ensuring faster data transmission despite the growing presence of malicious nodes. Similarly, the throughput analysis shows that ATBSRP maintains higher data transmission rates, whereas the other approaches exhibit a significant decline as the number of malicious nodes increases. Furthermore, the detection accuracy comparison highlights that the proposed method consistently achieves superior accuracy, efficiently identifying and mitigating threats in the network. These findings collectively indicate that ATBSRP enhances network reliability and security, outperforming the benchmarked approaches across key performance metrics.

The lightweight cryptographic mechanism used in ATBSRP is evaluated based on encryption time and key generation time, as shown in Table 7.

Table 7:

ECC performance evaluation

Key size (bits)Encryption time (ms)Decryption time (ms)Key generation time (ms)
1601.21.13.4
1921.61.44.1
2242.11.85.0
2562.72.36.2

ECC, elliptic curve cryptography.

The performance evaluation of the lightweight cryptographic mechanism in ATBSRP is analyzed based on key size, encryption time, decryption time, and key generation time. As the key size increases from 160 bits to 256 bits, encryption and decryption times gradually rise. For a 160-bit key, encryption takes 1.2 ms, while decryption takes 1.1 ms. At 256 bits, encryption time increases to 2.7 ms, and decryption time reaches 2.3 ms. This expected increase is due to the higher computational complexity of larger key sizes.

Key generation time also follows a similar pattern, rising from 3.4 ms (160-bit) to 6.2 ms (256-bit). The results indicate that while larger keys provide stronger security, they come at the cost of slightly increased computational overhead. However, the use of ECC ensures efficient performance, making it suitable for IoT environments where resource constraints are critical. The trade-off between security and computational efficiency is well-balanced.

V.
Conclusion

This research introduces ATBSRP, a robust framework designed to enhance security and efficiency in IoT-based networks. The proposed system integrates a trust evaluation module, anomaly detection module, lightweight cryptographic mechanism, and adaptive trust-based routing to mitigate routing attacks such as blackhole, Sybil, rank, and wormhole attacks. By leveraging direct and indirect trust scores alongside historical trust evaluation, ATBSRP ensures that routing decisions are based on reliable node behavior. The GMM and machine learning-based detection techniques further strengthen security by effectively identifying malicious nodes. Experimental evaluations demonstrate superior performance of ATBSRP in terms of packet delivery ratio (PDR), end-to-end delay, throughput, routing overhead, and detection accuracy compared to existing approaches. The results indicate that ATBSRP outperforms existing protocols, offering higher security, lower latency, and improved detection accuracy while maintaining optimal QoS. Beyond technical advancements, this research has significant contributions to human society by strengthening cybersecurity in wireless networks, which are essential for applications such as health care IoT, emergency communication, and smart city infrastructure. The enhanced trust-based security model ensures reliable data transmission in critical systems, preventing cyber threats that could disrupt essential services. Furthermore, the lightweight cryptographic mechanisms ensure energy-efficient security solutions, benefiting resource-constrained environments like remote health care monitoring and disaster response networks. By mitigating security threats in real-time and ensuring resilient communication, ATBSRP contributes to building safer, more secure digital ecosystems for society at large.

Language: English
Submitted on: Jan 10, 2025
Published on: Jul 14, 2025
Published by: Professor Subhas Chandra Mukhopadhyay
In partnership with: Paradigm Publishing Services
Publication frequency: 1 times per year

© 2025 K. Sangeetha, K. Arulanandam, published by Professor Subhas Chandra Mukhopadhyay
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.