Towards Automated Identification of Block Cipher Structures Using Machine Learning

In the contemporary digital landscape, where vast amounts of data are exchanged across untrusted networks, cryptography [1,2] serves as the cornerstone of information security. It ensures confidentiality, authenticity, and integrity—three essential pillars that uphold secure communication [3,4]. Among various cryptographic primitives, block ciphers have emerged as the fundamental building blocks of symmetric key encryption [5–7], where the same secret key is used for both encryption and decryption. A block cipher converts fixed-length plaintext blocks into ciphertext through multiple rounds of substitution, permutation, and key-dependent transformations, thereby achieving high degrees of diffusion and confusion as envisioned by Claude Shannon [8,9]. These operations collectively protect sensitive information against unauthorized access and cryptanalytic attacks [10].

Prominent examples of block ciphers such as Advanced Encryption Standard (AES) and Data Encryption Standard (DES) are based on well-established structural paradigms, primarily Substitution–Permutation Networks (SPN) and Feistel networks. These structural designs govern how confusion and diffusion are achieved within the encryption process and form the basis for cryptographic security analysis. Understanding and identifying the structural composition of a block cipher are crucial in numerous real-world scenarios such as digital forensics, malware analysis, and cryptographic auditing. The ability to infer whether a ciphertext originates from a Substitution–Permutation Network (SPN) or a Feistel Network provides valuable insights into the possible family of ciphers [11], facilitating further cryptanalysis and security evaluation. In Feistel structure, the data is divided into two halves and then processed. While in SPN, substitution layer is achieved through the s-box and its careful construction plays the key role in the overall security of the cipher [12].

In practice, block ciphers are deployed using secure modes of operation such as Cipher Block Chaining (CBC), Counter (CTR), and Galois/Counter Mode (GCM), which are designed to provide strong security guarantees, including indistinguishability under chosen-plaintext attacks (IND-CPA). Under this model, ciphertexts produced by a secure encryption scheme are expected to be computationally indistinguishable from random sequences, thereby preventing adversaries from inferring any meaningful information about the plaintext or underlying structure. However, the IND-CPA security notion is defined in an asymptotic and computational sense, assuming ideal conditions such as unlimited randomness, sufficiently large data, and adversaries bounded by polynomial-time computation. In contrast, real-world scenarios often involve finite-length ciphertext samples, limited data availability, and empirical analysis settings where statistical properties may deviate slightly from theoretical expectations.

The problem of identifying cryptographic algorithms or their structural properties from observed data has been studied in various contexts, particularly in network security, malware analysis, and protocol reverse engineering. Early approaches relied on handcrafted statistical features such as byte frequency distributions, entropy measures, and n-gram analysis to distinguish between encrypted, compressed, and plaintext data. Traditional analytical methods in cryptography rely heavily on heuristic or mathematically intensive procedures, such as differential and linear cryptanalysis [13–16]. These approaches demand significant computational effort and deep domain expertise, making them impractical for large-scale or automated cryptographic identification tasks. Consequently, there has been growing interest in leveraging data-driven approaches, particularly machine learning (ML), to automate cipher recognition and structural analysis. Recent advances in machine learning have demonstrated remarkable capabilities in detecting complex patterns in high-dimensional data. In the context of cryptography, machine learning has been applied to side-channel analysis, cryptanalysis, and algorithm identification tasks. However, most prior work has focused on scenarios where structural leakage is explicit, such as Electronic Codebook (ECB) mode, or where auxiliary information is available.

The combination of cryptanalysis using machine learning techniques has taken the field one step forward in advancing the automated cipher identification paradigm, thereby yielding an active research area. In this regard, preliminary efforts focused on classifying the behavior of the cipher based on the statistical relationships and frequency patterns exhibited by the bytes of the ciphertext. Early researchers, such as Torres et al. [17] and de Souza et al. [18], introduced the use of genetic algorithms and graph theory to sort not only cryptographic keys but also the underlying encryption algorithm. Additionally, Manjula [19] employed decision tree classifiers to distinguish between stream ciphers, block ciphers, and public-key ciphers, achieving a maximum accuracy of 75%, marking the beginning of the effectiveness of machine learning algorithms in cryptanalysis. Based on these early developments in the automated cryptanalysis domain, the use of supervised ML techniques brought a new level of precision to cipher classification. Swapna et al. [20] demonstrated the usability of Support Vector Machines for classifying encryption algorithms like DES, 3DES, AES, Blowfish, and RC2, while Barbosa et al. [21] attempted to inject a text modeling concept by applying a “bag of words” ideology to the structure of the encrypted text based on its linguistic nature defined by the content of the ciphertext.

These works proved that even if the ciphertext looks random, some statistical patterns exist that could be utilized for cipher recognition algorithms. These approaches often exploit statistical irregularities in real-world data, including deviations from ideal randomness due to finite sample sizes, padding schemes, or implementation artifacts. However, most prior studies either focus on algorithm identification in weak settings (e.g., ECB mode) or require auxiliary metadata, limiting their applicability to realistic cryptographic scenarios.

Further work has been done in the field of feature enhancement to achieve accurate outcomes. In this context, the work of Zhao et al. [22] discussed the use of the integration of Recursive Feature Elimination and XGBoost for better accuracy, obtaining a classification accuracy of 89.6% for ten block ciphers. Nagireddy and Murthy [23] explained the importance of histogram features in identifying different types of encryption algorithms based on these histogram features as powerful discriminators. Zhang et al. [24] and Yuan et al. [25] further explored the use of statistical ASCII code and convolutional features separately for classifying the aforementioned encryption algorithms based on recreating the characteristics of the ciphertext.

In addition to this, Ciphertext features based on NIST randomness has also been employed for cipher classification. As Yuan et al. [26] developed an ensemble model, HKNNRF (Hybrid KNN and Random Forest), for identifying block ciphers in ECB mode based on NIST randomness based ciphertext features with 69.5% binary and 34% five-class accuracy—better than the conventional models SVM, KNN and RF. These developments collectively underscored the promise of ML-based approaches in automating cipher structure identification.

More recent research has extended this line of inquiry through deep learning and hybrid neural frameworks. In particular, Xia et al. [27] constructed a feature-engineered residual neural network-based cryptographic algorithm identifier. By taking NIST randomness indices as input features, the model correctly identified six block and public-key ciphers. The proposed method had over 90% accuracy, displaying that it was robust and could be applied to cryptanalysis. While, Yuan et al. [28] proposed a transformer-based model that achieved notable performance gains over conventional ML classifiers by using improved NIST randomness testing approach. Similarly, Hu et al. [29] and Xie et al. [30] explored CNN-based and CNN–Transformer fusion architectures, achieving accuracies above 90% in ECB-mode experiments and demonstrating the interpretability of ciphertext bit patterns. Kaner et al. [31] used deep learning for randomness analysis instead of relying on NIST randomness with 87 precision and 97 recall and showed another more efficient way of randomness evaluation.

Likewise, Jeong et al. [32] conducted a comprehensive neural cryptanalysis of five widely used block ciphers, experimenting with the vulnerability of these ciphers through various modes of attack such as Encryption Emulation, Plaintext Recovery, and Key Recovery. Their experimental results with additional training data and sentence-based analysis demon-started that RNN-based models are better than other models like transformers in the recovery of keys and ciphertexts.

Despite their success, most of these efforts operate under the Electronic Codebook (ECB) mode—a simplified setting that processes blocks independently and fails to capture inter-block dependencies that occur in realistic cipher operation modes such as Cipher Block Chaining (CBC).

While the literature demonstrates substantial progress, several critical gaps persist. First, existing research has primarily evaluated block ciphers in ECB mode, where ciphertext blocks are independent. In contrast, CBC mode introduces dependency between consecutive blocks, reflecting the behavior of encryption systems in real-world deployments but posing additional challenges for feature-based classification [26]. Second, most prior studies rely on fixed or uniform ciphertext sizes, leaving open the question of how file size and entropy saturation influence classification performance. Third, comparative evaluations across multiple ML models under controlled and diverse data conditions remain scarce, limiting our understanding of which techniques generalize best across cipher families and data variations. Lastly, the majority of existing works focus on identifying specific algorithms or cipher modes rather than the underlying structural design, such as differentiating between SPN and Feistel architectures—an aspect that holds foundational significance in structural cryptanalysis. For instance Xia et al [33] presented the possibility of classifying block cipher structures—namely SPN and Feistel under ECB mode, which overlooks the complexities caused by chaining modes such as CBC. Hence, It is crucial to generalize such classification methods to CBC mode, where ciphertexts are chained and patterns less directly observable. Motivated by the above observations, this work aims at exploring the possibility of identifying block cipher structures, particularly SPN and Feistel design, from the features extracted in CBC-mode. To this end, we create a balanced dataset including sixteen benchmark block ciphers: five SPN-based and five Feistel-based. We extract both statistical and histogram-based features directly from the ciphertext and test several classical machine learning models, such as XGBoost, Random Forest (RF), Naive Bayes (NB), Logistic Regression (LR), and k-Nearest Neighbor (KNN). Our research is also extended to take into consideration the impact of ciphertext size and entropy on performance. In other words, we seek to discover the statistical fingerprints of these cryptographic architectures and what they can reveal about each structure, even in CBC mode.

This work introduces a data-driven framework for the structural identification of block ciphers, aimed at distinguishing between Substitution–Permutation Network (SPN) and Feistel architectures operating under Cipher Block Chaining (CBC) mode. The key contributions are summarized below:

• Unified Framework for Block Cipher Structure Identification: The proposed approach systematically identifies the structure of a block cipher based on statistics extracted directly from the ciphertext. By shifting the focus from algorithms to structure-level classification, the framework provides a more generalizable methodology for cryptographic analysis.

• Dataset Construction and Structural Feature Representation: This study constructs a balanced and diverse dataset designed to emulate real-world encryption scenarios, incorporating diverse data categories such as textual articles, social media, news content, network traffic, and programming code. The dataset is subsequently encrypted using sixteen symmetric block cipher algorithms, evenly divided between SPN and Feistel-based structures. To enable consistent and controlled comparison, each cipher is implemented under ECB and CBC modes of operation. Within this experimental framework, a comprehensive set of features is extracted, including statistical and histogram-based measures, to systematically assess properties such as randomness, diffusion depth, and block-wise dependence. A detailed statistical analysis was also performed, demonstrating that ciphertexts exhibit near ideal randomness while retaining weak finite-sample deviations.

• Model Evaluation and Performance Analysis: The framework employs five supervised learning models XG-Boost, Random Forest, Naïve Bayes, Logistic Regression, and k-Nearest Neighbor to evaluate structural classification performance. We take into account the impact of ciphertext size and entropy on performance. In our study, XGBoost and Random Forest demonstrate superior accuracy of approximately 94–95% for larger ciphertexts (i.e. 200 KB) and maintain stable results across varying file sizes which is validated using 5-fold cross validation, confirming its ability to capture subtle statistical distinctions between SPN and Feistel designs.

• Broader Impact and Analytical Insights: The results reveal that small statistical deviations are detectable by our employed ML models, even in case of high entropy CBC encryption. To thi end, we also performed feature importance and ablation studies in order to identify the mos discriminative characteristics of ciphertext data. These findings highlight the feasibility of automated structural cryptanalysis and provide an empirical basis for future work. Our work demonstrates that machine learning models can exploit weak statistical signals present in practical finite-length ciphertexts encrypted using advanced mode such as CBC, without violating established cryptographic assumptions. This work bridges the gap between theoretical cryptographic security and empirical data-driven analysis, opening new directions for research at the intersection of machine learning and statistical cryptanalysis.

In summary, our work adopts a feature-engineered and statistically grounded approach, using interpretable machine learning models such as XGBoost. By explicitly analysing feature importance and conducting ablation studies, this work provides transparent insights into the statistical properties of ciphertext. Furthermore, the experimental design incorporates comparison of encryption modes (ECB and CBC), diverse plaintext sources, and multiple keys, ensuring that the results reflect generalizable behaviour rather than dataset-specific artifacts. Our study explicitly considers full-strength block ciphers under standard and challenging modes of operation, focusing not on breaking security guarantees but on detecting weak statistical signals present in finite ciphertext samples.

The remainder of this paper is organized as follows. Section II details the methodology, including dataset generation, feature extraction, and classifier setup. Section III presents the experimental evaluation and discussion. Section IV concludes with analytical insights and directions for future research.

In the proposed approach, the plaintext is acquired, encrypted and then evaluated for binary class identification across two cipher structures SPN and Feistel. The plaintext data is obtained from multiple sources, ensuring a broad and representative sample of text data encompassing multiple categories of text including wikipedia articles, news feeds, tweets, network traffic and source codes in different programming languages. This diversity of text enables a more comprehensive analysis and enhances the robustness of the research findings. A detailed breakdown of these categories and their respective proportions is presented in Table 2. Details of dataset generated is given in:

• Wikipedia Articles: Wikipedia articles on diverse topics.

• Mixed News Articles: Real-world technology news articles. Dataset Structure consisted of title that is Headline of the news article and full text/content.

• Programming Codes: Programming codes in different languages from github repositories.

• Twitter Data: It contains 1,600,000 tweets extracted using the twitter api.

• Network Traffic Data: The data collected in a network section by performing packet captures at different hours, during morning and afternoon, over six days.

All plaintext data files are preprocessed to strip away formatting components including hyperlinks, citations, templates, and style markup, leaving plain textual content. The text data thus collected is divided into five categrories of 10Kb, 20Kb, 50Kb,100Kb and 200Kb respectively to assess the classification accuracy across increasing file size [26]. Total number of downloaded files belonging to all categories comprise of 534 files of plaintext. Each file was generated into 5 different file sizes and have average word length mentioned in Table 3.

After this the ciphertext is then meticulously constructed to reflect diverse plaintext by encrypting each file with sixteen different block ciphers listed in Table I and 10 keys, see Appendix B for details. Two distinct ciphertext data sets are constructed for ECB and CBC modes of operation of sixteen block ciphers resulting into a total of 534,000 ciphertext files. Table 2 shows dataset specifications.

Substitution-Permutation Network (SPN) and Feistel Structures are the two main construction design models for Bloc Ciphers. Both designs employ multiple rounds of operation of substitution, permutation and mixing keys but vary in design layout and mathematical foundations. Sixteen block ciphers were chosen for this research, eight based on an SPN architecture, and eight with Feistel construction. All chosen ciphers possess either 64-bit or 128-bit blocks and support key size of either 64 or 128 bits, which is perfectly compatible for constant comparison. 1) Substitution-Permutation Network (SPN): SPN structure works over the idea of utilizing substitution (non-linear) and permutation (linear diffusion) layers in sequence over several rounds. Every round generally includes the following steps:

• Substitution-Permutation Network (SPN): The SPN structure operates on the principle of applying substitution (non-linear) and permutation (linear diffusion) layers in sequence over multiple rounds [11]. Each round typically comprises the following steps:

  • XOR with round key

  • Substitution via S-boxes

  • Bitwise permutation

  Mathematically, one round of an SPN cipher is represented as (Equation (1)): 1X(i)=P( S(X(i−1)⊕K(i))){{\rm{X}}^{({\rm{i}})}} = {\rm{P}}\left( {{\rm{S}}\left( {{{\rm{X}}^{({\rm{i}} - 1)}} \oplus {{\rm{K}}^{({\rm{i}})}}} \right)} \right)

  where X ⁽ⁱ⁾ is the state after round i, K ⁽ⁱ⁾ is the round key, S(·) denotes the substitution layer, and P (·) denotes the permutation layer.

  This design promotes confusion and diffusion across the bit level structure of the message, and is employed in AES, ARIA, ARADI, SERPENT, SKINNY, KALYNA, RIJNDEAL and PRESENT ciphers.

• Feistel Network: The Feistel structure is a foundational design in symmetric block ciphers [11]. It begins by dividing the input plaintext block into two equal halves, typically denoted as L₀ (left) and R₀ (right). The encryption process proceeds through multiple rounds. In each round i, the new left half L_i becomes the previous right half R_i–1, and the new right half R_i is computed by applying a round function F to R_i–1 and the corresponding round key K_i, followed by an XOR with L_i–1 (Equation (2) and Equation (3)).2Li=Ri−1{{\rm{L}}_{\rm{i}}} = {{\rm{R}}_{{\rm{i}} - 1}} 3Ri=Li−1⊕ F(Ri−1, Ki){{\rm{R}}_{\rm{i}}} = {{\rm{L}}_{{\rm{i}} - 1}} \oplus {\rm{F}}\left( {{{\rm{R}}_{{\rm{i}} - 1}},{{\rm{K}}_{\rm{i}}}} \right)

  This architecture enables the same algorithm to be used for both encryption and decryption, with only the round key order reversed during decryption. A final swap of the last two halves is often applied to produce the ciphertext. This approach allows for non-invertible round functions while still ensuring that the overall structure is reversible. Widely adopted block ciphers such as DES, 3DES, Blowfish, KASUMI, TWINE, SIMON, SM4 and CAST are based on the Feistel structure due to its flexibility and robustness.

A feature extraction pipeline based on byte-level statistics, compression characteristics, and higher-order histogram features is employed to extract features from ciphertexts obtained from ECB and CBC modes of operation of encryption algorithms. Seven statistical features along with 24 bit histogram features were extracted and derived from each ciphertext file that define both byte-level and block-level behaviour across sixteen algorithms. Below are the seven features used; details are given in Appendix A.

• Entropy (Shannon Entropy)

• Mean Byte Value

• Standard Deviation of Byte Value

• Frequency of High Byte Values

• Frequency of Zero Bytes

• Most Frequent Byte

• Compression Ratio

• Ciphertext Intensity Distribution (24-bit Histogram)

To perform experiments, four different scenarios were created as shown in Table 4. For S1 an d S3 both initialization vector (IV) and Key were kept fixed in ECB and CBC mode. In S2 and S4, IV remained fixed and the plaintext was encrypted using 10 random Keys. These scenarios were employed to check performance of models under Key and modes variation.

This study used Five supervised ML models; XGBoost, Random Forest (RF), Naive Bayes (NB), Logistic Regression (LR), and K-nearest neighbours (KNN). Initially, the five supervised machine learning models were trained and tested using a split between train and test of 80:20, where 80% of the data has been used for the training set and 20% as a test set. The initialized random state is fixed for each model for consistency and reproducibility. A Multi Layer Perceptron (MLP) with 2 layers was also used as a proxy for Deep Learning (DL). A combination of statistical features and histogram metric, mentioned in the previous section were used as inputs for these models. These models were then k-cross validated with k=5 on accuracy, precision, recall and F1-score for both ECB and CBC modes. The overview of the complete methodology to train models is shown in Figure 1.

As mentioned in previous sections, experiments were performed under four different variations of scenarios (S1–S4) to maximize learning under practical conditions. Results for these four scenarios are discussed in depth below.

• S1: Mode CBC; Fixed IV, Fixed Key: For S1, maximum and minimum achieved accuracy was 93% (XGBoost) and 72% (NB) respectively. Performance of XGBoost and Random Forrest showed almost no variations with file size, while that of Naive Bayes also remained fixed at 72%, under varying file sizes. Logistic Regression and KNN and MLP showed considerable improvement in accuracy with increase in file size, see Table 5.

• S2: Mode CBC; Fixed IV, Random Key: For S2, XGBoost and RF remained indifferent to file size (same as in case of S1) with accuracy of 92-93%. Naive Bayes performance deviated slightly, however Logistic Regression and KNN followed the same trend of huge improvement with increase file size, refer to Table 6.

• S3: Mode ECB; Fixed IV, Fixed Key: Focusing on S3 (ECB mode), all of the models showed slight improvement with larger files, except NB which showed random behaviour in accuracy as file size increased to 200KB. XGBoost ultimately showed the best performance under largest files, for details see Table 7.

• S4: Mode ECB; Fixed IV, Random Key: For S4 (again ECB mode and with variation in keys), each model improved accuracy except Naive Bayes which underperformed for increased file sizes as shown in Table 8.

Above results show that for CBC mode, irrespective of variation in encryption parameters i.e. different keys, XGBoost and Random Forrest models out-performed other models while maintaining accuracy consistently for most of the cases. Logistics Regression, KNN and MLP showed huge improvement for larger file sizes. Naive Bayes on the other hand performed consistently low under same key while it showed random behaviour for random keys.

Considering ECB mode, for both fixed and random keys, all models showed improvement and maximum accuracy for larger file sizes. Here only Naive Bayes under-performed for larger files for the case of random keys.

It can be safely deduced that as the file size increased, the best performing models either maintained performance (CBC) or showed improvement (ECB). This is attributable to feature discriminablity arising from the statistical randomness exhibited by larger ciphertexts, especially in case of ECB mode.

Overall, the findings reported in this section verify that increasing ciphertext size results in increased classification accuracy in all models tested.

The confusion matrix illustrates a graphical representation of the classification behaviour between SPN and Feistel network architectures as shown in Figure 2 for S1 scenario and Figure 3 for S2 (S1 and S2 are both CBC modes). The provided confusion matrices illustrate the performance of five different models XGBoost, Random Forest (RF), Naive Bayes (NB), Logistic Regression (LR), and KNN based on actual versus predicted outcomes, tested exclusively on 200 KB files.

Figure. 2:

Confusion Matrices of all ML Models (a; XGBoost, b; RF, c; NB, d; LR, e;KNN) for structure classification under S1 for 200KB.

Figure 3:

Confusion Matrices of ML models (a; XGBoost, b; RF, c; NB, d; LR, e;KNN) for structure classification under S2 for 200KB.

The analysis reveals the following insights:

• XGBoost Performance: It can be concluded from the results that the XGBoost model has the better accuracy for classifying the instances into the respective classes, as well as a balanced low score on mis-classifications, see Figure 2b and 3b.

• RF Performance: RF gained almost equally good performance on classification, but was slightly biased in its cases of mis-classification of SPN as a Feistel structure in Figure 2a and 3a respectively.

• Logistic Regression Performance: Logistic Regression was not as good as XGboost. Confusion matrices are shown in Figure 2d and 3d for both scenarios and show mis-classification of both SPN and Feistel.

• KNN Performance:The accuracy of KNN also followed the same trend of LR, confusion matrix verifies this claim see Figure 2e and S2 3e respectively.

• Naïve Bayes Performance: The minimum accuracy was shown by Naive Bayes for both S1 and S2 as seen with its confusion matrices, where it almost failed to classify SPN correctly in CBC mode (it was no better than random guess), Figure 2c and 3c.

• Structure-Specific Trends: The predictions made by all models for the Feistel structure tend to work better than SPN predictions in case of CBC mode (both S1 and S2 scenarios).

An inter-feature correlation Matrix was computed to evaluate the redundancy and dependency among the extracted features as shown in Figure 4. It provides an in-depth analysis of the statistical relationships between the extracted ciphertexts features. The findings depict a noticeable interdependence between frequency of of high value bytes and the compression ratio and also a high correlation between (Shannon) entropy and histogram entropy.

Figure. 4:

Statistical relationships among ciphertext features

The mix of relationships in inter-feature correlation matrix suggests that these parameters collectively capture the lack of byte distribution uniformity and structural redundancy within the ciphertext. The correlation between the frequency of high-value bytes and compression ratio indicates that the encryption algorithms exhibit characteristic patterns in balancing high and low byte occurrences, potentially leaking their internal diffusion and substitution mechanisms, but one of these feature might be treated as redundant. Overall, this analysis emphasizes that most features are distinct in nature, and provide meaningful insight into the statistical structure and randomness properties of ciphertexts. Such observations can be instrumental in distinguishing between different cipher architectures and understanding their underlying data transformation characteristics.

Feature Distribution graph are also shown in Figure 5 which depicts how each feature showed variation across range of values for encrypted data with the exception of the most frequent bytes depicting uniform distribution of 256 byte values.

Figure. 5:

Feature Distribution over values

Test accuracy versus file size slightly increases in case of ECB (for all less MLP and Naive Bayes which show reverse trend). For high performing models (XGBoost / RF), Statistical features dominate in importance. As file size increases, statistical features become more stable predictor compared to histogram features. In case of CBC mode, statistical features again dominate for high performing ML models (XGBOOST and RF). Accuracy does not increase or decrease much with file size for these models due to CBC leading to better randomization. In comparison, low performing MLP and Naive Baye perform even worse for large file sizes. Overall histogram feature is not helpful and lead to lesser accuracy. Figure 6 shows performance of all models employing either statistical only or all features (i.e. statistical features with histograms).

Figure. 6:

Feature Analysis of Classifiers for CBC and ECB modes (a) Fixed key (b) Random Kyes

Test accuracy versus file size slightly increases in case of ECB (for all less MLP and NB which show reverse trend). This behaviour is more evident in ECB mode, where ciphertext blocks are independently processed and retain structural patterns. In contrast, the CBC mode results depict less accuracy as shown in Figure 7. This is due to the fact that in the CBC encryption mode, dependency between blocks is capable of making the structure within ciphertexts as random as possible, by eliminating distinctive traits. Yet ML models were able to distinguish slight variations in randomness, and achieved reasonable accuracy on account of structure classification. The proposed set of features, which emphasize statistical properties in ciphertext, perform in a consistent manner and are robust even for CBC encryption mode.

Figure. 7:

Performance Analysis of Classifiers for CBC and ECB modes (a) Fixed key (b) Random Key

Models accuracy is also evaluated against file sizes using 5-fold cross-validation where the median accuracy (Q50) across folds is reported as the central tendency. For every file size and model combination (except RF in S1, S2 scenario), there is minimal spread in results. This indicates low cross-validation variance, implying tight confidence intervals and high reliability of the median estimate, as shown in Figure 8. These plots further validates the earlier discussion in section 3.1.

Figure. 8:

Model Statistics and Performance Distribution (CBC) (a) Fixed key (b) Random Key

National Institute of Standards and Technology (NIST) SP 800-22 [34] test suite consists of 15 tests i.e. Frequency, Block Frequency, Cumulative Sums, Runs, Longest Run, Rank, FFT, Non Overlapping Template, Overlapping Template, Universal, Approximate Entropy, Random Excursions, Random Excursions Variant, Serial and Linear Complexity. It was applied on ciphertexts in both ECB and CBC mode to check randomness affects over the increasing file sizes. The official implementation downloaded from the NIST website was used and following parameters were set for analysis.

• Block Frequency Length - 128

• Non Overlapping Template Block Length - 9

• Overlapping Template Block Length - 9

• Approximate Entropy Block Length - 10

• Serial Block Length – 16

• Linear Complexity Sequence Length – 500

The results for all four scenarios are shown in Figures 9 and 10. Pass rate has been calculated against all file sizes considering α = 0.01. In ECB mode, pass rate is far less due to the lack of chaining methodology as expected, see Figure 10a and 10b. However, if we look at the results of CBC mode, slight degradation in pass rate is observed for some tests as the size of the file increases. In general 10KB cipher files obtained using CBC mode fail on more NIST features than larger 100 or 200KB cipher files, but still there is not statistically significant difference.

Figure. 9:

NIST Test Pass Rates (%) over File Size for CBC mode (S1, S2)

Figure. 10:

NIST Test Pass Rates (%) over File Size for ECB mode (S3, S4)

In fact, three features are consistently failing across all ciphers types and all file sizes. Hence, NIST tests only ”validate” that there are partial variations from true randomness that can be learnt by ML for classification across cipher (structure).

Recent studies have explored deep learning models such as convolutional neural networks and transformer architectures for cryptographic algorithm identification. These models can automatically learn complex patterns from raw ciphertext data and have demonstrated promising results in ECB-mode experiments. However, the present work focuses on classical machine learning techniques for several reasons:

• The dataset size is relatively moderate, making feature engineered approaches more stable.

• Classical models provide interpretable features.

• Computational requirements remain significantly lower than deep neural models.

• DL models are computationally expensive and add complexity with inference latency and hence not suitable in real-time detection scenarios.

Nevertheless, deep learning architectures represent an important direction for future research and may enable more sophisticated detection of structural patterns within ciphertext streams. This research primarily focused on exploiting statistical features and for comparison used one Deep Learning (DL) Model (i.e.two layer MLP) that also works with these features. DL model did not add value as indicated by the results shown in Table 9 for CBC and Table 10 for ECB. The statistical features already captured the discriminative information and XGBoost and RF models performed better. Adding baseline DL model that also operates on same features as other ML models, allowed comparison without fully committing to end- to-end deep learning on raw data.

In order to generate dataset and perform experiment, a compute node is used which included Intel Xenon E5-2674 v4 x 60 64 CPU Cores, 124GB RAM. Operating System installed is Linux Ubuntu 24.04.4 LTS, 537 Storage.