Speech Processing Using Dynamic Micro-Block Optimization Based on Deep Learning

The core idea of CTC is to introduce a special blank symbol and to allow output labels to repeat, thereby aligning the input sequence with the target label sequence and performing transcription. The blank symbol indicates that the model deems there is no emission at the current time step. During post-processing, adjacent identical labels are collapsed into one, and all blank symbols are removed. This mechanism effectively tackles the length-mismatch problem between acoustic feature streams (speech frames) and decoded symbol streams (phoneme or character labels), especially when the input is much longer than the output [8]. Its working principle relies on a neural network that directly maps raw audio signals to a text label sequence, eliminating the need for manual forced alignment between acoustic units and text. This end-to-end characteristic enables the system to process inputs and outputs of arbitrary length without external segmentation, while the correspondence between them is discovered directly from the data during training; as a result, the approach has gained broad acceptance across automatic speech-recognition tasks. The following equation gives the definition of the CTC loss function.1LossCTC=−lnP(y∣x){{\mathop{\rm Loss}\nolimits} _{CTC}} = - \ln P(y\mid x) 2P(y∣x)=∑π∈Q(y)P(π∣x)P(y\mid x) = \sum\limits_{\pi \in Q(y)} P (\pi \mid x)

In the equation, x denotes the input sequence, y represents the target output label sequence, and Q(y) encompasses the complete set of valid CTC paths that can be collapsed to y.

As an end-to-end sequence modeling technique, while CTC provides an end-to-end solution for sequence alignment in speech recognition, it also introduces certain limitations, particularly in terms of output label sparsity and model robustness under noisy conditions [8]. Although CTC effectively addresses the length mismatch between input and output sequences by introducing blank symbols and allowing label repetitions, this strategy inherently lowers the information density of the output sequence, potentially compromising model accuracy and robustness [8]. Recently, the rise of self-supervised learning (SSL) has significantly advanced CTC based models [9]. SSL learns representations rich in semantic information without relying on explicit labels, enabling the model to better capture semantic content within speech sequences.

RNN is specifically designed for sequential data. Unlike feed-forward architectures (MLP or CNN), RNN introduce a temporal recurrence that memorizes historical information and reuses it for the current prediction. Typical deployment domains span language understanding, sequential data prediction, and spoken-content analysis [10]. The key idea is that when processing the t step, the model refers to its internal state computed at the immediately preceding time step, thus acquiring memory capability.

In an RNN, the instantaneous hidden activation ht is jointly conditioned on the present input xt and the state vector retained from the prior step [11]. the corresponding update expression is given below.3ht=g(Whht−1+Wxxt+bh){h_t} = g\left( {{W_h}{h_{t - 1}} + {W_x}{x_t} + {b_h}} \right)

In this expression, W represents the learnable coefficient matrix, b indicates the intercept term, g corresponds to the nonlinear active transformation.

The decoding layer is typically computed as: 4yt=f(Wyht+by){y_t} = f\left( {{W_y}{h_t} + {b_y}} \right)

Here, f denotes the nonlinear activation mapping (soft max or linear), which produces the final prediction yt at time step t.

RNN offer natural advantages for modeling temporal data such as speech. Nevertheless, the architecture faces several critical challenges. The foremost is the latent. risk of vanishing or exploding gradients during training: when the network becomes deep and input sequences are long, back-propagated error gradients may decay excessively or grow explosively[12]. Such gradient pathology severely impair the learning efficiency of deeper layers. Second, this gradient instability limits the RNN’s ability to capture and exploit long-range dependencies across distant time steps in very long sequences.

As a crucial refinement of recurrent neural network, LSTM incorporates memory cells governed by gated units, enabling more effective modelling of temporal sequences [13]. Fig. 1 illustrates the LSTM architecture, it modulates layer-to-layer information flow via learnable gates, allowing gradients to propagate over extended periods in a relatively stable manner [14]. The canonical LSTM employs three learnable gates: namely, input, forget, and output controllers [15]. Specifically, the input gate governs the degree to which fresh information is admitted into the memory cell at the current time step. The forget gate scales the previous cell state, deciding what fraction of long-term context is preserved or discarded. The output gate selects the segments of the updated cell state that are revealed as the hidden vector to downstream components.

Figure 1.

LSTM Architecture Diagram

The corresponding mathematical formulations-cell-state update equations, gate activation functions and the final hidden-state computation— are detailed in Equations (5)-(10).5gt=ρ(Mg[ ht−1,αt ]+βg){g_t} = \rho \left( {{M_g}\left[ {{h_{t - 1}},{\alpha _t}} \right] + {\beta _g}} \right) 6at=ρ(Ma[ ht−1,αt ]+βa){a_t} = \rho \left( {{M_a}\left[ {{h_{t - 1}},{\alpha _t}} \right] + {\beta _a}} \right) 7O˜=tanh(MO[ ht−1,αt ]+βo)\tilde O = \tanh \left( {{M_O}\left[ {{h_{t - 1}},{\alpha _t}} \right] + {\beta _o}} \right) 8Ot=gt□Ot−1+at□O˜{O_t} = {g_t}{O_{t - 1}} + {a_t}\tilde O 9bt=ρ(Mo[ ht−1,αt ]+bo){b_t} = \rho \left( {{M_o}\left[ {{h_{t - 1}},{\alpha _t}} \right] + {b_o}} \right) 10ht=bt□tanhOt{h_t} = {b_t}\tanh {O_t}

In the above, α_t is the incoming vector at temporal index t. h_t–1; and Ot–1 are the previous latent and memory vectors. Õ is the candidate. Gates a_t, b_t, g_t control inflow, outflow and forget; M and β are their weights and biases. ρ is sigmoid, ⊙ denotes element-wise product.

Gradient-based optimization is the cornerstone driving training of deep acoustic models, and its core still relies on back-propagation to transmit the output error layer-by-layer and update the weights. According to the criterion of "the number of samples used before each parameter update," gradient-based optimization is commonly classified into three variants: full-batch, purely stochastic, and mini-batch learning [16]. Although they share the same parameter iteration form, as shown in Equation (11).11θt+1=θt−ηm∑i=1m∇θL(xi,yi;θt){\theta _{t + 1}} = {\theta _t} - {\eta \over m}\sum\limits_{i = 1}^m {{\nabla _\theta }} L\left( {{x_i},{y_i};{\theta _t}} \right)

In the equation, θt stands for the vector of trainable weights, m equals the quantity of samples in the present mini-batch, η signifies the step-size coefficient, the pair (xi, yi) corresponds to the acoustic feature and its ground-truth transcription, L gives the per-sample objective value, and ∇ denotes the gradient of that objective relative to θt.

However, for speech tasks Batch GD must traverse the entire datasets before computing a single gradient update, which is time-consuming and incompatible with online incremental learning, whereas SGD updates parameters sample-by-sample but produces high-variance trajectories that destabilize CTC loss convergence [17]. Consequently, current speech recognition systems universally adopt Mini-batch GD as a compromise: gradients are estimated on mini-batches of 16–256 utterances and parameters are updated once per batch, markedly reducing memory footprint and improving parallel efficiency [18], as shown in equation (12).12θt+1=θt−η| Bt |∑i∈Bim∇θL(xi,yi;θt){\theta _{t + 1}} = {\theta _t} - {\eta \over {\left| {{B_t}} \right|}}\sum\limits_{i \in {B_i}}^m {{\nabla _\theta }} L\left( {{x_i},{y_i};{\theta _t}} \right)

In the equation, bt denotes the index set of samples in the current mini-batch with size m; the remaining symbols are consistent with those in the traditional formulation.

However, this strategy typically performs “random shuffling + sequential slicing” only once per epoch, so the composition of each mini-batch remains frozen throughout training. By implicitly assigning identical importance to every frame–label pair, it lacks the ability to adapt the sampling distribution on the fly as training progresses and cannot exploit speech-specific metadata such as gender, accent, or signal-to-noise ratio for fine-grained scheduling. This static limitation directly motivates the subsequent proposal of DMBO.

Dynamic Micro-block Optimization (DMBO) refines conventional mini-batch training by shrinking the sampling unit into micro-blocks (much smaller in capacity than mini-batch) and reselecting their members on-the-fly before each gradient computation. DMBO fuses three instantaneous signals to rank candidate samples. Sample complexity: normalized frame-length, SNR or accent-distance; Gradient informativeness: magnitude of the current loss or gradient; Meta-information: gender/accent labels used to encourage homogeneous or heterogeneous blocks.

The lightweight scheduling procedure is summarized in Equation (13) and Equation (14).

First, each candidate i receives an instant score: 13ρi=αSi+β‖ ∇Li ‖+γmi{\rho _i} = \alpha {S_i} + \beta \left\| {\nabla {L_i}} \right\| + \gamma {m_i}

Here, s_i denotes the complexity score, L_i the gradient norm, and m_i the meta-information weight; α + β + γ = 1 are the corresponding mixing coefficients. After ranking all candidates by ρ_i, the top-m samples are selected to form the micro-block.

Next, the network weights are updated with the standard mini-batch rule applied to the micro-block.

Traditional mini-batch gradient descent typically employs uniform random sampling, a strategy that readily clusters highly similar samples into the same batch and inadvertently amplifies individual instances. In speech processing, acoustic factors such as accent, gender, and age further intensify this imbalance; if these attributes are ignored, the gradient estimates become systematically biased and the model’s generalization performance deteriorates. To address this, DMBO replaces purely random selection with a criterion-driven sampling framework that explicitly controls the distribution of accent, gender, and other meta-attributes when forming each micro-block, ensuring that every parameter update is based on a subset that is both representative and information-rich.

The conventional micro-block sampling mechanism follows a completely random procedure: it first reshuffles the datasets and then indiscriminately selects data instances. This approach does not prioritize instances based on their informativeness or difficulty, which could lead to second-best utilization of computational resources and slower convergence.

Algorithm 1 clearly presents the code implementation of this basic sampling method, highlighting its simplicity and the lack of adaptive capacity to the specific needs of the training process.

During the initialization phase of the algorithm, the output container Microblock_samples is set to empty, and the entire datasets is then loaded into a dynamic array all_files. This array is immediately shuffled in place and randomized. Subsequently, iteration count is obtained by dividing the overall sample quantity by the micro-block size parameter. A loop is initiated from 1 to k: in each it eration, an empty cache unit batch_samples is instantiated first, b elements are taken from the front of the shuffled sequence to fill this unit, the selected elements are synchronously removed from the original array to maintain distribution consistency, and the current micro-block is inserted into output collection. When the loop terminates, algorithm returns constructed sequence of fine-grained units Microblock_samples.

In the process of acoustic data handling, raw speech samples are divided into two separate subsets based on the speaker's gender: audio from female speakers and audio from male speakers. Based on this binary grouping structure, this study designs two differentiated sampling schemes: a homogeneous gender classification construction mechanism and a heterogeneous gender integrated construction mechanism (as shown in Algorithms 2 and 3). Under the homogeneous mechanism framework, a single fine-grained unit contains audio data blocks of only one gender (either female or male), and adjacent units are assembled with alternating genders; whereas the heterogeneous mechanism requires each unit to contain a balanced combination of gendered speech instances. Additionally, a dual randomization process is adopted: first, unbiased sampling of unit samples is performed, followed by a secondary shuffle of samples within the unit. This procedure can effectively disrupt the model's statistical dependence on the sequence order of speech, preventing incorrect associative learning due to gender arrangement patterns.

In the homogeneous gender grouping strategy (Algorithm 2), an empty sample set `Microblock_samples` is initialized first. Then, speech metadata is parsed to separate female and male speaker identifiers, and corresponding audio files are retrieved to form independent path lists. After performing independent random shuffling on the two path collections, the algorithm calculates iteration count derived from total samples divided by preset unit size. This process assembles micro-units in an alternating gender pattern: let the current iteration count be variable `i`, when `i` is even, a fixed number of elements are taken from the front of the shuffled female path list and imported into a temporary container, while the selected entries are synchronously removed from the original list; if `i` is odd, the equivalent operation is performed on the male path. Finally, the constructed individual units are added to the target collection, and when the loop terminates, a layered container structure composed of alternating pure gender blocks is returned.

From the algorithmic framework design, the initial data-loading mechanism of Algorithm 3 remains highly consistent with Algorithm 2; the principal difference lies in the logic used to construct the sample sequence during iteration. Specifically, within the loop-operation interval, the model adopts a heterogeneous strategy to generate micro-block unit samples. In particular, at the sub-stage, a balanced male and female speaker split audio paths are integrated into the target set batch_samples. This set is simultaneously constrained so that the sample size of each group equals one-half of the constant batch_size, ensuring a balanced gender ratio.

This study selects the VCTK speech datasets [19] as the primary experimental data source. The datasets are preferred because it comprehensively covers multi-dimensional speaker attributes — gender differences, regional accent characteristics, and age distribution parameters are all recorded in structured meta-data files, perfectly matching the proposed micro-block sampling methodology. Fig. 4 and Fig. 5 quantitatively show the statistical frequency distribution of speakers across different gender and accent categories.

Figure 4.

Gender-based distribution of datasets samples

Figure 5.

Accent-based distribution of datasets samples

Owing to computational constraints, only 30 % of the original datasets is retained for analysis (approximately 14,000 utterances). Selective acquisition is achieved by finely tuning the core algorithmic parameter micro-block_examples, and the inclusion criteria for individual utterances strictly follow the current sampling logic.

The resulting data set amounts to about 14 hours of valid audio. An additional 1,048 utterances are randomly drawn from the global speaker pool to form a validation set, with the corresponding entries simultaneously removed from the base datasets to establish an unbiased evaluation mechanism for micro-block sampling efficacy.

Before deep-learning modelling begins, a standardized preprocess pipeline is executed: all audio signals are first decoded at a 16 kHz sampling rate, and 12-order MFCC are extracted; the analysis window is set to 15 ms with a 5 ms frame shift. Concurrently, text data are cleaned by completely removing punctuation marks, ensuring that the training targets closely mirror the actual spoken labels.

Utilizing PyTorch's computational infrastructure, this investigation implements LSTM neural networks as its foundational architecture. Structural configuration consists on five sequential processing strata, each incorporating 600-dimensional latent space projection nodes. Parameter refinement employs an adaptive moment estimation algorithm. Our comprehensive speech-to-text conversion framework is schematically presented in Figure 3. To conduct methodical comparisons of microscopic segment sampling techniques, constituent unit dimensions remain invariant at sixty-four acoustic samples.

The experimental environment runs Windows 10 (64-bit) on 32GB of RAM, with an RTX 3070 GPU and a 12th generation Intel Core i5-12600K CPU.

This investigation quantitatively assesses the efficacy divergence between gender-binding methodologies (single-gender cohorts versus mixed-gender compositions) when bench-marked against conventional randomized micro-batch assignment protocols. The experimental configuration employed a fixed batch dimension of 32 units and adjusted the optimizer step size to the 10⁻³ magnitude. Upon completing 800 training iterations, the deep learning architecture exhibited performance characteristics visualized in Fig. 6 and aggregated metrics presented in Table 1.

Figure 6.

Train loss (left), LER (right) for gender-based strategies

This investigation quantitatively assesses the efficacy divergence between gender-binding methodologies (single-gender cohorts versus mixed-gender compositions) when bench-marked against conventional randomized micro-batch assignment protocols. The experimental configuration employed a fixed batch dimension of 32 units and adjusted the optimizer step size to the 10⁻³ magnitude. Upon completing 800 training iterations, the deep learning architecture exhibited performance characteristics visualized in Fig. 6 and aggregated metrics presented in Table 1.

Illustrative trajectories in Fig. 6 delineate the dynamic evolution of training cost and LER, while Table 1 synthesizes terminal performance indices across both training and validation phases. Analytical outcomes reveal that single-gender configurations significantly surpassed mixed-gender arrangements and control protocols in convergence velocity and error suppression efficacy. Notably, this strategy achieved a 9-percentage-point relative reduction in LER across training and testing corpora—a discrepancy satisfying 94% confidence interval validation and demonstrating statistical robustness.

Mechanistically, constraining micro-clustering units to homogeneous vocalization categories enhances different feature extraction within vocal tract configurations, attributed to segregated training optimization for gender-specific phoneme production mechanisms. Conversely, mixed-gender paradigms exhibited negligible deviation from randomized allocation due to balanced gender-wise utterance distribution in the datasets.

By dynamically optimizing the representation of regional accents within the speech recognition pipeline, this study systematically evaluates the efficiency gap between dialect-homogeneous clustering and dialect-heterogeneous integration strategies relative to conventional random sampling. Core experimental settings include a batch size of 32 acoustic units, a fixed optimizer step size of 10-3, and a full training schedule of 800 epochs. Convergence behaviour is visualized in Fig. 7, while terminal performance metrics are summarized in Table 2.

Figure 7.

Train loss (left), LER (right) for accent-based strategies

Heterogeneous accent integration demonstrates a significant advantage across training stages, achieving an absolute reduction of 9.2 ± 0.4 % in label error rate. This performance leap originates from maintaining topological isomorphism between the dialect distribution inside each micro-batch and the phonetic variability of the corpus. Specifically, when the Euclidean distance between the batch-level dialect feature vector and the corpus distribution is minimized, the network effectively extracts cross-dialect covariance features, raising the completeness of the acoustic embedding space by 17.4% as verified by Riemannian manifold distance measurements.

The underlying mechanism maps dialectal acoustic features onto a Riemannian manifold and optimizes the covariance matrix to eliminate the distributional bias inherent in random sampling. Across consecutive training cycles, the heterogeneous architecture generates complementary phonemic representations and reduces the entropy of the model confusion matrix by 23 % compared with the homogeneous strategy. Our findings demonstrate that controlling dialect concomitant variable can upgrade recognition accuracy without increasing model parameters, providing a provable optimization path for multi-dialect speech processing.

Synthesis of Table 1 and 2 reveals: Architectures employing vocal homogeneity confinement integrated with cross-regional phonological diversity systematically surpass comparative methodologies in both developmental and evaluative dimensions. Underlying this advantage are two synergistic principles: Initially, aggregated processing of sex-specific acoustic cohorts facilitates meticulous delineation of correlative patterns linking spectral time domain signatures to linguistic transcriptions. Subsequently, per-iteration assimilation of dialectal fluctuations enables coherent internalization of articulatory discrepancies during neural computation, thus consolidating extrapolated resilience across linguistic domains. Ergo, embedding biometric metadata pertaining to vocal traits and geographical affinities within micro-scale sampling frameworks profoundly augments terminal model proficiency.

Table 3 conducts a cross-methodological appraisal of the proposed metadata-driven sampling paradigm. By curating baseline studies devoid of language models and possessing direct comparability, their LER metrics were incorporated as references. Critical findings reveal that: Specific configurations of our framework achieve superior phonetic decoding accuracy on testing sets relative to established baselines, with the vocal homogeneity combined with accent heterogeneity strategy exhibiting particularly significant breakthroughs—surpassing all documented optimal values in extant literature. Further computational scale expansion (e.g., augmenting parameter dimensions) is projected to yield additional LER reduction by 0.8 percentage points.