Research on a Lightweight Small Object Detection Method Based on Lite-RFB Modules

SSD (Single Shot MultiBox Detector) (as shown in Figure 1) achieves a high-efficiency balance between speed (59 FPS) and accuracy (74.3 mAP) by simultaneously performing object classification and localization in a single forward pass through multi-scale feature map prediction and a preset default box mechanism.

Figure 1.

SSD Network Architecture

SSD employs an enhanced VGG-16 network as its backbone feature extractor (truncated at the Conv5_3 layer), constructing a multi-scale detection framework through cascaded extra convolutional layers (Extra Feature Layers). Its architecture enables hierarchical prediction through feature maps at varying resolutions (e.g., Conv4_3 to Conv10_2). Each layer simultaneously generates class scores and bounding box offsets via 3×3 convolutions (output channels: 4×(Classes+4)). The network covers targets of varying sizes across input scales ranging from 160 to 1600 pixels using hierarchically designed default boxes. Shallow layers with high-resolution features (e.g., 38×38 Conv4_3) focus on small object detection, while deep layers with large receptive fields handle large targets. Non-Maximum Suppression (NMS) filters 8,732 initial detections per class, achieving 74.3mAP detection accuracy and 59FPS real-time performance, demonstrating the efficiency-accuracy balance of single-stage detectors.

The SSD framework achieves multi-scale object detection by constructing multi-level feature maps (conv4_3 to conv9_2). Its core lies in the scale generation formula (1) for default boxes: 1Sk=Smin+Smax−Sminm−1(k−1),k∈[1,m] {S_k} = {S_{\min }} + {{{S_{\max }} - {S_{\min }}} \over {m - 1}}(k - 1),k \in [1,m]

Where ^{S_min = 0.2}, ^{S_max =0.9} (VOC dataset), and ^m denotes the feature map layer number (typically 6). The shallow feature conv4_3 (38×38) covers 15% of small objects (<50×50 pixels) in VOC with high resolution, but its limited semantic expressiveness results in only 65.2% mAP for this category (VOC2007 test). The RFB module enhances the receptive field through multi-branch dilatedconvolutions: 2FRFB=σ(∑i=13Conv3×3(Fin,d=i)) {F_{RFB}} = \sigma \left( {\sum\limits_{i = 1}^3 {{{{\mathop{\rm Conv}\nolimits} }_{3 \times 3}}} \left( {{F_{in}},d = i} \right)} \right) (^σ represents ReLU activation), boosting small object mAP to 68.9% on VOC while increasing parameters by 40%.

The RFB (Receptive Field Block) network represents a substantial enhancement to the SSD (Single Shot MultiBox Detector) framework for object detection. Its core innovation lies in introducing a biomimetic receptive field module designed to overcome the limitations of standard convolutional layers in terms of receptive field size and diversity, thereby significantly improving the model's robustness to scale variations.

2) Cascading Receptive Field Blocks and Feature Enhancement

This is key to enhancing model performance. The RFB module employs a multi-branch architecture where each branch utilizes dilated convolutions with varying dilation rates. This design enables rapid expansion of the receptive field without increasing parameters or reducing spatial resolution, thereby capturing broader contextual information.

RFB_a: Directly processes shallow features Conv4-3. Its design likely emphasizes smaller dilation rates, introducing appropriate context while preserving detail to optimize small object detection.

RFB1, RFB2, RFB3: Process deep features in a cascading manner. Deep features inherently possess large receptive fields. By combining different dilation rates, the RFB modules further simulate multi-scale receptive fields. This enables the model to simultaneously perceive both local features and global context of objects, leading to more accurate localization and classification—especially for targets in complex scenes.

Feature Pyramid Construction and Multi-Scale Prediction

After feature enhancement by the RFB module (Figure 2), the model undergoes sub-sampling through a series of standard convolutional layers (BASIC-Conv1 to BASIC-Conv4), naturally constructing a feature pyramid. Ultimately, the model selects feature maps at multiple scales for input to the detection layer:

Figure 2.

RFB Model Diagram

High-resolution feature maps: Derived from the Conv4-3 layer enhanced by RFB_a, used for detecting small objects.

Medium-resolution feature maps: e.g., outputs from RFB2 or BASIC-Conv1, used for detecting medium-sized objects.

Low-resolution feature maps: Output from BASIC-Conv3 or BASIC-Conv4, featuring the largest receptive field, for detecting large objects in images.

MobileNet is a lightweight deep neural network specifically designed for mobile and embedded devices. It employs depthwise separable convolutions to reduce computational load and enhance computational efficiency.

Key features of MobileNet include:

Low computational complexity with significantly fewer parameters compared to models like VGG and ResNet.

Suitability for low-computational-power devices such as smartphones, embedded systems, and IoT devices.

Suitability for transfer learning and easy integration with other tasks like object detection and semantic segmentation.

MobileNet-SSD in Figure3. Is designed for lightweight efficiency, integrating the MobileNet backbone with the SSD detection framework. Its processing flow is: Inputs undergo multi-stage convolutions (Conv0 to Conv11, etc.) to extract features. The MobileNet section drastically reduces computational load via depthwise separable convolutions (e.g., Conv1), generating feature maps at different resolutions (150×150×32 to 19×19×512). The SSD detection branch utilizes these multi-scale feature maps (e.g., 1×1× (Classes+4)) to predict object categories and bounding boxes via convolutions. Finally, Non-Maximum Suppression (NMS) filters the detection results, achieving both efficiency and accuracy in mobile object detection scenarios.

Figure 3.

MobileNet-SSD Model Diagram

The main drawbacks of MobileNet-SSD are: weak detection capability for small objects, as the backbone network retains insufficient fine-grained features, leading to missed detections in identifying and locating small-sized targets; limited robustness in complex scenes, where insufficient feature discrimination results in high false positive rates when encountering dense objects or complex backgrounds; Despite lightweight optimization, its parameter count and computational load remain relatively high, resulting in suboptimal real-time performance on ultra-low-power embedded devices; and its network architecture exhibits low flexibility, leading to functional expansionsor hardware adaptations.

The proposed Lite-RFB module reconstructs the traditional RFB through structured compression. Its parameters consist of three components:

1) Depth-separable convolution decomposition

3 Params Lite-RFB =3×3×k2︸Deepthwise +3C×C︸Point wise +C×C︸Fusion {\rm{Params}}{{\rm{ }}_{{\rm{Lile - RFB }}}} = \underbrace {3 \times 3 \times {k^2}}_{{\rm{Depephinise }}} + \underbrace {3C \times C}_{{\rm{Point nise }}} + \underbrace {C \times C}_{{\rm{Fusion }}}

Where C denotes the number of channels, and k represents the convolution kernel size (default: k = 3 ).

2) Theoretical compression ratio calculation

4η=1−3k2+4C3Ck2 \eta = 1 - {{3{k^2} + 4C} \over {3C{k^2}}}

When ^C = 256, the theoretical compression ratio ^ŋ = 62.5% (parameter count reduced from 36.8M to 13.8M).

The dataset employed in this study is derived from the PASCAL VOC dataset in Figure 4, a benchmark collection widely used for object detection and image segmentation tasks in computer vision.

Figure 4.

Experimental Workflow Diagram

The PASCAL VOC data and associated challenges were initiated under the EU-funded PASCAL2 Network of Excellence, focused on Pattern Analysis, Statistical Modeling, and Computational Learning. The annual competitions ran from 2005 to 2012 and were discontinued after 2012. As a result, the PASCAL VOC dataset consists of a sequence of yearly releases from 2005 to 2012, denoted as VOC2005, VOC2006, and so forth. It should be noted that after VOC2007, the test sets were no longer made publicly available. This paper specifically uses the combined VOC2007 and VOC2012 dataset, which includes 16,551 training images and 4,952 test images spanning 20 object categories, as listed in Table 1. Among the annotated objects, approximately 15% are small targets with dimensions smaller than 50×50 pixels.

The hardware platform and software environment relied upon in this experiment are shown in the Table 2. below:

Through systematic comparative testing, this experiment clearly demonstrates the complete processing pathways for three object detection models: After inputting 300×300 test images, the original SSD employs standard VGG16 feature extraction. SSD+RFB achieves feature enhancement through multi-branch dilated convolutions, while Lite-RFB utilizes depthwise separable convolutions for lightweight processing. All three converge at a unified multi-scale detection head, ultimately outputting quantified results including mAP, FPS, and memory consumption. This flowchart visually reveals architectural differences among models: the RFB module enhances small object detection accuracy through feature fusion but increases computational overhead, while lightweight designs optimize resource consumption while maintaining accuracy. This provides a visual basis for model selection across diverse application scenarios.

Experimental results are evaluated across three metrics: mAP (mean average precision), FPS (frames per second), and Parameters. Relevant calculation formulas are as follows:

Let the total number of classes be ^C, the number of correctly classified pixels in class ⁱ be ^TP_i (true positives), and the total number of pixels in classes ⁱ be ^N_i (including both correctly and incorrectly classified pixels).The mPA formula is: 6mPA=1C∑i=1CTPiNi mPA = {1 \over C}\sum\nolimits_{i = 1}^C {{{T{P_i}} \over {{N_i}}}}

Here, a higher mPA (%) indicates more balanced classification capability across all categories.

Let the total time required to process ^N frames of images be ^T (in seconds). In real-time monitoring, autonomous driving, and similar scenarios, a minimum FPS of 30 is typically required to ensure smooth video playback. The FPS calculation formula is: 7FPS=NT FPS = {N \over T}

The experimental results clearly demonstrate the performance trade-offs resulting from different optimization approaches, providing clear guidance for model selection (as shown in Figure 5).

Figure 5.

Experimental Results Comparison

3) Lite-RFB Model: Balancing Performance and Efficiency

The Lite-RFB model strives for an optimal balance between accuracy, speed, and model size. Its mAP is approximately 72.8%, slightly lower than SSD+RFB but still superior to the original SSD model, achieving reasonable accuracy retention. More importantly, it achieves a high FPS of 58, the fastest inference speed, while maintaining a lightweight model size of only 18.9 million parameters. This efficiency stems from lightweight designs like depth-separable convolutions, which substantially reduce computational load and parameter count while maintaining robust performance. Consequently, this model is ideal for resource-constrained scenarios—such as mobile deployments and edge computing devices—where high-efficiency real-time processing is required without compromising detection effectiveness.

After a comprehensive training regimen, the proposed Lite-RFB SSD algorithm was subjected to rigorous evaluation. The test set data, comprising unseen images from the PASCAL VOC dataset, were fed into the trained network model to generate detection results in Figure 6. As illustrated in the corresponding figures, the model successfully identified and localized multiple objects within complex scenes, such as distinguishing between a 'sofa' and a 'chair' on a modern architectural terrace, accurately detecting a 'train' against a natural backdrop, and precisely delineating a 'person' and a 'dog' in a close-up interaction. These visual outcomes not only demonstrate the model's robust classification capabilities but also validate its precision in bounding box regression, confirming the practical efficacy of the Lite-RFB architecture in real-world object detection tasks.

Figure 6.

Test Set Training Figure