A Dual-Antenna Mobile Robot Orientation Estimation System Based on Rssi Fingerprinting and Machine Learning Techniques

Indoor location is a fundamental element of modern systems for intelligent buildings and enterprises, robotics, and the Internet of Things (loT). There is a significant amount of interest in this subject, as evidenced by the number of publications that have been published in recent years [1,2].

Satellite navigation systems, including GPS, GLONASS, Galileo, and BeiDou, do not apply to indoor location, despite their high accuracy in open spaces. This is primarily an effect of the limited propagation of radio signals through building structures, which effectively prevents location or significantly reduces its accuracy [3].

Due to their ubiquity, relatively low implementation costs, and the ability to adapt existing infrastructure, radio technologies are very popular in indoor environments. Examples of such technologies include Bluetooth, Wi-Fi, RFID, and UWB (Ultra-Wide Band) [3–7]. Although the accuracy of such solutions is still limited, continuous research and standards development are being conducted to make the solutions more usable.

The majority of indoor location systems focus on determining the position of an object in two-or three-dimensional space. Frequently, the orientation (angular position) of the object relative to a fixed reference frame is crucial information in addition to its position. This enables the determination of the object's pose in a 6-degree-of-freedom space. This is important in the context of industrial asset management systems, indoor navigation systems, and autonomous robots [8].

Commonly, magnetometers are used as compasses to find the orientation of an object. This facilitates the determination of the object's orientation in relation to the magnetic poles of the Earth. There are a lot of sources of soft iron and hard iron interference in the industrial environment, so this solution might not work there [9,10]. Additionally, these solutions necessitate periodic calibration [10].

The Angle of Arrival (AoA) technique is one of the radio techniques used to determine orientation. It is a component of the Bluetooth standard called Direction Finding [11]. This solution is based on the utilization of antenna arrays and the measurement of the phase difference of the received signal between individual antennas of the array, the distance between which is already known (Figure 1). This solution is susceptible to multipath phenomena, has a high technological cost, and is subject to limitations in accuracy under NLOS (Non-Line of Sight) conditions [12]. An alternative approach is the utilization of optical detectors, which enable the simultaneous precise positioning at 6.55 cm and orientation with an accuracy of 0.51° [13]. However, this solution necessitates a specialized infrastructure and is restricted to operation in Line of Sight (LOS) conditions. In addition, the literature provides angle of arrival estimations that are derived from RSSI (Received Signal Strength Indicator) measurements and a rotating antenna system. Finding signal minima in the spatial characteristics of the antennas is the foundation of the estimation process. The results showed that the mean absolute error was 4° when the test was done outside and 6° when it was done inside a building [14]. This solution necessitates a distinct mechanical structure and employs numerous radio receivers that must communicate with one another to process data from each one.

Fig. 1.

Graphical representation of the angle of arrival

This article discusses the proposed solution, which is a switchable dual-antenna system that estimates orientation using machine learning fingerprinting techniques and RSSI value differences.

Cardinal directions (north, south, east, and west) were classified based on compass readings. The classification process employed the k-NN (k-Nearest Neighbors), SVM (Support Vector Machines), Random Forest (RNF), and Multi-Layer Perceptron (MLP) models. Sensitivity, precision, F1-score, and Matthews Correlation Coefficient (MCC) were determined to evaluate the quality of the models.

In contrast to a multi-receiver system, the device design is simplified by the use of a switchable dual-antenna system. In comparison to a single-antenna system, it is feasible to acquire additional information by exploiting the anisotropy of the antennas' directional characteristics. This translates to the capacity to ascertain the direction, analogous to the AoA method [14,15].

The subsequent sections of the article are structured as follows: Section 2 discusses the technical aspects of the approach and the methodology required to comprehend the work. Section 3 describes the process of evaluating the quality of the developed methods and their underlying principles. In Section 4, the results and an analysis of them for each model are shown. The conclusions are presented in Section 5.

The classification models were characterized by metrics in order to assess the results obtained using the developed models. Metrics such as precision, sensitivity, and F1 score were calculated. The analyzed problem is multi-class, and in addition to metrics for individual classes, micro- and macro-averages were also calculated [19–22].

The precision of a given class C is expressed by formula (1), it defines the ratio of correctly classified TP observations to all observations classified as class C, both correctly and incorrectly [20]. 1PrecisionC=TPCTPC+FPC{\rm{ }}Precisio{n_C} = {{T{P_C}} \over {T{P_C} + F{P_C}}} where: TP_C – number of observations correctly classified into class C, FP_C – number of observations incorrectly classified as class C.

The micro-average precision expressed by equation (2) is the ratio of the sum of all (in each class) observations correctly classified to the sum of all (in each class) observations classified to a given class [20]. Since the sums refer to all classes, ultimately, it is the ratio of all values correctly classified to all classifications. 2Precision¯micro =∑k=1KTPk∑k=1K(TPk+FPk)=∑k=1KTPkGrand Total{\overline { Precision} _{micro }} = {{\sum\nolimits_{k = 1}^K {T{P_k}} } \over {\sum\nolimits_{k = 1}^K {\left( {T{P_k} + F{P_k}} \right)} }} = {{\sum\nolimits_{k = 1}^K {T{P_k}} } \over {{\rm{ }}Grand Total{\rm{ }}}}

The macro-average precision expressed by the relation (3) is the average of the precision of individual classes [20]. 3Precision¯macro =1K∑k=1KPrecisionk{\overline { Precision} _{macro }} = {1 \over K}\sum\nolimits_{k = 1}^K {Precisio{n_k}}

The sensitivity of class C described by equation (4) defines the ratio of correct classifications to the sum of all classifications classified into class C [20]. It measures what fraction of true positive TP observations was detected. 4RecallC=TPCTPC+FNC{\rm{ }}Recal{l_C} = {{T{P_C}} \over {T{P_C} + F{N_C}}} where: TP_C – number of observations correctly classified into class C, FN_C – number of observations belonging to class C but classified incorrectly.

Analogously to micro-average precision, micro-average sensitivity is expressed as the ratio of the sum of all (in each class) observations correctly classified to the sum of all (in each class) classifications classified into a given class. Therefore, the value of micro-average sensitivity and precision are given the same value (5) [20]. 5Recall¯micro = Precision¯micro{\overline { Recall} _{micro }} = {\overline { Precision} _{micro }}

The macro-average sensitivity expressed by equation (6) is the average sensitivity of each class [20]. 6Recall¯macro =1K∑k=1KRecallk{\overline { Recall} _{macro }} = {1 \over K}\sum\nolimits_{k = 1}^K {Recal{l_k}}

The F₁ score (7) is a metric that represents the harmonic mean of precision and sensitivity [20]. It can therefore be expressed for each class, as well as for the micro- and macro-average. This metric provides a compromise between precision and sensitivity, especially in cases where both metrics have extreme values. To obtain a high score, both metrics must be high. The F₁ score ranges from 0 to 1, where 0 indicates poor model performance and 1 indicates an ideal model. 7F1=21Precision+1Recall{F_1} = {2 \over {{1 \over {{\rm{ }}Precision{\rm{ }}}} + {1 \over {{\rm{ }}Recall{\rm{ }}}}}}

Because precision, sensitivity, and F₁ scores are not unique to unbalanced sets, do not account for true negatives (TN), and are calculated for individual classes, the Matthews correlation coefficient (MCC) expressed by equation (8) [20,21,23] will also be calculated. This coefficient takes into account the TP, TN, FP, and FN scores, and does not favor a dominant class, which translates into robustness to an unbalanced dataset. This coefficient always has a maximum value of 1, while the minimum value depends on the number and distribution of classes – this means that the minimum value will range from -1 to 0. 8MCC=c·s−∑kKpk·tk(s2−∑kKpk2)·(s2−∑kKtk2)MCC = {{c\cdots - \sum\nolimits_k^K {{p_k}\cdot{t_k}} } \over {\sqrt {\left( {{s^2} - \sum\nolimits_k^K {p_k^2} } \right)\cdot\left( {{s^2} - \sum\nolimits_k^K {t_k^2} } \right)} }} where: c – total number of samples correctly classified, s – total number of classifications, p_k – number of times class k has been predicted, t_k – number of times class k actually occurred.

Four classification algorithms were used to process the collected data: k-NN, SVM (SVC - Support Vector Classifier), Random Forest, and MLP neural network. Each model was trained using training data collected offline. For each model, it was necessary to select model hyperparameters. The hyperparameters were selected using a grid search method, which involves examining all combinations according to a predefined list of parameter variants [24]. The models were implemented in Python using the scikit-learn package.

In the case of the SVM and Random Forest classifiers, due to their sensitivity to unbalanced training data sets, a weight correction was applied, expressed by equation (9) [25,26]. 9Classweight =samplesnum classcount .classfreq{\rm{ }}Clas{s_{weight }} = {{{\rm{ }}sample{s_{num }}} \over {{\rm{ }}clas{s_{count }}.clas{s_{freq }}}} where: Class_weight – class weight, samples_num – the number of all class elements, class_Count – number of all classes, class_freq – class frequency.

This correction is inversely proportional to the frequency of occurrence of a given class in the training data set [27,28].

4.1.

k-NN

The following hyperparameters were selected for the k-NN model [29]:

• –

  weights = 'uniform' – weight function used during prediction, all points are treated equally,

• –

  neighbors = 15 – number of neighbors taken into account,

• –

  algorithm = ‘auto’ – automatic selection of the algorithm for searching for nearest neighbors.

Table 4 presents the precision, sensitivity, and F₁ score for each k-NN class. The best-detected direction was west, containing 39 samples, representing 35.1% of the test set. It achieved the highest sensitivity of 0.64, meaning that 64% of the true west orientations were correctly identified. A precision of 50% means that half of the cases were false positives. The F₁ score indicates that the relationship between sensitivity and precision is balanced.

Tab. 4.

Class	Recall	Precision	F1
‘East’	0.46	0.41	0.43
‘North’	0.34	0.43	0.38
‘South’	0.13	0.33	0.19
‘West’	0.64	0.50	0.56

The south class achieved the worst result, achieving a sensitivity of 13%. This class contained 15 samples, representing 13.5% of the test set. Precision, on the other hand, was 33%.

The north and east directions achieved moderate performance, with the east direction achieving slightly better results.

Table 5 presents the global metrics of the classifier. All macro-averages and micro-averages indicate moderate classifier performance. The classifier achieved an accuracy (micro score) of 0.45. The MCC coefficient reached a value of 0.23, indicating poor classifier performance.

Tab. 5.

Precision macro	Recall macro	F1 macro	Micro score	MCC
0.42	0.40	0.39	0.45	0.23

The confusion matrix presented in Figure 10 shows that all directions are most frequently confused with west. East and north were each classified as west nine times, while south was classified as west seven times. An asymmetry in errors between south and west is visible. South was classified as west seven times, while west was never classified as south.

Fig. 10.

Confusion matrix for the k-NN model

4.2.

SVM

Class corrections were applied to the SVM (SVC) model as described in section 4. The selected hyperparameters were [27]:

• –

  Kernel = ‘poly’ – type of kernel used for calculations,

• –

  Regularization = 10 – the value of parameter C affecting the objective function, it limits overfitting,

• –

  Gamma = 1 – kernel function factor,

• –

  Degree = 4 – degree of the polynomial of the ‘poly’ kernel.

Table 6 presents the metric results for the SVC model. The classifier best classified the west and east directions. Both classes achieved the F₁ score of 0.44. The east direction achieved a sensitivity of 0.46, meaning that 46% of actual cases for this orientation were correctly classified. For the west direction, this was 44%. Precision was also similar in both cases, reaching 0.42 for the east direction and 0.45 for the west direction, respectively. This means that in over 50% of cases, these directions were incorrectly classified. The south direction achieved the worst performance. Sensitivity was 0.27 and precision 0.22. For the north direction, sensitivity was 0.31 and precision 0.38.

Tab. 6.

Class	Recall	Precision	F1
‘East’	0.46	0.42	0.44
‘North’	0.31	0.38	0.34
‘South’	0.27	0.22	0.24
‘West’	0.44	0.45	0.44

Table 7 presents the global metrics of the classifier. All macro-averages and micro-averages indicate moderate performance of the classifier. The classifier achieved an accuracy of 0.39. Despite the moderate metric values, the MCC coefficient reaches a value of 0.16, indicating poor performance of the classifier.

Tab. 7.

Precision macro	Recall macro	F1 macro	Micro score	MCC
0.37	0.37	0.37	0.39	0.16

Analyzing the confusion matrix in Figure 11 demonstrates that the east and south directions were most frequently misclassified as west. In the north direction, the errors were evenly distributed among the classes. The direction of the west was most frequently misclassified as north.

Fig. 11.

Confusion matrix for the SVM model

4.3.

Random Forest

Similarly to the SVM model, weight correction was applied. The model parameters were determined as follows [28]:

• –

  max_depth = None – maximum tree depth, none means no limit,

• –

  min_samples_leaf = 4 – minimum required number of samples to constitute a leaf node,

• –

  min_samples_split =10 – minimum number of samples required to split an internal node,

• –

  n_estimators = 50 – number of trees in the forest.

Due to the operation of the algorithm, in order to achieve deterministic behavior of the model, the seed in a random number generator was also selected and was set to 3013327208.

Table 8 presents the metrics for the Random Forest model. The west direction had the best sensitivity of 0.62. This means that 62% of actual cases of this orientation were correctly classified. More than 50% of cases classified as west were incorrect, with precision of 0.47. The east direction is distinguished by a precision of 0.37 and a sensitivity of 0.50, resulting in the F₁ score of 0.42. This indicates moderate performance for this class. Sensitivity was inferior to precision in the north class. The south class received zero results, indicating that it was never properly identified.

Tab. 8.

Class	Recall	Precision	F1
‘East’	0.50	0.37	0.42
‘North’	0.24	0.39	0.30
‘South’	0.00	0.00	0.00
‘West’	0.62	0.47	0.53

The global metrics presented in Table 9 show moderate classifier performance. The low MCC value of 0.16 indicates poor classifier performance.

Tab. 9.

Precision macro	Recall macro	F1 macro	Micro score	MCC
0.31	0.34	0.31	0.41	0.16

The confusion matrix (Figure 12) indicates that the most frequently confused directions were west and east. South was not correctly classified even once. There is a visible asymmetry in errors between the south and other classes. South was classified as east 7 times, east as south 0 times, south was classified as west 7 times, and west as south 2 times.

Fig. 12.

Confusion matrix for the Random Forest model

4.4.

MLP Classifier

The hyperparameters of the MLP classifier were determined as follows [30]:

• –

  solver = 'adam' – optimizer,

• –

  activation = 'relu' – activation function,

• –

  hidden_layers_size = (150, 75) – structure of the neural network,

• –

  max_iter = 50000 – maximum number of epochs.

Again, due to the operation of the algorithm, in order to achieve deterministic behavior of the model, the seed in a random number generator was also selected and was set to 496680515.

Table 10 demonstrates the MLP classifier metrics. The highest sensitivity was achieved for the east direction, at 0.54. This means that 54% of actual cases for this orientation were correctly classified. The precision for the east direction was 0.38, resulting in a combined F₁ score of 0.45. The west direction achieved a sensitivity of 0.46 and a precision of 0.47, resulting in the F₁ score of 0.47, thus better than the east direction. The south direction achieved the worst metrics, with a sensitivity of 13%, a precision of 0.20, and the F₁ score of 0.16. The north direction had a sensitivity of 0.34, a precision of 0.40, and a combined F₁ score of 0.37.

Tab. 10.

Class	Recall	Precision	F1
‘East’	0.54	0.39	0.45
‘North’	0.34	0.40	0.37
‘South’	0.13	0.20	0.16
‘West’	0.46	0.47	0.47

The overall model metrics are presented in Table 11. The macro-average sensitivity and precision were 0.37, and the F₁ score was 0.36. These results indicate moderate classifier performance. A low MCC value indicates overall poor classifier performance.

Tab. 11.

Precision macro	Recall macro	F1 macro	Micro score	MCC
0.37	0.37	0.36	0.41	0.18

The confusion matrix of the MLP classifier shown in Figure 13 indicates that the most frequently confused directions were east and west. The south direction was correctly classified in two cases, most often as east. The north direction was classified eight times each as west and east.

Fig. 13.

Confusion matrix for the MLP Classifier model

Further research is planned to enhance the quality of estimation by expanding the switchable multi-antenna system. The implemented dual-antenna switched array is characterized by simple design, low cost, and easy scalability. Increasing the number of antennas enables greater exploitation of spatial diversity through information multiplication, while avoiding unnecessary system complexity or cost increases in implementation.

To fully utilize the benefits resulting from the switchable multiantenna array, work will also be conducted on increasing the sampling frequency of signal measurements. Furthermore, increased sampling will enable effective estimation of the orientation of a moving object. This is intended to be achieved through the use of newer Wi-Fi standards and a redesign of the measurement infrastructure. Preliminary simulation studies have shown that sampling of 15 samples per location allows for a reduction of MAE value by more than a factor of 3 [31].

The results presented in Table 12 show exemplary results of location estimation based on RSSI measurements using selected classifiers. For each classifier, mean absolute errors (MAE), root mean square errors (RMSE), and standard deviations (Stdev) were calculated. Analysis of the results shows that it is possible to achieve MAE at the level of even 1.46 m, RMSE of 1.82 m and Stdev of 1.06 m when using an MLP network. This means that location estimation with moderate accuracy is possible, which can be applied in sector-based localization. The values presented in the table are slightly higher than similar studies performed in publications [32] (MAE ≈ 0.50 m, RMSE ≈ 0.76 m) and [33] (RMSE ≈ 0.247 m). In well-controlled indoor environments (good anchor geometry, minimal multipath/obstruction) error on the order of 0.2 – 1 m is possible. In more realistic or cluttered indoor environments, errors of a few metres (2–4 m) are common.

The results presented in Table 13 show the outcomes obtained for estimation with the use of a single antenna. It is evident that the application of data from both antennas results in improved results in all considered cases. For the MLP Classifier, a MAE difference of 0.18 m, RMS of 0.17 m, and a Stdev reduction of 0.04 m were obtained.

Further work anticipates a comprehensive estimation of both orientation and location of the tracked object based on received signal strength measurements.

The outcomes of experimental research on a mobile robot’s orientation estimation are presented in this article. The objective of the investigation was to evaluate the feasibility of determining orientation in relation to the cardinal directions (N, W, S, E) by analyzing the difference in RSSI values in a dual-antenna tracker that employs the Wi-Fi standard and machine learning fingerprinting techniques.

The results of the studies that have been presented indicate that it is feasible to determine orientation and location with moderate precision. The results obtained are significantly influenced by the imbalance of the training set, which results in differentiating the performance of the model among individual classes. The quality of the developed models can be enhanced by giving training data collection extra attention to guarantee that the final set is as balanced as possible.

Simulation studies [31] have shown that the performance of classifiers can be enhanced by increasing the number of signal samples measured at a specific location. However, this is constrained by the speed of RSSI measurement and the number of hardware resources available. Furthermore, this approach has the potential to mitigate the effects of interfering phenomena.

Further research is planned to enhance the quality of estimation and facilitate dynamic object tracking by expanding the switchable multi-antenna system and implementing the most recent Wi-Fi standards. The use of a multi-antenna switchable array allows maintaining low implementation cost and maximizing the benefits of antenna spatial diversity – more antennas enable obtaining more data correlated with orientation and location.