Introduction
Measurement of the symptoms present and the type of activities performed by the patients must be performed for accurate diagnosis of the disease or assessment of the severity of some diseases. Shortness of breath even during light daily physical activity has been reported in patients with chronic obstructive pulmonary disease (COPD) (Satoh et al. 2009). The timing of arrhythmias on an electrocardiogram (ECG) is correlated with the type of activity performed by patients while wearing the Holter ECG (Miao et al. 2015).
COPD is a chronic respiratory disease caused by smoking. It is the fourth leading cause of death worldwide (Lozano et al. 2012). Patients with COPD report feeling out of breath even after light daily physical activities, such as brushing teeth, eating meals, and taking the stairs (Satoh et al. 2009). Consequently, their physical activity level decreases (Kawagoshi et al. 2011). Low levels of physical activity is related to high readmission and mortality rates (Nguyen et al. 2014; Waschki et al. 2011). Thus, it is important to maintain the amount of physical activity for a better prognosis. The severity of COPD is measured using the 6-min walking test at present; however, this test is cumbersome for some patients with low respiratory capacity as it involves walking as far as possible in a straight line, covering a distance of 30 meters and returning to the starting position within a 6-min time frame (Holland et al. 2014). Moreover, the manner in which medical personnel communicate instructions to patients is not standardized, which influences the distance covered by the patient (Weir et al. 2013). Thus, the 6-min walking test is considered less objective. Therefore, the development of a new method that can automatically measure the types of physical activity of patients with COPD and the amount of each activity is necessary to objectively estimate the severity of COPD without imposing a burden on patients.
Holter ECG is a long-term dynamic ECG technology invented by the American physicist Norman J. Holter (Kennedy 2006). It is attached to the chest of the patient for 24 h to record the ECG continuously. Patients are instructed to perform their routine activities and fill in a record paper when they experience symptoms such as chest pain, palpitations, and shortness of breath. The time, symptoms, and activity triggering the symptoms are recorded (UNM Hospitals). Arrhythmias are observed when patients perform a certain activity; thus, it is important that the patients identify the type of activities they are performing. Physicians compare the ECG with the record paper for diagnosis. Patients must take notes by themselves in the Holter ECG method, and the diagnosis would be less accurate if they forget to do so. Therefore, the development of a new way to automatically identify the patients’ activities and record them is necessary such that patients can be relieved from the burden of recording, and the accuracy of the diagnosis can be improved.
An increasing number of studies have been conducted on the recognition of daily human activities in recent years, which are expected to be applied to the assessment and diagnosis of the beforementioned diseases. Acceleration data obtained using an accelerometer were used for machine learning to enable the automatic differentiation of daily activities. This approach is less burdensome and more objective than the conventional methods as it only requires patients to wear a device and lead their daily life. Acceleration data are automatically collected. Moreover, these accelerometers have been miniaturized in recent years; thus, they can be easily mounted on smartwatches and Holter ECG. Several studies have attempted to identify daily activities using an accelerometer (Kaneko, Yoshida & Yuda 2019; Ankita et al. 2021; Leotta, Fasciglione & Verri 2021). However, to the best of our knowledge, few studies have aimed to identify activities that can trigger shortness of breath in patients with COPD (Yamane et al. 2023) or listed in the Holter ECG recording paper. Triaxial accelerometers were attached to the dominant wrist and hip to identify COPD-related activities in our previous study. This method was developed further in the present study, and five nine-axis accelerometers were used simultaneously to further improve the recognition accuracy. In addition, some of the activities to be recognized were changed to ensure that more types of diseases could be targeted.
The goal of this study was to develop a simpler and more accurate method for the measurement of the severity and/or diagnosis of diseases wherein the type of activity of the patients is important, using a wearable device. Some activities were not recognized accurately in our previous study wherein triaxial accelerometers were used; thus, we assumed that attaching nine-axis accelerometers to five parts of the body would enable accurate recognition of most types of activities. However, it was important to investigate whether attachment to one part of the body would facilitate accurate recognition of activities, as attaching accelerometers to five parts of the body in daily life is not realistic. It was especially important to determine whether attachment to the non-dominant wrist (attachment position of the wristwatch type device) and chest (attachment position of the Holter ECG) would facilitate the accurate recognition of activities when the accelerometer was attached to only one part. We also aimed to determine the relationships between the attachment position of the accelerometer, the type of activity, and the accuracy of activity recognition to further improve accuracy. In short, two objectives were set in this study. The first objective was to assess the accuracy of activity recognition based on the attachment position of the accelerometer. The second objective was to evaluate the accuracy of activity recognition based on the type of activity.
Methods
Study population
Thirty healthy participants (13 males and 17 females) provided written informed consent to participate in the study. The mean age of the participants was 21.0 ± 0.87 years (range: 19–23 years). Twenty-nine participants among the 30 participants were right-handed, whereas the remaining one participant was left-handed. This study was conducted in accordance with the principles of the Declaration of Helsinki and was approved by the Ethics Committee of Okayama University (approval number: R2203-001, approved on April 14, 2022).
Device
The ActiGraph GT9X Link (ActiGraph LLC, Pensacola, FL, USA) is an activity monitor equipped with a three-axis accelerometer and complete inertial measurement unit (IMU). The IMU houses a secondary tri-axis accelerometer, tri-axis gyroscope, tri-axis magnetometer, and temperature sensor that measure the acceleration, rotational velocity, magnetic flux, and temperature, respectively. The IMU can provide information regarding rotation and position for advanced applications.
Procedure
Figure 1 illustrates the experimental procedure and data analysis.
Figure 1
Experimental procedure and data analysis.
The following five ActiGraph GT9X Link were attached on to the participants (Figure 2):
Figure 2
The five attachment positions of ActiGraph GT9X Link. i) On the dominant wrist: attached with the help of the link wristband. ii) On the non-dominant wrist: attached with the help of the link wristband. iii) On the chest: attached with the aid of a pouch and belt. iv) On the hip on the contralateral side of the dominant wrist: attached with the aid of a link belt clip. v) On the thigh on the contralateral side of the dominant wrist: attached with the aid of a pouch and belt.
One ActiGraph GT9X Link on the dominant wrist with the help of the link wristband.
One ActiGraph GT9X Link on the nondominant wrist with the help of the link wristband.
One ActiGraph GT9X Link on the chest with the aid of a pouch and belt.
One ActiGraph GT9X Link on the hip on the opposite side of the dominant wrist with the aid of a link belt clip.
One ActiGraph GT9X Link on the thigh on the opposite side of the dominant wrist with the aid of a pouch and belt.
The IMU recordings were performed at a sampling frequency of 100 Hz for all positions. The idle sleep mode was disabled. The participants performed nine activities in the following order: lying in the supine position, standing, sitting, taking a meal, brushing teeth, using the restroom, walking, ascending/descending the stairs, and running. Activities performed between taking a meal and brushing teeth were categorized as “other movements.” The activities from lying in the supine position to brushing teeth were performed in one room, whereas those after brushing teeth were performed in the same building. Running activities were performed in a gymnasium. All activities were performed for 2 min and self-paced wherever applicable. The participants returned to the first room after the completion of all activities, and the accelerometers were removed. The participants were provided with a JPY 1,000 prepaid card as a reward. All measurements, except for those recorded while using the restroom, were obtained simultaneously for two participants. Measurements were acquired individually while using the restroom.
Activity types
Nine activities were selected for this study from two categories of activities: basic activities (lying in the supine position, standing, sitting, walking, ascending/descending the stairs, and running) that were identified by a tri-axis accelerometer in previous studies (Sumikawa et al. 2018; Trost, Zheng & Wong 2014) and daily living activities during which patients with COPD reported shortness of breath in the 24-item questionnaire (Eakin et al. 1998) or those listed in the Holter ECG recording paper (taking a meal, brushing teeth, and using the restroom).
(i) Lying in the supine position
The participants took off their shoes and stood on a sheet prepared beside the bed. They laid down on the bed on their backs and maintained a resting posture for 2 min for measurement when the experimenter provided a signal. Subsequently, they stood on the original sheet when the experimenter provided another signal. The participants then put on their shoes. They were instructed to remain in the same resting position during the measurement. Movements related to changing their position from standing to supine and from supine to standing were not used for data analysis.
(ii) Standing
The participants were seated on a chair. They were instructed to stand up when the experimenter provided a signal and maintain a resting posture for 2 min. Subsequently, the participants sat on the chair when the experimenter provided a signal. They were instructed to remain in the same resting position during the measurement. Movements related to changing their position from sitting to standing and from standing to sitting were not used for data analysis.
(iii) Sitting
The participants were instructed to stand in front of a chair and sit on the chair when the experimenter asked them and maintain a resting position for 2 min. They stood up when the experimenter provided another signal. The participants were instructed to remain in the same resting position during the measurement. Movements related to changing their position from standing to sitting and from sitting to standing were not used for data analysis.
(iv) Taking a meal
The participants were instructed to pretend to eat a meal using empty dishes and chopsticks while looking at a picture of a meal in a sitting position. They performed the simulated behavior when the experimenter provided a signal and continued for 2 min. The task was completed when the researcher provided another signal. The participants were instructed to simulate behaviors such as chewing, changing dishes, and drinking. The participants were not provided any specific instruction regarding whether to say, “Thank you for the food,” with their hands clasped together before and after eating. When two participants underwent this measurement simultaneously, they were placed in a position such that their backs were facing each other to avoid distraction.
(v) Brushing teeth
The participants were instructed to brush their teeth at the tap. They prepared their toothbrushes by placing toothpaste on them and held the toothbrushes in their mouth when the preparation was complete. The participants began brushing their teeth when the experimenter provided a signal. The activity was measured for 2 min. They continued to brush their teeth until the researcher provided an additional signal. They gargled and cleaned up after the measurement was completed. The participants were instructed to bring their toothbrushes and were free to use their toothpaste. As some participants may have felt that the tap water in the room was not clean, bottled water and paper cups were provided for gargling. The participants were instructed to hold the toothbrush with their dominant hand at all times during the measurements.
(vi) Using the restroom
The experimenter led the participants to the entrance of the restroom. The participant entered the restroom at the signal and performed simulated actions, such as doing their business, wiping with toilet paper, and washing hands. The participants were given a stopwatch to wear around their necks and were instructed to take at least two minutes from entering the restroom to coming out. They were instructed to shift and use Western-style toilets. As the accelerometer was attached to the thigh, the participants performed the simulated movements with their pants on. When two participants performed the task simultaneously, one participant used the restroom, and the other waited in the room where they had brushed their teeth. When the first participant finished using the restroom, they returned to their original room, and the second participant performed the activity as the first participant.
(vii) Walking
The experiment was conducted in a corridor that allowed participants to take one lap around the courtyard and return to their original location. The participants started walking when the experimenter provided a signal. They changed direction and made another circuit in the opposite direction after circling the hallway and returning to their original location such that both rightward and leftward movements would be performed. This process was repeated until the walking time reached 2 min. They walked to the end of the lap on being told that it was their last lap. The participants were instructed to walk at a constant speed that was comfortable for them. Most participants completed three laps. When two participants walked simultaneously, one participant walked counterclockwise in the first lap, and the other participant walked clockwise in the first lap. They started walking at the same time but ended at different times depending on the walking speed of each participant. If only one participant was performing the task, the initial direction of walking was determined such that the total number of participants who walked counterclockwise on the first lap was the same as that of the participants who walked clockwise on the first lap.
(viii) Ascending/descending the stairs
This measurement was conducted in a six-story building, with the starting point in front of the stairs on the third floor. The participants were instructed to ascend/descend the stairs when the experimenter provided a signal. They ascended and descended the stairs for 2 min and continued until the end of the lap when they were told that this was the last round trip. The participants were instructed to walk at a constant speed that was comfortable for them. Most participants completed four round trips. When two participants performed the task simultaneously, one participant ascended/descended the stairs between the third and fourth floors starting from the top, and the other participant ascended/ descended the stairs between the third and second floors starting from the bottom. They started ascending/descending the stairs at the same time and finished at different times depending on their walking speed. When only one participant was performing the task, the initial direction of ascending/descending was determined such that the total number of participants who walked between the third and fourth floors was the same as that of the participants who ascended/descended the stairs between the third and second floors. The staircase had one landing between the third and fourth floors and between the third and second floors.
(ix) Running
The measurement was performed at a martial arts hall in a space with tatami mats. The participants ran in a counterclockwise circular motion. They started running when the experimenter provided a signal. They ran for 2 min and continued running until the experimenter provided another signal. They were instructed to run at a constant speed that was comfortable for them. The participants were allowed to take off their masks but were instructed not to speak without their masks to prevent the spread of the coronavirus infection. The participants were also free to remove their socks and run barefoot.
Data processing and feature extraction
Raw data were extracted from the devices using ActiGraph software (ActiLife Version 6.13.4). The data were saved in raw format as .gt3x files and converted to .csv format for data processing. The .csv file contained 11 columns: Timestamp, Accelerometer X, Accelerometer Y, Accelerometer Z, Temperature, Gyroscope X, Gyroscope Y, Gyroscope Z, Magnetometer X, Magnetometer Y, and Magnetometer Z. The data, except for timestamp and temperature, were segmented into sequences of 10-s non-overlap-time-domain or 10-s non-overlap-frequency-domain features (Liu, Gao & Freedson 2012), as shown below: mean, standard deviation, variance, maximum, minimum, root mean square, signal magnitude area, correlation between the axes, entropy, energy, kurtosis, skewness, median, interquartile range, and autoregressive. There was one measure per axis for the mean, standard deviation, variance, maximum, minimum, root mean square, entropy, energy, kurtosis, skewness, median, and interquartile range, whereas there were four measures per axis for autoregression. The correlation between axes was computed with a combination of x and y, x and z, y and z. The signal magnitude area was computed from the three axes. In total, 156 features were used for each 10-s window during model training and testing.
Model training and testing
Leave-one-subject-out (LOSO) cross-validation was used for model training and testing (Esterman et al. 2010). The dataset of 30 participants was split into two non-overlapping subsets: a training subset comprising the data of 29 participants and a test subset comprising the data of one participant. Label-encoding, an encoding technique that converts labels into numerical form to handle categorical variables, was used. The random forest classifier model (Breiman 2001) was trained using the training data. Random forest (RF) is a supervised learning algorithm wherein multiple decision trees are built and merged to obtain more accurate and stable predictions. This algorithm can be used to solve classification and regression problems (Donges 2023). The trained model was evaluated using the test data after training. The training and evaluation procedures were repeated 30 times while changing the combinations of the training and test subsets. Scikit-learn (Pedregosa et al. 2011), an open-source data analysis library for the Python programming language, was used for model training and testing. Python version 3.6.8 and scikit-learn version 0.23.2 were used in this study. The following scikit-learn functions were used: train_test_split, accuracy_score, random forest classifier, classification_report, and confusion_matrix. The hyperparameters include the following: n_estimators = 2,500, criterion = “gini,” max_depth = 30, min_samples_split = 2, min_samples_leaf = 1, and random_state = 42.
Model evaluation
The following indices were used to evaluate the performance of the classifier.
Precision: Precision is defined as the ratio of predicted positive cases that were actually correct. Precision was determined using the following equation: Precision = (True positive)/(True positive + False positive).
Recall: Recall is defined as the ratio of actual positive cases that were correctly classified. Recall was determined using the following equation: Recall = (True positive)/(True positive + False negative).
F-value: The F-value is defined as the harmonic mean of precision and recall. The F-value was determined using the following equation: F = 2 • (Precision • Recall)/(Precision + Recall).
F-value varies from 0 to 1 (Bull et al. 2018). An activity with an F-value of ≥0.7 was considered recognizable. Confusion matrices were constructed to summarize the prediction results of activity classification.
The participants wore a GT9X Link on their dominant wrist, nondominant wrist, chest, hip, and thigh. The following six methods were used to evaluate the performance of activity classification: (1) using data from the dominant wrist position for prediction (dominant wrist classifier), (2) using data from the non-dominant wrist position for prediction (non-dominant wrist classifier), (3) using data from the chest position for prediction (chest classifier), (4) using data from the hip position for prediction (hip classifier), (5) using data from the thigh position for prediction (thigh classifier), and (6) using data from all five positions (all classifiers).
Results
Acceleration data measured using the nine-axis accelerometers were used to examine the recognition accuracy of each activity. An activity was considered recognizable when the F-value was ≥0.7.
The F-values were >0.7 for all activities in all classifiers (using data from all five positions), dominant wrist, non-dominant wrist, and chest classifiers, whereas the F-values of the hip and thigh classifiers were <0.7 for some activities (Table 1). The F-values for lying in the supine position, walking, and running were >0.9 for all or five of the six classifiers; however, the F-values for sitting, taking a meal, using the restroom, and other movements were <0.9 for all or five of the six classifiers. Thus, lying in a supine position, walking, and running were considered well-identified. Sitting, taking a meal, using the restroom, and other movements were not well identified.
Table 1
Evaluation results. Emboldened values represent F-values of 0.9 or higher.
| PRECISION | RECALL | F-VALUE | ||
|---|---|---|---|---|
| Lying in the supine position | d-wrist | 0.9668 | 0.9791 | 0.9689 |
| nd-wrist | 0.9446 | 0.9737 | 0.9552 | |
| Chest | 0.9956 | 1.0000 | 0.9976 | |
| Hip | 0.9810 | 1.0000 | 0.9877 | |
| Thigh | 0.7483 | 0.7925 | 0.7445 | |
| All positions | 1.0000 | 1.0000 | 1.0000 | |
| Standing | d-wrist | 0.9548 | 0.9897 | 0.9662 |
| nd-wrist | 0.9350 | 0.9459 | 0.9306 | |
| Chest | 0.8528 | 0.8675 | 0.8094 | |
| Hip | 0.7305 | 0.7256 | 0.6920 | |
| Thigh | 0.9220 | 0.8741 | 0.8692 | |
| All positions | 0.9585 | 0.9949 | 0.9714 | |
| Sitting | d-wrist | 0.9137 | 0.8699 | 0.8728 |
| nd-wrist | 0.9171 | 0.8690 | 0.8769 | |
| Chest | 0.8119 | 0.7667 | 0.7591 | |
| Hip | 0.7232 | 0.7897 | 0.7336 | |
| Thigh | 0.7028 | 0.7103 | 0.6749 | |
| All positions | 0.9592 | 0.9615 | 0.9602 | |
| Taking a meal | d-wrist | 0.8616 | 0.8677 | 0.8584 |
| nd-wrist | 0.8650 | 0.9068 | 0.8712 | |
| Chest | 0.8442 | 0.8161 | 0.8187 | |
| Hip | 0.7119 | 0.6971 | 0.6669 | |
| Thigh | 0.8448 | 0.8340 | 0.8213 | |
| All positions | 0.9555 | 0.9393 | 0.9363 | |
| Brushing teeth | d-wrist | 0.9214 | 0.9051 | 0.9112 |
| nd-wrist | 0.8433 | 0.7979 | 0.8104 | |
| Chest | 0.9386 | 0.9171 | 0.9224 | |
| Hip | 0.8180 | 0.7944 | 0.7775 | |
| Thigh | 0.7347 | 0.7863 | 0.7457 | |
| All positions | 0.9471 | 0.9333 | 0.9372 | |
| Using the restroom | d-wrist | 0.7916 | 0.8806 | 0.8228 |
| nd-wrist | 0.7904 | 0.7634 | 0.7588 | |
| Chest | 0.8567 | 0.8569 | 0.8420 | |
| Hip | 0.8094 | 0.7224 | 0.7223 | |
| Thigh | 0.8620 | 0.8057 | 0.8146 | |
| All positions | 0.9529 | 0.9245 | 0.9309 | |
| Walking | d-wrist | 0.8964 | 0.9239 | 0.9064 |
| nd-wrist | 0.9295 | 0.9010 | 0.9039 | |
| Chest | 0.9761 | 0.9593 | 0.9661 | |
| Hip | 0.9328 | 0.9249 | 0.9205 | |
| Thigh | 0.9047 | 0.8652 | 0.8769 | |
| All positions | 0.9860 | 0.9636 | 0.9737 | |
| Ascending/descending the stairs | d-wrist | 0.9203 | 0.8957 | 0.8983 |
| nd-wrist | 0.8979 | 0.8920 | 0.8890 | |
| Chest | 0.9532 | 0.9785 | 0.9641 | |
| Hip | 0.9382 | 0.9294 | 0.9310 | |
| Thigh | 0.8855 | 0.9353 | 0.8990 | |
| All positions | 0.9632 | 0.9901 | 0.9756 | |
| Running | d-wrist | 1.0000 | 0.9974 | 0.9987 |
| nd-wrist | 0.9974 | 0.9897 | 0.9926 | |
| Chest | 1.0000 | 0.9889 | 0.9933 | |
| Hip | 0.9976 | 0.9923 | 0.9944 | |
| Thigh | 0.9797 | 0.9949 | 0.9855 | |
| All positions | 1.0000 | 1.0000 | 1.0000 | |
| Other movements | d-wrist | 0.7806 | 0.6952 | 0.7224 |
| nd-wrist | 0.7316 | 0.7423 | 0.7192 | |
| Chest | 0.7917 | 0.7777 | 0.7739 | |
| Hip | 0.7929 | 0.7600 | 0.7336 | |
| Thigh | 0.7719 | 0.7274 | 0.7280 | |
| All positions | 0.8777 | 0.8837 | 0.8704 |
Figure 3 presents the confusion matrices for the activity-type predictions provided by the non-dominant wrist, chest, hip, and thigh classifiers. As shown on the diagonal of the confusion matrix, the majority of each activity was classified correctly. However, some areas were misclassified.
Figure 3
Confusion matrices comparing predicted and correct activities. The rows correspond to the actual activities, whereas the columns correspond to the activities predicted by the activity classifier. (a) Non-dominant wrist classifier. The light blue underline indicates misclassification between taking a meal and other movements. The orange underline indicates misclassification between using the restroom and other movements. The red underline indicates misclassification between walking and ascending/descending the stairs. The purple underline indicates misclassification among brushing teeth, standing, and taking a meal. (b) Chest classifier. The light blue underline indicates misclassification between taking a meal and using the restroom. The orange underline indicates misclassification between using the restroom and other movements. The green underline indicates misclassification between standing and sitting. (c) Hip classifier. The light blue underline indicates misclassification among taking a meal, sitting, and using the restroom. The green underline indicates misclassification between standing and sitting. (d) Thigh classifier. The blue underline indicates misclassification between lying in the supine position and sitting.
Examples of raw acceleration, rotational velocity, and magnetic field data are presented in Figures 4, 5, and 6. The participants’ activities were recorded during the experiments and synchronized with the accelerometer output (marked as 0–9 in Figures 4, 5, and 6). Different types of activities and positions of the accelerometer resulted in different fluctuation patterns of acceleration, rotational velocity, and magnetic field strength values. The results of the F-values, confusion matrices, and raw acceleration data suggest that the recognition of the activities examined in this study is feasible when data from all five positions or only the dominant wrist, non-dominant wrist, or chest data are used.
Figure 4
Example of waveform data along the x axis. The red and green lines correspond to the data of the non-dominant wrist and chest data, respectively. The numbers represent the activities: 0, lying in a supine position; 1, standing; 2, sitting; 3, taking a meal; 9, other movements; 4, brushing teeth; 5, using the restroom; 6, walking; 7, ascending/descending the stairs; 8, running. (a) acceleration along the x-axis. (b) angular velocity along the x-axis. (c) magnetic field strength along the x-axis.
Figure 5
Example of waveform data along the y axis. The red and green lines correspond to the data of the non-dominant wrist and chest data, respectively. The numbers represent the activities: 0, lying in a supine position; 1, standing; 2, sitting; 3, taking a meal; 9, other movements; 4, brushing teeth; 5, using the restroom; 6, walking; 7, ascending/descending the stairs; 8, running. (a) acceleration along the y-axis. (b) angular velocity along the y-axis. (c) magnetic field strength along the y-axis.
Figure 6
Example of waveform data along the z axis. The red and green lines correspond to the data of the non-dominant wrist and chest data, respectively. The numbers represent the activities: 0, lying in a supine position; 1, standing; 2, sitting; 3, taking a meal; 9, other movements; 4, brushing teeth; 5, using the restroom; 6, walking; 7, ascending/descending the stairs; 8, running. (a) acceleration along the z-axis. (b) angular velocity along the z-axis. (c) magnetic field strength along the z-axis.
Discussion
Our activities are characterized by the movement of various parts of the body, which move depending on the type of activity. These characteristics are reflected in the waveforms of acceleration, rotational velocity, and magnetic field, which can be measured using a nine-axis accelerometer. The specificity or similarity of the waveform movements can improve or deteriorate the accuracy of activity recognition. Comparison of the F-values for each position revealed that the dominant wrist, non-dominant wrist, and chest showed high or moderate accuracy for all movements. Attachment to all five parts resulted in accurate recognition of almost all activities with F-values of >0.9. However, the F-values were low for the hip and thigh for some activities. As we are particularly interested in the non-dominant wrist and chest, which are the attachment positions of the wristwatch type device and Holter ECG, respectively, this section mainly focuses on these sites.
Based on the F-values, lying in a supine position, standing, walking, and running were well identified when the accelerometer was attached to the non-dominant wrist. This may be because lying in the supine position and standing were stationary movements, whereas walking and running were regular movements. The wrist remained stationary in the supine position and standing positions, and the acceleration, rotational velocity, and magnetic field waveforms exhibited constant values. In contrast, the arms moved back and forth regularly during walking and running, and the waveforms oscillated regularly. In particular, the amplitude values of the running waveform were much larger than those of the other activities, reflecting the intense movements of running, which may have contributed to the very high accuracy of running recognition.
Lying in a supine position, brushing teeth, walking, ascending/descending the stairs, and running were identified with high accuracy when the accelerometer was attached to the chest. This is possibly due to the movement of the chest caused by activities such as tooth brushing. The waveform oscillated finely with a constant value in the case of tooth brushing. As lying in the supine position was stationary, the waveform exhibited constant values. Walking, ascending/descending the stairs, and running were regular movements; thus, the waveforms oscillated regularly. As with the non-dominant wrist, the amplitude values of the waveforms, especially for running, were very different from those of other activities, which may have contributed to the very high accuracy of the recognition of running. A previous study that used an accelerometer built into a Holter ECG (Kaneko, Yoshida & Yuda 2019) also identified lying in the supine position and walking with good accuracy.
According to the confusion matrix of the non-dominant wrist, walking and ascending/descending the stairs were misclassified. Both movements involved swinging the arms back and forth; thus, distinguishing between them based on wrist movement alone was difficult. In particular, walking and ascending/descending the stairs exhibited almost the same motion on stair landings. To differentiate among activities that involve translation in only one dimension, such as walking, the correlation between the axes was selected as one of the features. For instance, walking and running can be distinguished from ascending/descending the stairs using correlation. Walking and running usually involve translation in one dimension, whereas ascending/descending the stairs involves translation in multiple dimensions (Ravi et al. 2005). However, walking and ascending/descending the stairs were not well classified. Extending the window frame from 10 s could improve the recognition accuracy, which would enable the window frames to contain information on movements observed in stair landings and normal stairs.
Brushing teeth was misclassified as standing or taking a meal. The non-dominant wrist made regular and small movements while brushing teeth owing to the influence of the dominant arm moving the toothbrush. However, these minute movements were not sufficiently large to be detected as acceleration and could have been misclassified as standing or taking a meal. Brushing teeth and standing were sometimes misclassified as the other (i.e., brushing teeth was sometimes misidentified as standing, and vice versa). Both activities had similar standing movements and waveforms, which may have led to them being misclassified. However, it must be noted that although brushing teeth was misclassified as taking a meal, taking a meal was rarely misclassified as brushing teeth. When consuming a meal, the non-dominant arm performed actions such as holding a bowl or grabbing a plate. These movements were characteristic of the arm and could be classified as taking a meal. In contrast, brushing teeth was not a characteristic of the arm and was likely misclassified as taking a meal. The waveform for taking a meal oscillated irregularly, reflecting the complex motions involved in taking a meal. Combined with a wearing position other than the wrist, it is possible to capture the movement characteristics of brushing teeth and improve the recognition accuracy.
Taking a meal and using the restroom were misclassified as other movements and other movements were misclassified as taking a meal and using the restroom. Taking a meal and using the restroom were misclassified as other movements owing to the irregular and complex hand movements. The waveforms of the movements oscillated irregularly, reflecting multiple movements. Taking a meal included movements such as lifting a bowl or grabbing a plate, whereas using the restroom included movements such as picking up toilet paper or washing hands. In this study, other movements were defined as movements between taking a meal and brushing teeth; however, the accuracy may differ on changing the definition.
According to the confusion matrix of the chest, standing and sitting positions were misclassified with each other, as the chest position in both positions was perpendicular to the ground. It might be useful to use the difference in the height of the center-of-gravity between standing and sitting positions to distinguish between these activities. However, it is difficult to determine the mean height of the center-of-gravity from acceleration. This is because if the relative motion remains the same, the time variation of the acceleration will also remain the same, even if the height of the center-of-gravity is different. Since these distinctions were difficult to make even in previous studies that used the tri-axis accelerometer built into the Holter electrocardiogram to classify activities, an analysis that included pre- and post-activity was attempted, and a convolutional neural network (CNN) was applied to track the changes in the height of the center-of-gravity at certain boundary points, such as sitting > standing and supine > sitting (Kaneko, Yoshida & Yuda 2019).
Taking a meal was misclassified as using the restroom, as both activities were performed while sitting down, and the chest movements were supposed to be similar. Using the restroom and other movements were misclassified with each other. Using the restroom was a complex movement that combined multiple movements; thus, it was probably classified as other movements.
The F-value for the hip position was <0.7 for standing and taking a meal. The confusion matrix indicated that standing and sitting were misclassified as the movements of standing and sitting were similar when measured at the hip. Taking a meal was misclassified as sitting and using the restroom, as these movements involved sitting movements. The F-value for the thigh position was <0.7 for sitting. The confusion matrix indicated that sitting and lying in a supine position were misclassified as the thigh position was parallel to the ground in both activities.
Smartwatches, including the Apple Watch, are equipped with nine-axis accelerometers and peripheral blood oxygen saturation measurement functions at present. The Apple Watch can provide SpO2 values not so different from that of medical devices under experimental conditions (Rafl et al. 2022). In this study, activities during which patients with COPD reported shortness of breath were identified with high accuracy by the wrist-worn nine-axis accelerometer, indicating the possibility of using a smartwatch to monitor the type of activities performed by patients with COPD and the peripheral blood oxygen saturation levels, which would allow to determine the severity of COPD. In other words, the severity of COPD is determined by measuring the degree of decrease in SpO2 while performing activities associated with the tendency to experience shortness of breath. Furthermore, in this study, a chest-worn nine-axis accelerometer was able to identify activities with high accuracy. The use of nine-axis accelerometers in Holter ECGs is not common at present. However, the identification of patient activities using a nine-axis accelerometer would reduce the difficulty in recording the patients’ activities and lead to a more accurate diagnosis of arrhythmia. Therefore, the implementation of nine-axis accelerometers in Holter ECGs is desirable in the future.
This study has some limitations. First, the participants were young healthy individuals. The activity patterns of young individuals differ from those of elderly individuals. Similarly, the activity patterns of healthy individuals differ from those of patients. In subsequent studies, we aim to investigate the accuracy of the accelerometer in identifying elderly individuals and patients. Second, the activities were performed under experimental settings. Therefore, they differed from the activities performed in daily life. It is possible that the results would differ if the data were collected under free-living conditions.
Conclusions
The main purpose of this study was to establish an automatic and accurate method to identify the activities of the patients using wearable devices, thereby facilitating the convenient measurement of the severity of disease and accurate diagnosis. As the first step towards this purpose, the extent to which a combination of accelerometers and machine learning can be used to identify activities that require automatic recognition in medical care was clarified to apply activity recognition using accelerometers in wearable devices. The activities during which patients with COPD reported shortness of breath and the activities described in the Holter ECG recording form were selected as the activities, and nine-axis accelerometers were attached to five locations on the body. A comparison of the recognition accuracy for each position where the accelerometer was attached revealed that the recognition accuracy was high for the dominant wrist, non-dominant wrist, and chest but low for the hip and thigh. A comparison of the recognition accuracy for each activity revealed that the recognition accuracy was high when lying in the supine position, standing, walking, and running when the accelerometer was attached to the non-dominant wrist. The accuracy was high for identifying lying in the supine position, brushing teeth, walking, ascending/descending the stairs, and running when the accelerometer was attached to the chest. These results indicate that activity recognition could be used to measure the severity of COPD and for more accurate detection of arrhythmia by automatically performing activity recognition from accelerometer data built into smartwatches (often attached on the non-dominant wrist) and Holter ECGs (often attached on the chest). This will improve the quality of medical care by reducing the burden on patients and medical staff and enabling more accurate treatment according to the severity of COPD and type of arrhythmia. This will be of great benefit to our society, where the number of patients is expected to increase with the aging population.
Data Accessibility Statement
Data are available from the authors on reasonable request.
Acknowledgements
We thank all the participants for their participation in our study.
Funding Information
This study was supported by JSPS KAKENHI (grant number: JP21K12787).
Competing Interests
The authors have no competing interests to declare.
Author Contributions
All authors conducted a literature search. M.M. and M.K. conceived the study design. M.K. recruited participants and collected acceleration data. T.Y. processed the acceleration data, extracted features, and carried out machine learning. T.Y. wrote the manuscript. M.M. reviewed the manuscript.