Figure 1.
The S-CTRNN utilized in this study. The S-CTRNN has five groups of neural units: input, context, output, variance, and PB units. Input neural units receive current sensory inputs xt. Based on the inputs, PB state pt, and context state ct, the S-CTRNN generates predictions about the mean yt and variance vt of future inputs in the output and variance units, respectively. Parameters, such as synaptic weights wij and the internal state of PB units, are optimized by minimizing negative log-likelihood as calculated using predictions about sensory states, their variance, and actual target sensory states ŷt.
Figure 2.
Ball interaction tasks in the training and adaptation phases. A) Four interactive behavioral patterns learned by a robot controled by an S-CTRNN with PB. The upper left and upper right figures show the right and left behaviors, respectively. The lower left and lower right figures show the self-play and attract behaviors. B) System overview during adaptive interaction between a robot and an experimenter. The solid lines for prediction and sensory input represent visual information about the ball position. The dotted lines represent proprioceptive information about the robot’s joint angles. The neural network generates predictions about sensory states yt and their variances vt based on current sensory inputs xt and also recognizes situations by updating PB activity online in the direction of minimizing the negative log-likelihood calculated using the predictions and the target signal (actual sensory feedback) ŷt.
Figure 3.
Generated time series data from interacting with the experimenter under normal conditions. The robot with a normal network (K = 0) successfully adapted to the changing situations (time steps 400–499, in red box) by flexibly switching its intention (PB state) in the direction of minimizing the increased negative log-likelihood. “Output joint” indicates predictions about selected four-dimensional joint angles. “Input vision,” “variance vision,” and “negative log-likelihood vision,” respectively, indicate the two-dimensional ball position and corresponding estimated variance and precision-weighted prediction error. The negative log-likelihood at time step t is the value after the postdiction process inside the error regression window between time steps t − W + 1 and t. PB indicates activation values of the two PB units. The joint-angle output of the time series was quantitatively assessed every 100 time steps, as described in Methods.
Figure 4.
Generated time series data from interacting with the experimenter under increased or decreased sensory variance conditions. A) Robot’s behavior under increased sensory variance condition (K = 8). With increased sensory variance, the robot’s intention was invariant through the interaction with a situation change (time steps 400–499, in red box) due to highly reduced precision-weighted prediction error, leading to a freezing behavior. B) Robot’s behavior under decreased sensory variance condition (K = −8). With decreased sensory variance, the robot experienced huge precision-weighted prediction error signals, and its intention first quickly changed and then fixed at a certain point, leading to an unlearned repetitive movement. Note that the ranges for negative log-likelihood shown in the graphs for the high-variance condition and the low-variance condition are different. The joint-angle output of the time series was quantitatively assessed every 100 time steps, as described in Methods. Abnormal behavioral patterns, including freezing and inappropriate repetitive behavior, were observed under both increased and decreased sensory variance conditions, and these figures show representative examples.
Figure 5.
Dynamics of internal PB states (upper figures) and error signals (bottom bar graphs) for each condition during the interactions shown in Figures 3 and 4. Colored dots in the upper figures represent PB dynamics during different periods of time (early: time steps 0–199; middle: time steps 200–499; late: time steps 500–699). Bottom bar graphs show the corresponding mean of the negative log-likelihood per time step during each time span. A) Flexible intention switching under normal condition during the interaction shown in Figure 3. During the situation change in the middle period, generated error signals caused intention switching, and error signals were successfully reduced during interaction in the new situation in the late period. B) Deficits in intention switching for high sensory variance during the interaction shown in Figure 4A. Even when the situation changed in the middle period, PB states were almost unchanged due to the underestimated precision of prediction error. C) Large shift of network behavior for low sensory variance during the interaction shown in Figure 4B. Internal PB states first dynamically fluctuated in the early period, but after the middle period, they became almost fixed at a certain value, although generated error signals were still very large.
Figure 6.
Changes in the robot’s behavior and negative log-likelihood associated with various levels of sensory variance. A) The occurrence rates of each behavioral trait over 120 trials for each variance level determined by a parameter K are shown. Behavioral traits observed at time step from 150 to 250 were assessed (see Methods). B) Negative log-likelihood per time step for each level of sensory variance is shown. Bars in the graph correspond to mean values over 120 trials for each parameter K. One-way repeated-measures ANOVAs indicated significant differences between the five conditions for the frequencies of the sum of the three abnormal behaviors, F (4, 36) = 51.0, p < 0.05, and levels of negative log-likelihood, F (4, 36) = 110.24, p < 0.05. Adjusting for multiple comparisons using the Holm–Bonferroni method, significant differences were found between the normal condition (K = 0) and other unusual variance conditions (K = −8, −4, 4, 8) in frequencies of abnormal behaviors, all p < 0.05. In addition, significant differences in levels of negative log-likelihood between all pairs were reported, all p < 0.05.