## Abstract

Stick balancing on the fingertip is a complex voluntary motor task that requires the stabilization of an unstable system. For seated expert stick balancers, the time delay is 0.23 s, the shortest stick that can be balanced for 240 s is 0.32 m and there is a ° dead zone for the estimation of the vertical displacement angle in the saggital plane. These observations motivate a switching-type, pendulum–cart model for balance control which uses an internal model to compensate for the time delay by predicting the sensory consequences of the stick's movements. Numerical simulations using the semi-discretization method suggest that the feedback gains are tuned near the edge of stability. For these choices of the feedback gains, the cost function which takes into account the position of the fingertip and the corrective forces is minimized. Thus, expert stick balancers optimize control with a combination of quick manoeuvrability and minimum energy expenditures.

## 1. Introduction

The importance of balance control for the elderly is underscored by the high mortality and morbidity associated with falls. Often the falls cannot be attributed to a slip or a trip, but are related to issues associated with weight transfer [1] and the ‘fear of falling’ syndrome [2]. Consequently, it has been suggested that losses of balance in the elderly may be related to failures to properly integrate information provided by sensory feedback with cortical internal models that have been refined through decades of balancing experiences [3,4].

The role of an internal model, or predictor feedback (PF), is to predict the sensory consequences of movements [5,6]. In doing so, the internal model makes it possible to make corrective movements faster than the feedback delay [7,8] and to possibly sense when an adverse event such as a fall is about to occur. Investigations into the development of an accurate and robust internal model which underlies expertise are made difficult, because typically years of practice are required. Consequently, current research has focused on a variety of voluntary eye–hand coordination tasks in which certain individuals are able to rapidly acquire exceptional skill [9,10]. As expertise develops, the accuracy and uniformity of task performance increases, but muscular activations [11] and overall brain activation decrease, except in those brain regions most essential for task performance [12,13].

Control theoretic studies for human balancing tasks, including slacklining [14] and stick balancing on the fingertip [15], associate expert balancing with states that minimize energy expenditure. However, a number of observations suggest that feedback for stick balancing is tuned towards the edge of instability [15–19] including the presence of power-law behaviours [15,20–24], and Weibull-type stick balancing survival statistics [25,26]. Recently, a similar conclusion has been reached from an analysis of stability radii for a model of human balance control during quiet standing [27].

Here, we provide the first evidence to show that control at the edge of stability minimizes energetic costs for stick balancing. Thus, expert stick balancers optimize control with a combination of quick manoeuvrability and minimum energy expenditures. These observations emphasize the importance of investigations into dynamical phenomena which occur at the edge of stability for understanding both the causes of falls and the development of strategies to minimize their occurrence.

## 2. Background

During stick balancing, the fingertip is continually moving and hence mathematical models take the form of a pendulum–cart system (figure 1) governed by
2.1where *θ* is the vertical displacement angle of the stick, *m* and *m*_{0} are, respectively, the mass of the stick and cart, is the acceleration of the cart (fingertip) and *f*(*t*) describes the control force. If the control force is zero (*f*(*t*) = 0), then elimination of the cyclic coordinate *x* and linearization around the upper fixed point yields
2.2where is the angular natural frequency of the pendulum hung downward.

The parameter is equal to 1 when and 4 when . During expert stick balancing, the wrist and fingers are held rigid and the movements of the arm occur at the elbow and shoulder [15,20,28]. The equivalence between the human arm mechanism and the pendulum–cart model can be established by relating the mass *m*_{0} of the cart to the inertia of the arm segments for an average human arm [29]. We estimated that *m*_{0} = 1.2 kg and hence *c* = 4 (see the electronic supplementary material for details).

The linearized equations of motion for the control of a pendulum–cart model are
2.3where *x* is the displacement of the fingertip from the typical starting point for stick balancing located in front of the subject (*L* being the total length of the arm). When the subject is seated with their back against the chair (this study), the displacements in *x* cannot be longer than the subject's arm, which yields for an average arm length of *L* = 0.67 m [29].

A dependence of *f*(*t*) on *x* makes it possible to investigate the role of sensory uncertainties and postural effects on arm movements [21,28,30] for stabilizing an inverted pendulum. The maximum control force is limited by where is the maximum acceleration of the fingertip, while the rate of change of the control force is limited by where is the maximum jerk. Experimental observations suggest that of the fingertip is and [31,32].

We considered two candidate choices of *f*(*t*).

### 2.1. Delayed state feedback

First, it is possible that the feedback is directly related to the delayed values of the position, velocity and acceleration. In control theory, this concept is called delayed-state feedback. An obvious choice is to use the most recently available values of , , and , , . Thus, we consider a proportional–derivative (PD) controller
2.4and a proportional–derivative–acceleration (PDA) controller
where , , , , and are, respectively, the proportional, derivative and acceleration control gains for the angular position *θ* of the stick and for the location *x* of the cart.

### 2.2. Predictor feedback

Second, we can assume that *f*(*t*) is involved in making a prediction of the actual state variables and hence we have PF [33]. It should be noted that PF corresponds to an internal model in the neuroscience literature [34] and is often associated with finite spectrum assignment in the engineering control literature [33].

In order to give the control force, it is most convenient to write (2.3) in the first-order form
2.6where
2.7with
2.8being the mass matrix and the stiffness matrix, respectively. We assume that the control force is readily provided by the efferent copies, and matrices *A* and *B* and the delay *τ* are also available for the nervous system with high accuracy as a result of a long-enough learning process. We anticipate that this is true for expert stick balancers. The state is predicted by the solution of (2.6) over the interval as
2.9

Note that this prediction uses the most recent available states and the control force issued over the interval , which is readily provided by the efferent copies. The PF force reads 2.10with 2.11

Thus, the control force can be written as where , , , are the elements of and . The first four terms represent the delayed state feedback, while the last term is associated with the weighted integral of the issued control force over the interval .

## 3. Methods

### 3.1. Stick balancing

Data were collected from 66 healthy undergraduate students (34 females and 32 males) between the ages of 18 and 24 who were free from balance disorders. The stick is an oak dowel with diameter 6.35 mm and lengths ranging from 0.2 to 0.91 m. The training protocol was designed to identify subjects with exceptional stick balancing abilities and included financial incentives [26]. Subjects were seated in a chair and were required to keep their back against the back of the chair at all times while facing a blank black screen. All subjects began by balancing a 0.56 m stick. Subjects were required to stick balance each day in the laboratory for as long as it took to accumulate 10–15 min of total balance time (BT), referred to herein as a practice session. Since the increase in the mean BT between two practice sessions performed on consecutive days was typically greater than the increase in mean BT between two practice sessions performed on the same day, we describe skill acquisition in terms of days of practice rather than total accumulated BT. After 2 days of unsupervised practice, subjects whose mean BT for 25 consecutive supervised stick balancing trials (day 3) was less than 10 s were dropped from the study. The remaining 40 subjects (21 females and 19 males) had daily supervised practice sessions in the laboratory. Fourteen subjects (14/66) were able to balance the stick longer than 240 s for at least one out of five trials by day 7 and by day 16 an additional 10 subjects had reached this milestone (24/66). Once a subject was able to balance a 0.56 m stick for 240 s, they began balancing sticks of different lengths. Six of the subjects from this group (6/24) are the experts reported in this study (see Results): three males: E1 (85 days), E2 (30 days), E4 (25 days) and three females: E3 (40 days), E5 (10 days), E6 (13 days). Typically, these subjects could balance sticks longer than 0.56 m for 240 s without additional practice. Sticks shorter than 0.56 m required additional days of practice: the shorter the stick the greater the number of days of practice required to achieve BT > 240 s.

### 3.2. Motion capture

A high-speed motion capture system (3 Qualisys Oqus 300 cameras, 500–1024 Hz) was used to measure the position of the reflective markers attached to each end of the stick (total mass of stick with markers is 6.3–20.5 g). Typically, data were low-pass filtered with a cut-off frequency of 50 Hz and then downsampled to 125 Hz. The vertical displacement angles were calculated as and where and are the displacement angle in the AP (anterior–posterior) and the ML (medial–lateral) direction, respectively, the subscripts b, t indicate the bottom and top markers attached to the stick and is the distance between the two markers. The power spectral density (PSD) of the fluctuations in and was determined using Matlab.

### 3.3. Time delay measurement

The time delay for stick balancing was measured from the responses to a sensory blank out [34]. Subjects were required to balance a 0.91 m stick on the surface of a table tennis racket while wearing liquid crystal (LC) glasses (figure 2*a*). The purpose of the table tennis racket is to minimize sensory inputs from cutaneous mechanoreceptors located in the fingertip. The LC glasses are equipped with LC optical beam shutters: two LC shutters (VX series, , Boulder Nonlinear Systems, Boulder, CO, USA) were crossed and taped over each lens of the safety glasses (4 LC shutters in total). The remainder of the viewing area of the laboratory glasses was covered by black electrical tape and the experiment was performed in a dimly lit room to ensure that during a visual blank out the subject could not see the position of the stick. A signal generator (Grass S-8800) sent a square-wave timing signal to each lens so that visual blank outs lasting 0.5–0.8 s are produced synchronously for both eyes (transparent → opaque LC shutter latency is less than 0.001 s; opaque → transparent latency is less than 0.005 s). During a visual blank out, the subject is instructed to ‘keep balancing’. Provided that the length of the blank out is longer than *τ*, but not so long that the subject cannot recover balance after the blank out is over, *τ* can be estimated as the time between the offset of the blank out and the first corrective movement. Trials in which eye blinks occurred were not used for the determination of *τ*. In order to minimize the effects of changes in the position of the table tennis racket which are uncorrelated to the blank out, we averaged trials (see the electronic supplementary material). The first corrective movement after the blank out is identified from the changes in the velocity of the fingertip (figure 2*b*).

### 3.4. Numerical simulations

Numerical simulations were written in Matlab using the semi-discretization technique [35], where with being the discrete time step and *r* being an integer. As the control problems for stick balancing mainly arise in the AP plane (see Results), we identified *θ* in the model with . Stick falls were identified when either *θ* exceeded ° or *x* exceeded . The discrete-time version of (2.12) with sampling period , given by
with
3.2corresponds to the tapped delay-line control proposed by Mehta & Schaal [34].

## 4. Results

Here we describe the experimental observations that support the model for stick balancing described in §2.

### 4.1. Time delay

Figure 2*b* shows that for a 0.5 s blank out we obtain (range 0.22–0.24 s for subjects E1, E3, E4). When the blank out was longer than 0.5 s, two of these subjects (E3, E4) could not re-establish stick balancing after the visual blank out. Subject E1 was able to keep the stick balanced even when the blank out lasted as long as 0.8 s. For this stick balancer, *τ* determined using blank outs in the range of 0.5–0.8 s was approximately the same. The time delay of 0.23 s is equal to that for the response of stick balancing to mechanical perturbations [34].

### 4.2. Sensory dead zone

Three observations indicate that the major control problems for stick balancing on the fingertip are in the saggital (AP) plane: (i) BT <5 s when expert stick balancers place an eye patch over one eye, (ii) the standard deviation for is larger than for (figure 3*a*) and this difference increases as *ℓ* decreases (figure 3*b*) and (iii) for novice stick balancers with mean BT ≈ 40–60 s, 72% of 246 stick falls while balancing a 0.56 m stick occur in the AP direction and for experts, 84% of 51 stick falls while balancing a 0.26 m stick occur in the AP direction.

We interpreted these observations in terms of a sensory dead zone, , for the detection of , where *Π* is the sensory threshold. Our estimation procedure for *Π* is motivated by the observation that the time history of shows irregular peaks at irregular time instances. We assumed that these peaks were the result of a free fall for time period *τ* after leaving the dead zone. The solution over the free-fall period can be given as , where is the time instant when the stick is on the edge of the dead zone, i.e. . Substitution of the parameters into the system matrix *A* according to (2.7) using gives the ratio . Thus, before starting corrective motions, *θ* increases by a factor of 1.78 after leaving the dead zone.

A sweeping window of length over the history of was used to check for the maximum peaks in each interval , where goes from to with being the length of the data. The minimum value of these maximum values is taken as an upper estimate for . Figure 3*c* shows the estimated for different window sizes For subjects E1–E4, there is a plateau between and 6 s. The more skilled expert stick balancers had the lower *Π*, 0.8° and 1°, respectively, for E1 and E2. We used the corresponding values of as an estimate of *Π* for these subjects.

The presence of the dead zone means that there is switching feedback, namely the feedback is turned on or off depending on whether is larger or smaller than *Π*. This means that the angular position perceived by the neural system is
4.1where is the stick's actual angle and *Π* is the functional sensory threshold. We assume that information related to and remains available [36].

### 4.3. Power spectral density

A consequence of switching feedback is that it generates oscillations [37–40]. Figure 3*d* shows that there is a peak in the PSD for the fluctuations in between (figure 3*d*). This peak was observed for subjects E1–E6 and could also be readily observed for less skilled subjects. A peak in this frequency range can also be seen for ; however, it is less prominent.

### 4.4. Feedback identification

A necessary condition for the stabilization of the upright position of an inverted pendulum by time-delayed feedback is that the length of the pendulum must be longer than a critical length, [41]. When *τ* is known, corresponds to the shortest pendulum that can be stabilized by the given feedback. Thus, by measuring it is possible to experimentally exclude some of the control concepts.

Figure 4 compares BT determined from five consecutive stick balancing trials as a function of *ℓ* for subjects E1–E6. If BT exceeded 240 s, the balancing trial was terminated and the subject was then asked to balance a shorter stick. All of these subjects could balance sticks when and no subject could accomplish this task when : subjects E1 and E2 could balance sticks as short as 0.32 m for 240 s. A sharp drop off of BT for has also been observed for pole balancing in one dimension [42]. Although we cannot determine with precision it is certainly no longer than 0.32 m and no smaller than 0.2 m.

The vertical dashed lines in figure 4 show determined using (2.3) with (4.1) when *f*(*t*) for PD, PDA and PF is given, respectively, by (2.4), (2.5) and (2.12). The were estimated using numerical simulations with five initial conditions: , for ) over a (four-dimensional) grid of the control gains , , , . For the PDA control, the acceleration gains were fixed as , . If at least one simulations out of lasted for 240 s without falling, then the balancing task was assessed to be successful, and the length of the stick was decreased. The critical length was selected to be the one for which the balancing task was successful, but for a stick 0.01 m shorter falling was observed for all the possible combinations of the control gains and for all initial conditions.

The measured appears to agree best with the determined for PDA control (figure 4). However, the human visual system is not very sensitive for detecting changes in acceleration [43]. This uncertainty will certainly shift the estimate of very much to the right [41]. Thus, it is more likely that the nervous system uses PF. For PF the difference between the estimated and measured values of is in large part due to uncertainties in the internal model and the unmodelled uncertainties in the sensory inputs (likely of the order of 5% [41]).

## 5. Model

The experimental observations suggest that the model for stick balancing is given by (2.3) where *f*(*t*) is given by (2.12), and is given by (4.1) subject to the constraints imposed on *x*, , and . Here, we illustrate the cardinal features of this model when , ° and choices of of the form
where is an initial angle (a more complete description will be given elsewhere). These choices of reflect two observations: (i) all stick balancing trials begin with the stick held stationary for a few seconds and (ii) the subject cannot reproduce a given because of the presence of the sensory dead zone.

There are four control gains: two for the control of , and two for the control of the position *x* of the fingertip, . If ° and there are no constraints on *x*, , and , then the corresponding linear stability region in the plane has a roughly rectangular shape (see dashed red curve in figure 5*a*). The longer BT for the nonlinear model with movement constraints and sensory threshold ° occur in the left portion of the linear stability region. The position of the dominant peak in the PSD depends on the values chosen for the gains (figure 5*b*). Peaks in the range of 0.6–0.8 Hz (figure 3*d*) are associated with values of the gains located in the lower left corner of the linear stability region. For the choices of the gains indicated by the point A, the time series (figure 5*c*) and the PSD (figure 5*d*) generated by the model are qualitatively similar to those observed experimentally for E1 (respectively, figure 5*e*,*f*).

The solutions of the model are microchaotic and exhibit a sensitivity to initial conditions (not shown). Microchoas is a phenomenon produced by deterministic time-delayed dynamical systems with a switching feedback [44,45] and hence is not observed when °. It is remarkable that a deterministic model generates a time series and PSD that qualitatively resembles those generated by a human stick balancer (see Discussion).

Figure 6 shows a set of stability diagrams representing the dynamic behaviour of balancing a 0.56 m stick in the four-dimensional parameter space of the control gains. It is observed that high BT can be achieved outside of the linearly stable region. This property is attributed to the intriguing interplay between the sensory dead zone, the movement constraints and the time delay as suggested previously by a simplified scalar discrete map model of balancing [38].

The yellow dots in figure 6 indicate the parameter points where the BT was 240 s. The size of the yellow dots shows the control cost [46] 5.1where the first term measures the variance of the cart displacement, the second term measures the variance of the control effort, , and and are the corresponding weights. The weight was set to 1 and the weight was adjusted such that, at the parameter point where the control cost is minimum, the contributions of the two terms in (5.1) are equal, i.e. . This condition gives . The smaller the size of the yellow dots, the smaller the control cost.

Comparison to experiments is performed based on three factors: the peak of the PSD of *θ*, the standard deviation of *θ* and the standard deviation of *x*. Light blue circles indicate the parameter points, where these three factors are close to the measured ones within ±10% deviation. Figure 6 shows that these points coincide with the points where the cost *C* is minimal. This suggests that the nervous system minimizes both the control effort and the fingertip displacement by tuning control at the edge of stability.

## 6. Discussion

The most important control problems for stick balancing on the fingertip in three dimensions are related to the long-time delay, the presence of a sensory dead zone for the estimation of and the capabilities of the fingertip to make sufficiently quick movements. The dead zone arises because the human visual system is not able to measure the depth of a moving target to the same accuracy that it can measure its azimuth and elevation [47,48]. Consequently, there are errors in the estimation of whose magnitude continually changes as the movements of the stick changes. The state-dependent nature of the errors arises, in part, because the accommodative reflex has a long latency, a slow response time and uses a dual mode type of feedback which combines both open- and closed-loop components [49]. In our model, we assumed that *Π* was constant. The advantage of this approximation is that the resulting model for stick balancing captures many of the experimental observations while remaining tractable. Thus, it is possible to compare observations with predictions.

It is likely that all sensory receptors possess a dead zone, namely a threshold below which changes in input are not reflected by changes in output [50]. Usually, the dead zone is very small and hence the presence of low-amplitude oscillations and microchaos is buried within the intrinsic noisy variability. However, for stick balancing the size of the dead zone is of the order of the magnitude of the observed fluctuations and hence its effects on balance control must be taken into account. The existence of sensory thresholds for balance control is supported by the beneficial effects of perturbations on stick balancing [51], postural sway [52] and gait stability [53,54]. From a mathematical point of view, the most important effect of the dead zone is that it eliminates the possibility of an equilibrium solution of (2.3). Thus, successful stick balancing is related to a complex bounded time-dependent state [51] which in our model is manifested as microchaos. Because the position of the fingetrip cannot be stabilized, physical constraints such as the length of the arm and the maximum acceleration and jerk of its movements become important determinants of the success of stick balancing. Indeed stick balancing is more easily performed while standing than sitting for many subjects [21]. The increase in BT with standing is likely related to the increase in the arm's reach, but may also arise because this posture enables control mechanisms related to the arm's torque to be implemented [28,55,56].

There are two sources of uncertainty in our model. First, as the internal model is continually refined with practice, it always contains some inaccuracies. As we mentioned in §4.4, the result of uncertainties in the internal model is to increase . The second source of uncertainty arises because of uncertainties in the perception of the angular displacement of the stick. A beneficial effect of the sensory dead zone is that it operates as a ‘noise gate’ to reduce the effects of the noise [57].

The small amplitude and complex noise-like dynamics generated by the model are due to microchaos and arise even though the model contains no noisy inputs. It is generated by interactions between the long-time delay and the sensory dead zone [38,44,45] and is observed whether the feedback is PD, PDA or PF. The sensitive dependence of microchaos on initial conditions may play a role in stick falling [58]. By contrast, there is a large literature on the effects of noise on balance and motor control (e.g. [6,15,18,24,46,59]). Is noise of deterministic chaotic or stochastic origin? This question cannot be answered experimentally since it is well established that deterministic chaotic dynamical systems can generate the same statistical properties that are typically associated with stochastic dynamical systems [60–62]. Thus, it should not be surprising that our conclusions obtained with a deterministic model of balance control can also be inferred from stochastic models of balance control [15,24]. However, our observations go one step further and suggest that variability in motor control may simply be the consequence of the presence of a time delay and a sensory dead zone. In other words, it is not necessary to hypothesize the existence of stochastic forces.

Our observations shed no light onto the nature of the control mechanisms used by less skilled stick balancers. The power-law behaviours described previously [15,20,21,23] are not observed when an expert (E1, E2) balances a 0.58 m stick (data not shown). However, we have observed that when the same experts balance a 0.28 m stick the distribution of accelerative movements made by the fingertip exhibits ‘broad shoulders’. Thus, it is possible that subjects use other types of control strategies to provide some control for stick balancing while an internal model is being learned, such as delayed state feedback [63], clock-driven switched feedback [55], noise-assisted control [15,24] or nonlinear types of controllers [16,17,23].

The search for optimality principles that either maximize or minimize some quantity related to sensorimotor control has a long history (for a review, see [59]). Our observations strongly support the concept that organisms are able to minimize energy expenditures and maximize manoeuvrability by moving about an unstable position. The surprising observation is that this control is achieved by tuning the internal model towards instability. We anticipate that our findings will have many implications for balancing control including the nature of falling in the elderly.

## Ethics

This study was approved by the institutional review board at Claremont McKenna College in accordance with the currently applicable US Public Health Service Guidelines. All participants provided informed consent for all research testing.

## Data accessibility

The derivation of *m*_{0} for the arm and information concerning the need for averaging in the time delay estimation are provided in the electronic supplementary material. The computer programs and stick balancing data used in this study are available at Dryad (https://datadryad.org/resource/doi:10.5061/dryad.73q8s).

## Authors' contributions

J.M. conceived, designed and coordinated the study and helped draft the manuscript; R.M. performed the time-delay measurements; M.Z. performed the estimation of the sensory dead zones; S.R. performed the estimation of the critical stick lengths; T.I. derived the mathematical model for stick balancing, wrote the computer programs and helped draft the manuscript. All authors gave final approval for publication.

## Competing interests

We declare we have no competing interests.

## Funding

This research was supported by the Hungarian National Science Foundation (T.I.) (OTKA-K105433) and National Science Foundation (J.M.) (NSF-0617072 and NSF-1028970). J.M. acknowledges support from the William R. Kenan, Jr Charitable Trust and the Invitation Award to Distinguished Scientists by the Hungarian Academy of Sciences.

## Acknowledgements

We thank G. Csernak, S. Gielen, B. D. Hatfield and G. Stepan for useful discussions, W. Cook and Boulder Nonlinear Systems for their assistance in constructing the visual blank out apparatus and S. Davis, E. Lopez, V. Kim, K. Mendoza-Ochoa and H. Wilson for technical assistance.

- Received March 12, 2016.
- Accepted May 17, 2016.

- © 2016 The Author(s)

Published by the Royal Society. All rights reserved.