Modelling of Human Control and Performance Evaluation using Artificial Neural Network and Brainwave

Conventionally, a human has to learn to operate a machine by himself / herself. Human Adaptive
Mechatronics (HAM) aims to investigate a machine that has the capability to learn its operator
skills in order to provide assistance and guidance appropriately. Therefore, the understanding of
human behaviour during the human-machine interaction (HMI) from the machine’s side is
essential. The focus of this research is to propose a model of human-machine control strategy
and performance evaluation from the machine’s point of view. Various HAM simulation
scenarios are developed for the investigations of the HMI.
The first case study that utilises the classic pendulum-driven capsule system reveals that a
human can learn to control the unfamiliar system and summarise the control strategy as a set of
rules. Further investigation of the case study is conducted with nine participants to explore the
performance differences and control characteristics among them. High performers tend to
control the pendulum at high frequency in the right portion of the angle range while the low
performers perform inconsistent control behaviour. This control information is used to develop
a human-machine control model by adopting an Artificial Neural Network (ANN) and 10-time-
10-fold cross-validation. Two models of capsule direction and position predictions are obtained
with 88.3% and 79.1% accuracies, respectively.
An Electroencephalogram (EEG) headset is integrated into the platform for monitoring brain
activity during HMI. A number of preliminary studies reveal that the brain has a specific
response pattern to particular stimuli compared to normal brainwaves. A novel human-machine
performance evaluation based on the EEG brainwaves is developed by utilising a classical target
hitting task as a case study of HMI. Six models are obtained for the evaluation of the
corresponding performance aspects including the Fitts index of performance. The averaged
evaluation accuracy of the models is 72.35%. However, the accuracy drops to 65.81% when the
models are applied to unseen data. In general, it can be claimed that the accuracy is satisfactory
since it is very challenging to evaluate the HMI performance based only on the EEG brainwave
activity.