non linear time series 1

non linear time series 1

(Parte 1 de 6)


Editor: Leon O. Chua University of California, Berkeley


Volume 3: Lectures in Synergetics V. I. Sugakov

Volume 34: Introduction to Nonlinear Dynamics* L Kocarev & M. P. Kennedy

Volume 35: Introduction to Control of Oscillations and Chaos A. L Fradkov & A. Yu. Pogromsky

Volume 36: Chaotic Mechanics in Systems with Impacts & Friction B. Blazejczyk-Okolewska, K. Czolczynski, T. Kapitaniak & J. Wojewoda

Volume 37: Invariant Sets for Windows — Resonance Structures, Attractors, Fractals and Patterns A. D. Morozov, T. N. Dragunov, S. A. Boykova & O. V. Malysheva

Volume 38: Nonlinear Noninteger Order Circuits & Systems — An Introduction P. Arena, R. Caponetto, L Fortuna & D. Porto

Volume 39: The Chaos Avant-Garde: Memories of the Early Days of Chaos Theory Edited by Ralph Abraham & Yoshisuke Ueda

Volume 40: Advanced Topics in Nonlinear Control Systems Edited by T. P. Leung & H. S. Qin

Volume 41: Synchronization in Coupled Chaotic Circuits and Systems C. W. Wu

Volume 42: Chaotic Synchronization: Applications to Living Systems E. Mosekilde, Y. Maistrenko & D. Postnov

Volume 43: Universality and Emergent Computation in Cellular Neural Networks R. Dogaru

Volume 4: Bifurcations and Chaos in Piecewise-Smooth Dynamical Systems Z T. Zhusubaliyev & E. Mosekilde

Volume 45: Bifurcation and Chaos in Nonsmooth Mechanical Systems J. Awrejcewicz & C.-H. Lamarque

Volume 46: Synchronization of Mechanical Systems H. Nijmeijer & A. Rodriguez-Angeles

Volume 47: Chaos, Bifurcations and Fractals Around Us W. Szemplihska-Stupnicka

Volume 48: Bio-Inspired Emergent Control of Locomotion Systems M. Frasca, P. Arena & L Fortuna

Volume 49: Nonlinearand Parametric Phenomena V. Damgov

Volume 50: Cellular Neural Networks, Multi-Scroll Chaos and Synchronization M. E. Yalcin, J. A. K. Suykens & J. P. L. Vandewalle

Volume 51: Symmetry and Complexity K. Mainzer



NONLINEAR SCIENCE% senesA voi.52 Series Editor: Leon 0. Chua


Michael Small Hong Kong polytechnic University

\[p World Scientific • • • • • • •

Published by

World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore 596224

USA office: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE

British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library.

APPLIED NONLINEAR TIME SERIES ANALYSIS Applications in Physics, Physiology and Finance

Copyright © 2005 by World Scientific Publishing Co. Pte. Ltd.

All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the Publisher.

For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 2 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher.

ISBN 981-256-117-X

Printed by Fulsland Offset Printing (S) Pte Ltd, Singapore

For Sylvia and Henry For Sylvia and Henry


Nonlinear time series methods have developed rapidly over a quarter of a century and have reached an advanced state of maturity during the last decade. Implementations of these methods for experimental data are now widely accepted and fairly routine, however genuinely useful applications remain rare. The aim of this book is to focus on the practice of applying these methods to solve real problems. It is my hope that the methods presented here are sufficiently accessible, and the examples sufficiently detailed, that practitioners in other areas may use this work to begin considering further applications of nonlinear time series analysis in their own disciplines.

This volume is therefore intended to be accessible to a fairly broad audience: both specialists in nonlinear time series analysis (for whom many of these techniques may be new); and, scientists in other fields (who may be looking to apply these methods within their speciality). For the experimental scientist looking to use these methods, MATLAB implementation of the underlying algorithms accompany this book.

Although the mathematical motivation for nonlinear time series analysis is fairly advanced, I have chosen to keep technical content in this book to a minimum. Postgraduate and advanced undergraduate students in the physical sciences should find the material reasonably easy to understand. This book may be read sequentially; skimmed in a pseudo-random order; used primarily as a reference; or, treated as a manual for the companion computer programs.

The applications

To illustrate the usefulness of nonlinear time series analysis, a wide variety of physical, financial and physiological systems have been considered.

vii viii Preface

In particular, several detailed applications serve as case studies of fruitful (and occasionally less fruitful) applications, and illustrate the mathematical techniques described in the text. These applications include:

• diagnosis and control of cardiac arrhythmia in humans (prediction of ventricular fibrillation); • characterisation and identification of aberrant respiratory control in infants (sleep apnea and Sudden Infant Death syndrome); • simulation and recognition of human vocalisation patterns;

• interpretation and prediction of financial time series data; and,

• quantitative assessment of the effectiveness of government control measures of the recent SARS crisis in Hong Kong.

The applications described in this book are drawn largely from my own work, and for this I offer no apology. These applications are the applications which interest me most, and with which I am most familiar. Some of the applications are particularly long and detailed: this is so that the reader can truly get a feel for the complexity of each example and the effort required to fit the methods to the problems. However, if one is primarily interested in the methods themselves, it is entirely possible to read this monograph as a textbook: one should simply skip (or skim) the long-winded application based sections.

The tools

The technical tools utilised in this book fall into three distinct, but interconnected areas: quantitative measures of nonlinear dynamics, Monte-Carlo statistical hypothesis testing (aka Surrogate data analysis), and nonlinear modelling.

In Chapter 1 we discuss nonlinear time series analysis from the perspective of time delay embedding reconstruction. We describe the current state-of-the-art in reconstruction techniques, together with my own view of how to "best" embed scalar time series data. Chapters 2 and 3 are concerned with the estimation of statistical quantities (mostly dynamic invariants) from time series data.

Quantitative measures, such as estimation of correlation dimension and

Lyapunov exponents have been described previously in standard texts. However, this material is included here for completeness: an understanding of these techniques is necessary for the surrogate data methods that follow.

Preface ix

More importantly, it is necessary to provide a current description of the most appropriate estimation techniques for short and noisy data sets. In particular, it should be noted that many of the more venerable techniques perform poorly when faced with the limitations of most experimentally obtained data i.e. short and noisy time series. Hopefully, some of the new methods we discuss will help alleviate these problems.

Three standard methods of Monte-Carlo statistical hypothesis testing (the method of surrogate data) have come into widespread use recently. In Chapters 4 and 5, these methods are described along with their many limitations and restrictions. Several new, state-of-the-art methods to circumvent these problems are also described and demonstrated to have useful application. Finally, a natural extension of surrogate data methods is the testing of model fit, and this is the third focus of this book.

Standard nonlinear modelling methods (neural networks, radial basis functions and so on) are the subject of numerous excellent texts. In Chapter 6 we focus on finding the best model and how to determine when a given model is "good enough". These are two problems that are not well addressed in the current literature. For this work, we utilise the surrogate data methods described previously, as well as information theoretic measures of both the model and the data. These information theoretic measures of the data are in turn related to, and form the foundation of, the dynamic invariants described in the introduction of this book.

Acknowledgment s

This volume is actually the result of many years work. The new methods presented here build on a broad and strong foundation of nonlinear time series analysis and nonlinear dynamical systems theory. In collating this material in a single volume I should thank many people. I would never have met this exciting research field if it were not for my PhD adviser, Dr. Kevin Judd (University of Western Australia, Perth), and also Dr. Steven Stick (Princess Margaret Hospital for Children, Perth). My interest in the analysis of human ECG rhythms was sown by Prof. Robert Harrison (Heriot-Watt University, Edinburgh), and my enthusiasm for financial time series analysis is due to the significant work of Prof. Marius Gerber (Proteus VTS). I should wish also acknowledge Prof. Michael Tse (Hong Kong Polytechnic University) for his help and also for suggesting this endeavour in the first place: without his incessant encouragement I would not have x Preface written this.

As is usual for a post-doctoral researcher, my work has been conducted in a variety of settings. Primarily, the work that serves as the foundation for this volume was conducted in the Centre for Applied Dynamics and Optimisation (CADO) in the Mathematics Department of the University of Western Australia; the Nonlinear Dynamics Group in the Physics Department of Heriot-Watt University (Edinburgh); and, the Applied Nonlinear Circuits and Systems Research Group in the Department of Electronic and Information Engineering at the Hong Kong Polytechnic University. I extend my warm thanks to my numerous friends and colleagues in all three centres.

Finally, I would like to acknowledge those who have contributed directly to this work: Dr. Tomomichi Nakamura persevered through an early draft version of much of this manuscript and gave me valuable feedback at each stage, and my students Mr. Xiaodong Luo and Mr. Yi Zhao assisted and provided me with valuable ideas and new ways of looking at most of the content of this work. Most important of all, my wife Sylvia endured the entire text of this volume and vastly improved both the grammar and style.

Finally, I am obliged to acknowledge various sources of financial support.

This work was supported in part by Hong Kong University Grants Council (UGC) Competitive Earmarked Research Grants (CERG) numbers PolyU 5235/03E and PolyU 5216/04E as well as a direct allocations from the Department of Electronic and Information Engineering (A-PE46 and APF95).

Feedbac k

Although the best way to really understand how these techniques work is to create your own implementation of the necessary computer code, this is not always possible. Therefore, I have made MATLAB implementations of many of the key algorithms and the various case studies are available from my website ( You are most welcome to send me your comments and feedback ).

Michael Small

Hong Kong January 25, 2005


Preface vii

1.1 Stochasticity and determinism: Why should we bother?2

1. Time series embedding and reconstruction 1 1.2 Embedding dimension 5 1.2.1 False Nearest Neighbours 6 1.2.2 False strands and so on 7 1.2.3 Embed, embed and then embed 8 1.2.4 Embed and model, and then embed again 9 1.3 Embedding lag 10 1.3.1 Autocorrelation 10 1.3.2 Mutual information 1 1.3.3 Approximate period 1

Improved modelling and superior dynamics 41 1.9 Summary 4

2. Dynamic measures and topological invariants 47 xii Contents

2.3 Application: Detecting ventricular arrhythmia69

2.2.3 Alternative encoding schemes 60 2.4 Lyapunov exponents and nonlinear prediction error 74 2.5 Application: Potential predictability in financial time series 80 2.6 Summary 82

3. Estimation of correlation dimension 85

3.2 Box-counting and the Grassberger-Procaccia algorithm87

3.1 Preamble 86 3.3 Judd's algorithm 90 3.4 Application: Distinguishing sleep states by monitoring respiration 95 3.5 The Gaussian Kernel algorithm 102 3.6 Application: Categorising cardiac dynamics from measured ECG 105 3.7 Even more algorithms Ill

4. The method of surrogate data 115

4.1 The rationale and language of surrogate data 116 4.2 Linear surrogates 120 4.2.1 Algorithm 0 and its analogues 121 4.2.2 Algorithm 1 and its applications 122 4.2.3 Algorithm 2 and its problems 123 4.3 Cycle shuffled surrogates 125 4.4 Test statistics 129 4.4.1 The Kolmogorov-Smirnov test 131

Contents xiii

5. Non-standard and non-linear surrogates 149

5.1 Generalised nonlinear null hypotheses: The hypothesis is the model 150 5.1.1 The "pivotalness" of dynamic measures 152 5.1.2 Correlation dimension: A pivotal test statistic — nonlinear hypothesis 153

5.6 Simulated annealing and other computational methods174

6. Identifying the dynamics 179

6.1 Phenomenological and ontological models 180 6.2 Application: Severe Acute Respiratory Syndrome:

Assessing governmental control strategies during the SARS outbreak in Hong Kong 181 6.3 Local models 195 6.4 The importance of embedding for modelling 198 6.5 Semi-local models 200 6.5.1 Radial basis functions 200 6.5.2 Minimum description length principle 201 6.5.3 Pseudo linear models 205 6.5.4 Cylindrical basis models 207 6.6 Application: Predicting onset of Ventricular Fibrillation, and evaluating time since onset 208

Bibliography 229 Index 241

Chapter 1

Time series embedding and reconstruction

Nonlinear time series analysis is the study of time series data with computational techniques sensitive to nonlinearity in the data. While there is a long history of linear time series analysis, nonlinear methods have only just begun to reach maturity. When analysing time series data with linear methods, there are certain standard procedures one can follow, moreover the behaviour may be completely described by a relatively small set of parameters. For nonlinear time series analysis, this is not the case. While black box algorithms exist for the analysis of time series data with nonlinear methods, the application of these algorithms requires considerable knowledge and skill on the part of the operator: you cannot simply close your eyes and press the button.

Moreover, the growing artillery of nonlinear time series methods still has fairly few tangible, practical, applications: "applications" that satisfy the mathematicians or physicists definitely exist, but fairly few of these applications merit the attention of a practising engineer or physician.

(Parte 1 de 6)