Effective recognition of change points in tropospheric time series with the Sequential Regime Shift Detector
Abstract
The process of homogenization employed for tropospheric time series, e.g. Integrated Water Vapour (IWV), involves the detection of breaks or shifts in the mean value of data. This is an extremely important task since trends estimated from tropospheric data applied nowadays to monitor changes in climate are affected by these breaks. Therefore, there is a need to develop the optimal algorithm to detect shifts. In this research, we present the results of analysis performed under the umbrella of the COST ES1206 action. We analysed 360 time series of IWV differences which were synthetized within COST sub-working group. They were based on real differences of IWV series between IWV estimated from Global Navigation Satellite System (GNSS) observations and the European Centre for Medium Range Weather Forecasts (ECMWF) operational and reanalysis (ERA interim) model. A number of 360 series contained different levels of complexity and was named: EASY, LESS-COMPLICATED and FULLY-COMPLICATED datasets. The first named included white noise, whilst the two remaining were affected by autoregressive (AR) process. The effective detection of shifts may be biased by AR noise, as many algorithms start to detect regime shifts related to AR process itself. We applied the Sequential Regime Shift Detector (SRSD) software as proposed by Rodionov (2005), which is an extended version of the Sequential T-test Analysis of Regime Shifts (STARS) software introduced to detect shifts in climate series. To overcome the bias arising from AR noise, we had firstly applied 'prewhitening' using the Ordinary Least Squares (OLS) method. Later, we detected shifts in the mean based on a Student's t-test from 'prewhitened' data. On the base on epochs of shifts, stepwise trend was removed and then we detected shifts in the variance using F-test analysis. Values of parameters: significance level p, cut-off length l and Huber's weight parameter H, which are essential to determine breaks, were estimated with multiple analytical tests. Using the SRDS method, we were able to detect 50% of offsets simulated for FULLY-COMPLICATED dataset. 33% of detected offsets was not simulated, meaning that the algorithm reported regime shifts related to AR noise. This number increases to 94% when 'prewhitening' process was omitted. Finally, we compared the results of the SRSD-based detection with the Integrated Cumulative Sum of Squares (ICSS), which is a very common algorithm for finding change points in the variance. For EASY dataset, we detected 47% of simulated offsets, for LESS-COMPLICATED and FULLY-COMPLICATED dataset, only 30% of simulated offsets was detected.
- Publication:
-
EGU General Assembly Conference Abstracts
- Pub Date:
- April 2018
- Bibcode:
- 2018EGUGA..20..318G