Statistical signal processing is one of the most important tasks in a large amount of areas of scientific studies, such as astrophysics, geophysics, and space physics. Phase recurrence analysis and long-range persistence are the two dynamical structures of the underlying processes for the given natural phenomenon. Linear and nonlinear time series analysis approaches (cross-correlation analysis, cross-recurrence plot, wavelet coherent transform, and Hurst analysis) are combined to investigate the relative phase interconnection and long-range correlation between solar activity and geomagnetic activity for the time interval from 1932 January to 2017 January. The following prominent results are found: (1) geomagnetic activity lags behind sunspot numbers with a phase shift of 21 months, and they have a high level of asynchronous behavior; (2) their relative phase interconnections are in phase for the periodic scales during 8-16 years, but have a mixing behavior for the periodic belts below 8 years; (3) both sunspot numbers and geomagnetic activity can not be regarded as a stochastic phenomenon because their dynamical behaviors display a long-term correlation and a fractal nature. We believe that the presented conclusions could provide further information on understanding the dynamical coupling of solar dynamo process with geomagnetic activity variation, and the crucial role of solar and geomagnetic activity in the long-term climate change.