Objective: Sliding and tapered sliding window methods are the most often used approaches in computing dynamic correlations in biomedical signals such as the brain resting-state fMRI. However, due to the discrete nature of windows, the window methods suffer spurious high-frequency fluctuations and the zig-zag pattern in dynamic correlations. Methods: To address the problem and obtain more stable correlation estimates, we propose a novel windowless approach for computing dynamic correlations via heat kernel smoothing. The heat kernel, the natural generalization of the Gaussian kernel to manifolds, is used to defined a smoothing kernel without boundary or end points. Results: We show that the proposed windowless approach smooths out the unwanted high-frequency noise in correlation estimations and is more stable in identifying and discriminating state spaces in resting-state fMRI. The proposed method is applied to the study of interhemispheric connectivity and whole-brain network analysis. Conclusion: We present a novel framework using heat kernel to compute the windowless dynamic correlation, which is more stable with less high-frequency fluctuations than windowed methods. Significance: The proposed windowless approach reduced the spurious rapid changes in the state space of brain connectivity, and identified the strongest connections in brain networks and symmetry in interhemispheric connectivity.