Cross Correlation - Clear Sky Detection

Auto correlation and cross correlation can both be used to extract information from a time series.   In auto correlation, two datasets, say x and y are created from a single stream/source, but separated by a time lag appropriate to the application.  Cross correlation is based on two separate data sources.

This work has not been reviewed and therefore should be treated with caution.

Cross correlation can be used identify "cloud free" intervals in a stream of solar radiation data.  Under a clear sky, solar radiation is a smooth time series which is zero at sunrise, peeks at noon and returns to zero at sunset.  The presence of clouds in the sky introduces disturbance into the time series.  As the "clear" sky and the "cloud" sky required different forms of analysis, it is useful to be able to separate them.

The contrived diagram below shows the solar irradiance during a day which starts with a clear sky, around noon some broken or overcast cloud passes over, possibly bringing rain after which the clouds disperse leaving a fine afternoon.

Clear sky global horizontal irradiance (GHI) can be estimated by several methods.  In this application we only need an approximation that produces a time series similar to the data for a clear day, the magnitude of the estimates is not critical.  GHI is a combination of direct normal irradiance (DNI) and diffuse horizontal irradiance (DHI).  DNI can be estimated using the Meinel formula (see reference below).

A similar correlation (see previous post "Diffuse Irradiance - Part 3" for details, provides an estimate for DHI:

These can be combined into an estimate for GHI using this formula.  In all three equations, the Air Mass is calculated using the plane parallel method.

The average value of the solar constant is 1370 w/m2.

If the time and location of the observations is known, the Air Mass (AM) can be estimated using Sun-Earth geometry.

If the sky is completely clear, there will be a high level of correlation between the estimated and observed data.  Solar radiation measurements are very sensitive to changes in the atmosphere, even a small amount of high level cloud which is barely visible to the human eye can introduce some disturbance into the time series.

The graph below shows the correlation between the estimated and observed datasets.  When the sky is clear, the correlation coefficient is close to one, when the Sun is obscured by cloud, it is similar to a random number.  For this application, the sky was assumed to be clear, if the absolute value of the correlation coefficient was greater than 0.95.

As with any statistical or numerical method, the results can vary according parameter settings and assumptions, in this case these include the duration of the correlation period and the critical value of the correlation coefficient.  The results from this methodology are similar to those obtained by selecting clear sky intervals by eye from plots of solar irradiance.

Reference:

A.B. and M.P. Meinel
Applied Solar Energy (Pages 46 to 48?)
Addison Wesley Publishing Company, 1976