2.1 The ASTER dataset for EUREC⁴A

ASTER is mounted aboard Terra, a polar‐orbiting satellite in a descending Sun-synchronous orbit with an Equator crossing time of 10:30 local solar time. Terra crosses the latitude of Barbados and the HALO flight circle area roughly at 14:25 UTC, while the tracks further east at about 43^∘ W are observed by ASTER an hour earlier. Figure 1 shows the location of measurements taken in the area east of Barbados from 7 to 18^∘ N and from 41 to 62^∘ W between 11 January and 19 February 2020. The data from the observed swaths are segmented in the form of 60×60 km² images, each corresponding to 9 s of observation time.

ASTER's visible and near-infrared (VNIR) radiometer pointing nadir has three bands in the range of 0.53–0.86 µm. The radiometrically calibrated and geometrically coregistered Level 1B data provide top-of-atmosphere monodirectional radiances at 15 m pixel resolution at the sub-satellite point. We use the band 3 radiance centered at 0.807 µm in the present study to define the total cloud cover. One image of band 3 radiances consists of 4200 pixels along-track and 4980 pixels across-track with, depending on the viewing angle, about 15.4 % swath edge pixels that are neglected within the further analysis, leaving about 17 684 552 pixels per image.

In our analysis we work with reflectance instead of radiance with the aim to reduce the influence of varying solar zenith angles θ₀ within the overpasses and slightly varying extraterrestrial solar irradiance E₀. The reflectance R is calculated from the radiance L as

We further draw comparisons to the ASTER cloud mask, which is based on several bands in the VNIR (Werner et al., 2016). The cloud mask works with thresholding tests and is representative for traditional passive remote sensing cloud-masking schemes such as the Moderate Resolution Imaging Spectroradiometer (MODIS) cloud detection scheme. In more detail, the algorithm uses three tests to distinguish between bright clouds and the dark ocean from thresholds applied to radiance values in the VNIR range. An additional test based on a band in the shortwave infrared (SWIR) is no longer applicable as the SWIR detector broke in 2007. Nevertheless, the three thresholding tests allow us to distinguish between confidently clear, probably clear, probably cloudy, and confidently cloudy pixels following the method described in Werner et al. (2016). Within the current study we combine the flags probably cloudy and confidently cloudy if we refer to cloudy regions according to the ASTER cloud mask. We omit a fifth test including ASTER's thermal band 14 (11.65 µm, 90 m pixel resolution) that is designed to detect cirrus-contaminated areas and Sun glint at the expense of a lower resolution. The observations during EUREC⁴A are luckily recorded at a minimum Sun reflection angle larger than 23^∘, making Sun glint highly unlikely.

Concerning cirrus cases, we decided to stay with the high resolution, but instead exclude images that have a high likelihood to be contaminated by cirrus clouds. A test based on the ratio of ASTER's thermal bands 13 and 14 is implemented following a publication by Hulley and Hook (2008). Next to cirrus, the test unfortunately also detects low thin clouds, the latter being the main actor of the current study, which we therefore want to keep in the dataset. Most importantly, we notice that our main results and statements change only marginally, indicating that cirrus does not have a strong impact on the current study. Nevertheless, we exclude images that have a chance of more than 10 % coverage by potential cirrus as defined by Hulley and Hook (2008), which leaves 380 images for our analysis.

2.2 WALES airborne lidar measurements

The WALES lidar instrument (Water Vapor Lidar Experiment in Space demonstrator; Wirth et al., 2009) is part of the remote sensing package that was on board the HALO research aircraft during EUREC⁴A (Stevens et al., 2019). The aircraft flew at about 9 km of altitude throughout of the campaign and thus below the typical altitude of cirrus clouds in the trades. We therefore do not expect any cirrus contamination in the WALES dataset. The high-spectral-resolution lidar measurements from the auxiliary channels of the instrument at 532 nm are well suited to investigate small and optically thin clouds due to the high instrument sensitivity to small particles ranging from aerosols to cloud droplets. The advantage of WALES compared to spaceborne active instruments such as the Cloud–Aerosol Lidar with Orthogonal Polarization (CALIOP) simply lies in the closer distance and thus higher sensitivity to low clouds as well as the much higher horizontal sampling due to the lower aircraft speed (0.2 versus 7 km s⁻¹). The resulting horizontal spatial resolution of the WALES cloud product is about 40 m during EUREC⁴A, which is slightly larger but commensurate with that of ASTER. CALIOP has been shown to struggle in detecting small clouds with cloud tops below 1 km (Leahy et al., 2012), while we find 29 % of clouds detected by WALES during EUREC⁴A to have cloud tops below 1 km.

Within the present study we use the cloud mask and cloud optical depth product described in Konow et al. (2021). In the dataset, a cloud is defined as having a backscatter ratio that exceeds 10. This threshold is lower compared to the studies by Gutleben et al. (2019) and Jacob et al. (2020) wherein the value was chosen to make the detection limit comparable to CALIOP. The lower value used in the present study nicely separates the highest possible signals originating from marine aerosol and any cloud-related signal that might include anomalously humidified aerosols and the smallest cloud droplets. WALES uses the high-spectral-resolution lidar technique (HSRL; Esselborn et al., 2008) to distinguish molecular from particle backscatter at 532 nm, which allows for the direct measurement of the (two-way) atmospheric transmission. The latter is proportional to the range (r) and atmospheric-density-corrected lidar signal R_M(r). To a first approximation the optical thickness is given by

The complete algorithm adds several corrections and is described in detail in Esselborn et al. (2008).

2.3 Surface wind speed estimates

For the methodology described in Sect. 3 we need surface wind speed estimates at 10 m of height for a given ASTER pixel. The fifth-generation European Centre for Medium-Range Weather Forecasts reanalysis (ERA5) provides hourly wind speed estimates on a global grid at 10 m of height (2D surface product), which would fit our needs but showed a significant underestimation compared to collocated dropsonde measurements during EUREC⁴A (JOANNE dropsonde dataset: George et al., 2021). The underestimation is in agreement with a study by Belmonte Rivas and Stoffelen (2019), who find a low bias in ERA5 surface winds in the trades. Nevertheless, wind speed estimates from the ERA5 profile product (hourly, 0.25^∘ grid; Hersbach et al., 2020) agree remarkably well with dropsonde measurements.

Thus, we use ERA5 wind speeds at the lowest-altitude pressure level of 1000 hPa, which corresponds to about 135 m above sea level on average based on the dropsonde dataset. We derive a correction that translates from 1000 hPa to 10 m based on a comparison of ERA5 wind speed at 1000 hPa and the 10 m wind speed from dropsonde measurements (Pearson correlation coefficient 0.88). A least-squares fit provides us with the coefficients to estimate the 10 m wind speed by

This wind speed is an average value representative for a 0.25^∘ grid cell. We therefore use measurements at the Barbados Cloud Observatory (BCO) to estimate the variance in wind speed within 0.25^∘ compared to the 15 m ASTER grid. The BCO is located at the easternmost point of the island of Barbados and has been shown to take measurements representative of an undisturbed marine trade wind boundary layer (Stevens et al., 2016). We use the standard surface wind speed measurements from a Vaisala WXT-520 to derive an estimate of the surface wind variance within 0.25^∘ (27.12 km at 13^∘ N), which translates to about an 80 min sampling period. We add a Gaussian perturbation according to the estimated wind variance of 1.63 m² s⁻² to the average wind speed within our further analysis. The campaign average wind speed corresponding to the ASTER image locations is 9.02 m s⁻¹.

3.1 A simplified clear-sky model (SCSM)

The cloud-free radiance over ocean in the visible range depends on a narrow set of parameters and can be estimated by simplified one-dimensional radiative transfer calculations. In Appendix A we describe the full set of equations and approximations made in calculating the cloud-free signal with our simplified clear-sky model (SCSM). We generally assume a single-layer atmosphere with constant air density and calculate the extinction of solar radiance from the top of the atmosphere to the ground and back to the sensor in space. How the light is reflected at the surface in the view direction of the sensor is characterized by the bidirectional reflection function, which depends on the surface wind speed and the generated ocean wave slope distribution. Here, we use the wind speed estimates described in Sect. 2.3 as input to the Cox and Munk parameterization to derive an average reflectance for a given surface condition (Cox and Munk, 1954).

We further need to know the aerosol optical depth (AOD) to estimate the extinction of direct and diffuse radiation on its path through the atmospheric column. Although the aerosol load does not vary much within a 60×60 km² ASTER image, the availability of aerosol information from measurements, even for an image-average AOD, is very limited. Therefore, we estimate an effective AOD in an optimization approach by including information from the ASTER dataset. We assume that the pixels labeled confidently clear in the ASTER cloud mask are a good first guess for cloud-free and shall serve as a reference for finding a suitable effective AOD such that the simulated cloud-free values are in close agreement with the selected ASTER pixel values.

In Fig. 2 we illustrate the cloud-free retrieval workflow. In detail, we randomly select 20 000 pixel (0.11 % of valid image pixels) from those defined confidently clear by the ASTER cloud mask (see Sect. 2.1) for a given ASTER image. Simulating 20 000 samples ensures a proper representation of the cloud-free distribution at a manageable computational cost. For those input pixel locations we run the cloud-free model with the corresponding sensor–Sun geometries, surface wind speed estimates, and a first guess on the AOD. We further optimize this image AOD value iteratively by minimizing the summed squared difference between simulated and observed reflectances. Here, we make use of SciPy's implementation of the limited-memory Broyden–Fletcher–Goldfarb–Shanno algorithm (LM-BFGS) with bounds (SciPy version 1.5.2). The resulting effective AOD value is representative for the reflectance distribution of a single ASTER image. From all evaluated ASTER images we find a campaign average effective AOD of 0.077 (±0.051).

From comparing simulated cloud-free reflectance distributions to selected observed ones for manually checked and seemingly cloud-free ASTER observations, we find two things. First, the distributions agree very well in terms of their expected value. Second, the simulated distributions are more narrow compared to the observed ones as the Cox and Munk parameterization returns average pixel reflectance R_mean. We therefore introduce a variability in brightness in a post-processing step. We calculate a kernel density estimate with normal kernels characterized by a standard deviation ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$ that is placed on each of the simulated reflectance values (Rosenblatt, 1956; Parzen, 1962). We derive a suitable value for ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$ from comparing simulated cloud-free reflectance distributions and corresponding ASTER images that have at minimum 97 % confidently clear pixels in the ASTER cloud mask. From 22 cases we calculate the average ${\mathit{\sigma }}_{R_{\mathrm{mean}}}=\mathrm{0.0026}$ from a least-squares optimization, again using the LM-BFGS algorithm. We use a constant value for ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$ for the whole dataset due to the lack of several cloud-free observations for various sensor–Sun geometries. However, the ASTER dataset is confined to a narrow set of sensor–Sun geometries and outside possible Sun glint observations such that we assume that a constant value is sufficient for our application.

3.2 Identifying optically thin clouds in all-sky observations

The output from our SCSM provides us with a distribution of cloud-free reflectance $p\left(R\right|F_{\mathrm{CLEAR}},B)$, which is the probability distribution of reflectance values R given that they originate from cloud-free area with the flag F=F_CLEAR and additional background conditions B. The background conditions include the sensor–Sun geometry, wind speed, and AOD and are covered by the SCSM by handling each image individually. In the following we evaluate the probabilities on an image basis and therefore omit the implicit condition on B in the notation. Further, we use standard notation whereby | means “given that” for conditional probabilities and “,” means “and” and symbolizes combined (or joint) probabilities. For example, the SCSM output is a conditional probability as the SCSM framework does not include any information on the general cloud-free fraction within one image.

In the following, we split the observed reflectance distribution of an ASTER image into the categories or flag values $F\in \mathit{\{}{F}_{\mathrm{CLEAR}},F_{\mathrm{OTC}},F_{\mathrm{CLOUD}}\mathit{\}}$. The ascending order of the flag values indicates the associated expected increase in reflectance. The darkest observed pixels originate form cloud-free ocean observations. Small cloud fragments and humidified aerosols slightly enhance the reflectance, though they are often undetected by the cloud-masking scheme. We characterize them as optically thin clouds (OTCs). The flag CLOUD refers to the cloudy pixels detected by the ASTER cloud-masking scheme (see Sect. 2.1). We know the CLOUD part of a distribution p(R,F_CLOUD) from the observation and we can infer the CLEAR contribution from the SCSM output. The all-sky reflectance distribution p(R) is built up by the arithmetic sum of combined probability distributions of R and the flag values F:

Each combined probability can be represented by the product of the corresponding conditional probability and the probability of the flag value, i.e., for cloud-free

The probability of cloud-free p(F_CLEAR) is the true cloud-free fraction in an observed image and is challenging to estimate. Note that the true cloud-free fraction is independent of the ASTER cloud mask. If we knew the cloud-free fraction p(F_CLEAR), Eqs. (5) and (4) together would fully describe the observed reflectance distribution p(R). In the following we describe our approach for estimating the unknown cloud-free fraction.

The first constraint is given by the fact that any probability must be within the range [0,1], and thus we can formulate for our case

We can approach the estimation of the cloud-free fraction p(F_CLEAR) from a conservative side by deriving the maximum possible p(F_CLEAR) such that Eq. (6) still holds. Thinking visually, we scale the simulated cloud-free distribution up until it touches the all-sky distribution p(R). At the reflectance $R=R^{\prime }$ (of unknown value) where the probability density functions (PDFs) touch, we are certain that the non-cloudy classified reflectances are actually due to cloud-free:

We can solve Eqs. (7) and (6) for p(F_CLEAR) (for details see Appendix B). While being mathematically concise, the described method faces a problem. It relies on the exact count of measurements in only a single reflectance bin R^′ and is thus especially susceptible to measurement and model uncertainties. We tackle this problem by extending and relaxing the condition stated in Eq. (7). We modify this first condition from a single value to an extended range of reflectance values. As Eq. (7) would be overdetermined for more than one reflectance value in the presence of measurement and model uncertainties, we demand that the equation approximates the value $\mathrm{1}-p\left(F_{\mathrm{CLOUD}}\right|R^{\prime })$ for reflectivity values measured and known to be caused by cloud-free skies.

In particular, we do this by a weighted linear regression, minimizing the term

with p(F_CLEAR) as the only free variable. The regression weight $w=p\left(R\right)p\left(R\right|F_{\mathrm{CLEAR}})$ is chosen to only consider measured reflectance p(R) that overlaps with the range of simulated cloud-free reflectance p(R|F_CLEAR). The product of both guarantees close agreement around the peaks of measured and simulated PDF.

The resulting estimate of p(F_CLEAR) is more robust in the presence of small measurement or model errors, but a direct consequence of this approximate matching is that Eq. (6) does not necessarily hold for all R^′′ anymore. As illustrated in Fig. 3 using dotted and dashed lines, we correct this by clipping the resulting probabilities to the allowed range. As this clipping effectively modifies the simulated reflectance distribution and is thus potentially dangerous, we need to ensure that this method indeed only compensates for small measurement uncertainties (i.e., of the order of a single digital sensor count). We can do this by comparing the expected value of the clear-sky reflectance p(R|F_CLEAR) before and after clipping. On average, this difference is 0.15 % and even in the worst (maximum) case, the clipping causes a shift of 0.0018 in reflectance units, which is well below one digital sensor count of about 0.004 reflectance units. Based on this analysis, we use the more stable regression and clipping method instead of a direct application of Eq. (7).

Further, the SCSM does not include cloud shadows on the ocean surface, which introduce a signal at very low reflectances in the observed distribution. Conceptually we add the low reflectance values originating from such shadowed areas to the cloud-free reflectance distribution p(R,F_CLEAR).

Figure 3Visualization of the approach for estimating the cloud-free fraction p(F_CLEAR) by optimization. The orange dotted and dashed lines show the processing steps leading to the filled orange cloud-free PDF. The blue lines are the respective residuals related to optically thin clouds and resulting from the all-sky (grey) minus the CLEAR (orange) and minus the CLOUD PDF (dark blue; not visible).

Download

In Fig. 4 we show combined probability distributions per flag for an ASTER observation on 31 January east of Barbados. The inset shows the reflectance image that we translate into the distribution using the method described above.

Figure 4Reflectance distribution corresponding to the ASTER observation shown in the inset recorded on 31 January 2020 at 14:08:05 UTC southeast of the HALO circle area at 11.37^∘ N, 53.86^∘ W. The cloud-free contribution is retrieved with the method (1) described in Sect. 3.2 and displayed by the orange curve, while pixel reflectances identified as cloudy from the ASTER cloud-masking algorithm are shown in dark blue. We attribute the light blue contribution to the distribution to optically thin clouds.

Download

3.3 Robustness of optically thin cloud estimation

Our target variables are the fraction and expected reflectance of optically thin clouds. The retrieval of cloud-free and subsequent optically thin clouds in ASTER images depends on visible cloud-free areas, which limits the evaluation of the full ASTER EUREC⁴A dataset to images with less than 85 % detected cloud cover in the cloud-masking algorithm (380 images).

Within the retrieval we have two main free parameters which can introduce uncertainty in our target values: the surface wind speed estimate and the assumed variability ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$ of simulated average pixel reflectances R_mean. We first have a look at the added variability. From a comparison of 22 manually checked cloud-free reflectance distributions (>97 % confidently clear pixels) to the simulated distributions we derived an average variance of 0.0026 (±0.0007). We apply the methodology described in this section for the average value, as well as for a 20 % lower (0.0020) and 20 % higher value (0.0031). Similarly, we add an artificial bias of ±20 % to the surface wind speed estimates and investigate the change in our target values. The average wind speed in our dataset is 9.02 m s⁻¹ (±2.38 m s⁻¹). The resulting deviations in our target values, the fraction p(OTC) and expected reflectance E(R|OTC) of optically thin clouds, that result from a bias in ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$ and/or the surface wind speed are stated in Table 1.

The fraction of optically thin clouds p(OTC) changes only slightly with a change in wind speed, showing an overestimation for a negative wind speed bias, meaning that a small part of the cloud-free distribution is wrongly attributed to optically thin clouds. For a positive wind speed bias the opposite is the case. The low uncertainties (4.6 % and −4.9 %) are a result of the retrieval setup including the optimization of AOD, which can partly compensate for a bias in wind speed. Changing the variability of simulated average pixel reflectances ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$ can narrow (negative bias in ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$) and broaden (positive bias in ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$) the cloud-free distribution and thus lead to strong overestimation or underestimation of p(OTC) as high as 13.1 % and −14.1 % (relative deviations). Combining the highest retrieval uncertainties from the two free parameters, the wind speed and the variability ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$, we can get a deviation in the estimated fraction of optically thin clouds of up to ±0.027 (relative: ±19.7 %).

The expected reflectance of optically thin clouds E(R|OTC) shows a smaller sensitivity to changes in the wind conditions and ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$ compared to the fraction of optically thin clouds discussed above. An underestimation in wind speed leads to a marginal underestimation in the expected reflectance as lower cloud-free reflectance is wrongly attributed to optically thin clouds. In the case of an overestimation in wind speed, the cloud-free reflectance distribution extends to higher reflectance values, which are missing in the estimated E(R|OTC), and thus leads to a high bias in E(R|OTC). A more narrow (negative bias in ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$) or broader (positive bias in ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$) cloud-free distribution can decrease or increase the expected reflectance of optically thin clouds up to −4.5 %. However, the combined deviation due to possible biases in wind speed and ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$ is still within the range of ± 0.0031 (± 5.5 %), which is smaller than the reflectance bin size of the original Level 1B ASTER data (least significant bit).

Table 1Deviations of the fraction Δp(OTC) and expected reflectance ΔE(R|OTC) of optically thin clouds for the two main free parameters from the clear-sky retrieval, the surface wind speed, and the variability ${\mathit{\sigma }}_{R_{\mathrm{mean}}}$. The two numbers in each cell state the absolute and relative difference to the reference case with no wind speed bias and ${\mathit{\sigma }}_{R_{\mathrm{mean}}}=\mathrm{0.0026}$, respectively.

Download Print Version | Download XLSX

4.1 Visualizing optically thin clouds in an ASTER image

To visualize the OTC area in an image we can define a threshold in reflectance similar to common cloud-masking algorithms. We construct a total cloud cover mask that includes pixels with a probability of that pixel reflectance to be cloudy $p\left(F_{\mathrm{TOTAL}\mathrm{\_}\mathrm{CLOUD}}\right|R=R_{\mathrm{pixel}})\ge \mathrm{0.9}$ with $F_{\mathrm{TOTAL}\mathrm{\_}\mathrm{CLOUD}}=F_{\mathrm{OTC}}\vee F_{\mathrm{CLOUD}}$. In the particular ASTER image shown partially in Fig. 5 all reflectance values greater than 0.049 satisfy that condition. The cloud mask derived with the cloud-masking algorithm by including several ASTER bands is shown in blue in panel (a), while the total cloud cover mask is shown by the contours in red in panel (b). The background reflectance image in panel (b) is adjusted in its reflectance range with the aim of enhancing the range reflectances related to OTC.

The figure visualizes how OTC is often classified in pixels surrounding detected clouds. Detraining clouds and anomalously humidified aerosols likely cause enhanced reflectances close to thicker clouds. Possible scattering of light at the sides of thicker clouds might additionally enhance the brightness of their surrounding areas. Such surrounding halos of optically thin clouds lead to (threshold-dependent) smoother cloud edges, an interesting result in the context of cloud boundaries and related fractal dimensions. Also, cloud structures tend to be more connected in the total cloud cover mask, leading to larger cloud objects with smooth reflectance transitions to the cloud-free ocean background. While there are numerous studies on cloud shapes, we rather focus on a statistical estimate of area coverage and the contribution of OTC to the total cloud cover in the remainder of this work.

Figure 5Visualization of the area corresponding to optically thin clouds. Shown are reflectances at 0.807 µm for a 1.5×1.5 km² selection of an ASTER image recorded on 5 February 2020 at 14:25:15 UTC. (a) The full physical range of reflectance values ranging from 0 to 1 with overlaid blue contours outlining the ASTER cloud mask. (b) Similar to (a) but with the color scale limited to the 10th and 90th percentile of reflectances attributed to total cloud cover including optically thin clouds. The red contours correspond to $p\left(F_{\mathrm{TOTAL}\mathrm{\_}\mathrm{CLOUD}}\right|R=R_{\mathrm{pixel}})\ge \mathrm{0.9}$.

Download

4.2 The OTC equivalence in lidar data

In Fig. 5 optically thin clouds are barely visible in the reflectance field in panel (a), suggesting that those clouds have a very low cloud optical thickness. Due to nonlinearities in the physical and radiative properties of small cumulus clouds and the large influence of 3D radiative effects, plane-parallel retrievals of microphysical properties do not work reliably and we cannot derive cloud optical thickness from ASTER measurements directly (Davies, 1978; Loeb et al., 1997; Várnai and Marshak, 2003; Marshak et al., 2006; Stevens et al., 2019; Kölling, 2020). However, we use the theoretical relationships that plane-parallel retrievals are based on to estimate an effective cloud optical thickness that could be detected by ASTER against the ocean surface background following the two-stream approximation by Lacis and Hansen (1974):

with the cloud albedo A, cloud optical thickness τ, and the asymmetry parameter g=0.85. In Fig. 6 we show the relationship stated in Eq. (9) of a plane-parallel cloud (black line) and add uncertainties from cloud 3D effects and the background ocean signal.

The average ocean reflectance during EUREC⁴A was 0.04 including single cases as high as 0.08. Due to additional variability in the ocean wave reflection we expect clouds with an albedo below 0.1 and corresponding cloud optical thickness below 1 to dissolve in the ocean signal. For clouds with cloud optical thickness larger than 1, 3D effects such as brightening and shadowing as well as photon loss through the cloud sides become relevant and can easily cause a factor of 2 error in the reflectance that spans a distribution around the plane-parallel estimate and that we indicate by the grey shaded area in Fig. 6 (Marshak et al., 2006; Stevens et al., 2019). Overall, we assume that due to natural variability in the background ocean signal and the cloud signal, clouds with optical thickness below 1 likely do not stand out from the ocean and the ASTER cloud mask is presumably insensitive to such optically thin clouds.

Figure 6Plane-parallel relationship between cloud albedo and cloud optical thickness following Lacis and Hansen (1974). The ocean reflectance is estimated from the ASTER observations during EUREC⁴A, while the uncertainty due to 3D radiative effects is a rough estimate from the literature (Marshak et al., 2006; Stevens et al., 2019).

Clouds with an optical thickness below 1 are thin enough for a lidar beam to penetrate through the cloud and provide a reliable estimate of the cloud optical thickness. We can therefore make use of WALES lidar measurements for supporting information on the abundance of optically thin clouds.

Figure 7 shows the distribution of cloud optical thickness measurements from WALES for days with local research flights. The peak at low cloud optical thickness values corresponds to optically thin clouds that the lidar beam manages to penetrate. A cloud with optical thickness of about 2.5 reduces the lidar signal below the cloud to more than $\mathrm{1}/\mathrm{100}$, and the method to derive the optical thickness still works. At night the range of retrieved optical thickness increases to about 3.5 due to a better signal-to-noise ratio above clouds without scattered sunlight. In thicker clouds the signal vanishes in the system noise. We aggregate all measurements from optically opaque and thick clouds in one bin as we have no information on the actual cloud optical thickness.

In WALES measurements we associate optically thin clouds with an optical thickness below 1. The campaign average cloud optical thickness of OTC is 0.37, and the median is 0.31. Optically thin clouds have on average a cloud-top height at 1.3 km of altitude (median 1.0 km). We further use the WALES measurements to derive a fractional cloud cover in time for optically thin clouds and compare the results to the optically thin cloud cover from ASTER in the following section.

Figure 7Cloud optical thickness distribution from WALES lidar measurements for all days with local research flights during EUREC⁴A resulting in 92 h of data. Panel (a) shows the frequency distribution of all days, while panel (b) additionally shows the cumulative distributions for individual days. The days are sorted by their increasing average cloud optical thickness that we associate with optically thin clouds (yellow to dark green). The split x axis visualizes the limited information on thick clouds that are optically opaque to the lidar.

Download

Table 2Cloud cover estimates during EUREC⁴A from 380 ASTER satellite observations (60×60 km²) at 15 m resolution on 17 d and from WALES lidar measurements recoded within 13 research flights (days) at about 40 m resolution in January and February 2020.

^* “Detected” refers to the ASTER cloud mask and, in the case of WALES data, to clouds with cloud optical thickness ≥1.

Download Print Version | Download XLSX

4.3 The contribution of OTC to the total cloud cover

From analyzing 380 ASTER images during EUREC⁴A we find an average total cloud cover of 42.6 %, combined of 28.5 % from detected clouds and 14.1 % from optically thin clouds (see Table 2). Based on the cloud-free retrieval uncertainties derived in Sect. 3.3 we estimate the uncertainty in ASTER optically thin cloud cover to be within the range of ±2.7 %. In Table 2 we state the respective numbers derived from WALES measurements. We explicitly note that a direct comparison is not reasonable as the two instruments and approaches show optically thin cloud areas from two different perspectives. However, what we can say is that WALES lidar measurements indicate high fractional coverage by optically thin clouds, similar to what we find from ASTER images.

In Sect. 4.1 we mentioned the possible influence of scattering at cloud edges, which can illuminate areas surrounding thicker clouds. Such 3D effects would influence our results based on ASTER data and lead to an overestimation of OTC-related cloud cover. As WALES is less affected by the 3D scattering at cloud edges but shows a higher fraction of optically thin clouds (42.4 %) relative to ASTER (33.1 %), the ASTER analysis does not seem to be unduly influenced by 3D radiative effects.

Our results based on ASTER and WALES measurements are lower compared to an analysis of optically thin marine clouds from CALIOP measurements by Leahy et al. (2012). The authors find a fraction of optically thin clouds in the trades to be as high as 84 %. From WALES measurements we derived an OTC fraction of 42.4 % for cloudy profiles with cloud optical thickness <1. If we include clouds with cloud optical thickness up to about 3, as is done in the study by Leahy et al. (2012), the OTC fraction in WALES data increases to 74 %. Estimates based on CALIOP data are likely to overestimate the OTC fraction due to the lower sensor resolution of 90 m footprints every 335 m. Leahy et al. (2012) derive a possible overestimation of OTC fraction of up to 25 % in the trades due to partially cloudy CALIOP footprints, which supports our findings in the current study of a lower but still significant contribution of optically thin clouds to the total cloud cover.

Figure 8Change in optically thin cloud cover with total cloud cover. The blue markers correspond to values derived from 380 ASTER images (60×60 m²), with the dark blue line following along the median values. The green markers correspond to daily averaged cloud cover estimates from WALES lidar measurements. The grey diagonal line shows the maximum possible contribution of optically thin clouds to the total cloud cover.

Download

We further notice that the area covered by optically thin clouds increases with detected cloud cover for low total cloud cover as shown in Fig. 8 and similarly stated in Leahy et al. (2012). The positive correlation up to 0.4 total cloud cover might be due to a combination of two features. First, optically thin cloud areas are often found surrounding detected clouds (see also Fig. 5). This idea is supported in a study by Koren et al. (2007), who find enhanced reflectances in solar irradiance measurements before and after an identified cloud originating from humidified aerosols and/or unresolved cloud fragments.

The second ingredient to the proposed positive correlation is the cloud field structure. Trade wind cumulus cloud fields at low cloud cover typically correspond to sugar or gravel type structures as described by Stevens et al. (2020), consisting of many small clouds with enough space in between that can be partly filled with undetected optically thin clouds. More clouds and a more cloud boundary therefore lead to more optically thin cloud area up to a point at which this relationship saturates at about 0.4 total cloud cover. The saturation might be due to larger clouds or cloud structures being surrounded by pronounced cloud-free regions. A recent study by Schulz et al. (2021) identifies the so-called flower and fish cloud patterns of having characteristic cloud-free areas between clouds. By constraint, the positive correlation turns negative above 0.7 total cloud cover as the cloud-free, OTC, and detected cloud cover always add up to 1, and high-cloud-mask cloud cover situations leave little space for optically thin clouds.

We conclude that optically thin clouds cover large parts of the trades, leading to a higher total cloud cover than assumed so far from passive satellite observations.

4.4 The cloud reflectance–cloud cover relationship in ASTER observations

Current climate models typically have a narrow range of cloud optical thickness that might affect model perturbation experiments due to the nonlinearity of cloud optical thickness and its albedo. Especially in low-cloud regions such as the trades, climate models underestimate the cloud cover while overestimating its average reflectance, a problem often called the “too few, too bright” low-cloud problem (Nam et al., 2012; Klein et al., 2013). While observations show a positive correlation of cloud cover and cloud reflectance, models show a reverse sign (Konsta et al., 2016).

We investigate the cloud cover–cloud reflectance relationship in Figs. 9 and 10. Figure 9a shows in blue curves the change in all-sky reflectance distribution with increasing cloud cover as defined by the ASTER cloud mask, while the red lines similarly show the change with increasing total cloud cover. We show two representative cloud cover ranges: a low range from 0.1 to 0.3 and a high range from 0.5 to 0.7. With increasing cloud cover, the reflectance distributions shift to higher values, meaning that the overall image is brighter (dashed versus solid lines). As expected, the reflectance distributions as defined by our method (red lines, including optically thin clouds) peak at lower reflectance values compared to their ASTER cloud-mask counterparts, meaning that the total cloud cover area is less bright on average when optically thin clouds are included.

Panel (b) shows an interesting new facet to the difference in total and cloud-mask cloudy areas. The distributions show how the total cloud reflectance relative to the total cloud area in the image depends on cloud cover. The comparison of low- and high-cloud-cover cases reveals that clouds are brighter with increasing cloud cover (dashed versus solid lines), which is in agreement with our perception of larger, deeper, and brighter clouds being present in high-cloud-cover situations. The change in cloud brightness with cloud cover is less pronounced if the total cloud cover is considered (red lines, including optically thin clouds) compared to the cloud-mask-only case (blue lines).

Figure 9Combined probability density functions (PDFs) of (a) all-sky reflectance from ASTER p(R|CC), binned according to the total (red) and cloud-mask (blue) cloud cover (CC). We define two representative cloud cover ranges: low CC (0.1 to 0.3) and high CC (0.5 to 0.7). Panel (b) shows the conditional probability of total cloud reflectance p(R|F_TOTAL, CC), given that they are within the range of low or high CC. Compared to (a), the distributions in panel (b) do not include the cloud-free contributions at low reflectance.

Download

We further investigate the expected cloud reflectance in relation to derived cloud cover values for all 380 ASTER images in Fig. 10. Both cloud-mask and total cloud cover exhibit positive correlations with respective cloud reflectance values, in agreement with findings in Konsta et al. (2016). We derive a campaign average cloud reflectance from total cloud cover of 0.15, with contributions from the ASTER cloud-mask clouds (average: 0.21) and optically thin clouds (average: 0.06), which agrees quite well with an average trade wind cumulus cloud reflectance of 0.15 derived from a combination of POLDER (Polarization and Directionality of the Earth's Reflectances) and CALIOP measurements in the study by Konsta et al. (2016). Based on the cloud-free retrieval uncertainty stated in Sect. 3.3, the uncertainty in expected reflectance of optically thin clouds is as low as 0.0031 and does not influence our results and conclusions drawn here.

The positive correlation in Fig. 10 for total cloud cover agrees well with the corresponding Fig. 6a in Konsta et al. (2016). As mentioned before, climate models show a reverse sign of this correlation together with a general underestimation of cloud cover and simultaneous overestimation of cloud reflectance. Next to the model intrinsic mechanisms leading to clouds that are too few but too bright, biases might be partially due to tuning the model based on traditional cloud masks that overestimate the cloud reflectance, especially in the frequent low-cloud-cover situations.

Figure 10Expected cloud reflectance corresponding to the ASTER cloud mask (blue) and the derived total cloud cover (red) from 380 ASTER images. The median cloud reflectances are given by the lines, and the dataset averages are visualized by the + marker and the respectively colored tick labels. The frequency distributions of cloud cover and cloud reflectance are shown on the top and right, respectively.

Download

5.1 Implication for CRE estimates

In temperature perturbation studies, cloud feedback defines how clouds adjust to a perturbation in surface temperature and whether this change amplifies or dampens the initial temperature perturbation. As such, it is tied to the cloud radiative effect (CRE), which is the difference in all-sky and cloud-free radiative flux at the top of the atmosphere, in the initial and in the perturbed climate.

In the trades, climate models show a less negative CRE in response to warming, indicative of a positive cloud feedback (Zelinka et al., 2020). Observational constraints based on satellite data at coarse resolution might be insensitive to sub-pixel-scale clouds and consequently lack a robust cloud-free signal. From our analysis we can estimate an upper bound on the error in CRE that might arise from a cloud-free signal that is contaminated by undetected optically thin clouds.

If we assume that the pixel reflectances corresponding to optically thin clouds from the present analysis are fully mixed into the cloud-free signal, we would overestimate the cloud-free reflectance and consequently underestimate the CRE. We derive a relative bias ΔCRE per image from the differences in all-sky L_ALL, cloud-free L_CLEAR, and “contaminated” cloud-free L_CLEAR+OTC expected radiance values.

Note that we use the simulated cloud-free L_CLEAR radiances as those do not contain the low radiances from cloud shadows on the ocean surface, which would cause a slight underestimation of the cloud-free radiance.

In principle, a mono-directional radiance L can be converted to a radiative flux F as is done by Clouds and the Earth's Radiant Energy System (CERES) radiative flux products with the following equation (Loeb et al., 2003; Su et al., 2015):

with the Sun θ_s and sensor view θ_v zenith angles, the azimuthal difference Φ, and the anisotropic factor f. The anisotropic factor is challenging to estimate and no suitable values are available for ASTER observations. However, if we assume isotropic scattering of cumulus cloud fields (f=1) we can translate the CRE bias into an effective radiative flux at 0.807 µm.

The mean CRE bias from the ASTER dataset amounts to −7.5 %, which roughly translates to about −2.2 W m⁻² (at 0.807 µm). The order of magnitude is significant and highlights the importance of an improved representation of optically thin clouds in future studies.

5.2 Optically thin clouds in the aerosol–cloud interaction context

First, we would like to revisit and confirm our distinction of aerosols and optically thin clouds from the Introduction to this article. We consider humidified aerosols to be part of the cloud-free signal. As both ASTER and WALES data suggest a total cloud cover well below 100 % (insensitive to the exact cloud threshold in WALES) we are confident that the described signal of optically thin clouds can only be due to anomalously humidified aerosols and cloud droplets. However, we do see a possibility that fossil clouds, in the form of lingering pockets of humidified aerosol, might be classified as optically thin clouds, too. We think the WALES analysis and the magnitude of the observed optical depths (from WALES) exclude this as a major contributor. Even if this inference was incorrect, we believe it would be more correct to think of cloud fossils as optically thin (and fading) clouds than as an aerosol signal, particularly since such signals will not scale with aerosol amount. We therefore discuss possible implications of undetected optically thin clouds for aerosol–cloud interaction studies in the following.

Aerosol–cloud interaction studies represent a topic in itself, and we will not go into great detail but rather want to show where optically thin clouds might need to be considered in these studies. One largely debated issue is the positive correlation of AOD and cloud cover as an indirect aerosol effect. The underlying principle is that hydrophilic aerosols can serve as cloud condensation nuclei and increase the cloud droplet number concentration. More aerosols might therefore reduce the precipitation formation rate and increase the cloud liquid water content and cloud lifetime (Albrecht, 1989). Whether this so-called cloud lifetime effect actually leads to increased cloud cover is largely debated (Loeb and Manalo-Smith, 2005; Kaufman et al., 2005; Stevens and Feingold, 2009; Gryspeerdt et al., 2016).

Some modeling studies suggest negligible or equally small enhancing or decreasing influences of aerosols on the cloud cover (Xue and Feingold, 2006; Quaas et al., 2008; Seifert et al., 2015), while others suggest a considerable effect (Quaas et al., 2009). Observational studies, on the other hand, mostly rely on coarse satellite observations and show deficiencies in the accuracy of aerosol and cloud retrievals as discussed in Quaas et al. (2020). The positive correlation in optically thin cloud cover and detected clouds in the current study suggests that some of the proposed sensitivity of cloud cover to AOD might reflect a high bias in cloud-free estimates that is interpreted as high AOD. In agreement with our perception, an observational study by Gryspeerdt et al. (2016) estimates meteorological covariations to account for 80 % of the often proposed AOD–cloud cover relationship with the additional note on shallow cumulus regions having a very weak relationship.

Independent of the cloud lifetime effect, a positive perturbation in aerosols increases the cloud droplet number concentration and thus the cloud brightness, which is commonly referred to as the Twomey effect (Twomey, 1959; Quaas et al., 2020). Increasing the brightness also increases the probability of undetected and optically thin clouds identified in the current study to cross the detection threshold of common cloud-masking schemes. We therefore speculate that the Twomey effect indirectly leads to positive AOD–cloud cover relationships found in previous studies. It might be interesting to investigate the AOD–cloud cover relationship based on a more comprehensive definition of total cloud cover including optically thin clouds.