Sky segmentation is a part of the scene parsing world in which algorithms seek to label or identify objects in images. However, due to being trained on ideal datasets, these algorithms face difficulty in non-ideal conditions 
. As deep learning methods might become more involved in real world applications, it becomes apparent that off-the-shelf methods are not always effective and reliability comes into question. Inspired by Mihail et al., who compared existing sky segmentation methods and sought to bring attention to the problem of ideal datasets, we focused upon approaching the challenges they mentioned through existing models. The challenges of outdoor scene parsing lie in the time of day, year, and varying weather conditions. Their SkyFinder dataset allowed us to pursue these challenges in order to obtain improved results. This work highlights the importance of this problem. We offer an improved model that will aid in challenging sky segmentation in the real world. In this work, we adopted state-of-the-art segmentation model  for the task of pixel level detection of the sky. Our task is different than semantic segmentation in a sense that we are only interested in one object i.e. sky. Sky, unlike other objects can be difficult to segment due to poor lightning (night time) and weather conditions where even humans are likely to fail (e.g., over dense fog, thunder storms, etc). Thus, we attempt to address this problem in this work.
Our contribution is as following. First, we evaluated an off-the-shelf state-of-the-art model  to demonstrate that existing models fail for different weather conditions, times of day, and other transient attributes . Second, we fine-tuned RefineNet-Res101-Cityscapes model on the SkyFinder dataset and obtained 12.26% improvement over off-the-shelf model in terms of misclassification rate (MCR). Third, taking advantage of the existing big dataset, we trained  solely on SkyFinder, which further improved accuracy and outperformed all baseline methods. Fourth, to study across datasets performance of our trained models, we selected a subset from SUN  dataset with the ’sky’ label. We then both fine-tuned and trained RefineNet-Res101 model on this subset of SUN dataset (we refer to it as the SUN-Sky dataset in this paper), and perform evaluation across models trained on both SkyFinder and SUN-Sky datasets, and report the results. Fifth, following , we investigate the effect of times of day and weather conditions on performance of our model and transient attributes. Sixth, we compare our analysis with  in terms of MCR and report impact of weather and times of day w.r.t mIOU scores. Finally, we determine the impact of noisy images like motion blur, Gaussian noise, etcetera on our model’s performance and report our results on robustness of our approach.
The rest of this paper is organized as follows. In section 2, we present a brief overview of existing works in this area. In section 3, we describe details of our plan including utilized datasets, and our models. Section 4 discusses the performance metrics we used to evaluate the trained models. Section 5 presents experimental findings on Sky segmentation task for both datasets and analysis of different attributes, times of day, weather conditions and noise on this task, followed by discussions in section 6.
2 Related Work
The history of scene parsing i.e., assigning each pixel of input image to one of the object labels , has evolved from its beginnings. Scene labeling methods , , , ,  mainly use local appearance information for objects being learned from training data 
. Although, previously this task has been addressed by using hand engineered features with a classifier, recent methods address this task by learning features using deep neural networks. From convolutional networks spawned fully convolutional networks which moved away from pixel level algorithms to whole-image-at-a-time methods. Afterward, the introduction of skip layers led to deep residual learning . Recently, residual nets became the backbone to scene parsing algorithms such as RefineNet , PSPNet , and more.
Specifically, the sky segmentation task can be helpful for a diverse variety of applications such as stylizing images using sky replacement , obstacle avoidance , and  since sky tells a lot about weather conditions. Current applications of sky segmentation range from personal to public use and more. Often times they are used in scene parsing [5, 17]
, horizon estimation, and geolocation. Other applications include weather classification, image editing [7, 15], weather estimation [21, 14], and more.  and  worked on weather recognition and used camera as weather sensors. Weather detection has also been used for image searching  where one can search outdoor scene images based on weather related attributes.
Dev used color based methods  for segmentation of sky/cloud images, whereas  proposed deep learning approach for segmenting sky. Like , we used the same baseline methods (Hoeim , Tighe  and Lu 
) to compare our results. Hoiem uses a single image to produce an estimate of scene geometry for three classes: ground, sky and vertical regions by learning underlying geometry of an image via appearance-based models. Tighe combines region-level features with SVM-based sliding window detectors for parsing an image into different class labels including ’sky’. Lu classifies an input image into two classes: sunny or cloudy. Their work uses sky detection as an important cue for weather classification. For detecting sky, they used random forests to produce seed patches for sky and non-sky, and then used graph cut to segment sky regions. uses their sky detector for reporting results which we also used for our comparisons on the SkyFinder dataset. Here we investigate the effectiveness of existing state-of-the-art segmentation methods for this specific problem. To select among the best contenders for this task, we evaluated off-the-shelf RefineNet  and PSPNet  methods on SkyFinder dataset, and chose  for our further experiments as it outperforms  with a large margin on SkyFinder dataset. We further take a deep insight on challenges (such as weather conditions, night time images, and noisy images) which are faced even when robust methods are used.
|Method||Split 1||Split 2||Split 3||Avg.|
Our approach is based on adopting semantic segmentation methods for the task of pixel-level sky detection. We used two datasets: SkyFinder  and SUN-Sky  for our work. Our approach outperforms baseline methods in the task of sky segmentation. We evaluated generalization capability of our models by performing across datasets evaluation. Also, we studied influence of various factors like transient attributes, weather conditions and noise on our model’s performance.
3.1.1 SkyFinder dataset
The SkyFinder dataset is a subset of the Archive of Many Outdoor Scenes (AMOS) dataset. Due to the full SkyFinder dataset not being available, we used 45 of the 53 cameras shared. This entailed 60K-70K images, with an average of 1500 images per camera. These images were of varying dimensions, quality, time of day, season, and weather. However, some cameras contained images which indicated the camera was experiencing technical difficulties or repairs. These few images were removed to focus on the challenges we wished to address. We then changed the sizes of the images to introduce a form of uniformity and make test evaluation faster by resizing them to be within the ranges of .
A single segmentation map was associated with each camera. This is due to each camera being stationary for at least a year in order to be included within in the dataset. See figure 1 for sample images from this dataset.
3.1.2 SUN dataset
The Scene UNderstanding dataset is comprised of a multitude of different scenes and the objects that make up said scenes. It is constantly being updated through community effort, thus adding to it’s already large nature. We primarily looked at this dataset for its ”sky” object classification which allowed us to begin comparing the SkyFinder dataset and results. Unlike SkyFinder, images are not grouped by camera, instead they are grouped by scene such as airport terminal, or church. Also, SkyFinder focused on images taken from stationary cameras, whereas SUN has a variety of images from a variety of viewpoints. For the purposes of this research, we focused on the subsection of the SUN database that had the object ”sky” labeled (about 20,000 images). We resized those images to be within the range of for improved test evaluation speed. In what follows, we refer to this subset as the SUN-Sky dataset. See figure 2 for few sample images from this dataset.
Performance results of SUNdb Finetune and SUNdb ImageNet.
3.2 Sky Segmentation
The model we used, RefineNet, was created by Lin . This model seeks to to retain detail throughout the reconstruction of the image and its segmented output unlike its predecessor and backbone, ResNet.
3.2.2 Off-the-Shelf RefineNet
In order to establish a baseline in RefineNet, we used RefineNet’s Res101 model that was trained on Cityscapes, a dataset of European cities for scene segmentation. The Cityscapes dataset is an ideal dataset for the Sky class, and as a result does well in ideal conditions in the SkyFinder dataset. After setting up the model and using it on a single Titan X GPU, we ran each of the 45 cameras through the model and evaluated solely on their sky classification ability, i.e. all other classes were construed as non-sky. We refer this baseline model as RefineNet-Res101-Cityscapes in the following.
To obtain proof of concept prior to running the entire dataset we focused on a smaller subset of the dataset. We took 10 random cameras and broke it into a train-val-test split. From each camera we took 75 images for training, 25 images for validation, and between 175-300 images for testing and evaluation.
Following the success of fine-tuning the model on the subset, we fine-tuned the RefineNet-Res101-Cityscapes model on the SkyFinder dataset. Being unable to find the same train-val-test split as Mihail , we split the dataset into our own train-val-test splits. Although, to keep our experiments as much consistent as possible to 
, we used the same number of test cameras in our experiments. Hence, our split consisted of 13 cameras used for testing, 4 cameras used for validation, and the remaining cameras used for training. We then shuffled the cameras in each section but kept the same number of cameras for each training, validation, and testing set, and repeated the aforementioned fine-tuning process two more times for a total of three fine-tuning trials. We used a learning rate of 5e-5 for each instance and trained the model for 10 epochs. After 10 epochs, the validation accuracy started leveling out. Thus, for consistency, we report all results on model trained till 10 epochs. We refer our fine-tuned models as RefineNet-Res101-SkyFinder-FT in our results.
3.2.4 Training with SkyFinder dataset
Finally, to take advantage of the big size of the dataset, we trained RefineNet-Res101 from scratch, where Res101 was initialized with pre-trained ImageNet model but trained the RefineNet on solely the SkyFinder dataset. We used the same three train-val-test splits as mentioned above (to allow for fair comparison) and trained at a learning rate of 5e-4 for over 10 epochs. Due to the flattening of the learning curve, we used the model at epoch 10 for testing in both instances. We refer to these models as RefineNet-Res101-SkyFinder in our results.
3.2.5 Training with Sun-Sky dataset
We broke down the sky-labeled sub-dataset of the SUN dataset into a randomly shuffled 60-20-20 split for our uses. After resizing the images to to train and test quickly, we then generated the ground truth segmentation masks by keeping only the sky class and treating any other classes as non-sky. To train and evaluate, we focused on a similar process as we did with the SkyFinder dataset. Training entails fine-tuning from the RefineNet-Res101-Cityscapes model, and training on a model initialized from ImageNet-Res101. Evaluation is comprised of calculating the average MCR and mIOU of the dataset. We finetuned RefineNet-Res101-Cityscapes model using the SUN-Sky dataset for 10 epochs at a learning rate of 5e-4, and subsequently 10 more epochs at a learning rate of 5e-5. For our second model, we initialized from ImageNet-Res101 and trained using the same split as we did for the previous model for 10 epochs at a learning rate of 5e-4. Much like the previous model, we again trained another 10 epochs at a lower learning rate of 5e-5.
Table 2 shows quantitative results for SUN-Sky dataset on both fine-tuned model (RefineNet-Res101-SUNdb-FT) and trained model (RefineNet-Res101-SUNdb). See fig. 14 for qualitative results on this dataset.
|Time of day||Split 1||Split 2||Split 3||Avg.|
To evaluate the accuracy of our results we used both the misclassification rate (MCR) defined in Mihail , and the mean intersection over union (mIOU). The use of both of these metrics allowed for the ability to compare our results to the results found in Mihail , which focused primarily on MCR results, and determine the overlapping accuracy of the segmentation outputs.
In this section, we discuss our experiments for sky segmentation, and studying impact of different conditions like times of day, weather situation, transient attributes and noise on performance of our model. Please note that, although  have done a similar study, we extend their work and also report mIOU in our experiments.
5.1 SkyFinder dataset
Since we evaluated RefineNet on SkyFinder dataset in three settings: off-the-shelf RefineNet, fine-tuned, and trained from the scratch, for reference, we use RefineNet-Res101-Cityscapes for off-the-shelf RefineNet model, initialized from Res101 and trained on Cityscapes. We then further fine-tuned same off-the-shelf model on SkyFinder dataset for all three splits and refer to the model as RefineNet-Res101-SkyFinder-FT. RefineNet when initialized from ImageNet pre-trained Res101 and trained on SkyFinder from the scratch is referred as RefineNet-Res101-SkyFinder. We find that RefineNet-Res101-SkyFinder outperforms the fine-tuned model which is clearly because the former takes an advantage of the large size of this dataset. For comparison with other baseline models (Hoiem , Lu , or Tighe ) mentioned in , we used the MCR scores for all three test splits and also report the average performance for all methods in table 1. To compute MCR scores for our baselines on our test sets, we fetch the MCR score from metadata provided with SkyFinder dataset after they evaluated these models.
For results in table 1, we evaluated RefineNet at test scale of 0.8 (default setting), which performs better than being evaluated at full scale (scale = 1.0). Please refer to tables 1 and 3 for comparison. We find that RefineNet-Res101-SkyFinder outperforms all baseline methods both in terms of mIOU and MCR scores. For qualitative evaluation, the images are selected from a few of the first test set of cameras and do not include visual results from Hoiem , Lu , or Tighe . While Mihail created their own ensemble using the combination of the three methods, their results were unreported for individual images.
Analysis on time of day and weather has been performed with full scale test evaluation.
5.1.1 Performance on Camera 10917
First, some background on SkyFinder’s Camera 10917: this specific camera is of a location in which the sky does not peek through anywhere. It depicts a quaint village and trees, but no sky. When SkyFinder was trained on the first two splits it did not contain this camera in its training data. As a result we consistently witnessed abysmal IOU values of 0 which results in performance degradation for test splits 1 and 2 because the model has not previously seen images with no sky. However, the model performs decent in terms of MCR values (below 10%). We believe that the IOU evaluation method in a case such as this is inefficient. Therefore we pay particular attention to the MCR in regards to this camera. For test split 3, this camera has been used for training our models, thus model performs better on the test set.
5.1.2 Performance for night time
Fig 3 shows visual improvement in results for night time images when compared with off the shelf RefineNet-Res101-Cityscapes and RefineNet-Res101-SkyFinder fine-tuned, which makes it obvious why our trained models win over the baseline methods in terms of MCR as well.
5.1.3 Performance for weather obscured images
In section 5.1.5, our results suggests that sky segmentation in night time is more challenging than during day hours. But, our final trained model on SkyFinder still improves in terms of mIOU over our established baseline models (RefineNet-Res101-Cityscapes and RefineNet-Res101-SkyFinder-FT). Fig. 4 shows that despite of images obscured due to dense fog, RefineNet-Res101-SkyFinder was able to perform reasonably well.
|Weather||Split 1||Split 2||Split 3||Avg.|
5.1.4 Performance analysis for different day times
We also wanted to see how the trained model performed during different times of day, similar to . As we use three test splits for reporting our results, we sorted each of them w.r.t hour of the day. Then for each sorted split, we evaluate the respective model on its test set and compute mIOU and MCR. We then calculate average over them (see table 3). Please note that while evaluating RefineNet-Res101-SkyFinder for each of its respective test set, we used scale=1.0 (i.e. full scale at test time) and report the numbers. We find similar pattern as discussed in  for RefineNet as well i.e., model achieves good performance during day time in terms of MCR, and performance decreases during start(early morning) and end of the day(night time), thus MCR increases. In terms of mIOU, we witness a decrease in performance towards the end of the day, but overall the performance seems consistent. Fig 5 shows illustration for mIOU during different day hours. For comparing RefineNet with the results in the baseline paper in this aspect, we fetched MCR scores for each split w.r.t hour and report results provided by . Fig.6 shows that although our trained model follows the same pattern, but our pixel-level sky detector performs significantly better than all three methods.
Please see figure 7 for qualitative results when we tested RefineNet-Res101-SkyFinder on sorted test split w.r.t time. Row 1 shows success cases where row2 shows failure cases for different times of day.
5.1.5 Performance analysis for weather conditions
Like , we aim to evaluate how well semantic segmentation model when trained as a sky classifier performs for different weather conditions. We split the testset with respect to different weather conditions as provided in metadata of SkyFinder dataset and evaluated RefineNet-Res101-SkyFinder on them. We run the same experiment for all three test splits and then compute the average scores for both mIOU and MCR (See table 4 for quantitative results). Looking at the average mIOU scores, we find that our model struggles for few weather conditions like snow and sleet. On the other hand our model performs better even when weather is cloudy, partly cloudy, or foggy. Interestingly, the model is able to perform really well during thunderstorms both in terms of mIOU and MCR.
SkyFinder dataset has some images without labels for weather conditions, we evaluated our model on them and report the results. We observe that mIOU is lower than expected for clear sky. This is due to the reason that a big share of clear sky images (1200+) are from camera 10917 where there is no sky visible, thus it lowers the mIOU at test time for splits 1 and 2. Split 3 has been trained on such images and therefore learns to tell when there is no sky in the image (92.61% mIOU for clear sky).Fig 5 shows mIOU scores averaged over all three splits for RefineNet-Res101-SkyFinder. Fig 8 compares MCR score for weather with different methods, and RefineNet significantly outperforms them. For calculating MCR for baseline methods, we use the MCR scores from SkyFinder metadata. This figure reports the average MCR score over all test splits. Please see figure 9 for qualitative results.
5.1.6 Performance analysis w.r.t transient attributes
Inspired by , we aimed to investigate how much transient attributes affect the performance of our trained model. Hence, we selected images from one of our test splits for four transient attributes: gloomy, clouds, cold and night. We selected images having high presence of these attributes i.e., thresholded above 0.8 and with low chances of their existence i.e., value is less than 0.2. We then tested RefineNet-Res101-SkyFinder on these subsets of images(2 for each attribute : low and high) and report the performance.
As expected, the model performs well for the images where the sky is not gloomy, cloudy, cold or night time. The model is most robust to clouds and performs well(both in terms of mIOU and MCR) even when clouds are largely present in the image. In terms of MCR, model’s performance is worse when we input a highly gloomy image (MCR=10.84) or when it is night time (MCR=10.25). In terms of mIOU, our trained model performs slightly better for gloomy images than night time images (see table 5, last row). We also report MCR scores from baseline methods for comparisons and find that Tighe performs poorly among all of the baselines in high presence of the above mentioned transient attributes. Lu seems robust among the baselines but our method significantly outperforms all of them for this experiment(see fig. 10). Figure 11 shows the histogram of MCR scores when each of these four transient attributes are high and low in the input images. See figure 12 for qualitative results of this experiment.
5.1.7 Performance analysis w.r.t noisy images
We are also interested in investigating the impact of noise on the task of Sky segmentation. We conducted this experiment since the image can be noisy due to various reasons in original settings as well. For this purpose, we selected 50 images for each camera from one of the test splits (total 650 images) and added different types of noise to them. We added motion blur, Gaussian noise, Poisson noise, salt & pepper, and speckle noise to our subset of images. We first evaluated performance without adding any noise and then compared our results with noisy images(see table 6). We find that our trained model is robust to Poisson noise, and highly sensitive to salt & pepper noise. Motion blur also affects mIOU (a drop of approx. 3.5%). Similarly, Gaussian noise and speckle noise also hurt performance (both in terms of mIOU and MCR). See figure 13 for sample images after adding different types of noise to input images.
|Original||Motion Blur||Gaussian||Poisson||Salt & Pepper||Speckle|
|Motion blur||Gaussian||Poisson||Salt & Pepper||Speckle|
5.2 Sun-Sky dataset
The SUN-Sky dataset is a subset of images from SUN dataset  of outdoor scenes having sky present in them. Using a similar approach as we used for training on SkyFinder dataset, we first evaluate RefineNet-Res101-Cityscapes model on SUN-Sky dataset, then fine-tune the same model on this dataset referred as RefineNet-Res101-SUNdb-FT, and lastly, we trained RefineNet-Res101- on SUN-Sky dataset from the scratch (referred to as RefineNet-Res101-SUNdb). We randomly split the images into 60%-20%-20% ratio for training, validation and testing respectively. When evaluated these three models on our test set, we find that off-the-shelf model performs better(mIOU = 61.69, MCR = 8.4) on this dataset as compared to its performance on SkyFinder dataset (mIOU = 48.5, MCR = 18.93), which maintains that SkyFinder is more challenging in nature as compared to SUN-Sky dataset. Interestingly, the fine-tuned model on this dataset outperforms the model when trained solely on SUN-Sky dataset with a slight margin (both in terms of mIOU and MCR) which is mainly because the SUN-Sky dataset is not big enough in size. Overall, we find that both fine-tuned and trained models on this dataset performs reasonably well as compared to off-the-shelf RefineNet-Res101-Cityscapes model (see table 2).
5.3 Cross datasets evaluation
To see generalization power of our models, we evaluated them cross datasets i.e., models trained on SkyFinder were evaluated on SUN-Sky dataset and vice versa. Table 7 shows results for both fine-tuned and trained model on each dataset and across other dataset. Looking at the results, we find that SkyFinder, due to its large size, gives better performance when we trained the model on it from the scratch. Interestingly, our fine-tuned models perform better than trained from scratch when evaluated across datasets. In terms of MCR, fine-tuned model on SkyFinder generalizes better than the model fine-tuned on SUN-Sky dataset(6.67 vs. 11.74). Please note, for SkyFinder dataset, results shown from all models are averaged over all test splits.
6 Discussion and Conclusion
We will first direct our focus to the original results of RefineNet’s res101 model trained on Cityscapes in Table 1. Both results incorporate the pretrained model’s incapability to properly access non-ideal images, particularly night images as addressed earlier. Upon finetuning on the model using the SkyFinder dataset, we note a drastic improvement in the mIOU and just over 10% decrease in the MCR. This is clearly as a result of including the non-ideal images. Upon training it on the SkyFinder dataset, we see a less drastic improvement in one split, but improvement nonetheless.
Despite the averages in Table 1, focusing on a singular camera in one of the data splits, we can see a drastic improvement. Camera 858, consists of 200 images and was not seen by the finetuned or ImageNet initialized model during training. The mIOU from the model trained on Cityscapes was 69.21%, and had an MCR of 18.16%. However, after finetuning the model, the mIOU and MCR results respectively quickly jumped up to 91.41% and 4.69%. Finally, after training on only 28 cameras of the data (again, not including camera 858) and initializing from ImageNet, the results improved slightly. The mIOU increased to 95.76% and the MCR dropped a little more to 2.30%. Other cameras show similar rates of improvement. There are some cameras however that prove difficult to segment for all methods and show a smaller rate of improvement.
The importance of these results lies in the ability to use these models in real-world applications. Using the original pre-trained model would result in poor quality segmentation outside of the ideal circumstances. Off-the-shelf methods must be modified in order to be used most effectively in the real world.
Understanding the impacts of these results, we also incorporate the results from Mihail et al’s findings for their baseline methods. While their own model’s results which reported an MCR of 12.96% across their own testing split has not been used to report the performance in terms of MCR on our test splits. In spite of this, we can still further prove the idea that existing models are still effective–so long as they have been modified to suit the task’s needs.
We also demonstrated that, overall, even state-of-the-art models struggle with challenging conditions like night time, variation in weather and other transient attributes. Although, our trained models still perform much better than prior methods in terms of MCR.
Following this work, we intend to look at other scene parsing models to evaluate their off-the-shelf methods, finetune them, and possibly train them on the SkyFinder dataset in order to compare the results to the above. We also plan to develop our own end-to-end sky segmentation model to also use for comparison. To possibly improve general results, cleaning of the dataset may need to occur, such as removing timestamps. Other future directions include using sky segmentation for applications such as weather classification and weather forecasting. Code, our trained models and data will be made available for further exploration of this area.
-  W.-T. Chu, X.-Y. Zheng, and D.-S. Ding. Camera as weather sensor: Estimating weather information from single images. Journal of Visual Communication and Image Representation, 46:233–249, 2017.
-  G. De Croon, C. De Wagter, B. Remes, and R. Ruijsink. Sky segmentation approach to obstacle avoidance. In Aerospace Conference, 2011 IEEE, pages 1–16. IEEE, 2011.
-  S. Dev, Y. H. Lee, and S. Winkler. Color-based segmentation of sky/cloud images from ground-based cameras. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(1):231–242, 2017.
-  K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition, 2016.
-  D. Hoeiem, A. A. Efros, and M. Hebert. Geometric context from a single image, 2005.
-  D. Hoiem, A. A. Efros, and M. Hebert. Geometric context from a single image. In Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, volume 1, pages 654–661. IEEE, 2005.
-  P.-Y. Laffont, Z. Ren, X. Tao, C. Qian, and J. Hays. Transient attributes for high-level understanding and editing of outdoor scenes, 2014.
-  G. Lin, A. Milan, C. Shen, and I. Reid. Refinenet: Multi-path refinement networks for high-resolution semantic segmentation, 2016.
C. Liu, J. Yuen, and A. Torralba.
Nonparametric scene parsing: Label transfer via dense scene
Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 1972–1979. IEEE, 2009.
-  J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation, 2015.
-  C. Lu, D. Lin, J. Jia, and C. K. Tang. Two-class weather classification, 2014.
-  R. P. Mihail, S. Workman, Z. Bessinger, and N. Jacobs. Sky segmentation in the wild: An empirical study. In Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on, pages 1–6. IEEE, 2016.
-  M. Roser and F. Moosmann. Classification of weather situations on single color images. In Intelligent Vehicles Symposium, 2008 IEEE, pages 798–803. IEEE, 2008.
-  L. Tao, Y. Luo, and X. Zheng. Weather recognition based on images captured by vision system in vehicle, 2009.
-  L. Tao, L. Yuan, and J. Sun. Skyfinder: attribute-based sky image search. In ACM Transactions on Graphics (TOG), volume 28, page 68. ACM, 2009.
-  J. Tighe and S. Lazebnik. Superparsing: scalable nonparametric image parsing with superpixels. Computer Vision–ECCV 2010, pages 352–365, 2010.
-  J. Tighe and S. Lazebnik. Superparsing: scalable nonparametric image parsing with superpixels, 2010.
-  J. Tighe and S. Lazebnik. Finding things: Image parsing with regions and per-exemplar detectors. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3001–3008, 2013.
-  J. Tighe, M. Niethammer, and S. Lazebnik. Scene parsing with object instances and occlusion ordering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3748–3755, 2014.
-  Y.-H. Tsai, X. Shen, Z. Lin, K. Sunkavalli, and M.-H. Yang. Sky is not the limit: Semantic-aware sky replacement. ACM Transactions on Graphics (TOG), 35(4):149, 2016.
-  Wei-TaChu, Xiang-YouZheng, and D.-S. Ding. Camera as weather sensor: Estimating weather information from single images. Journal of Visual Communication and Image Representation, 46(1):233–249, 2017.
-  J. Xiao, J. Hays, K. Ehinger, A. Oliva, and A. Torralba. Sun database: Large-scale scene recognition from abbey to zoo, 2010.
-  J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In Computer vision and pattern recognition (CVPR), 2010 IEEE conference on, pages 3485–3492. IEEE, 2010.
-  X. Yan, Y. Luo, and X. Zheng. Weather recognition based on images captured by vision system in vehicle. Advances in Neural Networks–ISNN 2009, pages 390–398, 2009.
-  A. P. Yazdanpanah, E. E. Regentova, A. K. Mandava, T. Ahmad, and G. Bebis. Sky segmentation by fusing clustering with neural networks. In International Symposium on Visual Computing, pages 663–672. Springer, 2013.
-  H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia. Pyramid scene parsing network, 2017.