Sierra Overhead Analytics presented at the Solar Power International Conference in 2018 on the topic of remote sensing use for construction progress monitoring on large solar projects. See below for the information on our study and a link to the poster.
Automating Solar Construction Site Inventory Analysis via Drones, Machine Learning, and Computer Vision
Author: Owen Ransom, PhD. Founder, Sierra Overhead Analytics.
Abstract
Recent developments in computer vision and machine learning have been applied to the problem of inventory analysis. Methods were developed and applied to a high resolution orthomosiac of images taken at a California based project. The image was scaled and geo rectified to RTK-GPS measured ground control points so that all objects could be identified and located within the job site. These methods have applications well beyond inventory analysis.
Introduction
In 2018, SOA has begun to incorporate machine learning and computer vision to simplify historically tedious and complicated tasks. Recently, we’ve partnered with EPCs, developers, and project owners to streamline in situ inventory analysis through utilization of high resolution georectified real time site photographs which can be deconstructed and analyzed through use of computer vision algorithms.
For the case presented here, SOA used this method for a 5MW project section, (approximately 25 acres) at an active solar construction site in Central California. Prior to the survey, a detailed examination and documentation of desired materials was generated in conjunction with the client. A sUAS with dual imaging sensors was utilized to conduct an examination of the target sites, and all objects in it.
Data was processed using a combination of commercial and proprietary methods. The resultant dataset was CM level accurate with an approximate ground sample distance of 1cm.
Methods and Materials
SOA utilized a DJI Matrice 210 with a high resolution sensor to gather the visible light photographs. Ground control was established using RTK-GPS on the boundaries of the test site. The site was flown at a height of 60 meters and required approximately 10 minutes of flight time to gather all aerial data.
Approximately 600 photos were geo rectified and merged using proprietary and commercial softwares to form an overall site image with a ground sample distance (GSD) of approximately 0.8cm , then automated and manual counts were undertaken to assess the number of objects in the visual dataset.
Using computational vision and machine learning methods we were able to determine both object type and counts, automatically. The high resolution image was detailed enough to give precise object dimensions as well as locations. Edge and feature detection allowed for differentiation between full and partially depleted stacks of items, as well as determine differences between box and H pile. Inserted pile could be assessed for installation procedure, including row alignment.
Results and Discussion
SOA was able to differentiate between each type of object based on size, appearance, and dimension. Imaged based automated and manual counts were delivered to the client, whose manual ground based survey verified the final result of the process.
The resultant count was over 99.9% accurate to the verified on site number. SOA believes this technology is a strong potential replacement for manned inventory analysis and is scalable to cover the needs of a wide range of project sizes.
Automated detection and analysis will improve as more data sets are used to train the machine learning model, leading to the ability to analyze and inspect up to 25 MW per day per two man team.
Conclusion
Drone based aerial imagery, when combined with computer vision and machine learning analysis of processed data presents a strong argument for the automation of construction site inventory analysis.
Additionally, we feel these methods can be applied to other problems or analyses. The use of UAS based remote sensing allows for a higher resolution dataset, providing a more robust analysis differentiation between features , and allowing for detection of smaller features than before.
Object detection and differentiation will continue to improve as more sites are analyzed and processed and reliance on transfer based learning is reduced. SOA’s feature extraction tools and image processing toolboxes can be applied to a wide variety of problems beyond what is discussed here.