- SPECIAL REPORTS
- THE MAGAZINE
The ability to capture, process and use aerial visual intelligence in either realtime or near realtime has become necessary. Gone are the days of waiting weeks or months for georeferenced images for GIS. Digital camera technology and onboard GPS have significantly reduced the processing time. In some cases, geospatial imagery and full motion video (FMV) can be viewed streaming in real time within the GIS. This has moved aerial visual intelligence from its traditional role as merely a basemap to use as an operational layer for GIS.
High-resolution orthophotography used to represent imagery of perhaps 1 meter pixel resolution or a little better. As cameras advanced in resolution and the ability to handle these larger datasets improved, ground pixel resolutions increased down to 1 foot and even a few inches in the best cases.
While camera technology has come a long way, so have the airborne platforms and corresponding flight characteristics. Traditional aerial photography has been captured at altitudes of up to several thousand feet by manned aircraft. Unmanned airborne systems (UAS, also known as unmanned aerial vehicles or UAVs) can safely fly at very low altitudes and much lower speeds, providing imagery with incredible detail--up to 2 centimeters ground pixel resolution.
In addition to high-resolution visual content, many of today’s projects require three-dimensional imagery. Both airborne and ground-based LiDAR have become standard methods of capturing 3D data. Imagery can be post-processed using this 3D information for better positional accuracy and orthorectification. It can also be used as a framework for the creation of stunning perspective views.
3D point clouds (similar to LiDAR) can be captured by small UAVs. Depending upon the altitude of the platform and the speed of the flight, some data that surpasses the capabilities of airborne LiDAR can be quickly and inexpensively captured. In fact, due to the flight characteristics of UAVs, some data that would not normally be captured in a traditional manned LiDAR mission can be seen in the point clouds. This includes areas underneath overpasses, bridges and other structures. While ground-based LiDAR could capture these features, it is not always feasible or cost-effective. For example, locations such as agricultural fields and fragile wetlands may not be accessible to ground-based vehicles, but they would be reachable by a UAV. Some archaeologists are even mapping the 3D detail of faded petroglyphs with low-flying UAV platforms, showing detail not previously seen.
Multispectral image processing for agriculture, wetlands and other applications can be performed using UAV-collected data. Due to the reduced expense of flying UAV platforms, this data can be captured at several stages of the growing season. One recent example is the use of UAV-collected multispectral imagery and 3D point cloud data for vineyard yield analysis in the Texas hill country. John Klier from Texas State University has been building and operating UAVs for GIS and remote sensing research for several years. “While it may lack the flight time and payload capacity of a manned platform, the UAV does have many advantages for smaller projects,” he says. “A UAV can be deployed quickly and inexpensively and can gather data that is at least as good as (and) sometimes better than manned platforms.”
The detail of the imagery and point cloud data are so fine that the vegetation density of the vines can be seen. Repeatability allows for ongoing analysis through the entire growing season so that any adjustments that may be needed in the irrigation, for example, can be made on-the-fly.
In the U.S., such applications remain uncommercialized due to Federal Aviation Administration restrictions; only government agencies and educational institutions with a certificate of authorization (COA) can fly UAVs. In regions of the world where flight restrictions are less stringent, UAV data capture has already become standard fare. One geospatial data provider that has experienced explosive growth overseas is Hawkeye UAV, based in New Zealand. (The company’s AeroHawk UAV was featured prominently on stage with Jack Dangermond during the opening session of the 2012 Esri International User Conference in San Diego.) “UAVs being used for aerial survey purposes have a disruptive impact to the geospatial market in that they have the potential to enable people to control a significant proportion of their aerial imaging needs, especially for smaller areas that have been economically unviable for traditional capture,” says Rowland Harrison, president of Hawkeye. “This enables change detection on a timelier basis for most economic activities.”
Since UAVs are capable of capturing very high-resolution 3D data for GIS, the applications of this technology also include volumetric analysis for applications such as mining, landfills, reservoir construction, flood damage assessments and other projects in GIS. The ability to provide quick turnaround of quality datasets and to affordably repeat the process throughout the lifecycle of a project is revolutionary.
Adding FMV to a GIS provides another layer of intelligence. The airborne video (infrared or natural color) is captured in a data format that includes metadata about the aircraft’s altitude, attitude, GPS location and the corresponding orientation information about the camera sensor. Given some basic geometric parameters, geospatial software can properly georeference and display the imagery on the GIS map in up to near real time (1-2 seconds delay for processing and transmission of the data).
This technique has been used by the military for UAV-captured FMV for several years from a variety of platforms, ranging from small hand-launched UAVs all the way up to the Predator and Global Hawk. “GIS applications represent a unique opportunity for the transition of FMV tools from military to commercial uses,” says SAIC Program Manager and U.S. Air Force Reserve Maj. Joseph P. Ferfolia. “SAIC developed our RAVEPro Apps for the defense community in order to enhance the FMV consumer’s experience by processing the video stream in near real time, generating a stabilized video feed and precisely georegistering the video frames on a GIS background map. Mosaics of user-designated segments of video can be saved and used as an updated base map.” This same video processing technology is being introduced to the civilian market for GIS.
Image stabilization is especially important for video captured by a small UAV because of the inherently less stable platform that is involved. Smaller UAVs cannot carry the heavier, more expensive gyrostabilized camera sensors that manned aircraft can carry, so the imagery must be stabilized digitally in order to make it more useful in GIS (and to avoid motion sickness when viewing it).
Esri recently released its FMV Toolbar for ArcGIS, which allows FMV to be directly incorporated into the GIS. This allows for the overlay of the flight lines and footprints of the video frames to be viewed on the map, including those from multiple airborne platforms. The ability to view the video through a basic DVD-style player lets the GIS operator navigate through prerecorded video.
Perhaps the most powerful capability that FMV brings to GIS is the ability to capture features directly from the georeferenced FMV player window. For example, a GIS analyst can play a video that was recorded along a highway, pipeline or electrical transmission line and pause the action at any time to capture features of interest. Those features are stored directly in the GIS. This heads-up digitizing method has been used on static orthoimagery for many years; now the same can be done with FMV.
So not only is FMV a source of visual intelligence, it is a revolutionary new data capture method for GIS. Just as GPS moved from its original role as a military navigation tool to a civilian mapping tool, FMV has the potential to become a standard data capture technique for GIS.
One of the aspects of FMV is that the camera need not be perpendicular to the ground. For example, an aircraft can be flown at a safe distance from a wildfire while capturing FMV at an oblique angle that shows the scene to viewers in an Incident Command Post (ICP) miles away in real time. This greatly enhances the common operational picture (COP). Whether dealing with wildfires, storm damage assessments or oil spills, having a source of real time visual intelligence for the COP is a game-changer.
One example of how FMV can enhance decision-making comes from the 2010 Deepwater Horizon oil spill in the Gulf of Mexico. During the early days of the response, the ICP relied on daily overflights which captured aerial imagery of the oil on the water. The output from that system required extensive post-processing and hand digitizing, which cost valuable time and, perhaps more importantly, the dedication of a GIS analyst in order to georeference and digitize the resulting bitmaps on a daily basis. The turnaround time for this critical piece of information (“where is the oil?”) was half a day at best. Obviously, by that time, the oil had moved. Using FMV from manned or unmanned aircraft, the same information could have been processed and viewed in near real time.
Both manned and unmanned aerial platforms can be used for the capture of all the necessary data types. The project characteristics dictate which aerial platform is best suited for a given application. UAVs provide higher-resolution of small areas of interest (AOIs) and are typically operated within line-of-sight of the ground station due to air safety considerations.
The U.S. federal government and many states are debating and implementing rules for the use of UAVs. Recent media attention has focused on the public perception of privacy issues. The idea that “drones” could be lingering overhead and spying on individuals has many running scared and is hindering the legitimate use of UAVs for mapping and surveying applications. The same arguments have been made in the past about privacy concerns from traditional aerial photography.
For now, the FAA flight restrictions are effectively suspending large-scale commercial UAV use while these privacy issues and some legitimate air safety issues are being worked out. It is likely that future rules and regulations will include some form of competency testing and standard collision avoidance policies and procedures. That being said, UAVs are in flight daily in the U.S., performing GIS data capture. Many of these projects involve university research for a variety of applications.
Working with the FAA and airspace restrictions requires a level of expertise that traditional pilots are accustomed to. Paul Morgan, a former naval aviator and CEO of Tactical Systems Engineering explains: “We work with the FAA to ensure legal and appropriate access into the national airspace for small, commercial unmanned systems. We also engage with academic institutions, state and local governments to attain the certification required to fly unmanned systems.”
Beyond the regulatory considerations, there is the practical consideration of the risk and cost of crashing a UAV--both to the operator and anyone or anything that may be damaged on the ground. So flying UAVs is not the sort of thing that anyone can get into after a visit to the local hobby shop or by ordering parts online. While there are many inexpensive cameras (both still and HD video) and several capable remote-controlled aircraft available, that is not all that is involved. Future operators should be prepared to undergo substantial training before incorporating UAVs into their data capture toolbox.
A substantial amount of post-processing and data management is also needed to turn the hundreds or even thousands of digital images or HD video into a meaningful dataset for GIS. Many of the pioneers using UAVs for aerial imagery capture found that the volume of images collected, and the need to georeference and catalog them quickly, made the technology less-than-useful during a time-critical incident. The volume of data can be overwhelming.
Professional data capture and processing applications that are used in photogrammetry and FMV can help deal with the “big data” issue; however, it comes at a cost that many users may not be willing or able to pay. For this reason, many GIS users will be ordering and receiving data deliveries from service providers that take on that overhead cost, just as they do now with traditional imagery.
Realtime and near realtime aerial visual intelligence is a disruptive technology for GIS. The technology transfer of UAVs and FMV from the military to commercial applications is similar to the emergence of GPS as a tool for mapping during the 1990s.
The affordability of UAV-collected imagery and FMV capture from manned and unmanned platforms allows for routine updates and/or ongoing monitoring of disasters and everyday asset management.
The rapid turnaround time, including the ability to provide streaming in real time, provides better temporal accuracy for data and better decision-making by GIS analysts and management. This means that aerial imagery has moved from its traditional use as a GIS basemap to that of an operational layer. Opportunities abound for everyone involved in the process, from data capture and processing to analysis and application.
|The Future of Manned Aircraft|
Manned platforms are more cost effective and capable for larger projects of increased duration, such as capture of long pipelines, electrical transmission lines or damage assessments of hurricanes, winter storms and wildfires.
Due to restrictions on UAVs, the manned platform will be a mainstay for a long time. One thing that has changed with the manned platform is the ability to fly multiple sensors at once, such as digital orthos and LiDAR. For projects that are extensive in area or length, manned platforms hold the advantage.