Paul Tetley - Spookfish Global Operations
Mark Trotman - ICON Technologies
Nick Perkins - ICON Technologies
Geospatial imagery from satellites and airborne platforms provides the feedstock for a remarkably wide range of commercial, government, and social applications. These range from the management of high-profile, international scale humanitarian and disaster relief programmes to everyday commercial applications that impact us all, often without our even being aware of it. As an example of this latter category, in many countries the business of installing renewable-energy photovoltaics to residential and commercial buildings relies exclusively on high-resolution geospatial imagery to provide an efficient, nonintrusive commercial estimating service.
Current technologies for capturing geospatial imagery are a commercial compromise. Satellites can capture geospatial imagery across entire continents, but at the expense of spatial resolution and the flexibility to image specific areas according to a custom, on-demand timetable. Existing aerial solutions can capture high-resolution imagery on demand, but are cost-prohibitive for repetitive, large-area application.
Australian listed company Spookfish Limited (ASX: SFI) recognised an opportunity to develop a unique airborne platform that produces cost-effective, high-resolution wide-area geospatial imagery on demand, delivering the best features of both satellite and existing aerial solutions from a single airborne platform. As cost-effective on-demand imagery at these scales and resolutions has previously been unavailable, Spookfish expects that this development will ultimately spawn a range of new applications.
Spookfish engaged NI Alliance Partner ICON Technologies to provide the system integration framework for the camera systems, which enable their technology vision.
The Spookfish Airborne Imaging Platform (SAIP) integrates proprietary components and commercial off-the-shelf devices to achieve a combination of image capture rate, imaging geometry, data redundancy, and resolution that has been previously unattainable. It requires that multiple cameras be precisely steered in continuous synchronised motion, with associated recording of camera pointing angles, platform orientation, and acceleration, for use in post-processing the pixel data. At typical operating altitudes and speeds, the cameras must be steered with millidegree precision, and data about camera angles and platform position must be logged with near-microsecond accuracy and precision.
Spookfish used a suite of open-source and proprietary computer-vision and photogrammetry techniques in the processing of the imagery to produce orthophotos, elevation data, and 3D models.
The initial system prototypes, and the first two generations of commercially deployed systems, are based on PXI Express, which acts as master controller for all system functions and provides the framework for timing and synchronising all the system subcomponents. The balance of the SAIP system consists of an externally mounted pod containing one or more camera tubes, each of which can contain up to four cameras. The planned product rollout includes systems with one, two, and three camera tubes.
Figure 1 is a schematic showing the major system components and their associated communications protocols.
The PXI Express system master comprises a PXIe-1082DC chassis, PXIe-8135RT real-time controller, PXI-6683H GPS/PPS synchronisation module, PXIe-6674T timing module, PXIe-7971R FPGA Module for FlexRIO, and PXIe-4464 sound and vibration module. Details of the cameras, encoders, application software, and other core system components are commercial-in-confidence to Spookfish Innovations and cannot be discussed in this paper.
Generation I SAIP comprises a single camera tube mounted to a single-engine light aircraft, such as the Cessna 210, and is currently flying commercially in Australia. It addresses application requirements ranging from 6 cm/pixel down to sub 1 cm/pixel resolution, operating at speeds up to 160 knots, and altitudes up to 12,000 feet. The system collects up to 5 terabytes of compressed imagery in a typical five- to six-hour flying day.
Generation I generated significant commercial interest culminating in the announcement in May 2016 of a partnership with US aerial solutions company EagleView, covering rights to deploy the Spookfish technology in its US operations. Generation II comprises two camera tubes designed to mount on a larger twin-engine aircraft like the Piper PA-23 (see Figure 2). It targets similar resolutions to Generation I, but operates at higher altitude and ground speed to increase productivity. Generation II is expected to be flying commercially in Australia and the USA in 2017.
Figures 3 and 4 are low-resolution print reproductions of original high-resolution images acquired at 6 cm/pixel and 1.5 cm/pixel, respectively.
The ongoing SAIP product rollout provides for systems capable of even higher operating altitudes and speeds in Generation III and beyond. This will require continuous improvements to the precision and control tolerances of the camera control system.
The Benefits of a True Development Platform
Spookfish’s product development journey at the time of writing (February 2017) has been underpinned by the flexibility of the NI platform. PXI Express was chosen as the initial development hardware because the combination of timing precision, data bandwidth, and IO expansion options comfortably covered the projected requirements at the start of the project. In practice, the system quickly evolved far beyond its originally envisioned performance specification as new requirements arose, but the flexible PXI Express framework accommodated this evolution within an acceptable development budget and timeline.
For example, at the start of the development programme there was no expectation of needing synchronised multiaxis accelerometer data, but when the requirement arose, it was a straightforward change to add a PXIe-4464 sound and vibration module, and extend the LabVIEW-based Camera Control System to utilise the new data source.
Similarly, initial expectations were that all data would only need to be logged with millisecond accuracy. However, as the project evolved, ICON Technologies and NI delivered a three orders of magnitude improvement in timing accuracy and precision over original expectations.
NI’s RIO technology was also crucial in meeting some of these shifting performance targets. As changes to requirements began to push the limits of some of the proposed commercial off-the-shelf interfaces, they were substituted with custom FlexRIO implementations that could be optimised for the task. Because RIO is a platform-wide technology, ICON Technologies could to assist Spookfish to maintain budget control by using an existing internal stock of CompactRIO components to develop and test much of the FlexRIO code prior to deployment.
The ruggedness and reliability of the PXI Express system also meant that the transition from development system to deployed commercial instrument was seamless. Generations I and II SAIP are deployed versions of the original PXI Express development system. These meet all current requirements for the deployed system.
However, now that all the design parameters are well understood, ICON Technologies is working with Spookfish to assess the potential port of the system to CompactRIO. Recent updates to the CompactRIO family, specifically time-sensitive networking and the NI-9469 chassis synchronization module, make such a port a real possibility. Porting to CompactRIO would reduce physical deployment and maintenance costs significantly for multiple systems.
Taking Geospatial Imagery to New Application Spaces
The PXI Express, FlexRIO, and CompactRIO technologies were fundamental to realising the Spookfish Airborne Imaging Platform, which uniquely delivers wide-area, high-resolution geospatial imagery on-demand, at a price point that has not been previously possible. It will expand the reach of existing applications for this type of data, and open opportunities to create whole new applications that have previously been considered unviable.
The flexibility of the NI platform was critically important during the development phase as the system specification evolved. In particular, the platform accommodated an increase of three orders of magnitude in the timing precision requirements from the project inception with relatively minor changes to the system hardware and only incremental evolution of the application software.
The ability to implement all stages of the product development life cycle, from conceptual prototype to airborne testbed, and final deployment as a commercial product, on a single platform was also an important factor to delivering the product successfully to market in the face of an extremely challenging development schedule.
Spookfish Global Operations