MANTIS Student, Mohammad Pashaei, Defends his Dissertation!

This past Friday, October 29th, Mantis Ph.D. candidate (now Dr.) Mohammad Pashaei has successfully defended his dissertation, Applications of Deep Learning and Multi-Perspective 2D/3D Imaging Streams for Remote Terrain Characterization of Coastal Environments. To learn more, read his dissertation abstract below.

ABSTRACT:

Updated and accurate geospatial information about land cover and elevation (topography) is necessary to monitor and assess the vulnerability of natural and built infrastructure within coastal zones. Advancements in remote sensing (RS) and autonomous systems extend surveying and sensing capabilities to difficult environments, enabling more geospatial data acquisition flexibility, higher spatial resolutions, and allowing humans to “see” in ways previously unattainable. Recent years have witnessed enormous growth in the application of small unmanned aircraft systems (UASs) equipped with digital cameras for hyperspatial resolution imaging and dense three-dimensional (3D) mapping using structure-from-motion (SfM) photogrammetry techniques. Rapid proliferation in light detection and ranging (lidar) technology has resulted in new scanning and imaging modalities with ever increasing capabilities such as geodetic-grade terrestrial laser scanning (TLS) with ranging distances of up to several kilometers from a static tripod. Full-waveform (FW) lidar systems have led to a significant increase in the level of information extracted from a backscattered laser signal returned from a scattering object. With these advancements in remote sensing capabilities, comes an exponential increase in potential information gain at the cost of greatly enhanced data complexity. New methods are needed to efficiently extract meaningful information from these data streams. In this regard, deep learning (DL) techniques, in particular, convolutional neural network (CNN), have recently outperformed state-of-the-art machine learning techniques in a wide range of applications including RS. This study presents three main contributions in the use of DL for exploitation of UAS-SfM and lidar data for coastal mapping applications: 1) Evaluation of different DCNN architectures, and their efficiencies, to classify land cover within a complex wetland setting using UAS imagery is investigated; 2) DCNN-based single image super-resolution (SISR) is employed as a pre-processing technique on low-resolution UAS images to predict higher resolution images over coastal terrain with natural and built land cover, and its effectiveness for enhancing dense 3D scene reconstruction with SfM photogrammetry is tested; 3) Full waveform TLS data is employed for point cloud classification and ground surface detection in vegetation using a developed DCNN framework that works directly off of the raw, digitized echo waveforms. Results show that returned raw waveform signals carry more information about a target’s spatial and radiometric properties in the footprint of the laser beam compared to waveform attributes derived from traditional waveform processing techniques. Collectively, this study demonstrates useful information retrieval from hyperspatial resolution 2D/3D RS data streams in a DL analysis framework.

MANTIS Student, Isabel Garcia, on TV News!

This week, the KRIS 6 News – Corpus Christi TV channel aired a report on Hispanic women in the STEM field as part of the celebration of Hispanic Heritage Month.

Two of CBI’s members, the Ph.D. students Marina Vicens-Miquel and Isabel Garcia (one of MANTIS’ very own students) were interviewed by the KRIS-TV’s reporters and talked about their research and their views on the representation of Hispanic women in the STEM field.

In her interview, Isabel talked about her passion for surveying and Math and explained her work on mapping with mobile lidar.

MANTIS in the Field after Hurricane Hanna

Larissa Freguete and Jacob Berryhill prepare the Phantom 4 RTK for flight

VRS base to collect RTK GPS data

Jacob Berryhill and Larissa Freguete unload equipment at survey site

Larissa Freguete and the Phantom 4 RTK

Jacob Berryhill preparing the Wingtra WingtraOne UAS for takeoff

Jacob Berryhill launching the Wingtra WingtraOne UAS

Craig Palmer assisted by Larissa Freguete distributing equipment at the field site

Dr. Michael Starek

Beginning Wednesday, July 29th, Mantis began the first of several field campaigns to the survey the the damage along the Texas coast from Hurricane Hanna. Mantis and the field operations crew from the Conrad Blucher Institute for Surveying Science are working together to collect UAS and mobile lidar data for the City of Corpus Christi and Nueces County. The UAS and mobile lidar data collected can be processed to generate high resolution (2 cm accuracy) interpolations of the post-storm ground surfaces. Presently, survey priority has been given to the beach between the horse path (south of Access Road 6) and Packery Channel (see map in photo gallery). With beach access points damaged by the hurricane, especially in more remote locations, UAS data will be paramount in aiding city planners as they assess the damage from the recent hurricane. This data will also help inform their decisions for repairing and rebuilding in the wake of Hurricane Hanna.

Mohammad Pashaei Defends Proposal!

Once again, amid the global pandemic, another Mantis student successfully defended a doctoral proposal. Today at 10 am, Mohammad Pashaei, virtually defended his dissertation proposal, Applications of Deep Learning and Multi-Perspective 2D/3D Imaging Streams for Remote Terrain Characterization of Coastal Environments. To learn more, read his abstract below.

Abstract

Recent years have witnessed enormous growth in the application of Unoccupied Aircraft Systems (UASs) equipped with hyper-resolution digital RGB cameras for mapping purposes with much higher accuracy than most traditional airborne and spaceborne technologies. UAS has the potential to provide geospatial data in raw image format instantaneously and inexpensively at local geographic scales. Large number of raw UAS images are later processed within a dedicated photogrammetry software. By applying Structure-from-Motion (SfM) photogrammetry on raw images, the software generates a very dense point cloud which represents the Earth’s surface using individual 3D points. Other geospatial products such as digital surface model (DSM) and orthomosaic image may later be generated. 

Furthermore, over the last decade, there has been a proliferation of commercially available light detection and ranging (lidar) systems, such as Terrestrial Laser Scanning (TLS), to directly measure precise distance to the object and its reflectance. TLS systems have been well-received in the geomatics engineering community because of their ability to collect large amount of data in discretized, highly accurate and precise 3D points format in a very short time without further processing, leading to a dramatic reduction in costs and faster local scale survey. Although it offers very high accurate 3D surface models, especially where other RS techniques may fail, such as very complex, inaccessible, and hazardous objects or areas, TLS suffers from occlusions which is highly probable in vegetated area.

In this research, a wetland environment, called Mustang Island Wetland Observatory, located on a barrier island along the southern portion of the Texas Gulf Coast, USA, is considered as the experimental field for wetland mapping and land cover classification task. UAS hyperspatial images (with cm to sub-cm ground sampling distance (GSD)) and lidar data (with sub-cm range accuracy) using a full-waveform TLS system are acquired over the wetland area to feed into algorithms which are developed for mapping and classification tasks. Additional experiments take place at other structurally complex coastal environments in the region that include a mixture of natural terrain and built features.

Land cover classification can be a very challenging task due to the spectral and spatial complexity of the study area. Specifically, for complex coastal wetlands, where targets show high inter-class similarities and intra-class variabilities in imagery, designing accurate and efficient features for traditional statistical and Machine Learning (ML) approaches may not be a simple task. Furthermore, radiometric distortions, boundary uncertainties among natural targets, and huge computational redundancies adds to the complexity of the problem. For land cover classification using hyperspatial UAS imagery, Deep Convolutional Neural Networks (DCNNs) approach, which has already shown the superiority of Deep Learning (DL) framework in many image analysis tasks, is proposed here. Our experiment examines different DCNN architectures and introduces the most efficient networks suitable for the RS image analysis task. Moreover, image super-resolution technique using DCNNs is recommended to predict high-resolution (HR) UAS images from corresponding low-resolution (LR) images. Spatial resolution enhancement of UAS images in combination with some level of noise reduction in resulting super-resolved (SR) image set leads to a significantly denser point cloud w.r.t LR image set with less uncertainty in SfM photogrammetry procedure. Finally, the potential of full-waveform TLS for accurate 3D modeling of the Earth’s surface and classifying ground and above ground targets is investigated by accurate analysis of the backscattered (BS) waveform within DL framework and examining its properties.

Edison Veloz Defends Thesis!

Despite the global pandemic, Mantis student, Edison Veloz, defended his thesis, Evaluation of Environmental Impacts Produced by Gold Mining on the Areas on the Surrounding Forest in Southwester Ecuador using Multispectral Satellite and UAS Imagery, on July 17th! Read his thesis abstract below to learn more.

ABSTRACT:

Mining is a dangerous activity that can cause environmental damage to flora and fauna due to the utilization of heavy metals. Ecuador has a long history of mineral extractions and nowadays the activity is increasing in many parts of the country. Environmentalists state that chemicals, such as cyanide and mercury, could cause alterations in vegetation health. This study utilizes satellite and Unmanned Aircraft System (UAS) based remote sensing to analyze impacts to vegetation health around a mining area located in Bella Rica within the El Oro province of the southwestern zone of Ecuador.

Vegetation can be analyzed and identified through many remote sensing techniques, one of them is the Normalized Difference Vegetation Index (NDVI). This band ratio index ranges from +1 to -1 and uses red and near-infrared (NIR) bands to identify the presence of healthy or stressed vegetation. In this study, a small rotary UAS equipped with a two-band sensor recording red and NIR reflectance and a separate red-green-blue (RGB) digital camera was used to gather data and determine if vegetation closer to the mine exhibited different NDVI patterns compared to vegetation located farther away. Spatial differences in NDVI patterns may indicate potential impacts of waste from mining operations . To provide a time series assessment of vegetation changes around the mine, satellite imagery from PlanetScope was acquired and analyzed to measure changes in NDVI throughout the last three years. PlanetScope uses an array of miniaturized satellites, called CubeSats, equipped with four-band multispectral sensors providing imagery at a resolution of 3 m ground sample distance (GSD). In comparison, spatial resolution of the UAS products, which is dependent on flying height, range from 2.97 cm GSD for the RGB camera to 11.4 cm GSD for the multispectral sensor. Satellite derived NDVI was statistically compared to UAS derived NDVI values to assess the impact of spatial resolution and sensor quality on NDVI measurement. Furthermore, the UAS acquired RGB imagery was processed using Structure from Motion (SfM) photogrammetry to derive a 3D reconstruction of the scene, referred to as a point cloud. Properties of the point cloud data were analyzed to determine if relationships exist between land cover structure and NDVI patterns captured in the UAS multispectral imagery.

From UAS based multispectral data, significant differences in NDVI values were found between vegetation close to the mining area and vegetation at longer distances (p < 0.05), indicating that mining waste could be altering NDVI values in the region. Satellite imagery analysis suggests that changes in NDVI are related to different human activities that have been developed inside the study area. UAS derived NDVI shows a strong linear relationship with PlanetScope derived NDVI (R = 0.91), suggesting that the low cost and light-weight sensor onboard the UAS was able to capture similar reflectance information but at much higher resolution.  UAS-SfM point cloud data was applied to measure spatial variation in point density and canopy height, and determine if these measures could serve as a proxy for NDVI to assess vegetation health impacts from the mining operation. Results varied with NDVI and point cloud density exhibiting a weak relationship (R = 0.04). This relationship held at multiple resolutions suggesting that scene texture and uniformity in the densification stage of SfM does not correlate well with variation in NDVI due to differences in canopy cover. Interestingly, point cloud density changes did show a connection to the type of vegetation with high values of point density occurring over the more densely canopied forest areas. In contrast to point cloud density, UAS-SfM derived canopy height measures exhibited much stronger correlation to the UAS multispectral NDVI values (R = 0.69).

Based on the available data and the examined time frame, this study has shown that mining activities have altered NDVI values in the surrounding vegetation at the study site. Moreover, this study has shown that a small UAS platform equipped with a low-cost multispectral sensor can provide similar NDVI values to satellite imagery, but at much higher resolution. The ability to fuse detailed UAS information at a local scale with high repeat frequency CubeSat remote sensing data provides an effective means for monitoring impacts of mining operations at local to regional scales. Finally, results suggest that 3D point cloud data generated from UAS-SfM photogrammetry can enable effective characterization of vegetation structure and canopy height around mining operations providing another tool beyond NDVI to monitor impacts on vegetation growth and health.

MANTIS Professors' Work Gets Press Coverage from Lidar News!

The Viewshed Simulation and Optmization for Digital Terrain Modelling with Terrestrial Laser Scanning article written in-part by MANTIS’ own Dr. Michael Starek and Dr. Tianxing Chu, and co-authors Dr. Helena Mitasova and Dr. Russel S. Harmon, had press coverage on the Lidar News blog!

Lidar News provides the most current information regarding 3D laser scanning, lidar, UASs, and photogrammetry. The aforementioned article was published on the Lidar News blog on June 15, 2020.

Kelsi Schwind Wins Blue Marble Geographics Scholarship!

The Blue Marble Geographics academic scholarship is awarded each year to a graduate student who has demonstrated proficiency in Global Mapper® in a research project. On February 6, 2020, Kelsi received this scholarship for her research integrating structure-from-motion (SfM) data, airborne topobathymetric lidar-derived data, and GIS techniques to assess the impacts of Hurricane Michael on Little St. George Barrier Island in Apalachicola, Florida.

MANTIS Achievements Winter 2019/2020

BRADLEY KOSKOWICH, GSCS PH.D. STUDENT, defends phd proposal

On 2/13/20, BradleyKoskowich of MANTIS Lab passed his PhD qualifying exams on his proposed topic, Efficiently Localizing Monocular Images Using Image Synthesis, Point Cloud Products & Keypoint Densification. See his abstract below for more details!

ABSTRACT: 

Human beings possess powerful visual abilities which are used to navigate space, coordinate our actions, and correlate the contents of a perspective image with an approximate knowledge of the location the image would have been taken from, intuitively and from little more than the image itself. This lends itself to further abilities to create logical conclusions about the nature of an inferred three-dimensional space based on a series of two-dimensional observations. Recreating this same process digitally is a thoroughly studied problem which has seen years of computer vision research, namely feature detection & description, structure-from-motion (SfM), and their applications in simultaneous localization and mapping (SLAM). Variations and permutations of SLAM solutions have been proposed since its initial inception, and as of recent years have leaned towards incorporating additional hardware to improve their results, such as inertial motion units and global navigation satellite systems. However, the nature of incorporating additional data dimensions in these SLAM solutions alters it such that it the problem becomes intractable in the event of catastrophic interference or data loss from any dimension. While there have been attempts to mitigate or even read hardware interference and leverage it as additional information, such methods are not robust to even a partial systems failure. And systems which have proven robust to such failures incur significant pre-processing or computational time overhead, none of which propose to be suitable for real-time applications. This proposal assesses the need to decouple the localization component of SLAM solutions from the rest of the processing pipeline, allowing for flexible input datasets with tolerance for extreme noise. This proposal details mechanisms which can be developed and combined specifically to enable high performance POSE estimation as part of this flexible localization process, later incorporated into a robust and tolerant SLAM solution as a hybrid of typical algorithms and neural networks.

MANTIS in Apalachicola

Since 2016, members of MANTIS Lab and CBI field crew have collaborated with the Apalachicola National Estuarine Research Reserve’s (ANERR) Megan Lamb for week-long field campaigns supported by National Oceanic and Atmospheric Administration’s (NOAA) subsidiary, the National Geodetic Survey (NGS). Each year, the crew gathers Terrestrial Laser Scanner (TLS) and Unmanned Aerial Systems (UAS) data for one site on the main barrier island, Saint George Island, and from several sites on “Little” Saint George Island (it is the western component of the island that was split away from Saint George Island by Bob Sike’s Government Cut in 1954). The Saint George Island site is NOAA’s Unit 4 SET site while the “Little” Saint George sites include several beach profiles (D341, R4, R41, and R29) and several historical photosites (Westpass, Bayside, and Sike’s Government Cut). This data collection is part of an NGS gulf-wide research initiative to develop and improve current Relative Sea Level Rise (RSLR) models for the gulf coast by gathering high-resolution spatial (elevation) data.

Map of the eight sites surveyed annually by MANTIS, CBI, and ANERR.

A gator sunning herself on “Little” Saint George Island.

The TLS crew setting up a base at Saint George Island.

Vapor 55 crew surveying one of the beach profiles on “Little” Saint George Island.

Aerial View of the bay side of “Little” Saint George Island.

This year’s trip began May 20th and ended May 27th and included MANTIS’ director, Michael Starek, CBI’s Research Engineering Associates, Alistair Lord and Zachary Hasdorff, MANTIS Lab Manager, Melanie Gingras, and MANTIS master’s students, Jake Berryhill and Kevin Wilson. During the week, the crew used new platforms including the Wingtra WingtraOne and Pulse Aerospace Vapor 55 as well as tried-and-true platforms from previous Apalachicola field campaigns including the DJI Mavic, DJI Pantom 4, and Riegl VZ400 Terrestrial Laser Scanner (TLS). When georeferenced using RTK GPS control points. This data will generate point clouds that provide high spatial resolution data to monitor elevation changes as small as a couple centimeters and othomosaic imagery with pixel sizes on the order of 2cm GSD or less.

The Apalachicola field crew after their last day of field work on “Little” Saint George Island: Megan Lamb (top bow), Melanie Gingras (bottom bow), Jake Berryhill (top middle), Zachary Hasdorff (bottom middle), Alistair Lord (top stern), Kevin Wilson (middle stern), and Michael Starek (bottom stern)

To see video footage from our YouTube channel:

MANTIS Achievements Spring 2019

Listed below are the achievements of MANTIS Lab for the Spring 2018 semester.

ISABEL GARCIA AWARDED ASPRS SCHOLARSHIP

In January, at the ASPRS Annual Conference MANTIS’s very own, Isabel Garcia, CMSS PhD student, won the American Society of Photogrammetry and Remote Sensing (ASPRS) 2019  Ta Liang Travel Scholarship ($2,000). Congratulations Isabel, you’ve made MANTIS proud!

CHUYEN NGUYEN DEFENDS DISSERTATION

On March 29th, Chuyen Nguyen became Dr. Chuyen Nguyen as she was award her PhD by her committee for her dissertation project titled “Development of Geodetic Imagining Techniques and Computational Approaches for Marsh Observation”. Congrats Chuyen!

MANTIS AT TEXAS CHAPTER ASBPA SYMPOSIUM

Christopher Reynolds and Kelsi Schwind of MANTIS lab attended and present at the Texas Chapter ASBPA Symposium. Kelsi Scwind gave an oral presentation on her PhD work and Chris presented a poster on his thesis project.

Christopher reynolds DEFENDS thesis

On May 2nd, Christopher Reynolds defended his thesis titled “Emerging Littoral Surveying Technologies for Coastal Resilience and Durability”. His work was supported by a Department of Defense scholarship, where he will begin work this summer. Congrats Chris and best wishes at the DoD!

Happy GIS Day from MANTIS!

Point cloud of Moody High School students who attended GIS Day.

For those who are not familiar with GIS—we’re surprised and happy you’re here!— GIS is a geographic information system and is designed to capture, store, manipulate, analyze, manage, and present spatial or geographic data. For those who are not familiar with GIS Day, GIS Day provides an international forum for users of geographic information systems (GIS) technology to demonstrate real-world applications that are making a difference in our society.

This year’s GIS took place November 14, 2018 but the first formal GIS Day took place in 1999. Esri president and co-founder Jack Dangermond credits Ralph Nader with being the person who inspired the creation of GIS Day. He considered GIS Day a good initiative for people to learn about geography and the uses of GIS. He wanted GIS Day to be a grassroots effort and open to everyone to participate (http://www.gisday.com/).

On November 14, 2018, Del Mar College hosted its 20th GIS Day. Every year Del Mar College hosts an event at the Center for Economic Development to bring GIS professionals, land surveyors, and high school students under one roof to exchange ideas and share information. One of the most unique and arguably the most important feature of this event is the attendance of high school students to drum-up interest in this fast-growing and ubiquitously important field. MANTIS is happy to be a part of this: MANTIS staff member, Melanie Gingras, and students, Kevin Wilson, Jake Berryhill, Isabel Garcia, and Kelsi Schwind, all volunteered to assist with the latter portion of this event. Kevin Wilson demonstrated the use and application of drone technology in GIS while Jake introduced students to the FLIR sensor. Melanie, Isabel, and Kelsi presented terrestrial LiDAR function to the students in an easy-to-digest real-time scan of the attending students which are depicted in the blog images.

GIS Day point cloud of the Delmar Center for Economic Development locale.