URAP

Assessing Tree Stress in Response to Extreme Heat Events Using Unmanned Aircraft Vehicle and Artificial Intelligence

Lu Liang, Professor  
Landscape Architecture and Environmental Planning  

Applications for Fall 2024 are closed for this project.

The summer of 2023 witnessed record-breaking temperatures, with July being marked as the hottest month on record. The southern United States bore the brunt of this heat catastrophe, exemplified by the Dallas-Fort Worth area's numerous daily high temperature records—20 days surpassing 105°F and 42 days exceeding 100°F. Prolonged heat waves, coupled with persistent drought conditions, pose threats to both human health and tree survival. The extensive decline in tree canopies within urban built environments has exacerbated heat exposure for residents. Furthermore, these impacts may disproportionately affect marginalized communities with inadequate precautionary and mitigation measures. An urgent need is collecting and analyzing crucial data for investigating how tree health conditions has been affected by this extraordinary heat disaster. This data will play a pivotal role in comprehending the accompanying shifts in heat exposure and evaluating disparities in community resilience.

We aim to use images from unmanned aircraft vehicles paired with artificial intelligence to monitor tree stress in response to extreme heat events. Our current research is focused on improving tree crown detection and segmentation using deep learning, so that we can more accurately identify the number, location, and extent of tree crowns in different environments and conditions. A number of deep learning models have previously been applied to the same or similar tasks, usually based on Faster R-CNN (for detection) or Mask R-CNN (for segmentation). We are attempting to apply newer, open-set object detection and segmentation models, GroundingDINO and Segment Anything Model (SAM), to the problem, to take advantage of their Transformer-based architectures and pretraining on massive image datasets. We are also adapting these models to incorporate non-RGB data, including Canopy Height, NDVI, and Red-Edge channels. Finally, we are developing and training a novel Box Decoder to attach to SAM that would allow it to take bounding boxes surrounding groups of trees and split them into bounding boxes for individual trees, so that segmentation can also take place on individual trees rather than groups.

Role: Students will write code to improve our current models for tree detection and segmentation, and carry out experiments evaluating these models against each other (e.g. with different hyperparameters, input channels, etc) as well as against other models. We expect new students to take some time to learn and become familiar with the existing code in our repository in order to improve upon and experiment with these models, but students are also free and encouraged to propose their own models and solutions to the tree detection and segmentation problem. To this end, they should expect to read, discuss, and possibly implement recent research papers on image detection and segmentation techniques, and apply these to our tree datasets.

Student will learn and gain experience:
- Adapting code from large pretrained models (e.g. Meta Research’s Segment Anything Model) to new problems and datasets;
- Reading and implementing current research papers in deep learning techniques;
- Carrying out experiments comparing model performance;
- Writing and presenting research process and results;
- Planning new objectives and experiments as the research continues.

Qualifications: This research will be of interest to students of Computer Science and related fields looking to apply Machine Learning to Earth Science and Geography.
Required:
- Proficiency in the Python programming language
- Familiarity with machine learning concepts and methods
- Willingness to propose and carry out new research ideas, e.g. proposing new models or experiments to answer research questions
Preferred, but can be developed during research:
- Familiarity with machine learning framework PyTorch (strongly preferred, all current models are implemented using this framework, though experience with other ML frameworks, e.g. TensorFlow, can also be useful in learning this)
- Familiarity with Python library NumPy, image libraries such as Open-CV or Pillow, graphing libraries such as Matplotlib
- Experience working on machine learning projects, especially image processing
- Experience adapting code from open-source projects
- Familiarity with Google Drive, Google Colab, and Github

Hours: 6-8 hrs

Off-Campus Research Site: Student should meet with the professor at least once per week, preferably in person, but can be online if schedule conflicts.

Return to Project List