Glenda Hui-En Tan

Summer Undergraduate Research Fellowship 2025

Cognitive decline associated with aging can present a myriad of challenges to everyday functions, such as in locating items, which can hinder an individual's ability to prepare meals. In this work, we propose a system that uses an overhead RGB-D camera and zero-shot object detection models to identify and track items in 3D as they are being used in a kitchen. Our pipeline employs several strategies to achieve stable object tracking and object permanence even despite transient occlusions. We further ground the 3D coordinates of each tracked item using their semantic kitchen locations, which we define using a highly scalable registration procedure. We evaluate our system on a set of eight videos of older adults (median age = 78), captured in two different kitchens, and achieve reasonable performance in complex, ecologically valid scenes. Finally, we demonstrate how our pipeline can inform interaction with a user through CookerLooker, a mobile app that helps users easily locate items in their kitchen. To help catch when items are misplaced, the app includes a module that evaluates the appropriateness of item placements, information that could further inform assistive interactions with the user. In integrating our pipeline with CookerLooker, we attempt to address common challenges faced by older adults in the kitchen, empowering them to age independently.

• Gold Award, Singapore Science and Engineering Fair 2021
• Singapore University of Technology and Design Research & Innovation Award: Healthcare
• Grand Prize Bronze Award, Global Youth Science and Technology Bowl 2021 Hong Kong
• Represented Singapore at Global Youth Science and Technology Bowl 2021 Hong Kong

Colorectal cancer is the most common cancer in Singapore and the third most common cancer worldwide. Blood in a person's stool is a symptom of this disease, and it is usually detected by the faecal occult blood test (FOBT). However, the FOBT presents several limitations - the collection process for the stool samples is tedious and unpleasant, the waiting period for results is about 2 weeks and costs are involved. In this research, we propose a simple-to-use, fast and cost-free alternative - a stool recognition neural network that determines if there is blood in one's stool (which indicates a possible risk of colorectal cancer) from an image of it. As this is a new classification task, there was limited data available, hindering classifier performance. Hence, various Generative Adversarial Networks (GANs) (DiffAugment StyleGAN2, DCGAN, Conditional GAN) were trained to generate images of high fidelity to supplement the dataset. Subsequently, images generated by the GAN with the most realistic images (DiffAugment StyleGAN2) were concatenated to the classifier's training batch on-the-fly, improving accuracy to 94%. This model was then deployed to a mobile app - Poolice, where users can take a photo of their stool and obtain instantaneous results if there is blood in their stool, prompting those who do to seek medical advice. As "early detection saves lives", we hope our app built on our stool recognition neural network can help people detect colorectal cancer earlier, so they can seek treatment and have higher chances of survival.

CMU Office of Undergraduate Research and Scholar Development Presentation Award 2025

With rapid advancements in artificial intelligence (AI), many companies have begun incorporating generative AI into their advertising strategies, producing unique and engaging commercials that transcend the limits of traditional advertising. Given the immense success of these campaigns, this research aims to investigate if AI-generated ads can outperform their human-made counterparts in increasing consumer demand, and if so, the factors contributing to their greater success. To answer these questions, we first conducted two case analyses on Nike's "Never Done Evolving" ft. Serena Williams campaign and Coca-Cola's "Y3000" campaign. Next, we devised a theoretical framework of three potential success factors for AI-generated ads (complexity, creativity, and familiarity). Finally, a week-long field study was conducted on 240 Carnegie Mellon University students across different years and majors to study the effectiveness of AI-generated ads in increasing club membership of campus organizations, with club membership serving as a proxy for consumer demand. In this study, participants were randomly partitioned into four groups, with each group surveyed on how likely they were to join a club after viewing one of four posters (three AI-generated posters based on each factor of success and one human-made poster). Through this study, we discovered that although AI-generated posters are generally more effective than human-made posters in capturing people's attention, their human-made counterparts are still more effective in increasing club membership. While AI-generated posters attract a larger audience, there is greater variation in the interest level of the audience. Future research can further explore the impact of AI advertisements on various aspects of marketing, such as click-thru rates and sales on different marketing platforms such as e-commerce.

Art restoration, the practice of returning a damaged artwork to its original condition, is challenging: each artwork requires a unique treatment and complex ethical boundaries may be crossed. However, this field can be transformed by today's artificial intelligence (AI) algorithms, which currently excel in artistic tasks. Hence, this research aims to elucidate the role of AI in art restoration, specifically in restoring damaged paintings. To achieve this, an AI image enhancer web application was tested on images of three damaged paintings, with its digital reconstructions benchmarked against the actual restorations using various technical and aesthetic metrics. Results show that AI currently plays the role of an assistant in the analysis stage of art restoration: while AI can propose feasible intervention methods and produce digital reconstructions to aid art restorers in restoration (accelerating, improving accuracy and lowering costs), it cannot guide them in navigating complex ethical boundaries. Art restorers are still responsible for striking the balance between improving an artwork's legibility and respecting the original creation. These findings showcase the potential of AI to the artistic and historical communities, spurring further research into developing more effective AI algorithms.

In October 2023, a group of 44 scientists hailing from several U.S. states, Canada, Poland, and Switzerland came together for a hybrid in-person and virtual hackathon. The event was jointly hosted by Carnegie Mellon University Libraries and DNAnexus, a California-based cloud computing and bioinformatics company. This collaborative effort revolved around the theme of "Data Management and Graph Extraction for Large Transformer Models in the Biomedical Space." In the spirit of fostering collaboration, participants organized themselves into five teams, which ultimately resulted in the successful completion of four hackathon projects. These projects encompassed a wide range of topics, from detecting features contributing to virus susceptibility to validating models using knowledge graphs. Repositories for the hackathon projects are available at github.com/collaborativebioinformatics. We hope that the insights and experiences shared by these teams, as detailed in the following manuscript, will prove valuable to the broader scientific community.

• Gold Award, Singapore Science and Engineering Fair 2022
• Represented Singapore at Regeneron International Science and Engineering Fair 2022 Atlanta USA

More infectious virus variants can arise from rapid mutations in their proteins, creating new infection waves. These variants can evade one's immune system and infect vaccinated individuals, lowering vaccine efficacy. Hence, to improve vaccine design, this project proposes Optimus PPIme - a deep learning approach to predict future, more infectious variants from an existing virus (exemplified by SARS-CoV-2). The approach comprises an algorithm which acts as a "virus" attacking a host cell. To increase infectivity, the "virus" mutates to bind better to the host's receptor. 2 algorithms were attempted - greedy search and beam search. The strength of this variant-host binding was then assessed by a transformer network we developed, with a high accuracy of 90%. With both components, beam search eventually proposed more infectious variants. Therefore, this approach can potentially enable researchers to develop vaccines that provide protection against future infectious variants before they emerge, pre-empting outbreaks and saving lives.

• Gold Award, Singapore Science and Engineering Fair 2022
• Represented Singapore at Regeneron International Science and Engineering Fair 2022 Atlanta USA

More infectious virus variants can arise from rapid mutations in their proteins, creating new infection waves. These variants can evade one's immune system and infect vaccinated individuals, lowering vaccine efficacy. Hence, to improve vaccine design, this project proposes Optimus PPIme - a deep learning approach to predict future, more infectious variants from an existing virus (exemplified by SARS-CoV-2). The approach comprises an algorithm which acts as a "virus" attacking a host cell. To increase infectivity, the "virus" mutates to bind better to the host's receptor. 2 algorithms were attempted - greedy search and beam search. The strength of this variant-host binding was then assessed by a transformer network we developed, with a high accuracy of 90%. With both components, beam search eventually proposed more infectious variants. Therefore, this approach can potentially enable researchers to develop vaccines that provide protection against future infectious variants before they emerge, pre-empting outbreaks and saving lives.