Capstone projects target vulnerabilities in AI systems and smart devices

Every year, students in Johns Hopkins University’s Security Informatics master’s program work with faculty and industry mentors to solve real-world problems. Recent semester-long capstone projects included research applying large language models to spam detection and an evaluation of security features in popular electronic devices.

Current spam detection methods struggle to adapt to spammers’ rapidly changing tactics. To solve this problem, Qiyao Tang developed A Spam Detection System Based on LLM.

Qiyao Tang.

Non-tech industries devote significant time and resources to training their employees to identify phishing messages and spam, which can lead to data breaches, compromised systems, and financial losses. Nevertheless, few tactics successfully defend against spam emails, and traditional rules-based, collaborative, and machine learning spam detection methods often cannot keep up with spammers’ tactics.

In his project, Tang evaluated large language models’ success in assessing vulnerabilities to adversarial attacks using magic words—specific phrases that can manipulate models’ output— and through cross-dataset testing—which evaluates how a model works across different datasets.

“While LLMs perform well in identifying spam, they can be easily tricked by clever adversaries and don’t work well when facing types of spam, they have never seen before,” Tang explains. “My research highlights the need for stronger defense mechanisms to make AI-based spam filters more reliable.”

Tang notes that in the future, more advanced models will be needed to build systems that can withstand similar spam and phishing attacks.

Zhe Yan.

Two other students’ reliance on smart devices inspired their project: Security Analysis of Popular IoT Devices: Identifying Vulnerabilities and Proposing Solutions.

“Our inspiration for this project came from our own daily reliance on these devices and the realization that many people are unaware of the potential security risks associated with them,” said team member Zhe Yan, who worked with Yuyang Lei to evaluate the vulnerabilities of the Google x Yale Smartlock, the Ring Smart doorbell, and the Oura ring. “We want to provide practical suggestions for both users and manufacturers to improve security.”

Yan and Lei analyzed the full process of developing, producing, and distributing each device, and used exploitation techniques and tools such as HackRF One (a device that analyzes and manipulates radio signals) and Buffalo Router (a wireless router that can be modified for security testing) to assess these smart home and wearable devices. This allowed them to investigate how the Google x Smartlock authenticates access,  the Oura ring transmits health data, and the Ring camera communicates with cloud servers.

Yuyang Lei.

Their study confirmed that well-known brands that develop smart home IoT devices have vulnerabilities and may not protect users’ personal information. As part of their work, the students identified potential paths for attacks as well as ways to mitigate them. Their study also provides actionable insights into how to improve IoT device security, highlight user awareness, develop secure design principles, and improve industry practices.

“The findings contribute to advancing IoT security research and highlight the urgent need for strengthened defenses in the ever-expanding IoT ecosystem,” explains Lei.