ADA is a wheelchair-attached robot arm that enables people with motor impairments to independently feed themselves. I worked closely with over a dozen end-users to identify their needs and priorities, learn robot-executable food acquisition actions, and develop and evaluate the end-to-end system. The system has fed end-users diverse meals of their choice—grilled salmon, pizza, salads—in diverse out-of-lab environments—a cafeteria, conference room, and office. I also led a 5-day deployment of the system in an end-user's home, where it fed him 10 meals across varied spatial, social, and activity contexts.
Stretch is a general purpose mobile manipulator for home environments. I developed autonomous features for Stretch's operator interface, on both the frontend and backend. One such feature was "click-to-pregrasp," where the operator clicks an object in the camera feed and Stretch aligns its gripper to manipulate that object. Other features included real-time robot-to-operator audio streaming and operator-to-robot text-to-speech. I also lowered the interface's latency, CPU usage, and memory usage, getting worst-case CPU usage from 904% down to 172%. See my blog post for more details.
Zipline's Platform 2 Droid is a precision urban drone delivery system. I investigated user perceptions of droid motion during delivery, and worked with the motion planning and controls teams to integrate the insights into the autonomy stack. For that research, I added several features to a simulation environment to generate realistic delivery videos. I also prototyped and evaluated multiple ways for workers to scan packages before loading them into the droid.
Robots inevitably experience failures. I developed models of human behavior that enable robots to determine who and when to ask for help to overcome these failures. I then developed a system that enables even low-spec mobile robots to effectively navigate in large environments, by levaraging human help. Our system navigated 1,200 ft of hallways for 32 hours, with only 0.5 hours of human help over the four days.
As sensors become more ubiquitous in the world, there is an opportunity for robots to leverage multi-modal data to learn socially appropriate ways to interact with people. In this work, I developed an end-to-end pipeline, using attention-based deep neural networks, that enables a robot to use sensors in a store to learn socially appropriate ways of interacting with multiple customers at once. This network outperformed the state-of-the-art by up to 30% in predicting socially appropriate responses.
Podi is a guide robot for people who are blind or low-vision. I studied user preferences for how the robot drops them off to their destination. For the robot to match those preferences, I developed a model of how users follow a guide robot and integrated the model into the robot's planning algorithm, enabling the robot to plan motions for the coupled human-robot system.
I was the instructor for CSE/STAT 416 Introduction to Machine Learning (ML), where I taught 80+ undergraduate and graduate students deep learning, regression, matrix factorization, and social considerations in ML. I managed a team of 5 TAs to prepare and teach sections, respond to student questions, and grade. Instructor evaluation: 4.7/5.0.
I was a guest lectureer in Japanese and American classrooms, teaching K-8 students about the others' culture. My lectures in Japanese classrooms focused on the American multiculturalism, landmarks, games, and songs. My lectures in American classrooms focused on Japanese temples, public transportation, history, and culture.
I co-founded and ran Teknowledge, a computer science (CS) outreach organization. We taught afterschool CS at under-resourced public schools and community centers. I developed Intro Python (6th – 11th grade) and Inquiry-based Machine Learning (10th – 11th grade) curricula, and advised teachers and administrators as they converted our curriculum into a mandatory 8th grade CS course. I managed communications, logistics, and training for 25 mentors, 130+ students, and 7 community partners. I also participated in an initial investigation of integrating K-12 computational thinking curricula into rural KY schools, an initiative that has since flourished.