Meta AI
Shipped v1 features and set v2 direction, with vision prototypes informing HW/SW plans.
- Led definition of AI features
- Defined AI requirements for gen 1 and 2 platforms
- Shipped AI-powered features for next-gen hardware
- Used multi-modal input for AI
- Built foundational AI interaction patterns
- Enabled new AI experiences
- Integrated Meta AI capabilities seamlessly
- Maintained consistency across Meta's ecosystem
- Established scalable AI design patterns for future hardware
Contextual AI
Combining AI with sensors on glasses creates a powerful platform for delivering compelling user experiences. As a manager and design lead, I led teams in developing numerous prototypes, defining the scenarios, interactions, and visual designs to showcase these capabilities.
In this example on Orion, AI utilizes object recognition and location data, employing world-locking to seamlessly position UI elements within the environment. The scenario demonstrates how AI recognizes food items and suggests a recipe in real time.
Meta AI
We took the first steps in integrating AI into glasses experiences, enabling on-the-go functionality powered by input from the point-of-view camera.
As the design lead, I played a key role in crafting the experience for Meta AI on Orion.
- Delivered under short turnaround timelines
- Drove intense collaboration with engineering teams
- Aligned closely with Meta AI to integrate platform capabilities
- Designed and shipped new interaction patterns
Generative UX
In the near future, AI will dynamically generate interfaces by leveraging app services, functioning as an intelligent agent. These UIs will be tailored to individual needs, preferences, and environments, offering a personalized, customized, and contextualized experience.
To illustrate this to leadership and engineering, I developed a Node.js/React prototype that renders live UI by integrating service APIs to fetch content. This image of the prototype shows step-by-step instructions entirely generated by AI, which autonomously determined text treatments, image placement, and column layout. The prototype supports various components, such as lists, buttons, and maps, showcasing the potential for versatile, AI-driven UI generation.
- Developed prototype
- Initiated large initiative
- Presented at conference
