Hi, I'm Arun Krishna Vajjala

Collaborated with Dr. Jeff Nichols, Dr. Amanda Swearngin, and Dr. Titus Barik on research in Generative UI and UI Understanding, developing tools that bridge the gap between conceptual design and functional prototypes.

Squiggle: Architected and developed Squiggle, a tool that records whiteboard meetings in real-time and renders fully functioning SwiftUI apps. The system uses OpenCV and Mac webcam integration to capture 10-second snapshots of whiteboards, processing conversational dialogue and whiteboarding deltas through a sophisticated 4-agent architecture to understand UI construction and wireframing intent. Users can walk in with an idea and leave with a deployable iPhone prototype.

ScreenStorm: Built ScreenStorm, an interaction-based generative tool for creating screen variations and implicit design systems. The platform enables designers to make direct modifications to high-fidelity UIs and selectively preserve or modify elements when exploring alternatives. The system learns design patterns implicitly through user interactions, maintaining consistent style across multiple app screens while dramatically expanding designers' exploration capabilities.

Both tools significantly enhanced designer productivity, with users reporting increased exploration of design alternatives and reduced friction in the ideation-to-prototype pipeline. The 4-agent architecture analyzed user changes, generation preferences, and design patterns to deliver contextually relevant suggestions.

Mentored by Dr. Christian Bird, Dr. Nicole Forsgren, and Dr. Rob Deline, I identified bottlenecks in the software deployment and build process, leveraging Machine Learning and Artificial Intelligence to automate and streamline workflows for developers.

Collaborated with developers to gather requirements, designing and building a user interface that integrates K-means clustering on build failures using Azure OpenAI embeddings. This groups failures for easy access and triage by on-call developers.

Deployed custom Large Language Models (LLMs) within an information-secure Azure environment (OpenAI GPT-4o) to proactively tackle explainability and traceability, significantly reducing manual inspection and fatigue.

Presented this project at an executive review with a Corporate Vice President (CVP), where the partner product team requested an immediate push to production and initiated a successful tech transfer due to its potential to improve developer efficiency.

FRAME Project: Created FRAME, addressing an industry need for enhanced UI layout comprehension by implementing a Neural Graph-based approach to create structurally motivated GUI embeddings. FRAME integrated state-of-the-art CLIP, BERT, and Graph embedding techniques, coupled with structure enhancing mathematical concepts in the Rips-Complex and Embedding Propagation.

GUIFix Project: Designed and implemented GUIFix, a developer tool leveraging LLMs to detect and automatically repair accessibility issues in Android applications. Developed a novel workflow utilizing Python and CLIP embeddings to localize and repair accessibility issues with minimal developer intervention.

MotorEase Project: Designed and implemented MotorEase, an automated tool to detect motor-impairment accessibility issues in mobile applications. Integrated state-of-the-art techniques in PyTorch computer vision, pattern-matching, and static analysis, achieving 87% accuracy in detecting accessibility violations at runtime.

Collaborated with a multi-disciplinary team of researchers, engineers, and surgeons to prototype a surgical voice assistant, focusing on improving end-user interactions (surgeons) and enhancing intraoperative workflows through machine learning and software integration.

Led the design and development of a wake-word detection model using TensorFlow, AWS SageMaker, and PyTorch. Developed a robust audio processing and feature extraction pipeline achieving 80% accuracy in detecting the wake-word "Hey, Alcon" in real-time input streams.

Successfully deployed the voice assistant across multiple operating room devices in the U.S., significantly improving user interaction and reducing manual input during surgeries, surpassing initial performance expectations.