Intelligent Prototyping Tools

Prototyping systems allows designers and developers to quickly get their ideas in front of real users, get feedback, and iterate on their designs. However, this is often itself a time-consuming process. This research direction aims to develop systems that let designers and developers prototype functional systems as quickly as they can describe them, providing unparalleled ability to quickly iterate on designs, even during live trials.

  • How can we better leverage natural language description in system creation? This can impact how we think about project specifications and the design lifecycle from initial idea to product specification.
  • How can we create effective intelligent prototyping tools? This has implications for designers and system builders.
  • How do we coordinate crowds of people to complete complex, open-ended tasks? This has the potential to impact a wide range of applications in crowdsourcing and coordination in large organizations.
  • How can we coordinate crowds with multiple types and levels of expertise? In Apparition, this will help crowds simultaneously simulate and program behaviors in real time.



Apparition [1] is a system for prototyping interactive systems using voice and sketch. By using a combination of human and machine intelligence behinds the scenes, Apparition immediately begins to improve interface quality, and even make described system behaviors "real". Collective crowd control [2] and novel interaction techniques are used to collaboratively generate behavior animations.


[1]  S.W. Lee, Y. Zhang, I. Wong, Y. Yang, S.D. O'Keefe, W.S. Lasecki. SketchExpress: Remixing Animations For More Effective Crowd-Powered Prototyping Of Interactive Interfaces. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2017). Quebec City, Canada.

[2]   W.S. Lasecki, J. Kim, N. Rafter, O. Sen, J.P. Bigham, M.S. Bernstein. Apparition: Crowdsourced User Interfaces That Come To Life As You Sketch Them. In Proceedings of the International ACM Conference on Human Factors in Computing Systems (CHI 2015). Seoul, Korea. p1925-1934. Best Paper Honorable Mention

[3]   W.S. Lasecki, K.I. Murray, S. White, R.C. Miller, J.P. Bigham. Real-time Crowd Control of Existing Interfaces. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2011). Santa Barbara, CA. p23-32.