Brown University Department of Computer Science Industrial Partners

Brown University Department of Computer Science Industrial Partners Program August 2002 Michael J. Black

Brown University Founded 1764; Third college in New England 7 th in the country. 556 faculty members 16, 606 undergraduate applicants – 1413 admissions. 55% female, 28% minority. August 2002 Michael J. Black

Brown Students 5, 674 undergraduates, 1, 343 graduate students 316 medical students Intended freshmen concentrations: * 36% science and math: * 26% humanities * 21% social sciences * 9% engineering. Many involved in research. August 2002 Michael J. Black

CS Department 22 Tenured and tenure-track faculty and growing 194 ugrads 94 graduating 900 students/term in CS classes 26 masters students 52 Ph. D students 17 graduating masters 7 graduating Ph. Ds August 2002 Michael J. Black

Industrial Partners Program Focused program with a small number of partners. Customize relationships with each partner company (varies over time). Close interactions through research recruiting symposia and meetings August 2002 Michael J. Black

Industrial Partners August 2002 Michael J. Black

Membership Annual fee: $25, 000. - directly supports research and instruction Recruiting assistance - job fairs, guaranteed place, special events, IPP colloquia, recruiting events, fliers, announcements…. Information dissemination - conduit! newsletter with research news, patenting, partner interactions, … August 2002 Michael J. Black

IPP Symposia Held twice a year (spring and fall) - speakers from academia and industry - topics of current interest - opportunity for pre-competitive cooperation with other partners - discussion and social opportunities Recent symposia: Computer and Network Security; Component Software and Technologies; Web Technologies; ECommerce; Computing in a Wireless World; … August 2002 Michael J. Black

Membership Invitations to CS seminars and colloquia. Visits to campus and meetings with faculty and students. Visits by faculty or students. Regular contact with Prof. Michael Black (Director) and Suzi Howe (Manager) to customize the relationship. http: //www. cs. brown. edu/industry/ipp/ August 2002 Michael J. Black

Research (today) Comp. Linguistics and Natural Language Understanding (E. Charniak, T. Hofmann, M. Johnson, I. Kontoyiannis) Database Management Systems (U. Cetintemel, D. Goldin, S. Zdonik) Theory of Networking, Concurrency, and Distributed Comp. (M. Herlihy, S. Krishnamurthi, A. Lysyanskaya, E. Upfal) Security (Distributed Data Authentication) (R. Tamassia, P. Klein, A. Lysyanskya, T. Doeppner) New User Interfaces (M. Black, J. Hughes, D. Laidlaw, A. van Dam, D. Zelter) August 2002 Michael J. Black

Research Agents and Ecommerce – - statistical learning, decision theory, game theory - agent-based information retrieval systems - auctions, commodity trading, portfolio optimization… - facilitate human-computer interaction (Tom Dean, Amy Greenwald, Thomas Hofmann, Eli Upfal, Pascal Van Hentenryck) August 2002 Michael J. Black

Research Mobile and Ubiquitous Computing – - data and resource management - sensor networks, mobile ad hoc networks, Internet-scale information systems - decentralized replication (overlay networks) for mobile and weakly connected environments - adaptive data dissemination in wireless networks - mobile computing and collaboration tools, note-taking, annotation, etc. (Ugur Cetintemel, Tom Doeppner, Don Stanford, Stan Zdonik) August 2002 Michael J. Black

Other Research Artificial Intelligence Combinatorial Algorithms Combinatorial Optimization Computational Biology Computational Geometry and Graph Drawing Computer Graphics Educational Software Electronic Documents and Hypermedia Internet Computing Nanotechnology Neuroinformatics and Brain Science Operating Systems and Distributed Systems August 2002 Parallel Computation Programming Languages Randomized Algorithms and Probabilistic Analysis Robotics and Computer Vision Scientific Visualization Scientific Computing Software Engineering Theory of Computation User Interfaces and VR Verification and Reliable Systems Michael J. Black

August 2002 Michael J. Black

Computing out of the Box Michael J. Black Department of Computer Science Brown University August 2002 Michael J. Black

WHITHER THE COMPUTER? Dell workstation, 2001 XEROX Alto 1973 August 2002 Michael J. Black

New Types of Interaction Give machines more information (awareness) about their users and their users’ environment. * speech and natural language * vision - faces, identity, expressions, gaze, … - pose, motion, gesture - action, setting, tasks, environment - interactions between people August 2002 Michael J. Black

Applications • Human-Computer Interaction • Surveillance • Motion capture (games and animation) • Video search/annotation • Work practice analysis. * detect moving regions * estimate motion * model articulated objects * model temporal patterns of activity * interpret the motion Social display of puzzlement August 2002 Michael J. Black

Analysis of Facial Motion Recognizing facial expressions from motion * passive user interfaces * active user interfaces With Yaser Yacoob August 2002 Michael J. Black

Recognizing Facial Motion August 2002 Michael J. Black

Gestural Interfaces August 2002 Michael J. Black

Capturing Meetings Input video Motion and gesture analysis Web interface With Shanon Ju August 2002 Michael J. Black

Why is it Hard? 2 D view of 3 D world Unusual poses. Large Motion Self occlusion. Deformation. Low contrast Ambiguous matches August 2002 Michael J. Black

Approach Bayesian formulation p(model | cues) = p(cues | model) p(cues) 1. Likelihood: a. probabilistic, learned from examples. b. model both people and generic scenes (explain entire image). 2. Prior: “learned” implicit probabilistic model uses ideas from texture synthesis. 3. Model/Search: particle filtering. August 2002 Michael J. Black

Tracking 3 D Motion With H. Sidenbladh August 2002 Michael J. Black

Modeling/Tracking Activities With H. Sidenbladh August 2002 Michael J. Black

Forbidden Planet, 1956 August 2002 Michael J. Black

BRAIN-COMPUTER INTERFACES “Mad” scientist Nancy Davis (Reagan) Brain “If I could find … a code which translates the relation between the reading of the encephalograph and the mental image …the brain could communicate with me. ” “Donovan’s Brain”, Curt Siodmak, 1942 August 2002 Michael J. Black

LANGUAGE OF THE BRAIN “If spikes are the language of the brain, we would like to provide a dictionary… perhaps even providing the analog of a thesaurus. ” Rieke, et al 1997. August 2002 Michael J. Black

A NEURAL PROSTHETIC mathematical algorithm EEG single and multineuron activity Voluntar y control signal Computer cursor and keyboard entry August 2002 Robotic arm Stimulation of Muscles, Spinal Cord, and Brain From: Mijail Serruya Michael J. Black

CELL ENSEMBLES Utah Array B Central Sulcus MI MI SMA PMA Arcuate PMA 5 mm 80µV 1 ms 100 electrodes, 400 mm separation 4 x 4 mm Implanted in the MI arm area of motor cortex August 2002 Michael J. Black

REAL TIME NEURAL CONTROL Target Neural control Linear filters built on-line. Mijail Serruya August 2002 Michael J. Black

ROBOT CONTROL From: Mijail Serruya August 2002 Michael J. Black

Beyond the Desktop Metaphor Biologically-embedded computing. new physical pathways for interacting with the world sensing and acting inside the body the computer learns about the brain while the brain is constantly changing Basic research in perception enabling machines to “see” their users. August 2002 Michael J. Black

Perceptual User Interfaces with Francois Berard August 2002 Michael J. Black

NEURAL IMPLANT Chronically implanted. Stable recording for 2 -3 years (but not necessarily the same cells every day) Spikes as well as local field potentials. August 2002 Michael J. Black

SINGLE UNIT ACTIVITY Spikes 2/1000’s second 1/10 mm August 2002 David Sheinberg Michael J. Black

CURRENT/FUTURE WORK * Learning low-dimensional linear models of conditional firing (“receptive fields”). * Comparison of Bayesian and non-Bayesian decoding methods (Kalman filter, particle filter, linear filter) * Incorporating local field potentials. * Analysis of more complex motions and statistical models. * Recognizing patterns of motion. * Plasticity. * Robot control (service robots, semi-autonomous). * Recording from multiple brain areas. * Human studies. August 2002 Michael J. Black

SUMMARY Bayesian model of neural activity in MI * probabilistic relationship between neural activity and events in the world. Non-parametric model computed using regularization * extensive cross-validation experiments * superior to previous methods Introduced particle filtering for the Bayesian inference of hand motion in non-overlapping 50 ms intervals * non-Gaussian likelihood and non-linear dynamics * supports more sophisticated analysis August 2002 Michael J. Black

THANKS M. Fellows, Neuroscience D. Sheinberg, Neuroscience N. Hatsopoulos, Neuroscience W. Patterson, Engineering A. Nurmikko, Engineering G. Friehs, Brown Medical School S. Geman, Division of Applied Mathematics S. Shoham, Princeton L. Paninski, NYU, Center for Neural Science Support: National Science Foundation, ITR Program. The Keck Foundation The National Institutes of Health August 2002 Michael J. Black

August 2002 Michael J. Black