Michael J. Black


Michael J. Black is an American-born computer scientist working in Tübingen, Germany. He is a founding director at the Max Planck Institute for Intelligent Systems where he leads the Perceiving Systems Department in research focused on computer vision, machine learning, and computer graphics. He is also a Distinguished Amazon Scholar and an Honorary Professor at the University of Tübingen.
Black is the only researcher in the field to have won all three major test-of-time prizes in computer vision: the 2010 Koenderink Prize at the European Conference on Computer Vision, the 2013 Helmholtz Prize at the International Conference on Computer Vision, and the 2020 Longuet-Higgins Prize at the IEEE Conference on Computer Vision and Pattern Recognition.

Research

Optical flow

Black's thesis reformulated optical flow estimation as a robust M-estimation problem. The main observation was that spatial discontinuities in image motion and violations of the standard brightness constancy assumption could be treated as outliers. Reformulating the classical optimization problem as a robust estimation problem produced more accurate results.
This “Black and Anandan” optical flow algorithm has been widely used, for example, in special effects. The method was used to compute optical flow for the painterly effects in What Dreams May Come and for registering 3D face scans in The Matrix Reloaded.
A version of this work received the IEEE Outstanding Paper Award at CVPR 1991 and the Helmholtz Prize at ICCV 2013 for work that has "stood the test of time".
His early focus on statistical modeling of motion, particularly at motion discontinuities, led to two other prize papers. His work with David Fleet on the "Probabilistic Detection and Tracking of Motion Boundaries" won honorable mention for the Marr Prize at ICCV'99. Black's work with Stefan Roth "On the spatial statistics of optical flow" received honorable mention for the Marr Prize at ICCV 2005.
His work with and on the “” was awarded the 2020 . The prize is given annually by the IEEE Pattern Analysis and Machine Intelligence Technical Committee for “Contributions in Computer Vision that Have Withstood the Test of Time.” The “secrets” paper helped establish the state of the art in the field and led to the widely used Classic+NL flow algorithm.

Robust statistics and image statistics

The "Black and Anandan" method helped popularize robust statistics in computer vision. This was facilitated by several papers that connected robust penalty functions to classical "line processes" used in Markov Random Fields at the time. Black and Rangarajan characterized the formal properties of robust functions that have an equivalent line-process form and provided a process to convert between these formulations Black and colleagues applied these ideas to image denoising, anisotropic diffusion, and principal component analysis
The robust formulation was hand crafted and used small spatial neighborhoods. The work on Fields of Experts with Stefan Roth removed these restrictions. They learned the potential functions of an MRF with large spatial cliques by modeling the field potentials as a product of experts. Their formulation can be viewed as a shallow convolutional neural network.

Layered motion estimation

In 1993, Black and Jepson used mixture models to represent optical flow fields with multiple motions . This introduced the use of Expectation Maximization to the field of computer vision.

Neural decoding and neural prosthetics

In the 2000s, Black worked with John Donoghue and others at Brown University to create the technology behind the BrainGate neural prosthetics technology. Black and colleagues developed Bayesian methods to decode neural signals from motor cortex. The team was the first to use Kalman filtering and particle filtering to decode motor cortical ensemble activity.  With these Bayesian decoding methods, the team demonstrated the successful point-and-click control of a computer cursor by a human with paralysis and the decoding of full arm and hand movement in non-human primates.

Human motion and shape

Black is best known for his work on human motion and shape estimation. With Hedvig Sidenbladh and David Fleet, he introduced the use of particle filtering for tracking 3D articulated human motion. This work was awarded the Koenderink Koenderink Prize for Fundamental Contributions in Computer Vision at ECCV 2000.
His current work focuses on modeling and estimating human shape and pose from images and video. His team was the first to fit a learned 3D human body model to multi-camera image data at CVPR 2007, under clothing at ECCV 2008, from a single image at ICCV 2009, and from RGB-D data at ICCV 2011.
His group produced the popular SMPL 3D body model and popularized methods for estimating 3D body shape from images. SMPL is widely used in both academia and industry and was one of the core technologies licensed by Body Labs Inc.

Differentiable rendering

Loper and Black popularized "differentiable rendering" which has become an important component of self-supervised training of neural networks for problems like facial analysis. Classical methods for analysis by synthesis formulate an objective function and then differentiate it. The OpenDR method was more generic in that it differentiated a graphics rendering engine using Automatic differentiation. This provided a framework for posing a forward synthesis problem and automatically obtaining an optimization method to solve the inverse problem.

Datasets

Black has contributed to several significant datasets. The Middlebury Flow dataset provided the first comprehensive benchmark for the field. The MPI-Sintel Flow dataset demonstrated that synthetic data was sufficiently rich and similar to real data to provide a rigorous benchmark and to be useful for learning optical flow.
The HumanEva dataset was the first dataset with ground truth 3D human poses in correspondence with RGB video of people in motion. The approach used a combination of optical motion capture and multi-camera video capture. This dataset enabled the field to evaluate accuracy and compare performance for the first time.
Related to human pose, shape, and activity, Black has contributed to the SURREAL dataset of human motions, the JHMDB dataset of human actions, and the FAUST dataset of 3D body shapes. FAUST received the Dataset Award from the Eurographics Symposium on Geometry Processing, 2016.

Employment

1985–1989: After his bachelor's degree, Black moved to the Bay Area and worked as a software engineer at GTE Government Systems and Advanced Decision Systems developing expert systems on the Xerox and Symbolics Lisp machines. During this time, he completed his Master's of Computer Science in Symbolic and Heuristic Computation through the Honors Co-Op Program at Stanford. His advisor was John McCarthy.
1989–1992: During this period, Black pursued his PhD at Yale and was supported by a NASA Graduate Fellowship. He completed his PhD at the NASA Ames Research Center in the Human Factors Research Division led by Andrew Watson. At Yale, he was advised by P. Anandan and Drew McDermott.
1992–1993: Black did post-doctoral work at the University of Toronto as an Assistant Professor of Computer Science. He was supervised by Allan Jepson. During his time there, he received the Computer Science Students' Union Teaching Award.
1993-2000: In 1993, Black joined the Xerox Palo Alto Research Center as a member of research staff. He worked in the Image Understanding Area, led by Daniel Huttenlocher. In 1996, he took over management from Huttenlocher. He started the Digital Video Analysis Area in 1998.  
2000–2011: In 2000, Black joined the faculty of Brown University as an Associate Professor of Computer Science. In 2004, he was promoted to Full Professor.
2011–present: In 2011, Black became a Scientific Member of the Max Planck Society and one of the founding directors of the new MPI for Intelligent Systems.
2017–present: In 2017, with the acquisition of Body Labs by Amazon, Black joined Amazon as a Distinguished Amazon Scholar on a part-time basis.

Administration

In addition to co-founding the MPI for Intelligent Systems, Black led the founding of the International Max Planck Research School for Intelligent Systems.
In 2015, he proposed an initiative that has since become Cyber Valley, which aims to make the Stuttgart-Tübingen region of Germany a world leader in AI research and applications. He is on the research consortium's Executive Board and serves as its spokesperson.

Entrepreneurship

In 2013, a team from Black's group spun out Body Labs which commercialized 3D body model technology for the clothing and games industry. Black was a co-founder and investor. Body Labs was acquired by Amazon.com in 2017.
In 2018, Meshcapade GmbH spun out of his group. The start-up focuses on licensing technology developed at MPI-IS and providing services.