Time-Travel Rephotography

Xuan Luo

University of Washington

Xuaner Zhang

UC Berkeley

Paul Yoo

University of Washington

Ricardo Martin-Brualla

Google Research

Jason Lawrence

Google Research

Steven M. Seitz

University of Washington, Google Research

Best viewed full screen in 1080p to see details.

Abstract

Many historical people are captured only in old, faded, black and white photos, that have been distorted by the limitations of early cameras and the passage of time. This paper simulates traveling back in time with a modern camera to rephotograph famous subjects. Unlike conventional image restoration filters which apply independent operations like denoising, colorization, and superresolution, we leverage the StyleGAN2 framework to project old photos into the space of modern high-resolution photos, achieving all of these effects in a unified framework. A unique challenge with this approach is capturing the identity and pose of the photo's subject and not the many artifacts in low-quality antique photos. Our comparisons to current state-of-the-art restoration filters show significant improvements and compelling results for a variety of important historical people.

Comparisons

We compare our method with baselines that combine image restoration, colorization, and super-resolution, where the corresponding colorization algorithm is used to name the baseline. Please see our paper for detailed description of the baselines. Best viewed full screen to see details.

Morph Sibling to Output

We can create the interesting effect of morphing the "modern" sibling to the output for the "historical" figures. Best viewed full screen in 1080p to see details.

Click here if you cannot view the video above.

Age Progression

We can make fun animation of how the historical people progress in age given multiple images of them across different times. Best viewed full screen to see details.

BibTeX

@article{Luo-Rephotography-2020,
  author    = {Luo, Xuan and Zhang, Xuaner and Yoo, Paul and Martin-Brualla, Ricardo and Lawrence, Jason and Seitz, Steven M.},
  title     = {Time-Travel Rephotography},
  journal   = {arXiv preprint arXiv:2012.12261},
  year      = {2020},
}

Acknowledgements

We thank Bo Zhang, Qingnan Fan, Roy Or-El, Aleksander Holynski and Keunhong Park for insightful advice.