Time-Travel Rephotography

Xuan Luo

University of Washington

Xuaner Zhang

Adobe Inc.

Paul Yoo

University of Washington

Ricardo Martin-Brualla

Google Research

Jason Lawrence

Google Research

Steven M. Seitz

University of Washington, Google Research



Best viewed full screen in 1080p to see details.

Abstract

Many historical people were only ever captured by old, faded, black and white photos, that are distorted due to the limitations of early cameras and the passage of time. This paper simulates traveling back in time with a modern camera to rephotograph famous subjects. Unlike conventional image restoration filters which apply independent operations like denoising, colorization, and superresolution, we leverage the StyleGAN2 framework to project old photos into the space of modern high-resolution photos, achieving all of these effects in a unified framework. A unique challenge with this approach is retaining the identity and pose of the subject in the original photo, while discarding the many artifacts frequently seen in low-quality antique photos. Our comparisons to current state-of-the-art restoration filters show significant improvements and compelling results for a variety of important historical people.

Media & Press


Two Minute Papers
GIZMODO
Hack a Day
量子位
果壳
腾讯网

Morph Sibling to Output

We can create the interesting effect of morphing the "modern" sibling to the output for the "historical" figures. Best viewed full screen in 1080p to see details.

Click here if you cannot view the video above.

BibTeX

@article{Luo-Rephotography-2021,
  author    = {Luo, Xuan and Zhang, Xuaner and Yoo, Paul and Martin-Brualla, Ricardo and Lawrence, Jason and Seitz, Steven M.},
  title     = {Time-Travel Rephotography},
  journal = {ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH Asia 2021)},
  publisher = {ACM New York, NY, USA},
  volume = {40},
  number = {6},
  articleno = {213},
  doi = {https://doi.org/10.1145/3478513.3480485},
  year = {2021},
  month = {12}
}

Acknowledgements

We thank Bo Zhang, Qingnan Fan, Roy Or-El, Aleksander Holynski and Keunhong Park for insightful advice.