Most of us have quietly curtailed our lockdown hobbies, but internet creator Teigan Reamsbottom is still getting up at 4 a.m. to curate camp-classic pop culture video clips. He also does his best to improve the video quality using a dedicated gaming computer he’s stuffed full of RAM and a software suite that uses machine-learning algorithms to preserve the clips’ original characteristics. 

In a world where movies can be vanished for tax write-offs and physical media can be scarce, it’s heartening to see pop-culture artifacts unearthed — even if it means that no ’80s or ’90s celebrity is safe. 

Exhibit A: from the heyday of “Sex and the City,” an interview with Kim Cattrall where she scat sings while her then-husband Mark plays an upright bass. The clip has long been an internet fixture (it was once the subject of an exhibit at Lower East Side gallery THNK1994). It’s a great example of Reamsbottom’s eye for the delightfully unhinged and sometimes cringe camp from a pre-TikTok world. And in a certain sector of the internet, its restoration was cause for celebration. Look at the restored version versus the blurry clip we’ve been obsessing over for years below:

However upscaling video — improving its quality so that a clip can fit on our higher-resolution screens — is a process that’s laced (and/or interlaced) with challenges. Capturing the data and converting it is labor-intensive and every method leave artifacts behind that impact how the new footage looks. As machine-learning models take human labor and judgment out of the equation, that danger increases.

Even professional 4K transfers can appear airbrushed or plastic, an unnecessarily yassified version of something that looked perfectly good in its original format. (If you’ve made it this far into this article, this is your PSA to make sure you’ve turned off motion smoothing on your television.) As Chris Person noted for The Aftermath on the pitfalls of AI video upscaling, “Why transfer a tape correctly when we can just have a computer guess badly instead?”

That is, in essence, what AI video software does. It guesses where someone’s face begins and ends, how their hands move, how light and moisture react on their skin. It does this very badly, but does it so many times that (the hope is) it eventually gets close enough to being mostly right. For Reamsbottom, it is the best solution for publishing clips in which source video is unavailable — and, for the kind of brief camp moments that might not necessarily draw the eye of professional restoration houses. 

He told IndieWire that it’s a constant trial and error, balancing the different AI models available and adjusting sharpness and shadows in order to dodge the things that upscaling programs are most likely to fail. Like teeth. 

“As it’s increasing detail on the face, the AI model will over-increase the detail on the teeth and you’ll be able to see dark lines between the individual teeth,” Reamsbottom said. “So i”It can look a little scary. Some of the time it just looks like they have really dark teeth all of sudden because it outlines every tooth so much.”

Reamsbottom has to play with the amount of detail, always trying to avoid, in his words, turning the video’s subject into a Pixar character. He also must account for challenges within the images themselves. “I did one of Phyllis Diller recently, which was really hard to upscale because her outfit was sequined,” Reamsbottom said. “Her face might look good, but all of a sudden, the sequins didn’t look good. You really have to do a lot of playing around.” 

“Playing around” doesn’t begin to capture the time this demands. Reamsbottom said it can take over a day to do one pass on a 30-minute video, even with a dedicated upscaling computer. And machine learning can’t make pixelation pristine.

“You need to have something that’s at least of medium quality to turn it into something really nice,” he said. “Even then it can be iffy, but some of the super low-quality, highly pixelated stuff I’ve been trying to work with? It’s tough. The faces are tough, teeth are tough, or, you know, a nose might go missing.” 

While there are generative edit “AI” features on new Samsung phones (and similar models coming to iPhone) that appear to work much faster, being able to truly represent reality well through deep learning models is still the province of folks who work with video professionally and those who, like Reamsbottom, can devote a lot of time and dollars to the effort.

The time demand may still be high, but the tools to do this aren’t expensive; Reamsbottom uses a $300 software suite from Topaz Labs. And when done right, the results can be incredibly rewarding.

Reamsbottom has been working with an archive of tapes of Connie Francis, donated to him by the family of a fan who recorded a huge amount of footage of the pop singer. “There’s footage of him and personal footage of Connie, and when you see it upscaled, it almost makes you emotional because it’s like you’re experiencing something for the first time,” Reamsbottom said. “It’s magical when you get the end result, and it looks super clear. It’s like you’re there again.”

Source link

Leave a Reply

Your email address will not be published. Required fields are marked *