Sun. Sep 25th, 2022
    en flag
    nl flag
    et flag
    fi flag
    fr flag
    de flag
    he flag
    ja flag
    lv flag
    pl flag
    pt flag
    es flag
    uk flag

    Content Assessment: Captivatingly Terrifying? One-Shot Megapixel Neural Head Avatars

    Information - 94%
    Insight - 95%
    Relevance - 88%
    Objectivity - 89%
    Authority - 90%

    91%

    Excellent

    A short percentage-based assessment of the qualitative benefit of the report highlighting a new and unique approach to image rendering from one-shot megapixel portraits.

    Editor’s Note: From time to time, ComplexDiscovery highlights publicly available or privately purchasable announcements, content updates, and research from cyber, data, and legal discovery providers, research organizations, and ComplexDiscovery community members. While ComplexDiscovery regularly highlights this information, it does not assume any responsibility for content assertions.

    To submit recommendations for consideration and inclusion in ComplexDiscovery’s cyber, data, and legal discovery-centric service, product, or research announcements, contact us today.


    Background Note: Shared for the non-commercial educational benefit of cybersecurity, information governance, and legal professionals, this recently published research report explains the first system for creating megapixel avatars from single portrait images. The report may be beneficial for investigators and litigators monitoring potential image-based rendering tools and technologies that may be used in deepfake creation. 

    Publication from arXiv*

    MegaPortraits: One-shot Megapixel Neural Head Avatars

    By Nikita Drobyshev, Jenya Chelishev, Taras Khakhulin, Aleksei Ivakhnenko, Victor Lempitsky, and Egor Zakharov

    Abstract

    In this work, we advance the neural head avatar technology to the megapixel resolution while focusing on the particularly challenging task of cross-driving synthesis, i.e., when the appearance of the driving image is substantially different from the animated source image. We propose a set of new neural architectures and training methods that can leverage both medium-resolution video data and high-resolution image data to achieve the desired levels of rendered image quality and generalization to novel views and motion. We demonstrate that suggested architectures and methods produce convincing high-resolution neural avatars, outperforming the competitors in the cross-driving scenario. Lastly, we show how a trained high-resolution neural avatar model can be distilled into a lightweight student model which runs in real-time and locks the identities of neural avatars to several dozens of pre-defined source images. Real-time operation and identity lock are essential for many practical applications head avatar systems.



    Introduction

    Neural head avatars offer a new fascinating way of creating virtual head models. They bypass the complexity of realistic physics-based modeling of human avatars by learning the shape and appearance directly from the videos of talking people. Over the last several years, methods that can create realistic avatars from a single photograph (one-shot) have been developed. They leverage extensive pre-training on the large datasets of videos of different people to create the avatars in the one-shot mode using generic knowledge about human appearance.

    Despite the impressive results obtained by this class of methods, their quality is severely limited by the resolution of the training datasets. This limitation cannot be easily bypassed by collecting a higher resolution dataset since it needs to be simultaneously large-scale and diverse, i.e., include thousands of humans with multiple frames per person, diverse demographics, lighting, background, face expression, and head pose. To the best of our knowledge, all public datasets that meet these criteria are limited in resolution. As a result, even the most recent one-shot avatar systems learn the avatars at resolutions up to 512 × 512.

    In our work, we make three main contributions. First, we propose a new model for one-shot neural avatars that achieves state-of-the-art cross-reenactment quality in up to 512 × 512 resolution. In our architecture, we utilize the idea of representing the appearance of the avatars as a latent 3D volume and propose a new way to combine it with the latent motion representations, which includes a novel contrastive loss that allows our system to achieve higher degrees of disentanglement between the latent motion and appearance representations. On top of that, we add a problem-specific gaze loss that increases the realism and accuracy of eye animation.

    Our second and crucial contribution is showing how a model trained on medium-resolution videos can be “upgraded” to the megapixel (1024 × 1024) resolution using an additional dataset of high-resolution still images. As a result, our proposed method, while using the same training dataset, outperforms the baseline super-resolution approach for the task of cross-reenactment. We are thus the first to demonstrate neural head avatars in proper megapixel resolution.

    Lastly, since many practical applications for human avatar creation require real-time or faster than real-time rendering, we distill our megapixel model into a ten times faster student model that runs at 130 FPS on a modern GPU. This significant speedup is possible since the student is trained for specific appearances (unlike the main model that can create new avatars for previously unseen people). Furthermore, the applications based on such a student model “locked” to predefined identities can prevent its misuse for creating “deep fakes” while at the same time achieving low rendering latency.

    Read the original post.


    Complete Report: MegaPortraits: One-shot Megapixel Neural Head Avatars (PDF) – Mouseover to Scroll

    MegaPortraits- One-Shot Megapixel Neural Head Avatars

    Read the original publication.

    *Shared with permission based on educational and non-commercial distribution under Creative Commons 4.0 International license.

    Publication Source:

    Nikita Drobyshev, Jenya Chelishev, Taras Khakhulin, Aleksei Ivakhnenko, Victor Lempitsky, and Egor Zakharov. 2022. MegaPortraits: One-shot Megapixel Neural Head Avatars. In Proceedings of the 30th ACM International Conference on Multimedia (MM ’22), October 10–14, 2022, Lisboa, Portugal. ACM, New York, NY, USA, 18 pages. https://doi.org/10.1145/3503161.3547838


    Additional Reading

    Source: ComplexDiscovery

     

    Have a Request?

    If you have information or offering requests that you would like to ask us about, please let us know and we will make our response to you a priority.

    ComplexDiscovery is an online publication that highlights cyber, data, and legal discovery insight and intelligence ranging from original research to aggregated news for use by cybersecurity, information governance, and eDiscovery professionals. The highly targeted publication seeks to increase the collective understanding of readers regarding cyber, data, and legal discovery information and issues and to provide an objective resource for considering trends, technologies, and services related to electronically stored information.

    ComplexDiscovery OÜ is a technology marketing firm providing strategic planning and tactical execution expertise in support of cyber, data, and legal discovery organizations. Focused primarily on supporting the ComplexDiscovery publication, the company is registered as a private limited company in the European Union country of Estonia, one of the most digitally advanced countries in the world. The company operates virtually worldwide to deliver marketing consulting and services.

    Leaning Forward? The CISA 2023-2025 Strategic Plan

    The purpose of the CISA Strategic Plan is to communicate the...

    Continuous Risk Improvement? Q3 Cyber Round-Up From Cowbell Cyber

    According to Manu Singh, director of risk engineering at Cowbell, "Every...

    A Comprehensive Cyber Discovery Resource? The DoD Cybersecurity Policy Chart from CSIAC

    The Cyber Security and Information Systems Information Analysis Center (CSIAC) is...

    Rapidly Evolving Cyber Insurance? Q2 Cyber Round-Up From Cowbell Cyber

    According to Isabelle Dumont, SVP of Marketing and Technology Partners at...

    Revealing Response? Nuix Responds to ASX Request for Information

    The following investor news update from Nuix shares a written response...

    Revealing Reports? Nuix Notes Press Speculation

    According to a September 9, 2022 market release from Nuix, the...

    Regards to Broadway? HaystackID® Acquires Business Intelligence Associates

    According to HaystackID CEO Hal Brooks, “BIA is a leader in...

    One Large Software and Cloud Business? OpenText to Acquire Micro Focus

    According to OpenText CEO & CTO Mark J. Barrenechea, “We are...

    On the Move? 2022 eDiscovery Market Kinetics: Five Areas of Interest

    Recently ComplexDiscovery was provided an opportunity to share with the eDiscovery...

    Trusting the Process? 2021 eDiscovery Processing Task, Spend, and Cost Data Points

    Based on the complexity of cybersecurity, information governance, and legal discovery,...

    The Year in Review? 2021 eDiscovery Review Task, Spend, and Cost Data Points

    Based on the complexity of cybersecurity, information governance, and legal discovery,...

    A 2021 Look at eDiscovery Collection: Task, Spend, and Cost Data Points

    Based on the complexity of cybersecurity, information governance, and legal discovery,...

    Five Great Reads on Cyber, Data, and Legal Discovery for September 2022

    From privacy legislation and special masters to acquisitions and investigations, the...

    Five Great Reads on Cyber, Data, and Legal Discovery for August 2022

    From AI and Big Data challenges to intriguing financial and investment...

    Five Great Reads on Cyber, Data, and Legal Discovery for July 2022

    From lurking business undercurrents to captivating deepfake developments, the July 2022...

    Five Great Reads on Cyber, Data, and Legal Discovery for June 2022

    From eDiscovery ecosystem players and pricing to data breach investigations and...

    Cooler Temperatures? Fall 2022 eDiscovery Business Confidence Survey Results

    Since January 2016, 2,874 individual responses to twenty-eight quarterly eDiscovery Business...

    Inflection or Deflection? An Aggregate Overview of Eight Semi-Annual eDiscovery Pricing Surveys

    Initiated in the winter of 2019 and conducted eight times with...

    Changing Currents? Eighteen Observations on eDiscovery Business Confidence in the Summer of 2022

    In the summer of 2022, 54.8% of survey respondents felt that...

    Challenging Variants? Issues Impacting eDiscovery Business Performance: A Summer 2022 Overview

    In the summer of 2022, 28.8% of respondents viewed increasing types...

    Nuclear Options? Ukraine Conflict Assessments in Maps (September 17 – 21, 2022)

    According to a recent update from the Institute for the Study...

    Mass Graves and Torture Chambers? Ukraine Conflict Assessments in Maps (September 12 – 16, 2022)

    According to a recent update from the Institute for the Study...

    On The Run? Ukraine Conflict Assessments in Maps (September 7 – 11, 2022)

    According to a recent update from the Institute for the Study...

    Tangible Degradation? Ukraine Conflict Assessments in Maps (September 2 – 6, 2022)

    According to a recent update from the Institute for the Study...