Fri. Aug 12th, 2022
    en flag
    nl flag
    et flag
    fi flag
    fr flag
    de flag
    he flag
    ja flag
    lv flag
    pl flag
    pt flag
    es flag
    uk flag

    Content Assessment: Captivatingly Terrifying? One-Shot Megapixel Neural Head Avatars

    Information - 94%
    Insight - 95%
    Relevance - 88%
    Objectivity - 89%
    Authority - 90%

    91%

    Excellent

    A short percentage-based assessment of the qualitative benefit of the report highlighting a new and unique approach to image rendering from one-shot megapixel portraits.

    Editor’s Note: From time to time, ComplexDiscovery highlights publicly available or privately purchasable announcements, content updates, and research from cyber, data, and legal discovery providers, research organizations, and ComplexDiscovery community members. While ComplexDiscovery regularly highlights this information, it does not assume any responsibility for content assertions.

    To submit recommendations for consideration and inclusion in ComplexDiscovery’s cyber, data, and legal discovery-centric service, product, or research announcements, contact us today.


    Background Note: Shared for the non-commercial educational benefit of cybersecurity, information governance, and legal professionals, this recently published research report explains the first system for creating megapixel avatars from single portrait images. The report may be beneficial for investigators and litigators monitoring potential image-based rendering tools and technologies that may be used in deepfake creation. 

    Publication from arXiv*

    MegaPortraits: One-shot Megapixel Neural Head Avatars

    By Nikita Drobyshev, Jenya Chelishev, Taras Khakhulin, Aleksei Ivakhnenko, Victor Lempitsky, and Egor Zakharov

    Abstract

    In this work, we advance the neural head avatar technology to the megapixel resolution while focusing on the particularly challenging task of cross-driving synthesis, i.e., when the appearance of the driving image is substantially different from the animated source image. We propose a set of new neural architectures and training methods that can leverage both medium-resolution video data and high-resolution image data to achieve the desired levels of rendered image quality and generalization to novel views and motion. We demonstrate that suggested architectures and methods produce convincing high-resolution neural avatars, outperforming the competitors in the cross-driving scenario. Lastly, we show how a trained high-resolution neural avatar model can be distilled into a lightweight student model which runs in real-time and locks the identities of neural avatars to several dozens of pre-defined source images. Real-time operation and identity lock are essential for many practical applications head avatar systems.



    Introduction

    Neural head avatars offer a new fascinating way of creating virtual head models. They bypass the complexity of realistic physics-based modeling of human avatars by learning the shape and appearance directly from the videos of talking people. Over the last several years, methods that can create realistic avatars from a single photograph (one-shot) have been developed. They leverage extensive pre-training on the large datasets of videos of different people to create the avatars in the one-shot mode using generic knowledge about human appearance.

    Despite the impressive results obtained by this class of methods, their quality is severely limited by the resolution of the training datasets. This limitation cannot be easily bypassed by collecting a higher resolution dataset since it needs to be simultaneously large-scale and diverse, i.e., include thousands of humans with multiple frames per person, diverse demographics, lighting, background, face expression, and head pose. To the best of our knowledge, all public datasets that meet these criteria are limited in resolution. As a result, even the most recent one-shot avatar systems learn the avatars at resolutions up to 512 × 512.

    In our work, we make three main contributions. First, we propose a new model for one-shot neural avatars that achieves state-of-the-art cross-reenactment quality in up to 512 × 512 resolution. In our architecture, we utilize the idea of representing the appearance of the avatars as a latent 3D volume and propose a new way to combine it with the latent motion representations, which includes a novel contrastive loss that allows our system to achieve higher degrees of disentanglement between the latent motion and appearance representations. On top of that, we add a problem-specific gaze loss that increases the realism and accuracy of eye animation.

    Our second and crucial contribution is showing how a model trained on medium-resolution videos can be “upgraded” to the megapixel (1024 × 1024) resolution using an additional dataset of high-resolution still images. As a result, our proposed method, while using the same training dataset, outperforms the baseline super-resolution approach for the task of cross-reenactment. We are thus the first to demonstrate neural head avatars in proper megapixel resolution.

    Lastly, since many practical applications for human avatar creation require real-time or faster than real-time rendering, we distill our megapixel model into a ten times faster student model that runs at 130 FPS on a modern GPU. This significant speedup is possible since the student is trained for specific appearances (unlike the main model that can create new avatars for previously unseen people). Furthermore, the applications based on such a student model “locked” to predefined identities can prevent its misuse for creating “deep fakes” while at the same time achieving low rendering latency.

    Read the original post.


    Complete Report: MegaPortraits: One-shot Megapixel Neural Head Avatars (PDF) – Mouseover to Scroll

    MegaPortraits- One-Shot Megapixel Neural Head Avatars

    Read the original publication.

    *Shared with permission based on educational and non-commercial distribution under Creative Commons 4.0 International license.

    Publication Source:

    Nikita Drobyshev, Jenya Chelishev, Taras Khakhulin, Aleksei Ivakhnenko, Victor Lempitsky, and Egor Zakharov. 2022. MegaPortraits: One-shot Megapixel Neural Head Avatars. In Proceedings of the 30th ACM International Conference on Multimedia (MM ’22), October 10–14, 2022, Lisboa, Portugal. ACM, New York, NY, USA, 18 pages. https://doi.org/10.1145/3503161.3547838


    Additional Reading

    Source: ComplexDiscovery

     

    Have a Request?

    If you have information or offering requests that you would like to ask us about, please let us know and we will make our response to you a priority.

    ComplexDiscovery is an online publication that highlights cyber, data, and legal discovery insight and intelligence ranging from original research to aggregated news for use by cybersecurity, information governance, and eDiscovery professionals. The highly targeted publication seeks to increase the collective understanding of readers regarding cyber, data, and legal discovery information and issues and to provide an objective resource for considering trends, technologies, and services related to electronically stored information.

    ComplexDiscovery OÜ is a technology marketing firm providing strategic planning and tactical execution expertise in support of cyber, data, and legal discovery organizations. Focused primarily on supporting the ComplexDiscovery publication, the company is registered as a private limited company in the European Union country of Estonia, one of the most digitally advanced countries in the world. The company operates virtually worldwide to deliver marketing consulting and services.

    The Tip of the Iceberg? New ENISA Report on Threat Landscape for Ransomware Attacks

    According to ENISA, this threat landscape report analyzed a total of...

    Consumers Paying the Price? Cost of a Data Breach Hits Record High According to New IBM Report

    According to IBM Security, the annual Cost of a Data Breach Report...

    Safeguarding ePHI? NIST Updates Guidance for Health Care Cybersecurity

    This new NIST Special Publication aims to help educate readers about...

    Countering Threat Actors? Using Social Network Analysis for Cyber Threat Intelligence (CCDCOE)

    According to the NATO Cooperative Cyber Defence Centre of Excellence (CCDCOE)...

    Revenue Headwinds? KLDiscovery Inc. Announces Second Quarter 2022 Financial Results

    According to Christopher Weiler, CEO of KLDiscovery Inc, “The second quarter...

    Beyond Revenue? DISCO Announces Second Quarter 2022 Financial Results

    According to Kiwi Camara, Co-Founder and CEO of DISCO, “We are...

    Live with Leeds? Exterro Completes Recapitalization in Excess of $1 Billion

    According to the press release, with the support of a group...

    TCDI Completes Acquisition of Aon’s eDiscovery Practice

    According to TCDI Founder and CEO Bill Johnson, “We chose Aon’s...

    On the Move? 2022 eDiscovery Market Kinetics: Five Areas of Interest

    Recently ComplexDiscovery was provided an opportunity to share with the eDiscovery...

    Trusting the Process? 2021 eDiscovery Processing Task, Spend, and Cost Data Points

    Based on the complexity of cybersecurity, information governance, and legal discovery,...

    The Year in Review? 2021 eDiscovery Review Task, Spend, and Cost Data Points

    Based on the complexity of cybersecurity, information governance, and legal discovery,...

    A 2021 Look at eDiscovery Collection: Task, Spend, and Cost Data Points

    Based on the complexity of cybersecurity, information governance, and legal discovery,...

    Five Great Reads on Cyber, Data, and Legal Discovery for July 2022

    From lurking business undercurrents to captivating deepfake developments, the July 2022...

    Five Great Reads on Cyber, Data, and Legal Discovery for June 2022

    From eDiscovery ecosystem players and pricing to data breach investigations and...

    Five Great Reads on Cyber, Data, and Legal Discovery for May 2022

    From eDiscovery pricing and buyers to cyberattacks and incident response, the...

    Five Great Reads on Cyber, Data, and Legal Discovery for April 2022

    From cyber attack statistics and frameworks to eDiscovery investments and providers,...

    Inflection or Deflection? An Aggregate Overview of Eight Semi-Annual eDiscovery Pricing Surveys

    Initiated in the winter of 2019 and conducted eight times with...

    Changing Currents? Eighteen Observations on eDiscovery Business Confidence in the Summer of 2022

    In the summer of 2022, 54.8% of survey respondents felt that...

    Challenging Variants? Issues Impacting eDiscovery Business Performance: A Summer 2022 Overview

    In the summer of 2022, 28.8% of respondents viewed increasing types...

    Downshift Time? eDiscovery Operational Metrics in the Summer of 2022

    In the summer of 2022, 65 eDiscovery Business Confidence Survey participants...

    Droning On? Ukraine Conflict Assessments in Maps (August 3 – 7, 2022)

    According to a recent update from the Institute for the Study...

    Assuaging Distress? Ukraine Conflict Assessments in Maps (July 29 – August 2, 2022)

    According to a recent update from the Institute for the Study...

    Momentum Challenges? Ukraine Conflict Assessments in Maps (July 24 – 28, 2022)

    According to a recent update from the Institute for the Study...

    Port Support? Ukraine Conflict Assessments in Maps (July 19 – 23, 2022)

    According to a recent update from the Institute for the Study...