A Joint Loss based Deep Learning Model for Enhanced Image Inpainting

Haowen Yu; Liming Chen

doi:10.1109/dasc/picom/cbdcom/cy55231.2022.9927923

A Joint Loss based Deep Learning Model for Enhanced Image Inpainting

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Research on Image inpainting has attracted growing attention as image data and applications become commonplace. Nevertheless, it is still a challenge to repair or restore missing parts of electronic images to its original state. In this paper, we develop a new deep learning based model by combining the two mainstream generative models: Auto-encoders and Generative Adversarial Networks to improve the effect of image inpainting. Specifically we design a novel joint loss function which take into consideration reconstruction loss in a Generator and adversarial loss in a Discriminator to improve the repaired effect of texture features and semantic information. The model and joint loss function were experimented and evaluated using the public face data set celebA. Initial results showed that our model and function has significantly improved the effect of repairment in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) metrics.

Original language	English
Title of host publication	2022 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech)
Publisher	IEEE
Pages	1-6
Number of pages	6
ISBN (Electronic)	978-1-6654-6297-6
ISBN (Print)	978-1-6654-6298-3
DOIs	https://doi.org/10.1109/dasc/picom/cbdcom/cy55231.2022.9927923
Publication status	Published (in print/issue) - 13 Sept 2022
Event	International Conference on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress - Falerna, Italy Duration: 12 Sept 2022 → 15 Sept 2022

Publication series

Name	Proceedings of the 2022 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech 2022

Conference

Conference	International Conference on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress
Country/Territory	Italy
City	Falerna
Period	12/09/22 → 15/09/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Keywords

Image inpaining
Auto-encode
Generative Adversarial Network
Deep learning
Training
Measurement
Visualization
PSNR
Semantics
Neural networks

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/dasc/picom/cbdcom/cy55231.2022.9927923

Cite this

Yu, H., & Chen, L. (2022). A Joint Loss based Deep Learning Model for Enhanced Image Inpainting. In 2022 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) (pp. 1-6). (Proceedings of the 2022 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech 2022). IEEE. https://doi.org/10.1109/dasc/picom/cbdcom/cy55231.2022.9927923

Yu, Haowen ; Chen, Liming. / A Joint Loss based Deep Learning Model for Enhanced Image Inpainting. 2022 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech). IEEE, 2022. pp. 1-6 (Proceedings of the 2022 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech 2022).

@inproceedings{051538d48e104d6694ca3cf2bb4e6039,

title = "A Joint Loss based Deep Learning Model for Enhanced Image Inpainting",

abstract = "Research on Image inpainting has attracted growing attention as image data and applications become commonplace. Nevertheless, it is still a challenge to repair or restore missing parts of electronic images to its original state. In this paper, we develop a new deep learning based model by combining the two mainstream generative models: Auto-encoders and Generative Adversarial Networks to improve the effect of image inpainting. Specifically we design a novel joint loss function which take into consideration reconstruction loss in a Generator and adversarial loss in a Discriminator to improve the repaired effect of texture features and semantic information. The model and joint loss function were experimented and evaluated using the public face data set celebA. Initial results showed that our model and function has significantly improved the effect of repairment in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) metrics.",

keywords = "Image inpaining, Auto-encode, Generative Adversarial Network, Deep learning, Training, Measurement, Visualization, PSNR, Semantics, Neural networks",

author = "Haowen Yu and Liming Chen",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; International Conference on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress ; Conference date: 12-09-2022 Through 15-09-2022",

year = "2022",

month = sep,

day = "13",

doi = "10.1109/dasc/picom/cbdcom/cy55231.2022.9927923",

language = "English",

isbn = "978-1-6654-6298-3",

series = "Proceedings of the 2022 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech 2022",

publisher = "IEEE",

pages = "1--6",

booktitle = "2022 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech)",

address = "United States",

}

Yu, H & Chen, L 2022, A Joint Loss based Deep Learning Model for Enhanced Image Inpainting. in 2022 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech). Proceedings of the 2022 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech 2022, IEEE, pp. 1-6, International Conference on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress, Falerna, Italy, 12/09/22. https://doi.org/10.1109/dasc/picom/cbdcom/cy55231.2022.9927923

A Joint Loss based Deep Learning Model for Enhanced Image Inpainting. / Yu, Haowen; Chen, Liming.
2022 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech). IEEE, 2022. p. 1-6 (Proceedings of the 2022 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Joint Loss based Deep Learning Model for Enhanced Image Inpainting

AU - Yu, Haowen

AU - Chen, Liming

PY - 2022/9/13

Y1 - 2022/9/13

N2 - Research on Image inpainting has attracted growing attention as image data and applications become commonplace. Nevertheless, it is still a challenge to repair or restore missing parts of electronic images to its original state. In this paper, we develop a new deep learning based model by combining the two mainstream generative models: Auto-encoders and Generative Adversarial Networks to improve the effect of image inpainting. Specifically we design a novel joint loss function which take into consideration reconstruction loss in a Generator and adversarial loss in a Discriminator to improve the repaired effect of texture features and semantic information. The model and joint loss function were experimented and evaluated using the public face data set celebA. Initial results showed that our model and function has significantly improved the effect of repairment in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) metrics.

AB - Research on Image inpainting has attracted growing attention as image data and applications become commonplace. Nevertheless, it is still a challenge to repair or restore missing parts of electronic images to its original state. In this paper, we develop a new deep learning based model by combining the two mainstream generative models: Auto-encoders and Generative Adversarial Networks to improve the effect of image inpainting. Specifically we design a novel joint loss function which take into consideration reconstruction loss in a Generator and adversarial loss in a Discriminator to improve the repaired effect of texture features and semantic information. The model and joint loss function were experimented and evaluated using the public face data set celebA. Initial results showed that our model and function has significantly improved the effect of repairment in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) metrics.

KW - Image inpaining

KW - Auto-encode

KW - Generative Adversarial Network

KW - Deep learning

KW - Training

KW - Measurement

KW - Visualization

KW - PSNR

KW - Semantics

KW - Neural networks

UR - http://www.scopus.com/inward/record.url?scp=85145355692&partnerID=8YFLogxK

U2 - 10.1109/dasc/picom/cbdcom/cy55231.2022.9927923

DO - 10.1109/dasc/picom/cbdcom/cy55231.2022.9927923

M3 - Conference contribution

SN - 978-1-6654-6298-3

T3 - Proceedings of the 2022 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech 2022

SP - 1

EP - 6

BT - 2022 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech)

PB - IEEE

T2 - International Conference on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress

Y2 - 12 September 2022 through 15 September 2022

ER -

Yu H, Chen L. A Joint Loss based Deep Learning Model for Enhanced Image Inpainting. In 2022 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech). IEEE. 2022. p. 1-6. (Proceedings of the 2022 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech 2022). doi: 10.1109/dasc/picom/cbdcom/cy55231.2022.9927923

A Joint Loss based Deep Learning Model for Enhanced Image Inpainting

Abstract

Publication series

Conference

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this