Lung cancer CT image generation from a free-form sketch using style-based pix2pix for data augmentation

Ryo Toda, Atsushi Teramoto, Masashi Kondo, Kazuyoshi Imaizumi, Kuniaki Saito, Hiroshi Fujita

Research output: Contribution to journalArticlepeer-review

10 Citations (Scopus)


Artificial intelligence (AI) applications in medical imaging continue facing the difficulty in collecting and using large datasets. One method proposed for solving this problem is data augmentation using fictitious images generated by generative adversarial networks (GANs). However, applying a GAN as a data augmentation technique has not been explored, owing to the quality and diversity of the generated images. To promote such applications by generating diverse images, this study aims to generate free-form lesion images from tumor sketches using a pix2pix-based model, which is an image-to-image translation model derived from GAN. As pix2pix, which assumes one-to-one image generation, is unsuitable for data augmentation, we propose StylePix2pix, which is independently improved to allow one-to-many image generation. The proposed model introduces a mapping network and style blocks from StyleGAN. Image generation results based on 20 tumor sketches created by a physician demonstrated that the proposed method can reproduce tumors with complex shapes. Additionally, the one-to-many image generation of StylePix2pix suggests effectiveness in data-augmentation applications.

Original languageEnglish
Article number12867
JournalScientific reports
Issue number1
Publication statusPublished - 12-2022

All Science Journal Classification (ASJC) codes

  • General


Dive into the research topics of 'Lung cancer CT image generation from a free-form sketch using style-based pix2pix for data augmentation'. Together they form a unique fingerprint.

Cite this