Chien-Wen Chen
ABSTRACT
For those people without artistic talent, they can only draw rough or even awful doodles to express their ideas. We propose a doodle beautification system named Doodle Master, which can transfer a rough doodle to a plausible image and also keep the semantic concepts of the drawings. The Doodle Master applies the VAE/GAN model to decode and generate the beautified result from a constrained latent space. To achieve better performance for sketch data which is more like discrete distribution, a shared-weight method is proposed to improve the learnt features of the discriminator with the aid of the encoder. Furthermore, we design an interface for the user to draw with basic drawing tools and adjust the number of reconstruction times. The experiments show that the proposed Doodle Master system can successfully beautify the rough doodle or sketch in real-time.
- Alex J Champandard . 2016. Semantic style transfer and turning two-bit doodles into fine artworks. arXiv preprint arXiv:1603.01768 (2016).Google Scholar
- Ian Goodfellow . 2016. Generative Adversarial Networks for Text. https://www.reddit.com/r/MachineLearning/comments/40ldq6/generative_adversarial_networks_for_text/Google Scholar
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio . 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680. Google ScholarDigital Library
- Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros . 2017. Image-to-Image Translation with Conditional Adversarial Networks. CVPR (2017).Google Scholar
- Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen . 2017. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 (2017).Google Scholar
- Diederik P Kingma and Max Welling . 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).Google Scholar
- Google Creative Lab . 2017. AutoDraw. https://experiments.withgoogle.com/chrome/autodrawGoogle Scholar
- Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther . 2015. Autoencoding beyond pixels using a learned similarity metric. arXiv preprint arXiv:1512.09300 (2015).Google Scholar
- Yong Jae Lee, C Lawrence Zitnick, and Michael F Cohen . 2011. Shadowdraw: real-time user guidance for freehand drawing ACM Transactions on Graphics (TOG), Vol. Vol. 30. ACM, 27. Google ScholarDigital Library
- Jonathan Long, Evan Shelhamer, and Trevor Darrell . 2015. Fully convolutional networks for semantic segmentation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3431--3440.Google Scholar
- Yongyi Lu, Shangzhe Wu, Yu-Wing Tai, and Chi-Keung Tang . 2017. Sketch-to-Image Generation Using Deep Contextual Completion. arXiv preprint arXiv:1711.08972 (2017).Google Scholar
- Laurens van der Maaten and Geoffrey Hinton . 2008. Visualizing data using t-SNE. Journal of machine learning research Vol. 9, Nov (2008), 2579--2605.Google Scholar
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox . 2015. U-net: Convolutional networks for biomedical image segmentation International Conference on Medical image computing and computer-assisted intervention. Springer, 234--241.Google Scholar
- Mihaela Rosca, Balaji Lakshminarayanan, David Warde-Farley, and Shakir Mohamed . 2017. Variational approaches for auto-encoding generative adversarial networks. arXiv preprint arXiv:1706.04987 (2017).Google Scholar
- Edgar Simo-Serra, Satoshi Iizuka, and Hiroshi Ishikawa . 2018. Mastering Sketching: Adversarial Augmentation for Structured Prediction. ACM Transactions on Graphics (TOG) Vol. 37, 1 (2018), 11. Google ScholarDigital Library
- Edgar Simo-Serra, Satoshi Iizuka, Kazuma Sasaki, and Hiroshi Ishikawa . 2016. Learning to simplify: fully convolutional networks for rough sketch cleanup. ACM Transactions on Graphics (TOG) Vol. 35, 4 (2016), 121. Google ScholarDigital Library
- A. Yu and K. Grauman . 2014. Fine-Grained Visual Comparisons with Local Learning Computer Vision and Pattern Recognition (CVPR). Google ScholarDigital Library
- A. Yu and K. Grauman . 2017. Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Images International Conference on Computer Vision (ICCV).Google Scholar
Index Terms
- Doodle Master: A Doodle Beautification System Based on Auto-encoding Generative Adversarial Networks
Comments