Skip to main content
SpringerLink
Log in

A review of design intelligence: progress, problems, and challenges

  • Review
  • Published:
Frontiers of Information Technology & Electronic Engineering Aims and scope Submit manuscript

Abstract

Design intelligence is an important branch of artificial intelligence (AI), focusing on the intelligent models and algorithms in creativity and design. In the context of AI 2.0, studies on design intelligence have developed rapidly. We summarize mainly the current emerging framework of design intelligence and review the state-of-the-art techniques of related topics, including user needs analysis, ideation, content generation, and design evaluation. Specifically, the models and methods of intelligence-generated content are reviewed in detail. Finally, we discuss some open problems and challenges for future research in design intelligence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Arjovsky M, Chintala S, Bottou L, 2017. Wasserstein generative adversarial networks. Proc 34th Int Conf on Machine Learning, p.298–321.

  • Aubry M, Maturana D, Efros AA, et al., 2014. Seeing 3D chairs: exemplar part-based 2D-3D alignment using a large dataset of CAD models. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.3762–3769. https://doi.org/10.1109/CVPR.2014.487

  • Ballester C, Bertalmio M, Caselles V, et al., 2001. Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans Image Process, 10(8):1200–1211. https://doi.org/10.1109/83.935036

    MathSciNet  MATH  Google Scholar 

  • Bertalmio M, Sapiro G, Caselles V, et al., 2000. Image in-painting. Proc 27th Annual Conf on Computer Graphics and Interactive Techniques, p.417–424. https://doi.org/10.1145/344779.344972

  • Bharadhwaj H, Park H, Lim BY, 2018. RecGAN: recurrent generative adversarial networks for recommendation systems. Proc 12th ACM Conf on Recommender Systems, p.372–376. https://doi.org/10.1145/3240323.3240383

  • Boden MA, 2009. Computer models of creativity. AI Mag, 30(3):23–34. https://doi.org/10.1609/aimag.v30i3.2254

    Google Scholar 

  • Brock A, Donahue J, Simonyan K, 2018. Large scale GAN training for high fidelity natural image synthesis. https://doi.org/1809.11096

  • Bruna J, Sprechmann P, LeCun Y, 2015. Super-resolution with deep convolutional sufficient statistics. https://doi.org/1511.05666

  • Chakrabarti A, Siddharth L, Dinakar M, et al., 2017. Idea inspire 3.0—a tool for analogical design. In: Chakrabarti A, Chakrabarti D (Eds.), Research into Design for Communities. Springer, Singapore, p.475–485. https://doi.org/10.1007/978-981-10-3521-0_41

    Google Scholar 

  • Champandard AJ, 2016. Semantic style transfer and turning two-bit doodles into fine artworks. https://doi.org/1603.01768

  • Chan C, Ginosar S, Zhou TH, et al., 2018. Everybody dance now. https://doi.org/1808.07371

  • Chen DD, Yuan L, Liao J, et al., 2018. Stereoscopic neural style transfer. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.6654–6663. https://doi.org/10.1109/CVPR.2018.00696

  • Chen LQ, Wang P, Dong H, et al., 2019. An artificial intelligence based data-driven approach for design ideation. J Vis Commun Image Represent, 61:10–22. https://doi.org/10.1016/j.jvcir.2019.02.009

    Google Scholar 

  • Ciesielski V, Barile P, Trist K, 2013. Finding image features associated with high aesthetic value by machine learning. Proc 2nd Int Conf on Evolutionary and Biologically Inspired Music, Sound, Art and Design, p.47–58. https://doi.org/10.1007/978-3-642-36955-1_5

    Google Scholar 

  • Cooper A, 1999. The Inmates Are Running the Asylum. SAMS, Indianapolis, USA.

    Google Scholar 

  • Cooper A, Reimann RM, 2003. About Face 2.0: the Essentials of Interaction Design. John Wiley & Sons, Indianapolis, USA.

    Google Scholar 

  • Dash A, Gamboa JCB, Ahmed S, et al., 2017. TAC-GAN-text conditioned auxiliary classifier generative adversarial network. https://doi.org/1703.06412

  • Datta R, Joshi D, Li J, et al., 2006. Studying aesthetics in photographic images using a computational approach. Proc 9th European Conf on Computer Vision, p.288–301. https://doi.org/10.1007/11744078_23

    Google Scholar 

  • de Gómez Silva Garza A, Maher ML, 1999. An evolutionary approach to case adaptation. Proc 3rd Int Conf on Case-Based Reasoning, p.162–173. https://doi.org/10.1007/3-540-48508-2_12

    Google Scholar 

  • de Silva Garza AG, 2019. An introduction to and comparison of computational creativity and design computing. Artif Intell Rev, 51(1):61–76. https://doi.org/10.1007/s10462-017-9557-3

    Google Scholar 

  • Deng J, Dong W, Socher R, et al., 2009. ImageNet: a large-scale hierarchical image database. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.248–255. https://doi.org/10.1109/CVPR.2009.5206848

  • Deng YB, Loy CC, Tang XO, 2018. Aesthetic-driven image enhancement by adversarial learning. Proc 26th ACM Int Conf on Multimedia, p.870–878. https://doi.org/10.1145/3240508.3240531

  • Donahue J, Krähenbühl P, Darrell T, 2016. Adversarial feature learning. https://doi.org/1605.09782

  • Dou Q, Zheng XS, Sun TF, et al., 2019. Webthetics: quantifying webpage aesthetics with deep learning. Int J Hum Comput Stud, 124:56–66. https://doi.org/10.1016/j.ijhcs.2018.11.006

    Google Scholar 

  • Dugosh KL, Paulus PB, Roland EJ, et al., 2000. Cognitive stimulation in brainstorming. J Pers Soc Psychol, 79(5):722–735. https://doi.org/10.1037/0022-3514.79.5.722

    Google Scholar 

  • Dumoulin V, Visin F, 2016. A guide to convolution arithmetic for deep learning. https://doi.org/1603.07285

  • Edelman RR, Hesselink JR, Zlatkin MB, 1996. MRI: Clinical Magnetic Resonance Imaging. Saunders, Philadelphia.

    Google Scholar 

  • Efros AA, Freeman WT, 2001. Image quilting for texture synthesis and transfer. Proc 28th Annual Conf on Computer Graphics and Interactive Techniques, p.341–346. https://doi.org/10.1145/383259.383296

  • Elgammal A, Liu B, Elhoseiny M, et al., 2017. CAN: creative adversarial networks, generating “art” by learning about styles and deviating from style norms. https://doi.org/1706.07068

  • Fang H, Zhang M, 2017. Creatism: a deep-learning photographer capable of creating professional work. https://doi.org/1707.03491

  • Faste H, Rachmel N, Essary R, et al., 2013. Brainstorm, chainstorm, cheatstorm, tweetstorm: new ideation strategies for distributed HCI design. Proc Conf on Human Factors in Computing Systems, p.1343–1352. https://doi.org/10.1145/2470654.2466177

  • Fu K, Murphy J, Yang M, et al., 2015. Design-by-analogy: experimental evaluation of a functional analogy search methodology for concept generation improvement. Res Eng Des, 26(1):77–95. https://doi.org/10.1007/s00163-014-0186-4

    Google Scholar 

  • Garabedian CA, 1934. Birkhoff on aesthetic measure. Bull Amer Math Soc, 40(1):7–10. https://doi.org/10.1090/S0002-9904-1934-05764-1

    MathSciNet  Google Scholar 

  • Gatys L, Ecker A, Bethge M, 2016a. Image style transfer using convolutional neural networks. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2414–2423. https://doi.org/10.1109/CVPR.2016.265

  • Gatys L, Ecker A, Bethge M, 2016b. A neural algorithm of artistic style. J Vis, 16(12):326. https://doi.org/10.1167/16.12.326

    Google Scholar 

  • Gero JS, 1990. Design prototypes: a knowledge representation schema for design. AI Mag, 11(4):26–36.

    Google Scholar 

  • Gilon K, Chan J, Ng FY, et al., 2018. Analogy mining for specific design needs. Proc CHI Conf on Human Factors in Computing Systems, p.121. https://doi.org/10.1145/3173574.3173695

  • Goel AK, Rugaber S, Vattam S, 2009. Structure, behavior, and function of complex systems: the structure, behavior, and function modeling language. AI Edam, 23(1):23–35. https://doi.org/10.1017/S0890060409000080

    Google Scholar 

  • Goldschmidt G, Smolkov M, 2006. Variances in the impact of visual stimuli on design problem solving performance. Des Stud, 27(5):549–569. https://doi.org/10.1016/j.destud.2006.01.002

    Google Scholar 

  • Gooch B, Gooch A, 2001. Non-photorealistic Rendering. A K Peters/CRC Press, New York, USA. https://doi.org/10.1201/9781439864173

    MATH  Google Scholar 

  • Goodfellow I, Pouget-Abadie J, Mirza M, et al., 2014. Generative adversarial nets. Proc 27th Int Conf on Neural Information Processing Systems, p.2672–2680.

  • Grudin J, Pruitt J, 2002. Personas, participatory design, and product development: an infrastructure for engagement. Proc 7th Biennial Participatory Design Conf, p.144–152.

  • Gulrajani I, Ahmed F, Arjovsky M, et al., 2017. Improved training of Wasserstein GANs. Advances in Neural Information Proc Systems, p.5767–5777.

  • Han J, Shi F, Chen LQ, et al., 2018. A computational tool for creative idea generation based on analogical reasoning and ontology. Artif Intell Eng Des Anal Manuf, 32(4):462–477. https://doi.org/10.1017/S0890060418000082

    Google Scholar 

  • Hao J, Zhou YJ, Zhao QF, et al., 2019. An evolutionary computation based method for creative design inspiration generation. J Intell Manuf, 30(4):1673–1691. https://doi.org/10.1007/s10845-017-1347-x

    Google Scholar 

  • Hartson R, Pyla PS, 2012. The UX Book: Process and Guidelines for Ensuring a Quality User Experience. Elsevier, Amsterdam. https://doi.org/10.1016/C2010-0-66326-7

    Google Scholar 

  • He KM, Sun J, 2014. Image completion approaches using the statistics of similar patches. IEEE Trans Patt Anal Mach Intell, 36(12):2423–2435. https://doi.org/10.1109/TPAMI.2014.2330611

    Google Scholar 

  • Hertzmann A, Jacobs CE, Oliver N, et al., 2001. Image analogies. Proc 28th Annual Conf on Computer Graphics and Interactive Techniques, p.327–340. https://doi.org/10.1145/383259.383295

  • Hong YJ, Hwang U, Yoo J, et al., 2019. How generative adversarial networks and their variants work: an overview. ACM Comput Surv, 52(1):10. https://doi.org/10.1145/3301282

    Google Scholar 

  • Huang HZ, Wang H, Luo WH, et al., 2017. Real-time neural style transfer for videos. IEEE Conf on Computer Vision and Pattern Recognition, p.7044–7052. https://doi.org/10.1109/CVPR.2017.745

  • Huang X, Belongie S, 2017. Arbitrary style transfer in realtime with adaptive instance normalization. Proc IEEE Int Conf on Computer Vision, p.1501–1510. https://doi.org/10.1109/ICCV.2017.167

  • Iizuka S, Simo-Serra E, Ishikawa H, 2017. Globally and locally consistent image completion. ACM Trans Graph, 36(4), Article 107. https://doi.org/10.1145/3072959.3073659

    Google Scholar 

  • Isola P, Zhu JY, Zhou TH, et al., 2017. Image-to-image translation with conditional adversarial networks. IEEE Conf on Computer Vision and Pattern Recognition, p.5967–5976. https://doi.org/10.1109/CVPR.2017.632

  • Jansen BJ, Jung SG, Salminen J, et al., 2017. Viewed by too many or viewed too little: using information dissemination for audience segmentation. Proc Assoc Inform Sci Technol, 54(1):189–196. https://doi.org/10.1002/pra2.2017.14505401021

    Google Scholar 

  • Jansson DG, Smith SM, 1991. Design fixation. Des Stud, 12(1):3–11. https://doi.org/10.1016/0142-694X(91)90003-F

    Google Scholar 

  • Jia J, Huang J, Shen GY, et al., 2016. Learning to appreciate the aesthetic effects of clothing. Proc 30th AAAI Conf on Artificial Intelligence, p.1216–1222.

  • Jia L, Becattini N, Cascini G, et al., 2020. Testing ideation performance on a large set of designers: effects of analogical distance. Int J Des Creat Innov, 8(1):31–45. https://doi.org/10.1080/21650349.2019.1618736

    Google Scholar 

  • Jiang SH, Fu Y, 2017. Fashion style generator. Proc 26th Int Joint Conf on Artificial Intelligence, p.3721–3727. https://doi.org/10.24963/ijcai.2017/520

  • Jing YC, Yang YZ, Feng ZL, et al., 2019. Neural style transfer: a review. IEEE Trans Vis Comput Graph, in press. https://doi.org/10.1109/tvcg.2019.2921336

  • Jo Y, Park J, 2019. SC-FEGAN: face editing generative adversarial network with user’s sketch and color. https://doi.org/1902.06838

  • Johnson J, Alahi A, Li FF, 2016. Perceptual losses for real-time style transfer and super-resolution. Proc 14th European Conf, p.694–711. https://doi.org/10.1007/978-3-319-46475-6_43

    Google Scholar 

  • Karras T, Laine S, Aila T, 2019. A style-based generator architecture for generative adversarial networks. The IEEE Conf on Computer Vision and Pattern Recognition, p.4401–4410.

  • Kaufman JC, Sternberg RJ, 2006. The International Handbook of Creativity. Edward Elgar Publishing, Cheltenham, UK.

    Google Scholar 

  • Keys R, 1981. Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Process, 29(6):1153–1160. https://doi.org/10.1109/TASSP.1981.1163711

    MathSciNet  MATH  Google Scholar 

  • Kim J, Lee JK, Lee KM, 2016. Accurate image superresolution using very deep convolutional networks. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1646–1654. https://doi.org/10.1109/CVPR.2016.182

  • Kingma DP, Welling M, 2013. Auto-encoding variational Bayes. https://arxiv.org/abs/1312.6114

  • Kong S, Shen XH, Lin Z, et al., 2016. Photo aesthetics ranking network with attributes and content adaptation. Proc 14th European Conf on Computer Vision, p.662–679. https://doi.org/10.1007/978-3-319-46448-0_40

    Google Scholar 

  • Krizhevsky A, Hinton G, 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report, University of Toronto, Toronto.

    Google Scholar 

  • Kwak H, An J, Jansen BJ, 2017. Automatic generation of personas using YouTube social media data. Proc 50th Hawaii Int Conf on System Sciences, p.833–842.

  • Larsen ABL, Sønderby SK, Larochelle H, et al., 2016. Autoencoding beyond pixels using a learned similarity metric. Proc 33rd Int Conf on Machine Learning, p.1558–1566.

  • LeCun Y, Bottou L, Bengio Y, et al., 1998. Gradient-based learning applied to document recognition. Proc IEEE, 86(11):2278–2323. https://doi.org/10.1109/5.726791

    Google Scholar 

  • Ledig C, Theis L, Huszár F, et al., 2017. Photo-realistic single image super-resolution using a generative adversarial network. IEEE Conf on Computer Vision and Pattern Recognition, p.105–114. https://doi.org/10.1109/CVPR.2017.19

  • Li C, Wand M, 2016. Precomputed real-time texture synthesis with Markovian generative adversarial networks. Proc 14th European Conf on Computer Vision, p.702–716. https://doi.org/10.1007/978-3-319-46487-9_43

    Google Scholar 

  • Li CC, Chen T, 2009. Aesthetic visual quality assessment of paintings. IEEE J Sel Top Signal Process, 3(2):236–252. https://doi.org/10.1109/JSTSP.2009.2015077

    Google Scholar 

  • Li HH, Wang JG, Tang MM, et al., 2017. Polarization-dependent effects of an Airy beam due to the spin-orbit coupling. J Opt Soc Am A, 34(7):1114–1118. https://doi.org/10.1364/JOSAA.34.001114

    Google Scholar 

  • Li XT, Liu SF, Kautz J, et al., 2019. Learning linear transformations for fast arbitrary style transfer. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.3809–3817.

  • Li YJ, Fang C, Yang JM, et al., 2017. Universal style transfer via feature transforms. Proc 31st Conf on Neural Information Processing Systems, p.386–396.

  • Liu GL, Reda FA, Shih KJ, et al., 2018. Image inpainting for irregular holes using partial convolutions. Proc 15th European Conf on Computer Vision, p.85–105. https://doi.org/10.1007/978-3-030-01252-6_6

    Google Scholar 

  • Liu H, Singh P, 2004. ConceptNet—a practical commonsense reasoning tool-kit. BT Technol J, 22(4):211–226. https://doi.org/10.1023/B:BTTJ.0000047600.45421.6d

    Google Scholar 

  • Liu MY, Huang X, Mallya A, et al., 2019. Few-shot unsupervised image-to-image translation. https://doi.org/1905.01723

  • Liu ZW, Luo P, Wang XG, et al., 2015. Deep learning face attributes in the wild. Proc IEEE Int Conf on Computer Vision, p.3730–3738. https://doi.org/10.1109/ICCV.2015.425

  • Lowdermilk T, 2013. User-Centered Design: a Developer’s Guide to Building User-Friendly Applications. O’Reilly, Beijing, China.

    Google Scholar 

  • Lu X, Lin Z, Shen XH, et al., 2015. Deep multi-patch aggregation network for image style, aesthetics, and quality estimation. Proc IEEE Int Conf on Computer Vision, p.990–998. https://doi.org/10.1109/ICCV.2015.119

  • Luo YW, Tang XO, 2008. Photo and video quality evaluation: focusing on the subject. Proc 10th European Conf on Computer Vision, p.386–399.

    Google Scholar 

  • Ma S, Liu J, Chen WC, 2017. A-lamp: adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment. Proc 30th IEEE Conf on Computer Vision and Pattern Recognition, p.722–731. https://doi.org/10.1109/CVPR.2017.84

  • Maguire M, Bevan N, 2002. User requirements analysis. In: Hammond J, Gross T, Wesson J (Eds.), Usability: Gaining a Competitive Edge. Springer, Boston, USA, p.133–148. https://doi.org/10.1007/978-0-387-35610-5_9

    Google Scholar 

  • Mai L, Jin HL, Liu F, 2016. Composition-preserving deep photo aesthetics assessment. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.497–506. https://doi.org/10.1109/CVPR.2016.60

  • Matthews T, Judge T, Whittaker S, 2012. How do designers and user experience professionals actually perceive and use personas? Proc Conf on Human Factors in Computing Systems, p.1219–1228. https://doi.org/10.1145/2207676.2208573

  • McCaffrey T, Krishnamurty S, 2015. The obscure features hypothesis in design innovation. Int J Des Creat Innov, 3(1):1–28. https://doi.org/10.1080/21650349.2014.893840

    Google Scholar 

  • McGinn J, Kotamraju N, 2008. Data-driven persona development. Proc Conf on Human Factors in Computing Systems, p.1521–1524. https://doi.org/10.1145/1357054.1357292

  • Miaskiewicz T, Kozar KA, 2011. Personas and user-centered design: how can personas benefit product design processes? Des Stud, 32(5):417–430. https://doi.org/10.1016/j.destud.2011.03.003

    Google Scholar 

  • Mikolov T, Chen K, Corrado G, et al., 2013. Efficient estimation of word representations in vector space. https://doi.org/1301.3781

  • Miller GA, 1995. Wordnet: a lexical database for English. Commun ACM, 38(11):39–41. https://doi.org/10.1145/219717.219748

    Google Scholar 

  • Mirza M, Osindero S, 2014. Conditional generative adversarial nets. https://doi.org/1411.1784

  • Miyato T, Kataoka T, Koyama M, et al., 2018. Spectral normalization for generative adversarial networks. Int Conf on Learning Representations.

  • Murray N, Marchesotti L, Perronnin F, 2012. AVA: a large-scale database for aesthetic visual analysis. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2408–2415. https://doi.org/10.1109/CVPR.2012.6247954

  • Nazeri K, Ng E, Joseph T, et al., 2019. Edgeconnect: generative image inpainting with adversarial edge learning. https://doi.org/1901.00212

  • Nelson BA, Wilson JO, Rosen D, et al., 2009. Refined metrics for measuring ideation effectiveness. Des Stud, 30(6):737–743. https://doi.org/10.1016/j.destud.2009.07.002

    Google Scholar 

  • Nielsen L, Hansen KS, Stage J, et al., 2015. A template for design personas: analysis of 47 persona descriptions from Danish industries and organizations. Int J Sociotechnol Knowl Dev, 7(1):45–61. https://doi.org/10.4018/ijskd.2015010104

    Google Scholar 

  • Niles I, Pease A, 2001. Towards a standard upper ontology. Proc Int Conf on Formal Ontology in Information Systems, p.2–9. https://doi.org/10.1145/505168.505170

  • Nilsback ME, Zisserman A, 2008. Automated flower classification over a large number of classes. Proc 6th Indian Conf on Computer Vision, Graphics & Image Processing, p.722–729. https://doi.org/10.1109/ICVGIP.2008.47

  • Odena A, Olah C, Shlens J, 2017. Conditional image synthesis with auxiliary classifier GANs. Proc 34th Int Conf on Machine Learning, p.4043–4055.

  • Pan YH, 2017. Special issue on artificial intelligence 2.0. Front Inform Technol Electron Eng, 18(1):1–2. https://doi.org/10.1631/FITEE.1710000

    Google Scholar 

  • Park T, Liu MY, Wang TC, et al., 2019. Semantic image synthesis with spatially-adaptive normalization. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2337–2346.

  • Pathak D, Krähenbühl P, Donahue J, et al., 2016. Context encoders: feature learning by inpainting. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2536–2544. https://doi.org/10.1109/CVPR.2016.278

  • Peeters JR, Verhaegen PA, Vandevenne D, et al., 2010. Refined metrics for measuring novelty in ideation. ID-MME Virtual Concept Research in Interaction Design, Article 4.

  • Perera D, Zimmermann R, 2019. CNGAN: generative adversarial networks for cross-network user preference generation for non-overlapped users. World Wide Web Conf, p.3144–3150. https://doi.org/10.1145/3308558.3313733

  • Pruitt J, Adlin T, 2005. The Persona Lifecycle: Keeping People in Mind Throughout Product Design. Elsevier, Amsterdam, p.724.

    Google Scholar 

  • Radford A, Metz L, Chintala S, 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. Proc 4th Int Conf on Learning Representations.

  • Reed SE, Akata Z, Yan XC, et al., 2016a. Generative adversarial text to image synthesis. Proc 33rd Int Conf on Machine Learning, p.1681–1690.

  • Reed SE, Akata Z, Mohan S, et al., 2016b. Learning what and where to draw. Advances in Neural Information Processing Systems, p.217–225.

  • Rigau J, Feixas M, Sbert M, 2008. Informational aesthetics measures. IEEE Comput Graph Appl, 28(2):24–34. https://doi.org/10.1109/MCG.2008.34

    Google Scholar 

  • Russell SJ, Norvig P, 2016. Artificial Intelligence: a Modern Approach. Pearson Education Limited, Harlow, Essex.

    MATH  Google Scholar 

  • Saleh B, Elgammal A, 2015. Large-scale classification of fine-art paintings: learning the right metric on the right feature. https://doi.org/1505.00855

  • Salimans T, Goodfellow IJ, Zaremba W, et al., 2016. Improved techniques for training GANs. Advances in Neural Information Processing Systems, p.2226–2234.

  • Salminen J, Sengün S, Kwak H, et al., 2017. Generating cultural personas from social data: a perspective of middle eastern users. Proc 5th Int Conf on Future Internet of Things and Cloud Workshops, p.120–125. https://doi.org/10.1109/FiCloudW.2017.97

  • Salminen J, Jansen BJ, An J, et al., 2018a. Are personas done? Evaluating their usefulness in the age of digital analytics. Persona Stud, 4(2):47–65. https://doi.org/10.21153/psj2018vol4no2art737

    Google Scholar 

  • Salminen J, Jung SG, An J, et al., 2018b. Findings of a user study of automatically generated personas. Proc Conf on Human Factors in Computing Systems, p.LBW097. https://doi.org/10.1145/3170427.3188470

  • Salminen J, Engün S, Jung SG, et al., 2019. Design issues in automatically generated persona profiles: a qualitative analysis from 38 think-aloud transcripts. Proc Conf on Human Information Interaction and Retrieval, p.225–229. https://doi.org/10.1145/3295750.3298942

  • Schwarz K, Wieschollek P, Lensch HPA, 2018. Will people like your image? Learning the aesthetic space. Proc IEEE Winter Conf on Applications of Computer Vision, p.2048–2057. https://doi.org/10.1109/WACV.2018.00226

  • Shah JJ, Kulkarni SV, Vargas-Hernandez N, 2000. Evaluation of idea generation methods for conceptual design: effectiveness metrics and design of experiments. J Mech Des, 122(4):377–384. https://doi.org/10.1115/1.1315592

    Google Scholar 

  • Simonyan K, Zisserman A, 2014. Very deep convolutional networks for large-scale image recognition. https://doi.org/1409.1556

  • Strohmann T, Siemon D, Robra-Bissantz S, 2017. brAInstorm: intelligent assistance in group idea generation. Proc 12th Int Conf on Design Science Research in Information System and Technology, p.457–461. https://doi.org/10.1007/978-3-319-59144-5_31

    Google Scholar 

  • Strothotte T, Schlechtweg S, 2002. Non-photorealistic Computer Graphics: Modeling, Rendering, and Animation. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

    Google Scholar 

  • Tang X, Wang ZW, Luo WX, et al., 2018. Face aging with identity-preserved conditional generative adversarial networks. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.7939–7947. https://doi.org/10.1109/CVPR.2018.00828

  • Tang XO, Luo W, Wang XG, 2013. Content-based photo quality assessment. IEEE Trans Multim, 15(8):1930–1943. https://doi.org/10.1109/TMM.2013.2269899

    Google Scholar 

  • Vandevenne D, Verhaegen PA, Dewulf S, et al., 2015. A scalable approach for ideation in biologically inspired design. Artif Intell Eng Des Anal Manuf, 29(1):19–31. https://doi.org/10.1017/S0890060414000122

    Google Scholar 

  • Varshney LR, Pinel F, Varshney KR, et al., 2019. A big data approach to computational creativity: the curious case of Chef Watson. IBM J Res Dev, 63(1):7:1–7:18. https://doi.org/10.1147/JRD.2019.2893905

    Google Scholar 

  • Verma P, Smith JO, 2018. Neural style transfer for audio spectograms. https://doi.org/1801.01589

  • Wang J, Yu LT, Zhang WN, et al., 2017. IRGAN: a minimax game for unifying generative and discriminative information retrieval models. Proc 40th Int ACM SI-GIR Conf on Research and Development in Information Retrieval, p.515–524. https://doi.org/10.1145/3077136.3080786

  • Wang TC, Liu MY, Zhu JY, et al., 2018. Video-to-video synthesis. https://doi.org/1808.06601

  • Wang WG, Shen JB, 2017. Deep cropping via attention box prediction and aesthetics assessment. Proc IEEE Int Conf on Computer Vision, p.2205–2213. https://doi.org/10.1109/ICCV.2017.240

  • Wang WN, Cai D, Wang L, et al., 2016. Synthesized computational aesthetic evaluation of photos. Neurocomputing, 172:244–252. https://doi.org/10.1016/j.neucom.2014.12.106

    Google Scholar 

  • Wang WS, Yang S, Zhang WS, et al., 2018. Neural aesthetic image reviewer. https://doi.org/1802.10240

  • Wang XT, Yu K, Wu SX, et al., 2018. ESRGAN: enhanced super-resolution generative adversarial networks. European Conf on Computer Vision, p.63–79. https://doi.org/10.1007/978-3-030-11021-5_5

    Google Scholar 

  • Wu JJ, Zhang CK, Xue TF, et al., 2016. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. Advances in Neural Information Processing Systems, p.82–90.

  • Xu T, Zhang PC, Huang QY, et al., 2018. AttnGAN: fine-grained text to image generation with attentional generative adversarial networks. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1316–1324. https://doi.org/10.1109/CVPR.2018.00143

  • Yan Y, Wang JR, Tang C, et al., 2019. Research on the development of contemporary design intelligence driven by neural network technology. In: Marcus A, Wang WT (Eds.), Design, User Experience, and Usability. Design Philosophy and Theory. Springer, Cham, p.368–381. https://doi.org/10.1007/978-3-030-23570-3_27

    Google Scholar 

  • Yang HY, Huang D, Wang YH, et al., 2018. Learning face age progression: a pyramid architecture of GANs. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.31–39. https://doi.org/10.1109/CVPR.2018.00011

  • Yang WM, Zhang XC, Tian YP, et al., 2019. Deep learning for single image super-resolution: a brief review. IEEE Trans Multim, 21(12):3106–3121. https://doi.org/10.1109/tmm.2019.2919431

    Google Scholar 

  • Yang Y, Zhuang YT, Wu F, et al., 2008. Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval. IEEE Trans Multim, 10(3):437–446. https://doi.org/10.1109/TMM.2008.917359

    Google Scholar 

  • Yi ZL, Zhang H, Tan P, et al., 2017. DualGAN: unsupervised dual learning for image-to-image translation. Proc IEEE Int Conf on Computer Vision, p.2868–2876. https://doi.org/10.1109/ICCV.2017.310

  • Yoon Y, Jeon HG, Yoo D, et al., 2015. Learning a deep convolutional network for light-field image super-resolution. Proc IEEE Int Conf on Computer Vision, p.57–65. https://doi.org/10.1109/ICCVW.2015.17

  • You S, You N, Pan MX, 2019. PI-REC: progressive image reconstruction network with edge and color domain. https://doi.org/1903.10146

  • Yu F, Zhang YD, Song SR, et al., 2015. LSUN: construction of a large-scale image dataset using deep learning with humans in the loop. https://doi.org/1506.03365

  • Yu JH, Lin Z, Yang JM, et al., 2018a. Free-form image inpainting with gated convolution. https://doi.org/1806.03589

  • Yu JH, Lin Z, Yang JM, et al., 2018b. Generative image inpainting with contextual attention. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.5505–5514. https://doi.org/10.1109/CVPR.2018.00577

  • Zakharov E, Shysheya A, Burkov E, et al., 2019. Fewshot adversarial learning of realistic neural talking head models. https://doi.org/1905.08233

  • Zeiler MD, Taylor GW, Fergus R, 2011. Adaptive deconvolutional networks for mid and high level feature learning. Proc IEEE Int Conf on Computer Vision, p.2018–2025. https://doi.org/10.1109/ICCV.2011.6126474

  • Zhang H, Xu T, Li H, et al., 2017. StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks. Proc IEEE Int Conf on Computer Vision, p.5907–5915.

  • Zhang H, Xu T, Li H, et al., 2019. StackGAN++: realistic image synthesis with stacked generative adversarial networks. IEEE Trans Patt Anal Mach Intell, 41(8):1947–1962. https://doi.org/10.1109/TPAMI.2018.2856256

    Google Scholar 

  • Zhang JJ, Yu JH, Zhang K, et al., 2017. Computational aesthetic evaluation of logos. ACM Trans Appl Perc, 14(3), Article 20. https://doi.org/10.1145/3058982

    Google Scholar 

  • Zhang R, Isola P, Efros AA, 2016. Colorful image colorization. Proc 14th European Conf on Computer Vision, p.649–666. https://doi.org/10.1007/978-3-319-46487-9_40

    Google Scholar 

  • Zhao H, Gallo O, Frosio I, et al., 2016. Loss functions for image restoration with neural networks. IEEE Trans Comput Imag, 3(1):47–57. https://doi.org/10.1109/tci.2016.2644865

    Google Scholar 

  • Zhu JY, Park T, Isola P, et al., 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proc IEEE Int Conf on Computer Vision, p.2242–2251. https://doi.org/10.1109/ICCV.2017.244

Download references

Acknowledgements

Figs. 5c, 5d, and 8 in this study were generated by the pre-trained models of Runway toolkit (https://runwayml.com).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China

    Yong-chuan Tang, Jiang-jie Huang, Meng-ting Yao, Jia Wei, Wei Li, Yong-xing He & Ze-jian Li

  2. Key Laboratory of Design Intelligence and Digital Creativity of Zhejiang Province, Hangzhou, 310027, China

    Yong-chuan Tang, Jiang-jie Huang, Meng-ting Yao, Jia Wei, Wei Li & Yong-xing He

  3. Zhejiang Lab, Hangzhou, 310012, China

    Yong-chuan Tang, Wei Li, Yong-xing He & Ze-jian Li

  4. Alibaba-Zhejiang University Joint Institute of Frontier Technologies, Hangzhou, 310027, China

    Ze-jian Li

Authors
  1. Yong-chuan Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiang-jie Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meng-ting Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jia Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yong-xing He
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ze-jian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong-chuan Tang.

Additional information

Compliance with ethics guidelines

Yong-chuan TANG, Jiang-jie HUANG, Meng-ting YAO, Jia WEI, Wei LI, Yong-xing HE, and Ze-jian LI declare that they have no conflict of interest.

Project supported by the National Science and Technology Innovation 2030 Major Project of the Ministry of Science and Technology of China (No. 2018AAA0100703), the National Natural Science Foundation of China (Nos. 61773336 and 91748127), the Chinese Academy of Engineering Consulting Project (No. 2018-ZD-12-06), the Provincial Key Research and Development Plan of Zhejiang Province, China (No. 2019C03137), and the Ng Teng Fong Charitable Foundation in the form of ZJU-SUTD IDEA Grant

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, Yc., Huang, Jj., Yao, Mt. et al. A review of design intelligence: progress, problems, and challenges. Front Inform Technol Electron Eng 20, 1595–1617 (2019). https://doi.org/10.1631/FITEE.1900398

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1631/FITEE.1900398

Key words

CLC number

Search

Navigation