Skip to main content

Advertisement

SpringerLink
Log in

Similarity-based approach for inventive design solutions assistance

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

With the increasing demand for inventive products, finding out inventive design solutions hidden in different industrial engineering domains has always been a challenge for engineers. In addition, patent documents are full of the latest inventive knowledge inside. In this paper, we rely on the assumption that an engineering problem may have an inventive practical solution in another scientific domain as long as they are similarly described. Therefore, we focus on applying machine learning techniques, more particularly neural networks to determine the similarity between patent problems. Technically, a trained bidirectional LSTM neural network, called Manhattan LSTM is integrated into our approach named SAM-IDM to predict the similarity between sentences. We experimentally show that Manhattan LSTM outperforms other baseline approaches in a labelled sample dataset of SNLI corpus. We then experiment our approach on a real-world U.S. patent dataset and we demonstrate that it presents promising results in terms of sentence similarity matching and inventiveness. An inventive design case is detailed to illustrate its performance and practicality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Reader which may refer to this link for the full patent https://patents.google.com/patent/US8847930B2/.

  2. https://bulkdata.uspto.gov/data/patent/grant/redbook/fulltext/2017/.

  3. https://www.kaggle.com/c/quora-question-pairs/data.

  4. https://nlp.stanford.edu/projects/snli/.

  5. https://drive.google.com/file/d/1JvrUuO4by_FzvyP-5gxKQAuyyQc9cadY/view.

References

  • Altshuller, G., Shulyak, L., & Rodman, S. (2002). 40 principles: Triz keys to technical innovation. Worcester, MA: Technical Innovation Center Inc.

    Google Scholar 

  • Altshuller, G. S. (1984). Creativity as an exact science: The theory of the solution of inventive problems. London: Gordon and Breach.

    Book  Google Scholar 

  • Bayer, J, Wierstra, D, Togelius, J, & Schmidhuber, J. (2009). Evolving memory cell structures for sequence learning. In International conference on artificial neural networks (pp. 755–764) Springer.

  • Benedetti, F., Beneventano, D., Bergamaschi, S., & Simonini, G. (2019). Computing inter-document similarity with context semantic analysis. Information Systems, 80, 136–147.

    Article  Google Scholar 

  • Bengio, Y., Simard, P., & Frasconi, P. (1994). Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2), 157–166.

    Article  Google Scholar 

  • Bultey, A., De Bertrand De Beuvron, F., & Rousselot, F. (2007). A substance-field ontology to support the TRIZ thinking approach. International Journal of Computer Applications in Technology, 30(1–2), 113–124.

    Article  Google Scholar 

  • Cavallucci, D., Rousselot, F., & Zanni, C. (2010). Initial situation analysis through problem graph. CIRP Journal of Manufacturing Science and Technology, 2(4), 310–317.

    Article  Google Scholar 

  • Cavallucci, D., Rousselot, F., & Zanni, C. (2011). Using patents to populate an inventive design ontology. Procedia Engineering, 9, 52–62.

    Article  Google Scholar 

  • Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12(Aug), 2493–2537.

    Google Scholar 

  • Costa, J. B., Silva-Correia, J., Ribeiro, V. P., da Silva, Morais A., Oliveira, J. M., & Reis, R. L. (2019). Engineering patient-specific bioprinted constructs for treatment of degenerated intervertebral disc. Materials Today Communications, 19, 506–512.

    Article  Google Scholar 

  • Cronier, P., Pietu, G., Dujardin, C., Bigorre, N., Ducellier, F., & Gerard, R. (2010). The concept of locking plates. Orthopaedics & Traumatology: Surgery & Research, 96(4), S17–S36.

    Google Scholar 

  • Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6), 391–407.

    Article  Google Scholar 

  • DeLongchamp, D. M., & Hammond, P. T. (2004). Highly ion conductive poly (ethylene oxide)-based solid polymer electrolytes from hydrogen bonding layer-by-layer assembly. Langmuir, 20(13), 5403–5411.

    Article  Google Scholar 

  • Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.

  • Ding, Z., Jiang, S., Ng, F., & Zhu, M. (2017). A new TRIZ-based patent knowledge management system for construction technology innovation. Journal of Engineering, Design and Technology, 15, 456.

    Article  Google Scholar 

  • Eranna, G., Joshi, B., Runthala, D., & Gupta, R. (2004). Oxide materials for development of integrated gas sensors—a comprehensive review. Critical Reviews in Solid State and Materials Sciences, 29(3–4), 111–188.

    Article  Google Scholar 

  • Fernando, T., Denman, S., McFadyen, A., Sridharan, S., & Fookes, C. (2018). Tree memory networks for modelling long-term temporal dependencies. Neurocomputing, 304, 64–81.

    Article  Google Scholar 

  • Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N., Peters, M., Schmitz, M., & Zettlemoyer, L. (2018). Allennlp: A deep semantic natural language processing platform. arXiv preprint arXiv:1803.07640.

  • Gasmi, H., Laval, J., & Bouras, A. (2019). Cold-start cybersecurity ontology population using information extraction with LSTM. In 2019 international conference on cyber security for emerging technologies (CSET), IEEE (pp. 1–6).

  • Girodon, J., Monticolo, D., Bonjour, E., & Perrier, M. (2015). An organizational approach to designing an intelligent knowledge-based system: Application to the decision-making process in design projects. Advanced Engineering Informatics, 29(3), 696–713.

    Article  Google Scholar 

  • Goldberg, Y. (2017). Neural network methods for natural language processing. Synthesis Lectures on Human Language Technologies, 10(1), 1–309.

    Article  Google Scholar 

  • Graves, A., Fernández, S., & Schmidhuber, J. (2005). Bidirectional LSTM networks for improved phoneme classification and recognition. In International conference on artificial neural networks (pp. 799–804). Springer.

  • Hao, J., Zhou, Y., Zhao, Q., & Xue, Q. (2019). An evolutionary computation based method for creative design inspiration generation. Journal of Intelligent Manufacturing, 30(4), 1673–1691.

    Article  Google Scholar 

  • Harris, Z. S. (1954). Distributional structure. Word, 10(2–3), 146–162.

    Article  Google Scholar 

  • Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.

    Article  Google Scholar 

  • Jardim-Goncalves, R., Sarraipa, J., Agostinho, C., & Panetto, H. (2011). Knowledge framework for intelligent manufacturing systems. Journal of Intelligent Manufacturing, 22(5), 725–735.

    Article  Google Scholar 

  • Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., & Liu, Q. (2019). Tinybert: Distilling bert for natural language understanding. arXiv preprint arXiv:1909.10351.

  • Kawaguchi, T., Seriguchi, K., Komatsu, H., Tanaka, K., & Kato, H. (2006). Shield box and shield method. US Patent App. 10/536,870.

  • Kenter, T., & De Rijke, M. (2015). Short text similarity with word embeddings. In Proceedings of the 24th ACM international on conference on information and knowledge management, ACM (pp. 1411–1420).

  • Kim, H. K., Kim, H., & Cho, S. (2017). Bag-of-concepts: Comprehending document representation through clustering words in distributed representation. Neurocomputing, 266, 336–352.

    Article  Google Scholar 

  • Kusiak, A. (2016). Put innovation science at the heart of discovery. Nature, 530(7590), 255.

    Article  Google Scholar 

  • Kusner, M., Sun, Y., Kolkin, N., & Weinberger, K. (2015). From word embeddings to document distances. In International conference on machine learning (pp. 957–966).

  • Mikolov, T., Karafiát, M., Burget, L., Černockỳ, J., & Khudanpur, S. (2010). Recurrent neural network based language model. In Eleventh annual conference of the international speech communication association.

  • Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.

  • Minaee, S., & Liu, Z. (2017). Automatic question-answering using a deep similarity neural network. In 2017 IEEE global conference on signal and information processing (GlobalSIP), IEEE (pp. 923–927).

  • Mnih, A., & Hinton, G. E. (2009). A scalable hierarchical distributed language model. In Advances in Neural Information Processing Systems (pp. 1081–1088).

  • Mueller, J., & Thyagarajan, A. (2016). Siamese recurrent architectures for learning sentence similarity. In Thirtieth AAAI conference on artificial intelligence.

  • Negny, S., Belaud, J. P., Robles, G. C., Reyes, E. R., & Ferrer, J. B. (2012). Toward an eco-innovative method based on a better use of resources: Application to chemical process preliminary design. Journal of Cleaner Production, 32, 101–113.

    Article  Google Scholar 

  • Ni, X., Samet, A., & Cavallucci, D. (2019). An approach merging the IDM-related knowledge. In International TRIZ future conference (pp. 147–158). Springer.

  • Pawar, A., & Mago, V. (2018). Calculating the similarity between words and sentences using a lexical database and corpus statistics. arXiv preprint arXiv:1802.05667.

  • Perren, S. M. (2002). Evolution of the internal fixation of long bone fractures: the scientific basis of biological internal fixation: Choosing a new balance between stability and biology. The Journal of Bone and Joint Surgery British, 84(8), 1093–1110.

    Article  Google Scholar 

  • Rahim, Z. A., Yusof, S. M., Bakar, N. A., & Mohamad, W. M. S. W. (2018). The application of computational thinking and TRIZ methodology in patent innovation analytics. In International conference of reliable information and communication technology (pp. 793–802). Springer.

  • Rajaraman, A., & Ullman, J. D. (2011). Mining of massive datasets. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Reimers, N., & Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084.

  • Renjith, S. C., Park, K., & Kremer, G. E. O. (2020). A design framework for additive manufacturing: Integration of additive manufacturing capabilities in the early design process. International Journal of Precision Engineering and Manufacturing, 21(2), 329–345.

    Article  Google Scholar 

  • Rousselot, F., Zanni-Merk, C., & Cavallucci, D. (2012). Towards a formal definition of contradiction in inventive design. Computers in Industry, 63(3), 231–242.

    Article  Google Scholar 

  • Sajana, A. R. (2011). A low-complexity intrusion detection algorithm for surveillance using PIR sensors in a wireless sensor network. PhD Thesis.

  • Sarica, S., Luo, J., & Wood, K. L. (2020). Technet: Technology semantic network based on patent data. Expert Systems with Applications, 142, 112995.

    Article  Google Scholar 

  • Schmidhuber, J., Wierstra, D., Gagliolo, M., & Gomez, F. (2007). Training recurrent networks by Evolino. Neural Computation, 19(3), 757–779.

    Article  Google Scholar 

  • Sheu, D. D., Chen, C. H., & Yu, P. Y. (2012). Invention principles and contradiction matrix for semiconductor manufacturing industry: Chemical mechanical polishing. Journal of Intelligent Manufacturing, 23(5), 1637–1648.

    Article  Google Scholar 

  • Shirwaiker, R. A., & Okudan, G. E. (2008). Triz and axiomatic design: A review of case-studies and a proposed synergistic use. Journal of Intelligent Manufacturing, 19(1), 33–47.

    Article  Google Scholar 

  • Sidorov, G., Velasquez, F., Stamatatos, E., Gelbukh, A., & Chanona-Hernández, L. (2014). Syntactic n-grams as machine learning features for natural language processing. Expert Systems with Applications, 41(3), 853–860.

    Article  Google Scholar 

  • Smirnov, A., Kashevnik, A., Teslya, N., Shilov, N., Oroszi, A., Sinko, M., Humpf, M., & Arneving, J. (2013). Knowledge management for complex product development. In IFIP international conference on product lifecycle management (pp. 110–119). Springer.

  • Souili, A., & Cavallucci, D. (2013). Toward an automatic extraction of IDM concepts from patents. In CIRP Design 2012 (pp. 115–124). Springer.

  • Souili, A., & Cavallucci, D. (2017). Automated extraction of knowledge useful to populate inventive design ontology from patents. In TRIZ–The Theory of Inventive Problem Solving (pp. 43–62). Springer.

  • Souili, A., Cavallucci, D., & Rousselot, F. (2015). A lexico-syntactic pattern matching method to extract IDM-TRIZ knowledge from on-line patent databases. Procedia Engineering, 131, 418–425.

    Article  Google Scholar 

  • Soumya George, K., & Joseph, S. (2014). Text classification by augmenting bag of words (bow) representation with co-occurrence feature. IOSR Journal of Computer Engineering, 16(1), 34–38.

    Article  Google Scholar 

  • Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., & Gamon, M. (2015). Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 conference on empirical methods in natural language processing (pp. 1499–1509).

  • Vojtáš, P. (2006). Fuzzy logic aggregation for semantic web search for the best (top-k) answer. Capturing intelligence (Vol. 1, pp. 341–359). Amsterdam: Elsevier.

    Google Scholar 

  • Wang, B., Duan, Y., Xin, Z., Yao, X., Abliz, D., & Ziegmann, G. (2018). Fabrication of an enriched graphene surface protection of carbon fiber/epoxy composites for lightning strike via a percolating-assisted resin film infusion method. Composites Science and Technology, 158, 51–60.

    Article  Google Scholar 

  • Wang, Z., Mi, H., & Ittycheriah, A. (2016). Sentence similarity learning by lexical decomposition and composition. arXiv preprint arXiv:1602.07019.

  • Whiteside, A., Shehab, E., Beadle, C., & Percival, M. (2009). Developing a current capability design for manufacture framework in the aerospace industry. In Proceedings of the 19th CIRP design conference–competitive design, Cranfield University Press.

  • Yan, W., Zanni-Merk, C., & Rousselot, F. (2011). An application of semantic distance between short texts to inventive design. In: KEOD (pp. 261–266).

  • Yan, W., Zanni-Merk, C., Rousselot, F., & Cavallucci, D. (2013). Ontology matching for facilitating inventive design based on semantic similarity and case-based reasoning. International Journal of Knowledge-Based and Intelligent Engineering Systems, 17(3), 243–256.

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the China Scholarship Council (CSC). The statements made herein are solely the responsibility of the authors.

Author information

Authors and Affiliations

  1. ICUBE/CSIP, INSA de Strasbourg, 24 Boulevard de la Victoire, 67084, Strasbourg, France

    Xin Ni & Denis Cavallucci

  2. ICUBE/SDC, INSA de Strasbourg, 300 Bd Sebastien Brant, 67412, Illkirch, France

    Ahmed Samet

Authors
  1. Xin Ni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ahmed Samet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Denis Cavallucci
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin Ni.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendices

A case studies

We introduce additional real solved use cases found in the literature. We highlight the output of SAM-IDM on these problems. SAM-IDM mines all similar problems and their patents using a fixed similarity threshold. These similar problems and corresponding solutions are investigated by experts to check their potential in creating an inventive solution.

Target Problem 1:“It is thus possible to prevent a decrease in the conductivity of the conductive film 421 due to oxidation.” (US9536627, Physics)

  • Similar Problem (Similarity Value: 0.83): “materials function as a heat sink due to the latent heat of vaporization.” (US9537344, Electricity)

  • Corresponding Solution: “materials transitioning from a liquid to a gas or vapor, or from a solid to a gas or vapor, could also be used as a heat sink, so long as the temperature associated with the phase change was in the desired range.”

  • Latent Inventive Solution Reference: From the corresponding solution according to the similar problem, we notice that Eranna et al. (2004) mentioned oxide materials for development of integrated gas sensors are related to the conductivity.

  • Similar Problem (Similarity Value: 0.83): “It may lead to a decrease in the yield in subsequent production steps.” (US9538663, Electricity)

  • Corresponding Solution: “connecting a sidewall of the wiring board to a sidewall in an opening of the metal frame by subjecting the metal frame to plastic deformation.”

  • Latent Inventive Solution Reference: With the hints of the corresponding solution, we notice that raising the mix ratio of the powdered metal is, in the conductive filler kneading method, able to obtain the desired surface resistance Kawaguchi et al. (2006). This might be useful to solve the target problem.

  • Similar Problem (Similarity Value: 0.82): “it is difficult to form a roughened surface.” (US9538642, Electricity)

  • Corresponding Solution: “Therefore, in the present embodiment, the above-described thin resin layer 1012 is formed through a method that uses resin contraction caused by heating.”

  • Latent Inventive Solution Reference: Based on the corresponding solution, we notice that a solid polymer electrolyte film from hydrogen bonding layer DeLongchamp and Hammond (2004) and a percolating-assisted resin film infusion method Wang et al. (2018) have been proposed to solve the related problem.

Target Problem 2: “Segmentation and analysis can add computational complexity and overhead.” (US9534958, Physics)

  • Similar Problem (Similarity Value: 0.86): “which can incur high computational complexity.” (US9538483, Electricity)

  • Corresponding Solution: “Some embodiments described herein provide a novel approach to the weighted sum-rate maximization in the MIMO interference network, and apply a novel and efficient algorithm with guaranteed monotonic convergence as well as an elegant way to establish rate duality between an interference network and its reciprocal.”

  • Latent Inventive Solution Reference: A Low-Complexity Intrusion Detection Algorithm is proposed by Sajana (2011) to solve the similar target issue.

Target Problem 3: “many of the devices also require the use of a pressure washer.” (US9533320, Performing Operations)

  • Similar Problem (Similarity Value: 0.87): “a hole of a microphone is blocked” (US9537985, Electricity)

  • Corresponding Solution: “The hall structure 100 according to the present invention can better reduce blocking of a hole of a microphone by combining the hole of the microphone and a hole of a speakerphone or receiver.”

  • Latent Inventive Solution Reference: The solution from US9537985 that combining two different holes is close to the target problem’s solution that the device consists of a connector and conduits in US9533320.

Target Problem 4: “Patient-specific implants are expensive to engineer and manufacture. Moreover, the plate can cause bone necrosis if the fit is too snug.” (US9532825, Human Necessities)

  • Similar Problem (Similarity Value: 0.81): “reservoirs are expensive and difficult to manufacture. ” (US9534198, Chemistry)

  • Corresponding Solution: “One aspect of the present invention is to provide an EC fluid cycling unit that enables fluid level control without the use of expensive ultrasonics or load cells.”

  • Latent Inventive Solution Reference: The bioprinted constructs for treatment of degenerated intervertebral disc Costa et al. (2019) that is similar with the fluid cycling unit has been proposed to solve the similar issue with the target problem.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ni, X., Samet, A. & Cavallucci, D. Similarity-based approach for inventive design solutions assistance. J Intell Manuf 33, 1681–1698 (2022). https://doi.org/10.1007/s10845-021-01749-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-021-01749-4

Keywords

Search

Navigation