Abstract
[Context and Motivation:] In current practice, existing traceability data is often underutilized due to lack of accessibility and difficulties users have in constructing the complex SQL queries needed to address realistic Software Engineering questions. In our prior work we therefore presented TiQi – a natural language (NL) interface for querying software projects. TiQi has been shown to transform a set of trace queries collected from IT experts at accuracy rates ranging from 47 % to 93 %. [Question/problem:] However, users need to quickly determine whether TiQi has correctly understood the NL query. [Principal ideas/results:] TiQi needs to communicate the transformed query back to the user and provide support for disambiguation and correction. In this paper we report on three studies we conducted to compare the effectiveness of four query representation techniques. [Contribution:] We show that simultaneously displaying a visual query representation, SQL, and a sample of the data results enabled users to most accurately evaluate the correctness of the transformed query.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Crosby, M.E., Stelovsky, J.: How do we read algorithms? a case study. IEEE Comput. 23(1), 24–35 (1990)
Gotel, O., et al.: Traceability fundamentals. In: Cleland-Huang, J., Gotel, O., Zisman, A. (eds.) Software and Systems Traceability, pp. 3–22. Springer, Heidelberg (2012). doi:10.1007/978-1-4471-2239-51
Jaakkola, H., Thalheim, B.: Visual SQL – high-quality ER-based query treatment. In: Jeusfeld, M.A., Pastor, Ó. (eds.) ER Workshops 2003. LNCS, vol. 2814, pp. 129–139. Springer, Heidelberg (2003). doi:10.1007/978-3-540-39597-3.
Kim, H.-J., Korth, H.F., Silberschatz, A.: Picasso: a graphical query language. IEEE Comput. 18, 169–203 (1988)
Kuusela, H., Paul, P.: A comparison of concurrent and retrospective verbal protocol analysis. IEEE Comput. 113(3), 387–404 (2000)
Li, F., Jagadish, H.: Constructing an interactive natural language interface for relational databases. IEEE Comput. 8(1), 73–84 (2014)
Li, Y., Yang, H., Jagadish, H.V.: Term disambiguation in natural language query for XML. In: Larsen, H.L., Pasi, G., Ortiz-Arroyo, D., Andreasen, T., Christiansen, H. (eds.) FQAS 2006. LNCS (LNAI), vol. 4027, pp. 133–146. Springer, Heidelberg (2006)
Mäder, P., Cleland-Huang, J.: A visual traceability modeling language. In: Petriu, D.C., Rouquette, N., Haugen, Ø. (eds.) MODELS 2010, Part I. LNCS, vol. 6394, pp. 226–240. Springer, Heidelberg (2010)
Mäder, P., Cleland-Huang, J.: A visual language for modeling and executing traceability queries. IEEE Comput. 12(3), 537–553 (2013)
Maletic, J.I., Collard, M.L.: TQL: A query language to support traceability. In: TEFSE 2009: Proceedings of the ICSE Workshop on Traceability in Emerging Forms of Software Engineering, pp. 16–20. IEEE Computer Society, USA (2009)
McFetridge, P., Groeneboer, C.: Novel terms and cooperation in a natural language interface. KBCS 1989. LNCS, vol. 444, pp. 331–340. Springer, Heidelberg (1990)
Parameswaran, A., Polyzotis, N., Garcia-Molina, H.: SeeDB: Visualizing database queries efficiently. Proc. VLDB Endowment 7(4), 325–328 (2013)
Pruski, P., Lohar, S., Aquanette, R., Ott, G., Amornborvornwong, S., Rasin, A., Cleland-Huang, J.: TiQi: towards natural language trace queries. In: IEEE 22nd International Requirements Engineering Conference, RE, Karlskrona, Sweden, 25–29 August 2014, pp. 123–132 (2014)
Pruski, P., Lohar, S., Ott, G., Goss, W., Rasin, A., Cleland-Huang, J.: TiQi: answering unstructured natural language trace queries. Requir. Eng 20(3), 215–232 (2015)
Rempel, P., Mäder, P., Kuschke, T., Cleland-Huang, J.: Mind the gap: assessing the conformance of software traceability to relevant guidelines. In: 36th International Conference on Software Engineering (ICSE) (2014)
Rodeghero, P., McMillan, C., McBurney, P.W., Bosch, N., D’Mello, S.K.: Improving automated source code summarization via an eye-tracking study of programmers. In: 36th International Conference on Software Engineering, ICSE 2014, Hyderabad, India, May 31–June 07, pp. 390–401 (2014)
Sharif, B., Falcone, M., Maletic, J.I.: An eye-tracking study on the role of scan time in finding source code defects. In: Proceedings of the Symposium on Eye-Tracking Research and Applications, ETRA 2012, Santa Barbara, CA, USA, 28–30 March 2012, pp. 381–384, (2012)
Sharif, B., Maletic, J.I.: An eye tracking study on the effects of layout in understanding the role of design patterns. In: 26th IEEE International Conference on Software Maintenance (ICSM), 12–18 September 2010, Timisoara, Romania, pp. 1–10 (2010)
Stolte, C., Tang, D., Hanrahan, P.: Polaris: a system for query, analysis, and visualization of multidimensional relational databases. IEEE Comput. 8(1), 52–65 (2002)
Uwano, H., Nakamura, M., Monden, A., Matsumoto, K.: Analyzing individual performance of source code review using reviewers’ eye movement. In: Proceedings of the Eye Tracking Research & Application Symposium, ETRA, San Diego, California, USA, 27–29 March 2006, pp. 133–140 (2006)
Acknowledgment
The work in this paper was partially funded by the US National Science Foundation Grant CCF:1319680.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Lohar, S., Cleland-Huang, J., Rasin, A. (2016). Evaluating the Interpretation of Natural Language Trace Queries. In: Daneva, M., Pastor, O. (eds) Requirements Engineering: Foundation for Software Quality. REFSQ 2016. Lecture Notes in Computer Science(), vol 9619. Springer, Cham. https://doi.org/10.1007/978-3-319-30282-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-30282-9_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-30281-2
Online ISBN: 978-3-319-30282-9
eBook Packages: Computer ScienceComputer Science (R0)