Abstract:
We conducted experiments for improving the estimation of importance of a bigram. We will have the benefit in the information retrieval for using the indexed list of bigra...Show MoreMetadata
Abstract:
We conducted experiments for improving the estimation of importance of a bigram. We will have the benefit in the information retrieval for using the indexed list of bigrams in Japanese. Importance is determined by verifying if the bigram is included in a title. In this study, this importance is expressed by the probability that the bigram appear in title provided that it also appears in abstract. Dealing with less frequently occurring data is always a challenge to estimate the probability, due to the danger of generating false results. Using direct estimation, we estimated the conditional probability by adding a regularization parameter A. We compared our method against the ones used in Apriori and Maximum Likelihood Estimation to see the improvement. We also observed the changes in the result by changing the A. Our method was found to be effective, and the required parameter was also robust.
Date of Conference: 29 January 2020 - 01 February 2020
Date Added to IEEE Xplore: 09 April 2020
ISBN Information:
Print on Demand(PoD) ISSN: 2374-314X