Conceptual Classification and Browsing of Internet FAQs Using Self-Organizing Neural Networks

Abstract

Widespread use of computer and internet leads to an abundant supply of information, so that many services for facilitating fluent utilization of the information have appeared. However, many computer users are not so familiar with such services that they need assistant systems to use the services effectively. In case of the internet portal services, users’ e-mail questions are answered by operator, but the increasing number of users brings plenty of burdens. At the time of writing this paper, more than 5 million people use the Hanmail net that is the biggest portal service in Korea and users’ questions per day come to about 200 cases. It is redundant and time-consuming to respond to duplicated questions by hand, and even worse user may not satisfy with the response time. Automatic processing of users’ questions might be not only efficient for operators who can avoid redundant task but also satisfiable for users.

In this paper, we propose a two-level self-organizing map (SOM) which automatically responds to the users’ questions on internet, and helps them to find their answer for themselves by browsing the map hierarchically. The system consists of two parts: classification and browsing subsystems. The classification system also consists of two parts. The first part is preprocessing and keyword clustering which help to encode the input vector for the next classification module. In case of keyword clustering, SOM reduces a variable length question to a normalized vector. Keyword clustering SOM plays the similar role of the thesaurus which discriminates the synonyms. The second part is classifying the queries and matching them with the corresponding answers by another SOM called document classification SOM. The browsing system is based on the completely learned document classification SOM. It helps users to search their answer conceptually by developing the system hierarchically with topology-preserving property of SOM.

Experiments with real world data from Hanmail net show the usefulness of the proposed method. The size of keyword clustring SOM is fexed as 10×10 and the size of document classification SOM is fixed as 150×150. The accuracy is 95.01% for training data and 82.7% for test data with 4.7% error rate.

This research was supported by Brain Science and Engineering Research Program sponsored by Korean Ministry of Science and Technology.