Fraud Detection in Online Market Research

Abstract

The key distinguishing approach in this paper is to utilize random sampling on the training set for optimal solution to maximize the fraud coverage as well as minimize the false positive rate. We have experimented with the variety of optimal solutions to discover a different bad actors segment. We have adapted a partial labeled data in the industry setting together with self developed set of SQL based rules in order to compensate in timely manner for just enough labelled data available for supervised learning model to detect fraud users before they negatively impact our business. Here, at DISQO, market research firm that provides raw data to our partners and clients as well as is a reputable panel for consumers to share their feedback on variety of brands and products, we were facing challenges related to noisy labelled data. Thus, set of rules were developed to assess every user against fraud in the following grade, A (red, very suspicious), B (yellow) and C (green). We started with a simple grading system. Then, after the optimal problem was formulated to maximize the fraud detection on the random sampled training set we were solving for optimal solution, and collected all of these solutions to average out and design our final solution in order to detect Fraud with better precision and improved recall from $26\%$ to $52\%$. Lastly we have developed a methodology to combine these optimal coefficient solutions in order to have a well generalized fraud detection model as averaging the coefficients next to the dynamic labels via Logistic Regression. However, we have achieved the best results when we solved for the optimal fraud coverage segment and trained on the hand picked number of classifiers to learn the separation in the data between bad and good actors. Then we have created a fraud vector of 5-dimensions, that consisted of the probabilities retrieved from hand picked classifiers based on the optimal solutions (we had 3 fraud segments retrieved from optimal solutions), one of the fraud vector’s dimension contained the CNN probability, other two were XGBoost and Logistic based probability, and kept the auto-encoder reconstruction error as another fraud vector dimension. At the end, we compare fraud vector magnitude on every users to assess quickly the fraud overall risk, we use every classifier probability and auto-encoder reconstruction error as fraud dimensions.

Supported by DISQO.