Text-Visualizing Neural Network Model: Understanding Online Financial Textual Data

Abstract

This study aims to visualize financial documents to swiftly obtain market sentiment information from these documents and determine the reason for which sentiment decisions are made. This type of visualization is considered helpful for nonexperts to easily understand technical documents such as financial reports. To achieve this, we propose a novel interpretable neural network (NN) architecture called gradient interpretable NN (GINN). GINN can visualize both the market sentiment score from a whole financial document and the sentiment gradient scores in concept units. We experimentally demonstrate the validity of text visualization produced by GINN using a real textual dataset.

Theoretical Analysis of the II Algorithm

Let $\varOmega _{dw}^{(k)}$ be a set of words included in the kth cluster and included in the polarity dictionary, $D^{(p)}$ and $D^{(n)}$ be the positive and negative document sets, $\partial {w}_{k, i}^{(2)*}$ be the ith component of $\partial \varvec{w}_{k}^{(2)*}$, $p^{-}(w_{k,i})$ be $p \left( j \in D^{(n)} | z_{j, i}^{(1, k)} > 0\right)$, $p^{+}(w_{k,i})$ be $1 - p^{-}(w_{k,i})$, and $\partial \varvec{H}^{(j, t)}$ be the gradient value of $\varvec{H}^{(j, t)}$ in Update. Then,

Proposition 1

If we utilize Update for the parameter updates, then,

$$\begin{aligned} \left\{ \begin{array}{cc} E[\partial {w}_{k, i}^{(2)*}]< 0 &{} \left( \frac{p^{+}(w_{k,i})}{p^{-}(w_{k,i})}> \frac{E[|\varDelta ^{(2)*}_{j, k}|| z_{j, i}^{(1, k)} = 1 \cap j \in D^{(n)}]}{E[|\varDelta ^{(2)*}_{j, k}| | z_{j ,i}^{(1, k)} = 1 \cap j \in D^{(p)}]}\right) \\ E[\partial {w}_{k, i}^{(2)*}] > 0 &{} \left( \frac{p^{+}(w_{k,i})}{p^{-}(w_{k,i})} < \frac{E[|\varDelta ^{(2)*}_{j, k}|| z_{j, i}^{(1, k)} = 1 \cap j \in D^{(n)}]}{E[|\varDelta ^{(2)*}_{j, k}| | z_{j ,i}^{(1, k)} = 1 \cap j \in D^{(p)}]}\right) \end{array} \right. . \end{aligned}$$

(1)

is established. Proposition 1 indicates that if Cond 1: the values of ${t^+}$ and ${t^-}$ are sufficiently large, and Cond 2: for every word $w_{k,i^{+}} \in \varOmega _{dw}^{(k)} \cap \varOmega _{pw}^{(k)}$, and $w_{k,i^{-}} \in \varOmega _{dw}^{(k)} \cap \varOmega _{nw}^{(k)}$, the initial values of $w^{(2)}_{k,i^{+}}$ and $w^{(2)}_{k,i^{-}}$ given by Init are positive and sufficiently large, and negative and sufficiently small, respectively, are met for every k, then, the II algorithm is expected to award each positive word $\in \varOmega _{pw}^{(k)}$ (negative word $\in \varOmega _{nw}^{(k)})$ a positive (negative) sentiment score.

Let ${\varvec{H}^{d}}^{(j, t)}$ be $\varvec{H}^{(j, t)} - {\varvec{H}^{*}}^{(j, t)}$. Then, the following propositions important for explaining the market mood predictability of GINN are established.

Proposition 2

If the initial values of $|\varvec{W^{(3)}}|$ and $|\varvec{W^{(4)}}|$ are sufficiently small (Cond 3) and for every $j \in \varOmega ^{(t)}_m$, the values of $\varvec{z}^{(2)}_{j}$ are $\left\{ \begin{array}{cc} positive &{} (j \in D^{(p)}) \\ negative &{} (j \in D^{(n)}) \end{array} \right.$, then, the first and second row vector values of $\partial \varvec{H}^{(j, t)}$ are positive and negative respectively, and $\frac{\sum _{j \in \varOmega ^{(t+1)}_m} \Vert {\varvec{H}^{d}}^{(j, t+1)} \Vert _{1} }{\sum _{j \in \varOmega ^{(t+1)}_m} \Vert \varvec{H}^{(j, t+1)}\Vert _{1}} \le \frac{\sum _{j \in \varOmega ^{(t+1)}_m} \Vert {\varvec{H}^{d}}^{(j, t)} \Vert _{1} }{\sum _{j \in \varOmega ^{(t+1)}_m} \Vert \varvec{H}^{(j, t)}\Vert _{1}}.$

Proposition 3

If, for every k, Cond 1–3 are established, the values $|\varOmega _{pw}^{(k, t^+)}|$, $|\varOmega _{nw}^{(k, t^-)}|$ and $|\varOmega _m|$ are sufficiently large, then, $\lim _{t \rightarrow \infty } \frac{\sum _{j \in \varOmega ^{(t)}_m} \Vert {\varvec{H}^{d}}^{(j, t)} \Vert _{1} }{\sum _{j \in \varOmega ^{(t)}_m} \Vert \varvec{H}^{(j, t)}\Vert _{1}} = 0.$

See the supplementary material (See footnote 2) for the proofs and the details.