Text structuring methods based on complex network: a systematic review

Abstract

Currently, there is a large amount of text being shared through the Internet. These texts are available in different forms—structured, unstructured and semi structured. There are different ways of analyzing texts, but domain experts usually divide this process in some steps such as pre-processing, feature extraction and a final step that could be classification, clustering, summarization, and keyword extraction, depending on the purpose over the text. For this processing, several approaches have been proposed in the literature based on variations of methods like artificial neural network and deep learning. In this paper, we conducted a systematic review of papers dealing with the use of complex networks approaches for the process of analyzing text. The main results showed that complex network topological properties, measures and modeling can capture and identify text structures concerning different purposes such as text analysis, classification, topic and keyword extraction, and summarization. We conclude that complex network topological properties provide promising strategies with respect of processing texts, considering their different aspects and structures.

Appendices

Appendix 1: Complex network measures

• Accessibility ${\alpha }_{i}^{h} =exp\left(-\sum {p}_{i,j}^{\left(h\right)}\text{log}{p}_{i,j}^{\left(h\right)}\right)$, where ${p}_{i,j}^{\left(h\right)}$ is the probability of a random walker to reach a given node j departing from i, in h steps.

• Aggregation coefficient ${S}_{i}=\frac{{K}_{i}}{\left(\genfrac{}{}{0pt}{}{{D}_{i}}{2}\right)}=\frac{2{K}_{i}}{{D}_{i}({D}_{i}-1)}$, where ${D}_{i}$ represents the degree of node ${v}_{i}$, and ${K}_{i}$ represents the degree of aggregation of node $V$ and can be defined as: ${K}_{i}=\sum _{{v}_{i}\cdot {v}_{k}\cdot {v}_{k}}{\beta }_{i\cdot j\cdot k}$, where ${\beta }_{i\cdot j\cdot k}$ represents the connection between node ${v}_{i}$ and ${v}_{j}$ and ${v}_{k}$ $(i\ne j\ne k)$.

• Assortativity ${\Gamma }=\frac{\frac{1}{\varvec{M}}\sum _{\varvec{j}>\varvec{i}}{\varvec{k}}_{\varvec{i}}{\varvec{k}}_{\varvec{j}}{\varvec{a}}_{\varvec{i}\varvec{j}}-{\left[\frac{1}{\varvec{M}}\sum _{\varvec{j}>\varvec{i}}\frac{1}{2}({\varvec{k}}_{\varvec{i}}+{\varvec{k}}_{\varvec{j}}){\varvec{a}}_{\varvec{i}\varvec{j}}\right]}^{2}}{\frac{1}{\varvec{M}}\sum _{\varvec{j}>\varvec{i}}\frac{1}{2}({\varvec{k}}_{\varvec{i}}^{2}+{\varvec{k}}_{\varvec{j}}^{2}){\varvec{a}}_{\varvec{i}\varvec{j}}-{\left[\frac{1}{\varvec{M}}\sum _{\varvec{j}>\varvec{i}}\frac{1}{2}({\varvec{k}}_{\varvec{i}}+{\varvec{k}}_{\varvec{j}}){\varvec{a}}_{\varvec{i}\varvec{j}}\right]}^{2}}$, where M is the number of edges of the network, and ${a}_{ij}=1$ if nodes $i$ and $j$ are connected and ${a}_{ij}=0$ otherwise.

• Assortative coefficient $r=\frac{\sum _{jk}jk\left({e}_{jk}-{q}_{j}{q}_{k}\right)}{{\sigma }_{q}^{2}}$, where ${e}_{jk}$ is the joint probability distribution of the remaining degrees of the two nodes j and k, ${q}_{k}$ is the distribution of the remaining degrees and ${\sigma }_{q}^{2}$ is the maximal value of function for perfectly assortativity network.

• Average clustering coefficient $C=\frac{1}{N}\sum _{i}{c}_{i}$, where ${c}_{i}$ is the clustering coefficient of vertex $i$.

• Average connection density ${K}_{den}=\frac{\left|E\right|}{\left|{N}^{2}\right|-\left|N\right|}$, where $\left|E\right|$ is the cardinality of the set of edges and $\left|N\right|$ is the cardinality of the set of nodes.

• Average degree $\langle K \rangle =\frac{K}{N}$, where $N$ is the number of vertices, $N=\left|V\right|$, and $K$ is the number of edges, $K=\left|E\right|$.

• Average path length $L=\sum _{i,j}\frac{{d}_{ij}}{N(N-1)}$, where ${d}_{ij}$ is the shortest path between $i$ and $j$, and $N$ is the number of vertices.

• Average shortest path when the network contains node k, the average shortest path of the network is ${L}_{k}=\frac{\sum _{i\ne j}{d}_{ij}}{N}(i,j\in V)$ and when the network does not contain the node k, the average shortest path of the network L is: $L=\frac{\sum _{i\ne k\ne j}{d}_{ij}}{N-1}(i,j\in V)$.

• Average shortest path length $L=\frac{1}{N(N-1)}\sum _{i\ne j}{d}_{ij}$, where ${d}_{ij}$ is the shortest path between $i$ and $j$, and $N$ is the number of vertices.

• Betweenness (B) ${B}_{i} ={\sum }_{s,t}\frac{{g}_{s,t}^{i}}{{g}_{s,t}}={\sum }_{j}{w}_{ji}$, where ${g}_{s,t}^{i}$ is the number of shortest paths connecting nodes s and t that include node i, and ${g}_{s,t}$ is the number of shortest paths connecting s and t, for all pairs s and t.

• Betweenness centrality ${bc}_{i}=\sum _{t\ne k\ne i}\frac{{\sigma }_{tk\left(i\right)}}{{\sigma }_{tk}}$, where ${\sigma }_{tk}$ is defined as the total number of shortest paths from node t to k, and ${\sigma }_{tk\left(i\right)}$ is the number of shortest paths from node $t$ to $k$ passing through node $i$.

• Cliques it represents complete subgraphs.

• Comprehensive Eigenvalue $CE=\alpha \varDelta L+\beta \varDelta C+\gamma {C}_{b\left(k\right)}$, where $\varDelta L$ is the shortest path variation of node $k$, $\varDelta C$ is the clustering coefficient variation and ${C}_{b\left(k\right)}$ is the betweenness of node $k$ and $\alpha +\beta +\gamma =1$.

• Coreness is a network degeneracy property that decomposes the graph ($G$) into a set of maximal connected subgraphs ${G}_{k}$, where $k$ denotes the core, such that nodes in $G$ have degree at least $k$ within the subgraph and ${G}_{k}\subseteq {G}_{k+1}.$ Coreness of a node is the highest core to which it belongs.

• Closeness centrality ${C}_{i}={l}_{i}^{-1}=n/{\sum }_{j}{d}_{ij}$, where ${l}_{i}$ is the average distance from node $i$ to all the other nodes, and ${d}_{ij}$ is the length of a geodesic path connecting nodes $i$ and $j$.

• Clustering coefficient it can be calculated as ${c}_{i}=2{n}_{i}/({k}_{i}^{2}-{k}_{i})$, where ${k}_{i}$ is the degree of vertex $i$, ${n}_{i}$ is the number of neighbors of $i$.

• Clustering coefficient variation clustering coefficient variation of node $k$ is $\varDelta C=\left|{C}_{k}-C\right|$, where ${C}_{k}$ is the average clustering coefficient when the network contains node $k$ and $C$ is the average clustering coefficient when the network does not contain node $k$.

• Degree of a vertex it quantifies the number of immediate neighbors of a vertex $i$ and can be obtained as: ${k}_{j} ={\sum }_{j}{A}_{ij}$, where ${A}_{ij}$ is the adjacency matrix such that ${A}_{ij}$ is 1 if there is an edge from vertex $j$ to vertex $i$, otherwise 0. In a directed network, each vertex has two degrees. The out-degree is the number of outgoing edges going out of from a vertex ${k}_{i}^{out} ={\sum }_{j}{A}_{ji}$ and the in-degree is the number of incoming edges onto a vertex ${k}_{i}^{in} ={\sum }_{j}{A}_{ij}$. The total degree of the vertex is the sum of its in- and out-degree ${k}_{i}^{tot}={k}_{i}^{in}+{k}_{i}^{out}$.

• Degree centrality ${degree}_{i}=\frac{1}{(n-1)}\sum _{j\ne i}{m}_{ij}$, where ${m}_{ij}=1$ if node $i$ is connected to node $j$.

• Degree distribution the degree distribution of a network, $p\left(k\right), k=0, 1,$ ..., measures the proportion of nodes in the network having degree $k$. Formally: $p\left(k\right)=\frac{{n}_{k}}{n}$.

• Diameter is the largest of all longest paths, $max\left\{{d}_{ij}\right\}$, where ${d}_{ij}$ represents the network diameters.

• Density $D=\frac{K}{N(N-1)}$, where $N$ is the number of vertices, $N=\left|V\right|$, and $K$ is the number of edges, $K=\left|E\right|$.

• Eccentricity the eccentricity of a node $i$ is a centrality index equal to the maximum length of all shortest paths from $i$ to the other nodes in the network.

• Efficiency ${E}_{glob}\left(G\right)=\frac{1}{N(N-1)}\sum _{i\ne j\in G}\frac{1}{{d}_{ij}}$, where ${d}_{ij}$ is the shortest path between vertices $i$ and $j$, $N$ is the number of vertices, $N=\left|V\right|$, and $G$ is the graph composed of vertices $V$ and a set of edges $E$. Since efficiency is also defined for disconnected graphs, the local efficiency can also be analyzed and is defined as the average efficiency of the local subgraphs and can be formally stated as: ${E}_{loc}\left(G\right)=\frac{1}{N}\sum _{i\in G}{E}_{glob}\left({G}_{i}\right), i\notin {G}_{i}$, where ${G}_{i}$ is the subgraph of the neighbors of vertex $i$.

• Eigenvector centrality the eigenvector centrality assigns a value to a given node $i$ proportional to the sum of the eigenvector centrality values of the nodes connected to $i$.

• Frequency of labelled motifs ${n}_{w,m}=\frac{{\tilde{n}}_{w,m}}{{n}_{m}}$, where ${\tilde{n}}_{w,m}$ is the total number of occurrences of node $w$ in motif m and ${n}_{m}$ is the total number of occurrences of motif $m$, irrespective of any node labels.

• Fraction of reciprocal connections mathematically, a reciprocal connection exists if ${M}_{i,j}>0$ and ${M}_{j,i}>0$ for $i\ne j$, where $M$ is the weight matrix. The ratio of the number of reciprocal edges to the number of edges from the network is a measure known as the fraction of reciprocal connections (FRC).

• Generalized Accessibility ${\alpha }_{i}^{\left(\infty \right)}=exp\left(-\sum {P}_{i,j}\text{log}{P}_{i,j}\right)$, where $P$ is the probability transition of all the pairs of nodes i and j.

• Generalized selectivity ${C}_{D-in/out}^{w\alpha }={k}^{in/out} \times \left(\frac{{s}_{i}^{in/out}}{{k}_{i}^{in/out}}\right)={k}_{i}^{in/out}\times {\left({e}_{i}^{in/out}\right)}^{\alpha }, \alpha >0$, where ${s}_{i}^{in/out}$ is the in-strength and out-strength of the node $i$ defined by ${s}_{i}^{in/out}=\sum _{j}{w}_{ij/ji}$, where ${w}_{ij/ji}$ represents the weight of link, and ${e}_{i}^{in/out}$ is the selectivity (or average strength) defined by ${e}_{i}^{in/out}=\frac{{s}_{i}^{in/out}}{{k}_{i}^{in/out}}$.

• Intermittency it is not a traditional network measurement, but it has a strong relationship with the concept of cycle length in networks, and is used to measure how periodically a word is repeated.

• Load centrality it is similar to betweenness centrality but considering weights on edges.

• Local efficiency ${E}_{loc}=\frac{1}{N}\sum _{i\in G}{E}_{glob}\left({G}_{i}\right)$, $i\notin G$, where ${G}_{i}$ is the subgraph of the neighbors of $i$.

• Locality index ${l}_{i}=\frac{{N}_{i}^{int}}{{N}_{i}^{int}+{N}_{i}^{ext}}$, where ${N}_{i}^{int}$ is the number of internal connections of the neighbors of a node $i$ and ${N}_{i}^{ext}$ is the number of edges between the ${k}_{i}$ neighbors of $i$, plus the ${k}_{i}$ edges that connect node $i$ to its neighbors.

• Matching index ${\mu }_{i,j}=\frac{{\sum }_{k\ne i,j}{a}_{ik}{a}_{jk}}{\sum _{k\ne j}{a}_{ik}+\sum _{k\ne i}{a}_{jk}}$, where ${a}_{ij}$ is an element of the adjacency matrix, and ${a}_{ij}=1$ if nodes $i$ and $j$ are connected.

• Modularity $Q=\frac{1}{2m}{\sum }_{i=1}^{n}{\sum }_{j=1}^{n}\left[{a}_{ij}-\frac{{k}_{i}{k}_{j}}{2m}\right]\delta \left({c}_{i}, {c}_{j}\right)$, where $m$ is the number of edges, $n$ is the number of nodes, $\delta \left({c}_{i}, {c}_{j}\right)=1$ if the nodes i and j are from the same class (community) and $\delta \left({c}_{i}, {c}_{j}\right)=0$, otherwise. This measurement ranges from $-\infty \le Q<1$. For $Q>0$, the number of edges inside the communities is greater than the expected.

• Neighborhood it quantifies the amount of nodes in the h-th concentric level around node $i$.

• Radius is the smallest of all longest paths, $in\left({d}_{ij}\right)$, where ${d}_{ij}$ represents the network diameters.

• Selectivity ${\theta }_{i}=\frac{{s}_{i}}{{k}_{i}}$, where ${s}_{i}$ is the strength of vertex $i$ and ${k}_{i}$ is the degree of vertex $i$.

• Shortest path the value of shortest path of given nodes $i$ and $j$ is defined by ${SP}_{i}=\sum {d}_{ij}$, where ${d}_{ij}$ represents the network diameters.

• Shortest path variation the shortest path variation of node k is the difference between the shortest path of the network when the network contains k and when it does not contain, $\varDelta L=\left|{L}_{k}-L\right|$.

• Symmetry ${S}_{i}^{\left(h\right)}=\frac{exp\left(-\sum {p}_{i,j}^{\left(h\right)}\text{log}{p}_{i,j}^{\left(h\right)}\right)}{\left|{H}_{h}\left(i\right)\right|+{\sum }_{r=0}^{h-1}{\eta }_{r}}$, where ${H}_{h}\left(i\right)$ is the set of all nodes in the $h - th$ hierarchic level of node $i$, $\left|{H}_{h}\left(i\right)\right|$ is the number of nodes in ${H}_{h}\left(i\right)$, and by considering a given hierarchic level $r$, ${\eta }_{r}$ is the number of nodes without edges connecting to the next hierarchical level.

• Strength ${s}_{i}^{in/out}=\sum _{j}{w}_{ij/ji}$, where $w$ is weight adjacency matrix in which ${w}_{ij/ji}$ represents the weight of link.

• Strength of a node $strength\left({v}_{i}\right) ={\sum }_{j}{w}_{ij}={\sum }_{j}{w}_{ji}$, where ${w}_{ij}$ is the corresponding entry in the weight matrix W for edge (${v}_{i}$, ${v}_{j}$)

• Strength centrality ${SC}_{i}=\sum {w}_{ij}$, where ${SC}_{i}$ is the value of strength centrality of the given node $i$ and ${w}_{ij}$is the similarity value between node $i$ and $j$.

• Transitivity the transitivity of a network is the fraction of all possible triangles present in the network. Possible triangles are identified by the number of triads (two links with a shared vertex), where $T=(3\#triangles)/(\#triads)$.

• Weight distribution it can be computed throughout probabilistic density function represented by a histogram of weights.

Appendix 2: Evaluation metrics

• Accuracy $accuracy=\frac{TP+TN}{TP+FP+FN+TN}$.

• Area under the curve (AUC)–Receiver operating characteristic (ROC) curve is a performance measurement for classification problem at various thresholds settings. ROC is a probability curve and AUC represents degree or measure of separability. The ROC curve is plotted with true positive rate (TPR) against the false positive rate (FPR) where TPR is on y-axis and FPR is on the x-axis.

• F1-score $F1=\frac{2PR}{P+R}$, where $P$ is precision and $R$ is recall. There are also F2-score, $F2 =\frac{5PR}{4P+R}$, and Fβ-score, $F\beta =\left(1+{\beta }^{2}\right)\frac{PR}{{\beta }^{2}P+R}$, but these weren’t used in the papers addressed in this study.

• Link perplexity Perplexity is a measure of a model’s ability to generalize to unseen data. It is defined as the reciprocal geometric mean of the likelihood of a test corpus of a given model and the collapsed state of the Markov chain. It can be used to evaluate the topic dependency relationship structure through the link perplexity, which can be formally stated as: $linkPerp=exp-\frac{{\sum }_{i=P+1}^{L}{log}p\left({\tilde{l}}_{i}\right|{l}_{1:P})}{\left|L\right|}$, where ${l}_{1:P}$ represents the topic relationship links from a topic dependency network and $p\left({l}_{i}\right|{l}_{1:P})$ is the relationship predictive distribution of the remaining dependency relationships and $L$ is a set of links of a topic relationship network.

• Macro average F1 value $acroF1=\frac{MacroP\times MacroR\times 2}{MacroP+MacroR}$ .

• Macro average recall rate $MacroR=\frac{1}{n}\sum _{i=1}^{n}{R}_{i}$, where ${R}_{i}$ is the recall rate of class $i$ text, and $n$ is the total number of categories.

• Macro average accuracy rate $MacroP=\frac{1}{n}\sum _{i=1}^{n}{P}_{i}$, where ${P}_{i}$ is the accuray rate of class $i$ text and $n$ is the total number of categories.

• Precision $P=\frac{A\cap B}{A}$, where $A$ is collection of documents (words) extracted by an algorithm and $B$ is collection of documents (keywords) in text files.

• p-value represents the likelihood of obtaining the corresponding accuracy rate in a random classification.

• Recall $R=\frac{A\cap B}{B}$, where $A$ is collection of documents (words) extracted by an algorithm and $B$ is collection of documents (keywords) in text files.

• Recall-Oriented Understanding for Gisting Evaluation $uge-N=\frac{\sum _{S\in \left\{referenceSummaries\right\}}{\sum }_{{gram}_{n}\in S}{count}_{match}\left({gram}_{n}\right)}{\sum _{S\in \left\{referenceSummaries\right\}}{\sum }_{{gram}_{n}\in S}count\left({gram}_{n}\right)}$, where n is the length of the n-gram and ${gram}_{n}$, ${count}_{match}\left({gram}_{n}\right)$ is the maximum of n-grams co-occurring in the candidate summary.