Characterizing Dementia Caregivers’ Information Exchange on Social Media: Exploring an Expert-Machine Co-development Process

Abstract

Social media platforms have introduced new opportunities for supporting family caregivers of persons with Alzheimer’s disease and related dementias (ADRD). Existing methods for exploring online information seeking and sharing (i.e., information exchange) involve examining online posts via manual analysis by human experts or fully automated data-driven exploration through text classification. Both methods have limitations. In this paper, we propose an innovative expert–machine co-development (EMC) process that enables rich interactions and co-learning between human experts and automatic algorithms. By applying the EMC in analyzing ADRD caregivers’ online behaviors, we illustrate steps required by the EMC, and demonstrate its effectiveness in enhancing human experts’ representations of ADRD caregivers’ online information exchange and developing more accurate automatic classification models for ADRD caregivers’ information exchange.

A Appendix

1.1 A.1 Importance Score calculation

The keyword importance score I is a measure that is model-dependent, so we introduce the implementation of I within the three classification models:

• Our linear kernel SVM model can be written as $c=signal(b+W^Tx)$, where $W=(w_1,w_2, ... w_k)$ are the weights for the features in the model. abs(wi) represents the importance of the feature in the model [9], so it is selected as the I score.

• When expressed in log-space, classification based on Multinomial Naïve Bayes model can be written as Formula 4, where $b=log(p(c_j))$ and $w_ji=log(p_ji)$. Weighted Average Pointwise Mutual Information (WAPMI) calculated from $w_ji$ is a good measurement to evaluate the importance of the feature [41], so it can be used as the I score. Note that the WAPMI score is between keyword t and model m, and it is different from MI(t, c) where c is a category.

  $$\begin{aligned} log(p(c_j|x))&= log(p(c_j)\prod \limits _ip_ji^xi) \nonumber \\&= log(p(c_j)) +\sum \limits _ix_ilog(p_ji) \nonumber \\&= b+ W^t_j x \end{aligned}$$

  (4)

• The Xgboost model generates a forest of decision trees $T=(t_1,t_2,...,t_n)$, where a feature $x_t$ is used to split a branch $b_{ij}$ within a tree. The information gain $gain(x_t,b_{ij})$ for feature x in branch $b_{ij}$ can be used to measure whether the split is good. By calculating the average gain $gain(c_t)$(see Formula 5) across all the trees in the forest, we can measure how feature $c_k$ affects the whole model.$gain(c_k)$ is the most important score for Xgboost feature selection  [11]. Thus, we use this score as the I score.

$$\begin{aligned} gain(c_t)=\frac{\sum \limits _{i=1}^n\sum \limits _{j=1}^{b_i}gain(x_i,b_{ij})}{\sum \limits _{i=1}^nb_i} \end{aligned}$$

(5)

1.2 A.2 Abbreviations

ADRD :

Alzheimer’s disease and related dementias

EMC :

Expert–machineco-development

AE :

Automatic Exploration

IML :

Interactive Machine Learning

HIW :

Health Information Wants

EOL :

End-of-life

IAEI :

Interactive Auto Exploration Interface

KT :

Keyword Tuning

I :

Important Score

MI :

Mututal Infomration Score

KF :

Keyword Frequency

PG :

Potnetially Good Recommendation Group

PB :

Potnetially Bad Recommendation Group

LF :

Low Frequency Recommendation Group

NK :

New Keywords Recommendation Group

AMI :

Average Mutual Information Score

nDCG :

Normalized Discounted Cumulative Gain

ID :

Initial Dataset

TEST :

Test Dataset

RD :

Recommendation Dataset

RD :

Recommendation Dataset