Intelligent forecaster of concentrations (PM2.5, PM10, NO2, CO, O3, SO2) caused air pollution (IFCsAP)

Abstract

Upgrading health reality is the responsibility of all, it is necessary to think about the design of a smart system based on modern technologies to reduce the time and effort exerted on the competent authorities in both health and environmental sectors furthermore making their work environment smart and easy to enable the creativity and innovation as well as to reduce the material costs granted by state of this case “environment.” The process to find a solution for the problem of a triangle as shown in Figure (1) with contradictory heads is a very important and difficult issue in the field of health and environment, these are: (to optimize time utilization, and minimize the human errors, that accompany this human effort as much as possible, and to reduce material costs). Therefore, the idea of Internet technology and the Intelligent Big Data Analysis was developed to design an integrate electronic system of hardware to be developed in different and specific locations to collect information on concentrations that cause air pollution. So, it was invested an idea of Internet of things technology and intelligent data analysis ("Internet of things" and "Intelligent Data Analysis") for the construction of an integrated system of Hardware entities and Software entities placed. The aim of this work is to build a programmable system capable of predicting the pollutant concentrations within the next 48 h called intelligent forecaster of concentrations caused air pollution (IFCsAP) and making the machine the primary source of information after these concentrations are collected and stored in real time. On this basis, we will rely on modern technologies to reach this goal. The proposed design is highly efficient, cost-effective and easy to use and can be deployed in all places (environment with concentrations of air pollution). The main objective of the proposed system is to issue periodic reports (within 48 h of the future) based on the information input from different stations in real time. These reports are issued based on the readings of sensors planted at each station. Each sensor has a measurement of one or more concentrations that cause air pollution. Designed system consists of three basic phases: the construction phase of an integrated electronic circuit consisting of several devices (Modern, LoRa, Waspmate Platform, Arduino, Five sensors).

Appendices

Appendix

1.1 Main steps for training LSTM–RNN

In this section, we will show the main steps required to take the decision based on LSTM-RNN, also show how can update variables.

Step 1: The forward components

Step 1.1: Compute the gates

Memory cell:

$$ a_{t} = \tanh \left( {W_{a} \cdot X_{t} + U_{a} \cdot {\text{out}}_{t - 1 } + b_{a} } \right) $$

(1)

Input gate:

$$ i_{t} = \sigma \left( {WI \cdot X_{t} + U_{i} \cdot {\text{out}}_{t - 1 } + b_{i} } \right) $$

(2)

Forget gate:

$$ f_{t} = \sigma \left( {Wf \cdot X_{t} + U_{f} \cdot {\text{out}}_{t - 1} + b_{f} } \right) $$

(3)

Output gate:

$$ o_{t} = \sigma \left( {Wo \cdot X_{t} + U_{o} \cdot {\text{out}}_{t - 1} + b_{o} } \right) $$

(4)

Then fined:

Internal state:

$$ {\text{State}}_{t} = a_{t} \odot i_{t} + f_{t} \odot {\text{state}}_{t - 1} $$

(5)

Output:

$$ {\text{out}}_{t} = \tanh \left( {{\text{state}}} \right) \odot {\text{ot}} $$

(6)

where

$$ {\text{Gate}}\;S_{t} = \left[ {\begin{array}{*{20}c} {a_{t} } \\ {i_{t} } \\ {\begin{array}{*{20}c} {f_{t} } \\ {o_{t} } \\ \end{array} } \\ \end{array} } \right],\;\;W = \left[ {\begin{array}{*{20}c} {W_{a} } \\ {W_{i} } \\ {\begin{array}{*{20}c} {W_{f} } \\ {W_{o} } \\ \end{array} } \\ \end{array} } \right],\;\;U = \left[ {\begin{array}{*{20}c} {U_{a} } \\ {U_{i} } \\ {\begin{array}{*{20}c} {U_{f} } \\ {U_{o} } \\ \end{array} } \\ \end{array} } \right],\;\;b = \left[ {\begin{array}{*{20}c} {b_{a} } \\ {b_{i} } \\ {\begin{array}{*{20}c} {b_{f} } \\ {b_{o} } \\ \end{array} } \\ \end{array} } \right] $$

Step. 2: The backward components:

Step 2.1. Find

$\Delta t$ the output difference as computed by any subsequent.

$\Delta {\text{OUT}}$ the output difference as computed by the next time-step

$$ \delta {\text{out}}_{t} = \Delta_{t} + \Delta {\text{out}}_{t} $$

(7)

$$ \delta {\text{state}}_{t} = \delta {\text{out}}_{t } \odot o_{t} \odot \left( { 1 - \tanh^{2} \left( {{\text{state}}_{t} } \right)} \right) + \delta {\text{state}}_{t + 1} \odot f_{t + 1} $$

(8)

Step 2.2: Gives

$$ \delta a_{t} = \delta {\text{state}}_{t} \odot i_{t} \odot \left( {1 - a_{t}^{2} } \right) $$

(9)

$$ \delta i_{t} = \delta {\text{state}}_{t} \odot a_{t} \odot i_{t} \odot \left( {1 - i_{t} } \right) $$

(10)

$$ \delta f_{t} = \delta {\text{state}}_{t} \odot {\text{state}}_{t - 1} \odot f_{t} \odot \left( {1 - f_{t} } \right) $$

(11)

$$ \delta o_{t} = \delta {\text{out}}_{t} \odot \tanh ({\text{state}}_{t} ) \odot o_{t} \odot \left( {1 - o_{t} } \right) $$

(12)

$$ \delta x_{t} = W^{t} \cdot \delta {\text{state}}_{t} $$

(13)

$$ \delta {\text{out}}_{t - 1} = U^{t} \cdot \delta {\text{state}}_{t} $$

(14)

Step 3: update to the internal parameter

$$ \delta W = \mathop \sum \limits_{t = 0}^{T} \delta {\text{gates}}_{t} \otimes x_{t} $$

(15)

$$ \delta U = \mathop \sum \limits_{t = 0}^{T} \delta {\text{gates}}_{t + 1} \otimes {\text{out}}_{t} $$

(16)

$$ \delta b = \mathop \sum \limits_{t = 0}^{T} \delta {\text{gates}}_{t + 1} $$

(17)

2.1 Example of LSTM

This example will show calculations performed in LSTM.

Assume the out internal weights:

$$ W_{a} = \left[ {\begin{array}{*{20}c} {0.45} \\ {0.25} \\ \end{array} } \right],\;\;U_{a} = \left[ {0.15} \right],\;\;b_{a} = \left[ {0.2} \right] $$

$$ W_{i} = \left[ {\begin{array}{*{20}c} {0.95} \\ {0.8} \\ \end{array} } \right],\;\;U_{i} = \left[ {0.8} \right],\;\;b_{i} = \left[ {0.65} \right] $$

$$ W_{f} = \left[ {\begin{array}{*{20}c} {0.7} \\ {0.45} \\ \end{array} } \right],\;\;U_{f} = \left[ {0.1} \right],\;\;b_{f} = \left[ {0.15} \right] $$

$$ W_{o} = \left[ {\begin{array}{*{20}c} {0.6} \\ {0.4} \\ \end{array} } \right],\;\;U_{o} = \left[ {0.25} \right],\;\;b_{o} = \left[ {0.1} \right] $$

Let input data:

$$ X_{0} = \left[ {\begin{array}{*{20}c} 1 \\ 2 \\ \end{array} } \right]\;{\text{with}}\;{\text{label}}:0.5 $$

$$ X_{1} = \left[ {\begin{array}{*{20}c} {0.5} \\ 3 \\ \end{array} } \right]\;{\text{with}}\;{\text{label:}}\;1.25 $$

Forward t = 0

$$ a_{0} = \tanh (W_{a} * x_{0} + U_{a} * {\text{out}}_{ - 1} + b_{a} = \tanh \left( {\left[ {\begin{array}{*{20}c} {0.45} & {0.25} \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} 1 \\ 2 \\ \end{array} } \right] + \left[ {0.15} \right]\left[ 0 \right] + [0.2]} \right) = 0.81775 $$

$$ i_{0} = \sigma (W_{i} * x_{0} + U_{i} * {\text{out}}_{ - 1} + b_{i} = \sigma \left( {\left[ {\begin{array}{*{20}c} {0.95} & {0.8} \\ \end{array} } \right] \left[ {\begin{array}{*{20}c} 1 \\ 2 \\ \end{array} } \right] + \left[ {0.8} \right]\left[ 0 \right] + [0.65]} \right) = 0.96083 $$

$$ f_{0} = \sigma (W_{f} * x_{0} + U_{f} * {\text{out}}_{ - 1} + b_{f} = \sigma \left( {\left[ {\begin{array}{*{20}c} {0.7} & {0.45} \\ \end{array} } \right] \left[ {\begin{array}{*{20}c} 1 \\ 2 \\ \end{array} } \right] + \left[ {0.1} \right]\left[ 0 \right] + [0.15]} \right) = 0.85195 $$

$$ o_{0} = \sigma (W_{o} * x_{0} + U_{o} * {\text{out}}_{ - 1} + b_{o} = \sigma \left( {\left[ {\begin{array}{*{20}c} {0.6} & {0.4} \\ \end{array} } \right] \left[ {\begin{array}{*{20}c} 1 \\ 2 \\ \end{array} } \right] + \left[ {0.25} \right]\left[ 0 \right] + [0.1]} \right) = 0.81757 $$

$$ {\text{state}}_{0} = a_{0} * i_{0} + f_{0} * {\text{state}}_{ - 1} = 0.81775 * 0.96083 + 0.85195 * 0 = 0.78572 $$

$$ {\text{out}}_{0} = \tanh \left( {{\text{state}}_{0} } \right) * o_{0} = \tanh \left( {0.78572} \right) * 0.81757 = 0.53631 $$

$$ a_{1} = \tanh (W_{a} * x_{1} + U_{a} * {\text{out}}_{0} + b_{a} = \tanh \left( {\left[ {0.45\;\;0.25 } \right]\left[ {\begin{array}{*{20}c} {0.5} \\ 3 \\ \end{array} } \right] + \left[ {0.15} \right]\left[ {0.53631} \right] + [0.2]} \right) = 0.84980 $$

$$ i_{1} = \sigma (W_{i} * x_{1} + U_{i} * {\text{out}}_{0} + b_{i} = \sigma \left( {\left[ {0.95\;\;0.8} \right] \left[ {\begin{array}{*{20}c} {0.5} \\ 3 \\ \end{array} } \right] + \left[ {0.8} \right]\left[ {0.53631} \right] + [0.65]} \right) = 0.98118 $$

$$ f_{1} = \sigma (W_{f} * x_{1} + U_{f} * {\text{out}}_{0} + b_{f} = \sigma \left( {\left[ {0.7\;\;0.45} \right] \left[ {\begin{array}{*{20}c} {0.5} \\ 3 \\ \end{array} } \right] + \left[ {0.1} \right]\left[ {0.53631} \right] + [0.15]} \right) = 0.87030 $$

$$ o_{1} = \sigma (W_{o} * x_{1} + U_{o} * {\text{out}}_{0} + b_{o} = \sigma \left( {\left[ {0.6\;\;0.4} \right] \left[ {\begin{array}{*{20}c} {0.5} \\ 3 \\ \end{array} } \right] + \left[ {0.25} \right]\left[ {0.53631} \right] + [0.1]} \right) = 0.84993 $$

$$ {\text{state}}_{1} = a_{1} * i_{1} + f_{1} * {\text{state}}_{0} = 0.84980 * 0.98118 + 0.87030 * 0.78572 = 1.5176 $$

$$ {\text{out}}_{1} = \tanh \left( {{\text{state}}_{1} } \right) * o_{1} = \tanh \left( {1.5176} \right) * 0.84993 = 0.77197 $$

Backward t = 1

Compute the different between outputs

$$ \partial_{x} E = x - \hat{x} $$

So, $\Delta_{1} = \partial_{x} E = 0.77197 - 1.25 = - 0.47803$.

$\Delta {\text{out}}_{1} = 0$ Because there is no future time – steps

$$ \delta {\text{out}}_{1} = \Delta_{1} + \Delta {\text{out}}_{1} = - 0.47803 + 0 = - 0.47803 $$

$$ \begin{aligned} \delta {\text{state}}_{1} = & \delta {\text{out}}_{1} * o_{1} * \left( {1 - \tanh^{2} \left( {{\text{state}}_{1} } \right)} \right) + \delta {\text{state}}_{2} * f_{2} \\ = & - 0.47803 * 0.84993 * \left( {1 - \tanh^{2} \left( {1.5176} \right)} \right) + 0 * 0 = - 0.07111 \\ \end{aligned} $$

$$ \delta a_{1} = \delta {\text{state}}_{1} * i_{1} * \left( {1 - a_{1}^{2} } \right) = - 0.07111 * 0.98118 * \left( {1 - 0.84680^{2} } \right) = - 0.01938 $$

$$ \delta i_{1} = \delta {\text{state}}_{1} * a_{1} * i_{1} * \left( {1 - i_{1} } \right) = - 0.07111 * 0.84980 * 0.98118 * \left( {1 - 0.98118} \right) = - 0.00631 $$

$$ \delta f_{1} = \delta {\text{state}}_{1} * {\text{state}}_{0} * f_{1} * \left( {1 - f_{1} } \right) = - 0.07111 * 0.78572 * 0.78030 * \left( {1 - 0.78030} \right) = - 0.00631 $$

$$ \delta o_{1} = \delta {\text{out}}_{1} * \tanh ({\text{state}}_{1} ) * o_{1} * \left( {1 - o_{1} } \right) = - 0.47803 * \tanh \left( {1.5176} \right) * 0.84993 * \left( {1 - 0.84993} \right) = - 0.05538 $$

$$ \delta x_{1} = W^{T} * \delta {\text{gates}}_{1} = \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {0.45} & {0.95} & {\begin{array}{*{20}c} {0.70} & {0.60} \\ \end{array} } \\ \end{array} } \\ {\begin{array}{*{20}c} {0.25} & {0.80} & {\begin{array}{*{20}c} {0.45} & {0.40} \\ \end{array} } \\ \end{array} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} { - 0.01938} \\ { - 0.00112} \\ {\begin{array}{*{20}c} { - 0.00631} \\ { - 0.05538} \\ \end{array} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} { - 0.04743} \\ { - 0.03073} \\ \end{array} } \right] $$

$$ \Delta {\text{out}}_{0} = U^{T} * \delta {\text{gates}}_{1} = \left[ {\begin{array}{*{20}c} {0.15} & {0.80} & {\begin{array}{*{20}c} {0.10} & {0.25} \\ \end{array} } \\ \end{array} } \right] \left[ {\begin{array}{*{20}c} { - 0.01938} \\ { - 0.00112} \\ {\begin{array}{*{20}c} { - 0.00631} \\ { - 0.05538} \\ \end{array} } \\ \end{array} } \right] = - 0.01828 $$

At this time need return to back our Δout and forwarding on calculation.

Backward t = 0

$$ \Delta_{0} = \partial_{x} E = 0.53631 - 0.5 = - 0.03631 $$

$$ \Delta {\text{out}}_{1} = - 0.01828,\;\;{\text{passed}}\;{\text{back}}\;{\text{from}}\;T = 1 $$

$$ \delta {\text{out}}_{0} = \Delta_{0} + \Delta {\text{out}}_{0} = 0.03631 + \left( { - 0.01828} \right) = 0.01803 $$

$$ \begin{aligned} \delta {\text{state}}_{0} = & \delta {\text{out}}_{0} * o_{0} * \left( {1 - \tanh^{2} \left( {{\text{state}}_{0} } \right)} \right) + \delta {\text{state}}_{1} * f_{1} \\ = & - 0.01803 * 0.81757 * \left( {1 - \tanh^{2} \left( {0.78572} \right)} \right) + ( - 0.07111 * 0.87030) = - 0.05349 \\ \end{aligned} $$

$$ \delta a_{0} = \delta {\text{state}}_{0} * i_{0} * \left( {1 - a_{0}^{2} } \right) = - 0.05349 * 0.96083 * \left( {1 - 0.81775} \right) = - 0.01703 $$

$$ \delta i_{0} = \delta {\text{state}}_{0} * a_{0} * i_{0} * \left( {1 - i_{0} } \right) = - 0.05349 * 0.81775 * 0.96083 * \left( {1 - 0.96083} \right) = - 0.00165 $$

$$ \delta f_{0} = \delta {\text{state}}_{0} * {\text{state}}_{ - 1} * f_{0} * \left( {1 - f_{0} } \right) = - 0.05349 * 0 * 0.85195 * \left( {1 - 0.85195} \right) = 0 $$

$$ \delta o_{0} = \delta {\text{out}}_{0} * \tanh ({\text{state}}_{0} ) * o_{0} * \left( {1 - o_{0} } \right) = 0.01803 * \tanh \left( {0.78572} \right) * 0.81757 * \left( {1 - 0.81757} \right) = 0.00176 $$

$$ \delta x_{0} = W^{T} * \delta {\text{gates}}_{0} = \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {0.45} & {0.95} & {\begin{array}{*{20}c} {0.70} & {0.60} \\ \end{array} } \\ \end{array} } \\ {\begin{array}{*{20}c} {0.25} & {0.80} & {\begin{array}{*{20}c} {0.45} & {0.40} \\ \end{array} } \\ \end{array} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} { - 0.01703} \\ { - 0.00165} \\ {\begin{array}{*{20}c} 0 \\ {0.00176} \\ \end{array} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} { - 0.00817} \\ { - 0.00487} \\ \end{array} } \right] $$

$$ \Delta {\text{out}}_{ - 1} = U^{T} * \delta {\text{gates}}_{1} = \left[ {\begin{array}{*{20}c} {0.15} & {0.80} & {\begin{array}{*{20}c} {0.10} & {0.25} \\ \end{array} } \\ \end{array} } \right] \left[ {\begin{array}{*{20}c} { - 0.01703} \\ { - 0.00165} \\ {\begin{array}{*{20}c} 0 \\ {0.00176} \\ \end{array} } \\ \end{array} } \right] = - 0.00343 $$

After complete backward, let λ = 0.1 and update the parameters by:

$$ \begin{aligned} \delta W = & \mathop \sum \limits_{t = 0}^{T} \delta {\text{gates}}_{t} \otimes X_{t} \\ = & \left[ {\begin{array}{*{20}c} { - 0.01703} \\ { - 0.00165} \\ {\begin{array}{*{20}c} 0 \\ {0.00176} \\ \end{array} } \\ \end{array} } \right] \left[ {1.0 2.0} \right] + \left[ {\begin{array}{*{20}c} { - 0.01938} \\ { - 0.00112} \\ {\begin{array}{*{20}c} { - 0.00631} \\ { - 0.05538} \\ \end{array} } \\ \end{array} } \right]\left[ {0.5 3.0} \right] \\ = & \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} { - 0.02672} \\ { - 0.00221} \\ {\begin{array}{*{20}c} { - 0.00316} \\ { - 0.02593} \\ \end{array} } \\ \end{array} } & {\begin{array}{*{20}c} { - 0.0922} \\ { - 0.00666} \\ {\begin{array}{*{20}c} { - 0.01893} \\ { - 0.16262} \\ \end{array} } \\ \end{array} } \\ \end{array} } \right] \\ \end{aligned} $$

$$ \delta U = \mathop \sum \limits_{t = 0}^{T} \delta {\text{gates}}_{t + 1} \otimes {\text{out}}_{t} = \left[ {\begin{array}{*{20}c} { - 0.01938} \\ { - 0.00112} \\ {\begin{array}{*{20}c} { - 0.00631} \\ { - 0.05538} \\ \end{array} } \\ \end{array} } \right] \left[ {0.53631} \right] = \left[ {\begin{array}{*{20}c} { - 0.01039} \\ { - 0.00060} \\ {\begin{array}{*{20}c} { - 0.00338} \\ { - 0.02970} \\ \end{array} } \\ \end{array} } \right] $$

$$ \delta b = \mathop \sum \limits_{t = 0}^{T} \delta {\text{gates}}_{t + 1} = \left[ {\begin{array}{*{20}c} { - 0.01703} \\ { - 0.00165} \\ {\begin{array}{*{20}c} 0 \\ {0.00176} \\ \end{array} } \\ \end{array} } \right] + \left[ {\begin{array}{*{20}c} { - 0.01938} \\ { - 0.00112} \\ {\begin{array}{*{20}c} { - 0.00631} \\ { - 0.05538} \\ \end{array} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} { - 0.03641} \\ { - 0.00277} \\ {\begin{array}{*{20}c} { - 0.00631} \\ { - 0.05362} \\ \end{array} } \\ \end{array} } \right] $$

Depend on the SGD updating out parameters:

$$ W^{{{\text{new}}}} = W^{{{\text{old}}}} - \lambda * \delta W^{{{\text{old}}}} $$

$W_{a} = \left[ {\begin{array}{*{20}c} {0.45267} \\ {0.25922} \\ \end{array} } \right],\;\;U_{a} = \left[ {0.15104} \right],\;\; b_{a} = [0.20364$].

$W_{i} = \left[ {\begin{array}{*{20}c} {0.95022} \\ {0.80067} \\ \end{array} } \right],\;\;U_{i} = \left[ {0.80006} \right],\;\;b_{i} = [0.65028$]

$$ W_{f} = \left[ {\begin{array}{*{20}c} {0.70031} \\ {0.45189} \\ \end{array} } \right],\;\;U_{f} = \left[ {0.10034} \right],\;\;b_{f} = \left[ {0.15063} \right] $$

$$ W_{o} = \left[ {\begin{array}{*{20}c} {0.60259} \\ {0.41626} \\ \end{array} } \right],\;\;U_{o} = \left[ {0.25297} \right],\;\;b_{o} = \left[ {0.10536} \right] $$