Estimation of Sulfonamides Concentration in Water Based on Digital Colourimetry

Abstract

Overuse of antibiotics is causing the environment to become polluted with them. This is a major threat to global health, with bacteria developing resistance to antibiotics because of it. To monitor this threat, multiple antibiotic detection methods have been developed; however, they are normally complex and costly. In this work, an affordable, easy to use alternative based on digital colourimetry is proposed. Photographs of samples next to a colour reference target were acquired to build a dataset. The algorithm proposed detects the reference target, based on binarisation algorithms, in order to standardise the collected images using a colour correction matrix converting from RGB to XYZ, providing a necessary colour constancy between photographs from different devices. Afterwards, the sample is extracted through edge detection and Hough transform algorithms. Finally, the sulfonamide concentration is estimated resorting to an experimentally designed calibration curve, which correlates the concentration and colour information. Best performance was obtained using Hue colour, achieving a relative standard deviation value of less than 3.5%.

1 Introduction

Antibiotic resistance is becoming more common, which is a serious threat to the health of everyone in the world. The treatment of infections becomes more difficult when the microorganisms responsible are resistant to antibiotics, with increases in treatment time, risk of the spread of the infection and, ultimately, risk of death [1].

Due to the misuse and overuse of antibiotics in animal husbandry, farming and, of course, treatment of humans, environmental waters are getting polluted with them. This pollution causes bacteria to be constantly exposed to antibiotics, developing resistance to them, and subsequently disseminating antibiotic resistant genes [2]. To monitor this pollution, several antibiotic detection methods have been developed, such as: high performance liquid chromatography tandem mass spectrometry (HPLC-MS/MS), HPLC with other detectors, electrophoresis with different detectors, immunoassays, and colourimetry [3]. These commonly used methods are expensive and difficult to use on a large scale, which discourages the deployment of them in monitoring programs. Thanks to advances made in computer vision-based analytical chemistry and the advent of the smartphone, it is now possible to create affordable and easy to use solutions for large scale environmental monitoring, based on digital colourimetry [7].

Colourimetry is the method of ascertaining the concentration of a compound in a solution by analysing its colour. Traditionally, a spectrophotometer (device that measures absorbance in a given wavelengths of UV or visible light) is used for this purpose. However, some work has been done to more easily determine compound concentrations using digital colourimetry. With image processing techniques it is quicker to analyse the colour of a solution and can be done outside of a laboratory setting. This approach, although practical, is not very precise due to photographs having device dependent colour values and changes in illumination affecting the captured colours. Therefore, a preprocessing step of colour correction is needed so that the digital colourimetry is consistent across devices. However there is not a perfect answer to this problem, since colour correction (CC) techniques use assumptions and approximations that are not always correct [4].

Concerning environmental applications, smartphone use based only in software code and application development have been proposed. For instance, exposure to air pollution considering spatially and temporally variation of population was implemented with help of mobile and wireless devices that yield information about where and when people are present. Hence, collective activity patterns were determined using counts of connections to the cellular network, allowing the estimation of population-weighted exposure to fine particulate matter (PM2.5) in New York City, USA [5]. Moreover, an application that models environmental concentrations of fine particulate matter (PM2.5), coarse particulate matter (PM10) and ozone concentrations, calculating personal health risks at the smartphone’s current location, was proposed for utilisation in the state of Oregon, USA [6].

Combination of chemistry and smartphones has been underexploited in the environmental area. Evaluation of nitrite concentration and pH determination in combination with a low-cost paper-based microfluidic device has been proposed, using a dedicated application that acquires information from seven sensing areas, containing different immobilised reagents, to produce selective colour changes when a sample solution is placed in the sampling area. Under controlled conditions of light, using the flash of the smartphone as a light source, the image captured with the built-in camera is processed using a customised algorithm for multi-detection of the coloured sensing areas. The developed image processing allowed reducing the influence of the light source and the positioning of the microfluidic device in the picture, but no report about field application is given [7]. Moreover, a portable chromium(III) ion detection system based on a smartphone readout device based on ELISA protocol has been proposed without application to field analysis [8].

To the best of our knowledge, mobile devices have not been used so far for antibiotic screening in environmental waters. In this work, a new approach for antibiotic pollution monitoring is proposed, using digital colourimetry on data acquired with smartphone cameras. To estimate the concentration of sulfonamides, a calibration curve is used, correlating concentrations to the Hue colour value. The colour of the sample is corrected using a reference target, so that there is colour constancy between images of the same concentration.

2 Methodology

The objective of this work is to detect the colour of the sample in photographs taken with a smartphone and estimate its sulfonamide concentration.

Colour correction is needed to ensure colour is consistent between photographs under different illuminations or from different devices. Colour correction can be done without a reference target. Well known examples of these methods are the White Patch, Gray-World, Gray-Edge and Gamut-Mapping. However, using the colour chart usually achieves better results, since there is knowledge of what the colours of the patches should be. In this sense, in order to facilitate the colour correction, an x-rite ColorChecker Passport was included during the photographs acquisition. It has a few targets to use has a reference for colour correction, but, in this work, only the classic target chart with 24 patches of different colours arranged in a 6 by 4 grid, with known ground truth values, is used.

A segmentation of the ColorChecker patches and the sample is done, followed by a colour correction step, before extracting the colour of the sample. Finally, by comparing this colour value to a set of standard samples with known concentrations, the concentration of the sample is estimated, a method known as calibration curve in analytical chemistry. However, there are multiple colour spaces in which the colour value can be extracted, so a study between the components of the RGB, XYZ, CIELAB and HSV colour spaces is performed to find which component results in the best concentration versus colour model.

2.1 Data

For this work, a database was built with photographs of the sample next to the colour chart, set up as shown in Fig. 1. The samples vary in sulfonamide concentration, from 0 to 150 $\mu$g/L, which provide different colour in a concentration dependent manner.

Fig. 1.

Example photograph from the database.

For the extraction procedure of the sulfonamides, the mixed-mode ion exchange polystyrene divinylbenzene sulfonated (SDB-RPS) disk from Empore-3M (Bellefonte, PA, USA) was cut (13 mm diameter) and placed inside a polypropylene disk holder, (Swinnex R filter holder, SX0001300, EMD Millipore Corporation, Billerica, MA). Four units were attached in parallel to propulsion tubes (Tygon R , 1.02 mm i.d.) fitted in a peristaltic pump (Gilson Minipuls 2, Villiers-le-Bel, France). Standards and samples (10 mL) were loaded at 0.8 mL min-1. The reagent, that gives colour to the sample, used is p-dimethylaminocinnamaldehyde (DMACA). DMACA working solution (0.22 g $\hbox {L}^{-1}$) was prepared by dilution of DMACA stock solution in chloroform (1:1, v/v).

Each disk was photographed twice by three different smartphone cameras, meaning 24 photographs for each sulfonamide concentration (8 from each smartphone). The smartphones used were the Xiaomi A1 (12 MP, f/2.2 + 12 MP, f/2.6), Samsung Galaxy J5 2017 (13 MP, f/1.7) and Huawei P Smart (13 MP, AF + 2 MP, depth sensor). The photographs were captured with the default settings.

2.2 Segmentation

In order to automatically detect the colours from the reference target and the sample, a segmentation algorithm is needed to isolate both regions of interest. First, the region of the chart is detected and cropped from the initial image (see Fig. 2). Then, the 24 patches are segmented to extract their colour values (see Fig. 3). Finally the disc is detected (see Fig. 5), followed by the extraction of the sample colour (see Fig. 6).

Colour Chart: The colour chart is detected after converting the image to grayscale, using a multilevel Otsu thresholding to generate 3 thresholds and using the lowest one to binarise it (B in Fig. 2). The lowest threshold is used because, outside the patches, the colour chart is black. Afterwards the object with the larger area is chosen, in case other objects are detected. Finally, knowing the orientation in which the chart is placed, it is split in two and the desired side is kept (classic target chart with 24 patches of different colours arranged in a 6 by 4 grid), see Fig. 2.

Fig. 2.

Steps to isolate the classic 24 patch target. A – Original image in grayscale. B – A binarised. C – Complement of B. D – Holes filled. E – Bounding box around the desired colour target. F – Grayscale classic colour chart.

Patches: After cropping the chart from the grayscale image, it is binarised using an Otsu thresholding, followed by the exclusion of large and small objects in order to isolate all the 24 patches (C in Fig. 3). Since the patches encompass a large range of colours, it is difficult for a threshold to binarise all of them. Therefore, the missing patches are estimated by completing the missing spots in the 6 by 4 grid of patches, using the centroids of the detected patches (D in Fig. 3).

Fig. 3.

Steps to estimate the centroids of the 24 patches. A – Grayscale classic colour chart. B – A binarised. C – Patches isolated. D – Estimation of missing patches centroids.

After determining the centroids for the 24 patches, a square bounding box is cropped around each one. The influence of the size of the bounding box was empirically tested and it was verified that using smaller squares does not significantly alter the colour values extracted. Therefore, since the detection of the patches might not be perfect, the bounding box is smaller than the patch, as demonstrated in Fig. 4). To extract the colour values, the median is chosen, removing the influence of outlier pixels.

Fig. 4.

Final crop for each patch.

Disk: First, the colour chart is covered, using the previous detection, so that it does not influence the sample detection. Then, a Canny edge detection is applied to detect the disk (B in Fig. 5). A dilation is applied to ensure that the edges of the disk are connected and then a fill is done so that the middle of the disk is covered (C in Fig. 5). Since this dilation greatly expands the edges, the initial segmentation is done around the disk. Figure 5 shows the initial edge detection, the result of the dilation and fill and how the disk is cropped from the initial image.

Fig. 5.

Initial detection of the disk. A – Region without the colour chart. B – Edge detection. C – Edges dilated and filled. D – Disk detection bounding box. E – Cropped disk.

Sample: Using the circular Hough transform, the disk is more precisely detected (A in Fig. 6). In low concentrations of sulfonamides, the sample presents very little colour, making it hard to distinguish from the rest of the disk. To differentiate the colour in those cases, the image of the disk is converted to grayscale and normalised between 0 and 1, followed by a histogram equalisation (D in Fig. 6). Then, a binarisation is done with the lowest threshold generated from a 3-multilevel Otsu thresholding (E in Fig. 6). Figure 6 shows how the initial crop is done around the disk, followed by a more precise detection of the disk and then the sample for colour extraction, using the median of all values.

Fig. 6.

Sample detection steps. A – Disk detection with Hough transform. B – Cropped disk. C – Grayscale B. D – C after normalisation and histogram equalisation. E – D binarised. F – Sample detected.

2.3 Colour Correction

In order to choose the most appropriate method for colour correction task, the following methods were compared, taken into account the standard deviation of the colour of the samples after colour correction:

• $*$ Weighted Grey Edge (WGE). The grey edge algorithm is based on the assumption that the average edge difference in a scene is achromatic. The weighted grey edge algorithm incorporates weighting of different types of edges (specular, shadow or material edges) [9].

• $*$ Illuminant Estimation Using White Patch (White). Estimating the illuminant as the colour of the white patch, since it should be white and any deviation is due to the illumination.

• $*$ Illuminant Estimation Using Achromatic Patches (Neutral). Same as above, but using the average colour between the 6 achromatic patches instead of just the white patch.

• $*$ Colour Correction Matrix RGB to RGB. This approach here presented uses the RGB values extracted from all the 24 color patches of the color rendition chart and compares them to the reference values provided by the chart’s manufacturer that were precisely measured with a spectrophotometer. By performing a least square regression between these two sets of values, the parameters that will more likely approximate the measured RGB values with the reference values are estimated [10].

• $*$ Colour Correction Matrix RGB to XYZ. Unlike the RGB to RGB transform described before, this method uses a RGB to XYZ transformation. Therefore obtaining the colour corrected XYZ image, which is the linear, device independent version of the raw camera output [11].

3 Results

3.1 Colour Correction

Table 1 compares the colour correction methods for each colour component, using standard deviation as a metric. A low standard deviation implies a better colour correction, showing that samples with the same concentration have a similar colour. For this comparison, images of four concentrations (0, 5, 25, 100 $\mu$g/L) were selected, and manual annotation of the patches and sample was performed, so that it is independent of the segmentation algorithm.

Table 1. Comparison of standard deviations for different colour correction methods in different colour spaces.

This comparison demonstrates that the RGB to XYZ colour correction matrix is the best method overall. Figure 7 shows an example of colour correction using this method.

Fig. 7.

Original image (left) and colour corrected image (right) using RGB-XYZ.

Further examination is needed to choose the best colour component, so calibration curves were plotted for each colour component and analysed.

3.2 Calibration Curves

Calibration curves were done using all the samples from the database, since their sulfonamide concentrations are known. In this step, the automatic segmentation algorithm described in Sect. 2.2 was applied, followed by the RGB to XYZ correction.

Increasing amounts of sulfonamides were retained in the solid support upon passage of more concentrated solutions. Upon addition of colour developing reagent (DMACA), the intensity of colour must be proportional to the amount of sulfonamides retained in the extracting disk. Therefore, a linear relationship between colour and concentration should exist.

Analysing the monotony of data, all colour components (in all colour spaces) showed monotony for data collected for solutions containing between 40–150 $\mu$g/L, meaning that the increase of sulfonamides retained in the solid support was correlated to an increase (or to a decrease) in the value of the component. For lower concentrations of sulfonamide (0–20 $\mu$g/L), only components G, H, a* and b* showed a monotonic behaviour. This means that these components are suitable for determination at lower concentrations, which could not be detected by analysis of other colour components.

In Fig. 8, the calibration curves for these four components are demonstrated. For each sulfonamide concentration (0, 5, 10, 15, 20, 40, 50, 100 and 150 $\mu$g/L) the average colour and the standard deviation are represented and a linear trendline is applied.

Fig. 8.

Calibration curves for Green, Hue, a* and b* colour components using the RGB-XYZ method.

As Fig. 8 demonstrates, the data does not appear to be well described by a linear regression globally. Therefore, studying the data separately for the lower concentrations (0–20 $\mu$g/L) and higher concentrations (40–150 $\mu$g/L) might give better results. Figure 9 shows how this division was done using the Hue component as an example. With this, the linear approximation appears to better represent the data.

Fig. 9.

Divided calibration curves for the Hue colour component using the RGB-XYZ method.

Afterwards, the relative standard deviations (RSD) were calculated for each concentration and colour component. The RSD is calculated by dividing the standard deviation by the absolute value of the mean and multiplying by 100, as shown in Eq. 1. It presents the standard deviation as a percentage of the mean. It is important for the data to have a low RSD, because it indicates a high precision.

$$\begin{aligned} RSD = \frac{std}{\left| mean \right| }*100 \end{aligned}$$

(1)

Table 2 shows that the components Green and Hue have the highest precision, although the RSD for Green becomes worse for higher concentrations, while Hue maintains its low RSD. Therefore, the concentration versus Hue calibration curve achieves the best performance. The 9694.8 RSD for $\hbox {a}^*$ and 427.8 RSD for $\hbox {b}^*$ happen because the mean is close to zero at those concentrations for those colour components.

Table 2. Relative standard deviations for each colour component and concentration.

4 Conclusions

This work was done with the aim of developing an algorithm to estimate sulfonamide concentration based on colour, that can be used for monitoring antibiotic pollution in environmental waters.

The methodology is composed of a segmentation, colour correction and calibration curve step. The segmentation step is split in two, colour patches and sample segmentation. The patches are necessary for the colour correction step, while the colour of the sample is needed to estimate the concentration of sulfonamides. The colour correction step uses a colour correction matrix that converts from RGB to XYZ. This colour correction method presented the best results due to using the colour chart as a known reference and converting to a device independent colour space. In order to choose which colour space and component is the best to estimate the sulfonamide concentration, an analysis of calibration curves was performed. The Hue and Green components presented the highest precision, with Hue being more consistent across all concentrations, never surpassing 3.5% relative standard deviation.

These results are promising, however there is still work to be done. New data will be acquired and used to validate the calibration curve. The developed algorithms will be incorporated in a smartphone application, so that it becomes a readily available alternative for sulfonamide detection.