mechanical and machine predictions can outperform human experts

Using Instagram data from 166 individuals, researchers applied machine learning tools to successfully identify markers of depression. Statistical features were computationally extracted from 43,950 participant Instagram photos, using color analysis, metadata components, and algorithmic face detection.

Resulting models outperformed general practitioners’ average unassisted diagnostic success rate for depression. These results held even when the analysis was restricted to posts made before depressed individuals were first diagnosed. Human ratings of photo attributes (happy, sad, etc.) were weaker predictors of depression, and were uncorrelated with computationally-generated features. These results suggest new avenues for early screening and detection of mental illness.

Source: https://epjdatascience.springeropen.com/articles/10.1140/epjds/s13688-017-0110-z

Clinical judgment refers to the typical procedure long used by applied psychologists and physicians, in which the judge puts data together using informal, subjective methods. Clinicians differ in how they do this: The very nature of the process tends to preclude precise specification.

Mechanical prediction, including statistical prediction (using explicit equations), actuarial prediction (as with insurance companies’ actuarial tables), and what we may call algorithmic prediction (e.g., a computer program emulating expert judges), is by contrast well specified.

To compare the accuracy of clinical and mechanical (formal, statistical) data-combination techniques, researchers performed a meta-analysis on studies of human health and behavior. On average, mechanical-prediction techniques were about 10% more accurate than clinical predictions. Depending on the specific analysis, mechanical prediction substantially outperformed clinical prediction in 33%-47% of studies examined.

Although clinical predictions were often as accurate as mechanical predictions, in only a few studies (6%-16%) were they substantially more accurate. Superiority for mechanical-prediction techniques was consistent, regardless of the judgment task, type of judges, judges’ amounts of experience, or the types of data being combined. These data indicate that mechanical predictions of human behaviors are equal or superior to clinical prediction methods for a wide range of circumstances.

Source: http://zaldlab.psy.vanderbilt.edu/resources/wmg00pa.pdf

Leave a comment