Presentation at CHI 2017

Bonnie Anderson and Dan Bjornn attended CHI 2017 this week at Denver, Colorado to present, “What Do We Really Know about How Habituation to Warnings Occurs Over Time? A Longitudinal fMRI Study of Habituation and Polymorphic Warnings.” CHI is widely considered the premier conference in the field of human–computer interaction.

We were fortunate to share the session with some of our favorite usable security researchers. The presentation went well and the questions from the audience allowed us to talk about our future research streams.

Whereas previous studies (including our own work) examined habituation at a single point in time, this study observed habituation over five consecutive days using fMRI and eye tracking simultaneously. This allowed us to measure not only the decrease in users’ attention to warnings over the course of the workweek, but also another core characteristic of habituation: response recovery—the increase in user response after a rest period during which the stimulus is absent.

We found that people habituated rapidly to repeated warnings within a single laboratory session (both in terms of decreased neural activity and fewer eye fixations). However, we observed a recovery effect of attention from one day to the next when warnings were withheld. Unfortunately, this recovery effect wasn’t enough to offset the overall pattern of habituation across the workweek. More positively, we found that a polymorphic warning with only four variations was able to significantly sustain attention over time.

Our results also add credibility to prior point-in-time studies by showing that the pattern of habituation they reported holds across a workweek, indicating that cross-sectional habituation studies can be useful proxies for habituation overtime. Additionally, the eye-tracking and fMRI results very similar, suggesting that eye tracking is a valid and cost-effective alternative to fMRI for studying the mental process of habituation.

From the abstract:

A major inhibitor of the effectiveness of security warnings is habituation: decreased response to a repeated warning. Although habituation develops over time, previous studies have examined habituation and possible solutions to its effects only within a single experimental session, providing an incomplete view of the problem. To address this gap, we conducted a longitudinal experiment that examines how habituation develops over the course of a five-day workweek and how polymorphic warnings decrease habituation. We measured habituation using two complementary methods simultaneously: functional magnetic resonance imaging (fMRI) and eye tracking.

Our results show a dramatic drop in attention throughout the workweek despite partial recovery between workdays. We also found that the polymorphic warning design was substantially more resistant to habituation compared to conventional warnings, and it sustained this advantage throughout the five-day experiment. Our findings add credibility to prior studies by showing that the pattern of habituation holds across a workweek, and indicate that cross-sectional habituation studies are valid proxies for longitudinal studies. Our findings also show that eye tracking is a valid measure of the mental process of habituation to warnings.

Article Download

Download a PDF of the article here.

Neurosecurity Lab at USENIX Enigma 2017

Tony presented recent work of the Neurosecurity Lab at USENIX Enigma 2017, a TED-style security conference held annually in the San Francisco Bay Area.

USENIX Enigma is interesting because its attendees and presenters are equal parts security academics and practitioners, with significant representation from Silicon Valley companies. It also has a strong emphasis on presentation quality.

Presentation Video

The talk received news coverage in a number of outlets, including:

Three New Research Articles on Habituation to Security Warnings

The Neurosecurity Lab has three recently published or accepted research articles on habituation to security warnings: (1) Journal of Management Information Systems (JMIS), (2) Decision Support Systems (DSS), and (3) ACM Conference on Human Factors in Computing Systems (CHI) 2017.

Journal of Management Information Systems

Our article, “From Warning to Wallpaper: Why the Brain Habituates to Security Warnings and What Can Be Done About It,” was published in December. It is an expanded version of our 2015 CHI paper. JMIS is widely recognized as one of the top three journals of the field of Information Systems.

The article reports the findings of two separate laboratory experiments that used fMRI and mouse cursor tracking to show how users habituate to security warnings. The fMRI experiment showed how neural activity in the visual processing center of the brain decreases precipitously with repeated exposures to a warning. We also found that a polymorphic warning design that repeatedly changed its appearance was resistant to the effects of habituation.

From the abstract:

Warning messages are fundamental to users’ security interactions. Unfortunately, research has shown that they are largely ineffective. A key contributor to this failure is habituation: decreased response to a repeated warning. Previous research has inferred the occurrence of habituation to warnings or measured it indirectly, such as through the proxy of a related behavior. Therefore, there is a gap in our understanding of how habituation to security warnings develops in the brain. Without direct measures of habituation, we are limited in designing warnings that can mitigate its effects.

In this study, we use neurophysiological measures to directly observe habituation as it occurs in the brain and behaviorally. We also design a polymorphic warning artifact that repeatedly changes its appearance in order to resist the effects of habituation. In an experiment using functional magnetic resonance imaging (fMRI; n = 25), we found that our polymorphic warning was significantly more resistant to habituation than were conventional warnings in regions of the brain related to attention.

In a second experiment (n = 80), we implemented the top four most resistant polymorphic warnings in a realistic setting. Using mouse cursor tracking as a surrogate for attention to unobtrusively measure habituation on participants’ personal computers, we found that polymorphic warnings reduced habituation compared to conventional warnings. Together, our findings reveal the substantial influence of neurobiology on users’ habituation to security warnings and security behavior in general, and we offer our polymorphic warning design as an effective solution to practice.

Article Download

Download a PDF of the article here.

Decision Support Systems

Our article, “Your Memory Is Working Against You: How Eye Tracking and Memory Explain Habituation to Security Warnings,” was published in December. This study examines habituation to security warnings in a laboratory experiment using eye tracking.

Habituation was measured in terms of the eye movement-based memory (EMM) effect, a neurophysiological manifestation of habituation in which people unconsciously scrutinize previously seen stimuli less than novel stimuli. Consistent with our JMIS findings, we also found that participants habituated less in terms of eye fixations to a polymorphic warning compared to conventional warnings.

From the abstract:

Security warnings are critical to the security of end users and their organizations, often representing the final defense against an attack. Because warnings require users to make a contextual judgment, it is critical that they pay close attention to warnings. However, research shows that users routinely disregard them. A major factor contributing to the ineffectiveness of warnings is habituation, the decreased response to a repeated warning. Although previous research has identified the problem of habituation, the phenomenon has only been observed indirectly through behavioral measures. Therefore, it is unclear how habituation develops in the brain in response to security warnings, and how this in turn influences users’ perceptions of these warnings.

This paper contributes by using eye tracking to measure the eye movement-based memory (EMM) effect, a neurophysiological manifestation of habituation in which people unconsciously scrutinize previously seen stimuli less than novel stimuli. We show that habituation sets in after only a few exposures to a warning and progresses rapidly with further repetitions. Using guidelines from the warning science literature, we design a polymorphic warning artifact which repeatedly changes its appearance. We demonstrate that our polymorphic warning artifact is substantially more resistant to habituation than conventional security warnings, offering an effective solution for practice. Finally, our results highlight the value of applying neuroscience to the domain of information security behavior.

Article Download

Download a PDF of the article here.

CHI 2017

Our article, “What Do We Really Know about How Habituation to Warnings Occurs Over Time? A Longitudinal fMRI Study of Habituation and Polymorphic Warnings,” is forthcoming at CHI 2017, held this year in Denver, Colorado. CHI is widely considered the premier conference in the field of human–computer interaction.

Whereas previous studies on habituation (including our JMIS and DSS studies above) examined habituation at a single point in time, this study observed habituation over the course of a workweek in five daily experimental sessions. We measured habituation using fMRI and eye tracking simultaneously, validating that eye tracking is a useful, non-obtrusive method for measuring habituation.

We found that people habituated rapidly to repeated warnings within a single laboratory session (both in terms of decreased neural activity and fewer eye fixations). However, we observed a recovery effect of attention from one day to the next when warnings were withheld. Unfortunately, this recovery effect wasn’t enough to offset the overall pattern of habituation across the workweek. More positively, we found that a polymorphic warning with only four variations was able to significantly sustain attention over time.

From the abstract:

A major inhibitor of the effectiveness of security warnings is habituation: decreased response to a repeated warning. Although habituation develops over time, previous studies have examined habituation and possible solutions to its effects only within a single experimental session, providing an incomplete view of the problem. To address this gap, we conducted a longitudinal experiment that examines how habituation develops over the course of a five-day workweek and how polymorphic warnings decrease habituation. We measured habituation using two complementary methods simultaneously: functional magnetic resonance imaging (fMRI) and eye tracking.

Our results show a dramatic drop in attention throughout the workweek despite partial recovery between workdays. We also found that the polymorphic warning design was substantially more resistant to habituation compared to conventional warnings, and it sustained this advantage throughout the five-day experiment. Our findings add credibility to prior studies by showing that the pattern of habituation holds across a workweek, and indicate that cross-sectional habituation studies are valid proxies for longitudinal studies. Our findings also show that eye tracking is a valid measure of the mental process of habituation to warnings.

Article Download

Download a PDF of the article here.

Future Work on Habituation

Despite the publication of these three articles, we’re not done with the topic of habituation yet. We recently completed a three-week field experiment that examines habituation in terms of reduced warning adherence behavior. This study is currently under peer review. We have also begun a pilot test to examine how the effects of habituation generalize from familiar notifications to novel warnings that share visual similarities. Across all of these studies, we’re seeking to find ways to reduce habituation so that warnings don’t lose their efficacy over time.

Bruce Schneier Visits the BYU Neurosecurity Lab

We recently had the pleasure of hosting author and security thought-leader, Bruce Schneier, at the Neurosecurity Lab. We know Bruce from presenting at the Workshop on Security and Human Behavior (2014, 2015, and 2016), which he co-chairs. Bruce has also featured our work on his blog, Schneier on Security.

We gave Bruce a tour of the MRI Facility:

As part of the tour, we scanned Bruce’s brain in the MRI scanner:

Best of all, Bruce gave a fantastic lecture to our students on security and the Internet of Things:

Thanks, Bruce, for visiting us at BYU!

On the Top of the World (Y Mountain)

The Neurosecurity Lab hiked to the top of Y Mountain, an 8,572 ft (2,613 m) mountain named for the 380 ft (116 m) “Y” insignia representing BYU.

It was a beautiful, clear fall morning. Below are some pictures we took.

On top of Y Mountain, overlooking BYU campus and Provo, UT.

The trail leading from the Y to the top of Y Mountain.

A panorama of Rock Canyon from the north summit.

Jeff, Brock, Bonnie, and Dan near the cliffs of the north summit.

After returning to the Y Mountain trailhead.