Mapping the Emotional Dimension:
Measuring Human Behavior Across Space & Time to Inform Tourism & Leisure Management
Garrett C. Millar
Hi everyone, thank you for coming today, and more importantly thank you for your contributions to my last 3 weeks here. My name is Garrett Millar, I'm a PhD student in Geospatial Analytics at North Carolina State University. Geospatial Analytics, also called Spatial Analytics, is the science of WHERE things HAPPEN.
LETS GET STARTED!!
First, I wanted to share my research background and previous research work. I got my Bachelors in Psychology, and then went straight into a PhD in Human-Computer Interaction. During this time, I researched how people use technology, how they experience it, and how they learn from it.
When I transitioned to the Geospatial Analytics program, I took with me this same sort of research ideal, but instead of how people interact with and experience technology, I wanted to find out how people interact with and experience space. With what we'll be discussing today, that space is experienced in the context of tourism and leisure, which I study to help manage those experiences.
Specifically, when people experience tourism and leisure, they feel something, and they feel something in a particular place. And to manage these experiences, you have to know not only that they felt something, but where in their experience they felt it. And SO, I'll be showing you today, where tourism and leisure participants FELT something.
My Netherlands Experience
Stroopwafel & analysis experience rambles . . .
Represents my past 3 weeks here:
Mainly because most of time here spent analysing data, coding, and visualizations.
Which is what that drawing is of, I was trying to figure out a way to appraoch analysing some new data I had just gotten.
I like to draw things out when I can, anf this specifically was for the Nuenen musuem data. As I wanted to draw out the floor plan with the locations of the beacon sensors that were used to collect the musuem visitors' locations.
When i finished drawing these beacons, I started think, they look a lot like my newly-found love here: STROOPWAFELS.
The second I came to this realization I hopped on my bike, went to the market, and picked up 3 bags of stroopwafels....
NEXT SLIDE
My Netherlands Experience
2 OF THE BAGS WERE EATEN THAT SAME NIGHT.
This is a bit silly and funny, but I think its a perfect way to let you all know a bit more about my experiences since I've been here (Note: beer next to me is not pictured)
Wanted to give overview of presentation . . .
2 projects to be discussed, and if time, talk about the next steps.
We first have CHIPS, a project that I've been working on to study and map the emotional arousal of cyclists while biking down a cycling highway between Tilburg & Waalwijk.
Then, there is Nuenen, a project measuring the emotional arousal and locations of its visitors, for the both indoor, and outdoor exhibits.
And last hopefully we have some time to talk about what all this work will look like further down the road.
Emotions
Why do emotions matter?
Measuring emotion
Mapping emotional experiences
Lets start from the top: why emotions? And why are we trying to map them?
Well, emotions play a major role in our day-to-day lives, as they play a large part in how we perceive and experience the world.
People can experience a wide range of different emotions, but usually depend on the context, or the place, in which they are experienced.
In fact, emotions are nothing more than a physiological response to some sort of stimuli, which tend to arise from our environment.
So, as tourism and leisure managers, we need to have a way to look at what people are feeling not just in general, but exactly where they are feeling it, so we can design that place to be the emotional experience that we want to deliver.
With this in mind, lets look at the cycling highway data.
Look at these videos. How do they make you feel? Look at the differences in the places. How do the differences in the places make you feel?
Call on someone, "Imagine yourself in this scenario. What are your initial thoughts and / or feelings?"
Look at these videos. How do they make you feel? Look at the differences in the places. How do the differences in the places make you feel?
Call on someone, "Imagine yourself in this scenario. What are your initial thoughts and / or feelings?"
Look at these videos. How do they make you feel? Look at the differences in the places. How do the differences in the places make you feel?
Call on someone, "Imagine yourself in this scenario. What are your initial thoughts and / or feelings?"
Methodology
Study Area — Netherlands
These are all scenes from the Tilburg Waalwijk cycling highway.
Now these videos are from the experiences of a few of the 12 people that rode this highway while their locations and emotional arousal were collected.
Lets map where they went, what kind of places they went through, And how they felt while doing it.
First, lets map where they went. This shows you in detail the highway they took between Tilburg and Waalwijk.
Methodology
Framework for Cyclists' Emotional Experiences
Then, lets look at what kind of places they went through, where were they?
First, we have different types of roads; such as smaller pedestrian roads, and then larger motorways
Then, theres different kinds of land use.
Places like forests, parks, businesses, and colleges.
And from putting roads and land use types together, we can create a colored map that shows what kind of areas they went through, which are then used to analyze their experiences statistically.
Methodology
Buffers as an Environment Interaction Metric
But before we can analyze their experiences statistically, we need to consider how they experience the environment, specifically, how they felt going through those places.
Well its very, very important to note, we dont experience the environment within the very immediate space around us, for instance, me giving this presentation, I am not only experiencing this table right in front of me and the floor right around my feet.
I'm trying to consider all of my audiences' different viewpoints, paying attention to any potential confused or disagreeing faces, while at risk of being distracted by a siren that may happen to pass by the building.
So we have to generate what are known as buffers. And what these do is basically stretch the sampled location point across a wider area (100m) to help us better account for peoples' wide range of poosbile interactions with the environment.
For instance, a cyclist may see a large intersection 50m ahead, and already begin to show a spike in emotion.
We can then use these more accurate environment experience points to continue our statistical investigation ...
Methodology
Mapping Emotion
And an inital exploration of that statistical process is shown here. We can put on top of the map we just discussed, each participants' range of emotional arousal.
That can be seen with these blue and red routes that are colored by emotional arousal (blue = low; red = high).
NEXT SLIDE -->
Methodology
Mapping Emotion cont'd
Overall, showing how emotional each person was at each individual point on the map. You cant see very much from that, so whats a way to find out more?
Results
Descriptive Statistics: Road Types
Well, we can statistically average their emotional experiences different kinds of land.
Different road types are shown here.
High emotional levels seen in pedestrian roads and tracks graded as level 4 (rough natural terrain).
Low emotional levels seen in motorway links and residential roads.
Results
Descriptive Statistics: Land Use by Type
As well as each type of land . . .
High emotional levels seen in bicycle parking and wetlands
Low emotional levels seen around christian churches and playgrounds...
BUT... problem with # of sampled points (Christian)
Results
Descriptive Statistics: Land Use by Group
So, its better to group these into larger land use categories. For instance: green, urban, and water.
This helps us get a broader look into what sort of environments can be attributed to cyclists' emotional experiences.
Results
Regressions: Ordinary Least Squares & Spatial Autoregressive Modeling
Comparing coefficient values and standard errors between regression models.
Altitude
0.031
0.001
0.01
0.00
0.028
0.001
Speed
0.009
0.01
0.001
0.002
0.001
0.014
Urban Areas
0.008
0.002
-0.169
0.001
-0.428
0.002
Green Areas
0.081
0.019
0.008
0.016
0.004
0.024
Water
-0.48
0.012
0.032
0.01
0.023
0.025
So here is a bunch of numbers representing the statistical analysis. And what these numbers specifically represent are the statistical relationships between different kinds of land use, namely urban, green, and water. And how emotional people were in those types of land.
In the left column, you see your normal, run-of-the mill statistical analysis (regression). Yet, it doesnt mean a thing. Why? (Ask audience - Ondrej should raise his hand)
Pretty much what Dr. Mitas has said . . .
Because the standard regression doesnt account for how two points in space that are close to each other, fundamentally have more to do with each other than two points that are far away from one another. This is known as spatial autocorrelation.
To better frame this effect, lets think about two coffee drinkers sitting next to each other at a table in a coffee shop. They're both sitting next to the same french press of coffee. So, they must both be huge coffee addicts right??? Well, what if the french press is only for one of them, in fact the other doesnt even drink coffee and tends to prefer tea?
Well, our standard run-of-the mill statistical regression (ordinary least squares) does not account for this spatially assumed correlation between two points.
So we of course need to account for the fact that just because things are close to one another, does not mean they directly affect one another.
I'm sure we've all heard the famous saying of : "CORRELATION DOES NOT EQUAL CAUSATION"
Well, we're interested in the causation in this case, and so we need to account for this effect of spatial autocorrelation.
The Spatial Lag Model is one way of controlling for this, but the Spatial Error model controls for it even better.
We can see in the SEM that cyclists are signifcantly more emotionally excited when they are pedaling up a hilly cycleway.
ALSO, they are more calm or less aroused in urban areas AS COMPARED to green areas or water areas.
The altitude finding makes sense. However, I'm not really sure what the urban areas finding means.
So I'm hoping Paul, if you're in the audience, to enact your urban design expertise and maybe offer up an interpretation or thoughts on why we may be seeing people be more calm in urban areas in this study??
[RE-READ UP ON DIFFERENCES BETWEEN SLM & SEM]
If you haven't already begun thinking this yourself, I wanted to use a visualization to demonstrate just how complex this data we are working with is.
This is a space-time cube, showing the data of one cyclist in this case, but is also possilble to show an average of all cyclists.
At the bottom of the space time cube we have a map, and those two dimnesions (width and depth), they show where a person is in space, and the height dimension shows them moving across time.
And the color of this weird-looking noodle, is showing you their emotional arousal.
If you turn this on its side, it would be diagonal meaning the person was moving at a constant speed, except when we can see these kinks, which show a person stopped, and in this case some of these stops are colored by high emotional experiences.
In this case, a stopped cyclist showing high emotional levels could indicate a cyclist needing to stop and wait to cross a large intersection.
A visualization such as this really helps us in the interpretation of why someone may be experiencing high emotional arousal.
However, its still not the best . . . NEXT SLIDE
Interactive Application Development
Due to these issues,
I made an app which can be used to look at where people are feeling things, and really zoom in on it, so that a manager can go, "Ok, a person felt something here, whats there??"
But, researchers can also use it to discover interesting patterns when exploring newly collected data, to then better direct where they should focus their research efforts.
So, just to quickly demonstrate . . . .
On the left, we can toggle back and forth between these different cyclists, which is an important thing to note due to the variability in skin conductance that can be exhibited across multiple people.
So, starting with Cyclist # 1, and looking at the colored markers displayed on the map (blue = low arousal, red = high arousal), we can see that they were experiencing some high levels of arousal in this area.
We can also coordinate what we're seeing on the map, with the skin conductance chart in the top-right corner.
[HOVER MOUSE OVER CHART TO DISPLAY NUMERICAL DATA]
This chart will graphically display all data that is currently loaded into the map frame. As I zoom in and out, and pan around in the map, it will automatically update (as you can see here). v
Now lets throw a couple more cyclists into the mix to see if others are experiencing similar states of arousal, or if Cyclist 1 is just an overall anxious person.
[CLICK CYCLIST 3]
So here, it appears these two are experiencing similar levels of high physiological arousal. Lets zoom out a bit and add another cyclist to get a better idea of the bigger picture.
[ZOOM OUT]
[CLICK CYCLIST 10]
So this area [MOVE OVER WITH MOUSE], seems to be causing some heightened states of physiological arousal. Seeing this, we of course would like to explore what is present in the environment that might be causing such reactions. To do so, I can first click one one of the markers ...
[CLICK POINT RIGHT BEFORE WATER] ...
to view additional information such as the speed the cyclist was going at that specific point, as well as the current elevation (NL is extremely flat), the total distance travelled to that point, specific skin conductance levels, and which cyclist it is.
But, as you may have noticed, a street view image is also concurrently and automatically displayed in the bottom right corner. With this, it is easier to understand what sort of environment the cyclist is interacting with and experiencing while exhibiting high (or low) levels of emotion.
[ZOOM TO SELECTED POINT] --> [ROTATE MAP VIEW, TILT, etc]
While the application is interesting visually, and is a good way to kill time (and I'm definitely not speaking from personal experience or anything), its main purpose is a research tool, designed to help researchers (THIS IS YOU ALL) better direct what they should focus their research efforts on.
With the way the system's structure is designed, most spatiotemporal data, especially emotion data, can be easily inputted and explored. For example, the analyses just discussed, as well as the visualizations and interactive application presented, can be applied to other interesting, but similar domains such as mapping emotional data collected inside and outside museums.
This brings us to our next project in Nuenen at the Vincent Van Gogh Musuem.
Visitors' location and emotional experiences were again collected while experiencing both inside and outside exhibits at the museum.
Web Mapping
Here is the outdoor musuem data.
It is showing individual peoples' emotional arousal as they experienced the outdoor exhibits at the musuem.
Again, blue represents low emotional arousal, red represents high emotional arousal.
We can also take a grand average of all participants' emotional arousal by location. SHOWN HERE (last selection made in video)
Outdoors: Methodology
Spatial Gridding & Averaging
The grand average I just showed can be done in a more spatial way, where a spatial grid is first overalaid on the map (you can see blended pixels or squares now which represent this grid).
Then, an emotional arousal average is taken for each one of these pixels.
This means all conductance data to have been sampled at any location that falls within one of those pixels, they are averaged together and represented with what is known as a raster (something to display continuous data across a grid)
Outdoors: Methodology
Spatial Gridding & Averaging
Road Map
SpatialLY averaged emotional arousal on a road map
Outdoors: Methodology
Spatial Gridding & Averaging
Road Map
Satellite
SpatialLY averaged emotional arousal with a satellite view
Outdoors: Methodology
Spatial Gridding & Averaging
Road Map
Satellite
Topographical
SpatialLY averaged emotional arousal on a topographical map
Spatial Gridding & Averaging
and the interactive version
Indoors: Methodology
Beacon Location Processing
Generate Beacon Coordinates
Now, lets talk about the indoor portion of the Van Gogh musuem.
If you remember my stroopwafel drawing from earlier, I mentioned that beacons were used to collect peoples' locations during their visit.
In order to use these for visualizations and analyses, I had to figure out where they were placed, specifically their spatial coordinates (latitude and longitude).
Indoors: Methodology
Beacon Location Processing
Generate Beacon Coordinates
Generate Buffers
I then performed a similar spatial technique we discussed for the cycling data, generating buffers around each beacon.
This was done as a way to better associate visitors' collected emotional data, with the beacons that they were closest to at the time they were feeling something....
.... this helps us identify where visitors felt something within the musuem, and at which exhibits.
Indoors: Methodology
Beacon Location Processing
Generate Beacon Coordinates
Generate Buffers
Crop Buffers by Floor Plans
The overall area of these buffers were then corrected (or cut) by accounting for the musuem's floor plans.
Indoors: Methodology
Indoor Beacon Heatmap: Ground Floor
Finally, we can take these created buffers, associate them with the visitors' collected emotional data, and colorize them by the intensity of their emotional experiences throughout the musuem.
Indoors: Methodology
Indoor Beacon Heatmap: First Floor
(SAME FOR FIRST FLOOR)
Taking a look at the floor plan with these beacon heat maps, we can start to identify which exhibits people felt strongly about something, and those they didnt feel strongly about anything at all.
These are the closets (blue circles), and these are the hats (red circles).
Looking at this, for this data, we can conclusively say that closets are not really making people emotional, but that hats are.
Indoors: Methodology
What Exhibits are Visitors Responding to Emotionally?
You can see those two exhibits here.
Interactive Application Development
Wrapping Up
What have we learned?
Mapping emotions is useful
Spatially analyzing and visualizing emotional data is a tough nut to crack
Whats next? Where do we go from here?
So, to go over what we've talked about and hopefully learned today . . .
We've learned why mapping emotional experiences is important, especially in the case of tourism and leisure management.
But that doing so isn't all peaches and stroopwafels.
So, whats next?
Next Steps?
More robust statistcal analyses
^^ along with exploring different spatial methods (viewsheds vs buffers)
Continued development of intuitive visualizations
Further interactive application development
And the last thing I have to say today, which leads us nicely into questions, which is: What would you all like to see?
Cycling: Methodology
GPS Data
. . . . . NOTES HERE . . . . . .
Cycling: Methodology
Skin Conductance
. . . . . NOTES HERE . . . . . .
Cycling: Results
Descriptive Statistics: Road Groups
Nuenen Outdoors: Methodology
GPS Data
. . . . . NOTES HERE . . . . . .
Nuenen Outdoors: Methodology
Skin Conductance
. . . . . NOTES HERE . . . . . .
Mapping the Emotional Dimension:
Measuring Human Behavior Across Space & Time to Inform Tourism & Leisure Management
Garrett C. Millar
Hi everyone, thank you for coming today, and more importantly thank you for your contributions to my last 3 weeks here. My name is Garrett Millar, I'm a PhD student in Geospatial Analytics at North Carolina State University. Geospatial Analytics, also called Spatial Analytics, is the science of WHERE things HAPPEN.
LETS GET STARTED!!
First, I wanted to share my research background and previous research work. I got my Bachelors in Psychology, and then went straight into a PhD in Human-Computer Interaction. During this time, I researched how people use technology, how they experience it, and how they learn from it.
When I transitioned to the Geospatial Analytics program, I took with me this same sort of research ideal, but instead of how people interact with and experience technology, I wanted to find out how people interact with and experience space. With what we'll be discussing today, that space is experienced in the context of tourism and leisure, which I study to help manage those experiences.
Specifically, when people experience tourism and leisure, they feel something, and they feel something in a particular place. And to manage these experiences, you have to know not only that they felt something, but where in their experience they felt it. And SO, I'll be showing you today, where tourism and leisure participants FELT something.