Pre-proposal presentation

Maps & Minds: Geospatial Analytics for Studying Human Cognition

Garrett C. Millar

Preliminary Oral Examination

November 4th, 2020

Initial thank you's to each member
With that said, lets keep this relaxed with an open discussion, meaning please dont hold your questions until the end! Lets get some interesting conversations going!!
As you all know, I'm extremely interested in how people think and behave. Specifically human cognition and behavior; and I always have been. My PhD research started in Psychology--studying how people interact with technology, so that technology can be improved. Yet, I didn't really care much for the context which people's cognition and behavior we're being studied in. At least, I wasn't all that passionate about it, that is, computer and software applications. I saw bigger problems that could benefit from a deeper understanding and even possible solutions. Problems more related to the world. Geo-spatial problems. The environment, climate change, people's detachment from the natural world, the list goes on.
I basically wanted (want) to understand, from the human perspective, where these problems were coming from, and why. So I started thinking of questions like: How we can learn about the world, how we interact and experience it, and how we can help protect and sustain it. Questions about the world that, if answered, could be used to improve it. The same idea applied in my initial Psychology research, improving technology by studying how we use it.
Well, turns out, you need to know a thing or two about the world to be able to even dream about starting to answer questions like those. And then know even more about the complex spatial stuff that take place on it! But, once I became a full-time Gesopatial Analytics student, and picked up a thing or two, I began discovering more and more exciting, and really interesting ways to start trying to answer my questions about people and the world.
I wanted to tackle these questions, not only from the human perspective, but now with newly-learned, geospatial analytic approaches. And those that know me pretty well, know I have trouble over-involving / over-committing myself to research projects. So I've made an effort to keep this brief for all of our sakes ;D
That said, today I will basically be showing you some different, but equally exciting ways I've developed and have used to began exploring, or navigating if you will, 'The Mind'. And thus, comes my PhD Dissertation Research title: Navigating the Mind: Geospatial Analytic Approaches for Assessing Human Cognition
Discussion of title (working) and its implications ?
- Colon can be taken out (Ross) -- hehe

Helena Mitasova
Committee Chair
Geospatial Analytics

Ross Meentemeyer
Committee Member
Geospatial Analytics

Laura Tateosian
Committee Member
Geospatial Analytics

Aaron Hipp
Committee Member
Geospatial Analytics

Pre-CGA Background & Experience

Designed and ran studies on student learning and intelligent tutoring systems in science domains
Measured student learning using eye-tracking, physiological devices, and facial recognition software (emotions)

MetaTutor IVH Project Designed and tested intelligent multi-agent hypermedia systems for collecting student learning data (e.g., eye-tracking, log-files, facial expressions of emotions) to support STEM learning.

Crystal Island Project Worked with colleagues in Computer Science to design studies examining how students learn about science and literacy from game-based learning environments.

Navigating the Mind: Geospatial Analytic Approaches for
Assessing Human Cognition

Completed Work

Chapter 1 Tangible Landscape: A Hands-on Method for Teaching Terrain Analysis

Other Work Increasing Underrepresented High School Students' STEM Career Awareness and Interest: An Informal Geospatial Science Program

Ongoing Work

Chapter 2 Space-time Analytics of Human Physiology for Urban Planning

Chapter 3 PoPS: A Spatial Decision-Making Support Framework for Plant & Pest Management Scenarios

And now, my working Dissertation Outline
And before we discuss the chapters and what they involve, I wanted to reiterate my three questions posed on the title slide. Those were:
1. How we can learn about the world
2. How we interact and experience it
3. How we can help protect and sustain it
These three questions have formed the Dissertation titles being shown to you now:
1. To start, we consider how we can learn about the world, in this case by using tangible user interfaces in the hopes they can help people learn about spatial concepts. So, we have Dissertation Chapter 1 : Tangible Landscape: A Hands-on Method for Teaching Terrain Analysis)
  - Note that other work not included as a dissertation title still deals with the overaching learning question (GAPS)
2. Then, we move onto How we interact and experience the world, investigated usng mobile sensing technologies in Dissertation Chapter 2: Space-time Analytics of Human Physiology for Urban Planning
3. Lastly, we look into ways we can help protect and sustain the world, specifically the ability to predict and manage the spread of plant disease in Dissertation Chapter 3: PoPS: A Spatial Decision-Making Support Framework for Plant & Pest Management Scenarios
We'll of course be discussing each of these chapters in further detail for the rest of today, but wanted to ensure the connection was made between my overaching questions, and the chapters to-be-in my dissertation.

Chapter 1 Tangible Landscape: A Hands-on Method for Teaching Terrain Analysis

Research Objective: Test the effectiveness of a hands-on method for teaching spatial concepts using Tangible Landscape
- Tested students’ acquisition & transfer of knowledge
- Examined students’ ratings of the system’s usability & user experience

Our first chapter comes from my very first work done here at the center (research with Tangible Landscape), and is basically the research idea that brought me here in the first place: advancing education about spatial concepts though tangible user interfaces (TUIs)
Basically, is it easier, or even simply more fun, to learn about the world and its related spatial concepts if you get your hands a little dirty in the process (hands-on approach) ?
To do this, we used Tangible Landscape to develop 3 tangible teaching lessons to teach the concepts of hydrology (water flow), geomorphology (referenced to as Landforms), and grading (or, Earthwork / Cut & fill).
And in efforts to test the effectiveness of using a TUI to teach spatial concepts (specifically Terrain Analysis), these lessons were used in a pilot study, where graduate Landscape Architecture students were tested before & after each tangible lesson in order to examine any increases in spatial learning.
Ratings of Tangible Landscape's usability and user experience were also collected.

Interaction, Feedback, & Example Solutions

Evaluation Materials

Topographic Map Assessment (TMA) Assessed students’ acquisition & transfer of spatial skills.
Tangible Lesson Assessments: Landforms Measured student’s knowledge specific to content in the landforms lesson
Tangible Lesson Assessments: Cut & Fill Measured student’s knowledge specific to content in the cut and fill lesson

the Topographic Map Assessment,
2 Tangible Lesson Assessments,
and a User Experience Survey.

And I will describe each in detail....
So, we administered the Topographical Map Assessment (developed by Newcombe & colleagues) to measure students' overall acquisition & transfer of spatial skills, given to participants in the 1st session & 2 weeks after the last session
- And what I mean by "Spatial Skills" here is that students are able to understand how elevation is encoded on topographic maps & how 3D terrain shape is represented on maps)
2 Tangible Lesson assessments were also adminstered before and after each tangible lesson (minus Water Flow; TMA had hydrology questions..), which measured student’s spatial knowledge specific to the content being taught in each tangible lesson.
- So, the landforms assessment tested them on geomorphology concepts introduced to them with Tangible Landscape, and the cut & fill assessment was structured the same way, just for eath moving / grading concepts instead.

Results & Contributions

Tangible Landscape supports both improved user experience as well as marginal, task-specific knowledge building
Several implications for the design and implementation of tangible teaching methods for learning about Landscape Architecture (and other topics)
Acknowledgements
- Software design & development: Drs. Anna Petrasova, Vaclav Petras, Payam Tabrizian, and Brendan Harmon
- Study implementation: Carla Delcambre of the Landscape Architecture department and her Grading and Drainage course at North Carolina State University
Reference
- Millar, G. C., Tabrizian, P., Petrasova, A., Petras, V., Harmon, B., Mitasova, H., & Meetenmeyer, R. K. (2018). Increasing Underrepresented High School Students' STEM Career Awareness and Interest: An Informal Geospatial Science Program American Geophysical Union, Fall Meeting 2018, December 12, 2018.

Our findings showed overall, that participants performed significantly better on the Cut & Fill (earth moving) assessment after having completed the analogous task with Tangible Landscape.
And this can potentially be explained by the ability to directly feel, grasp, and manipulate the various tangible materials as well as the projected visual feedback which helped them better understand the effects of their changes.
Also, results from the User Experience Survey highlight how Tangible Landscape allows users to try, see & feel, and directly experience multiple variations of a given solution.
** Specifically, action is reversible with Tangible Landscape, and this encourages users to explore without risk of consequence.
Acknowledgements
I lastly wanted to mention my gratitude for being able to have our paper published in, and presented at the ACM SIGCHI Conference on Human Factors in Computing Systems, the premier international conference for Human-Computer Interaction. Our paper, was even awarded an Honorable Mention Award for CHI 2018, ranking it among the top 5% of all submissions to the conference!!!

Other work Increasing Underrepresented High School Students’ STEM Career Awareness and Interest: An Informal Geospatial Science Program

Research Objective: Develop activities with Tangible Landscape and other related curricula lessons to increase underrepresented high school students' spatial thinking and interest in GIS (STEM)
Lead evaluative procedures for:
- Improving student competence in science
- Nurturing student enthusiasm for science
- Interesting students in research or other science-related careers

I also wanted to briefly go over some of my work that won't necessarily lead to any dissertation chapters, but was the very first project I was on as a student here at the center: GAPS - Geospatial Applications for Problem Solving
And is another way I've explored how we can learn about the world. This time more focused on how high school students can learn about it, and more importantly be excited about doing so!!
So, GAPS is a free Saturday program for high school students interested in using the latest cutting-edge technologies in the geospatial sciences to solve real-world problems, and in learning about all the exciting benefits of a career in the geospatial sciences.
And my main responsibilities included:
1. developing and running activities with Tangible Landscape and other related curricula lessons to increase underrepresented high school students' spatial thinking and interest in GIS (STEM) and...
2. Lead the evaluative efforts on the program as a whole, basically to see whether or not the program was an effective way to get students excited about geospatial science and science in general
3. I did this by focusing on the three overall goals you see on the slide, (which will revisit here in a moment):
  1. Improving student competence in science
  2. Nurturing student enthusiasm for science
  3. Interesting students in research or other science-related careers

Developed Activities

Water Flow		Trail Planning
Landforms		Channeling
Cut & Fill		Ponding

See Developed TL Activity Website: gcmillar.github.io/Tangible-Landscape-Activities-GAPS/index.html

I just wanted to quickly let you get an idea of what these TL activities really consisted of:
1. Water Flow: Students are asked to assume the role of a landscape contractor as they try to find appropriate villa construction sites across a mountainous territory.
2. Trail Planning: Students are assigned to plan a low-intensity hiking route in the Smoky Mountains meant for seniors and children.
3. Landforms: Students use given sand to create the required landforms and label them with the colored markers.
4. Channeling: Students are tasked with reshaping the topography with the overall goal of diverting all the simulated water from the “construction site” to the “stormwater drain” located downhill.
5. Cut & Fill: students must modify the given landscape using cut and fill projection (basic and advanced)
6. Ponding: Students are given a limited amount of sand to build a damn on a stream to impound the maximum volume of water possible.

Results & Contributions

Goal 1: Improving student competence in science

Average 3.39 (out of 4) on being able to explain background material during project presentations

Goal 2: Nurturing student enthusiasm for science

56% of participants indicated an increased interest in learning science

Goal 3: Interesting students in research or other science-related careers

80% of parents stated their child gained skills to use in a STEM career

Acknowledgements

Program facilitation: Drs. Eric Money, Kyle Bunds, Helena Mitasova

Reference

Millar, G. C., Money, E.S., Bunds, K.S., Mitasova, H., & Meetenmeyer, R. K. (2018). Increasing Underrepresented High School Students' STEM Career Awareness and Interest: An Informal Geospatial Science Program American Geophysical Union, Fall Meeting 2018, December 12, 2018.

Chapter 2 Space-time Analytics of Human Physiology for Urban Planning

Research Objective: Using mobile sensing to improve the understanding of how people experience their environment.
How?
- Examining spatial variation in people’s physiological responses in relation to the surrounding environment, to provide urban planners objective metrics on how individuals experience urban design elements.
CHIPS cycle highway project (with Ondrej Mitas, Joost de Kruijff, Lisette Hoeke, Paul de Coevering)
CELTH Storysperience project (with Ondrej Mitas, Moniek Hover, Marcel Bastiaansen, Wilco Boode)

Now, with the research that is admittedly my personal favorite, and not to mention the most geospatal analytic intensive, we can begin discussing how we interact with and experience the world (dissertation chapter 2)
Instead of investigating how people interact with computers, I started to become interested in how people not only interact with the world, but also how they experience it.
So to do this, I started levaraging mobile sensing technologies like smart watches to try and improve our understanding of not only how people behave, but more importantly, how they experience their environment.
This is done with additional hopes that understanding how we experience our cities, can naturally give rise to more informed design and development of our cities, thus resulting in the ability to call our cities "Smart"
That is, cities that are able to operate in a sustainable, efficient and intelligent manner, encompass hopes of more efficient and greener living, increasingly oriented toward people's needs and a resulting betterment of their daily lives.

I was given the opportunity to explore this research with two projects:
1. CHIPS: Cycle Highways Innovation for smarter People Transport and Spatial Planning
2. CELTH: Centre of experience, leisure, tourism, & hospitality
3. And while I wont be discussing this 2nd one, CELTH, I wanted to note how many amazing opportunities it provided me:
  - I was introduced to concepts of indoor mapping . . .

CHIPS: Cycle Highway Project

Test new wayfinding concepts (signage)
Participants: 12
Type of bike: e-bike
Cycle track: 18 km cycle highway
Before and after implementing new wayfinding

Now my research on this topic stems from a much larger project, the CHIPS project which spans Belgium, Germany, the Netherlands and the UK.
And this project particularly involves infrastructure adaptation to increasing e-bike use.
Basically, existing cycling infrastructure is not up to the standards required by heavy e-bike use.
Just think how much faster an e-cyclist is in comparison to your standard manual bike.
Signs (directions) are too small, cycle paths cant properly account for the presence of both regular and e-bike use (e-bikes passing by fast)
And so to begin exploring how these cycle highways might be adapted to account for increasing e-bike use, researchers at Breda University of Applied Sciences ran a small study to collect some data along a cycling track between Tilburg and Waalwijk ... NEXT SLIDE

Study Area & Cycle Track

Data Collection

Credit: Lisette Hoeke | Breda University of Applied Sciences

Briefly, describe / list the data collection efforts and methods here first . . .
Mention stress last, then say the following :
- So while human emotion is a very complex phenomenon, and extremely hard to pin down, there are physiological indicators that our bodies exhibit which can be used to infer whether or not someone is feeling an emotion in the first place. To break it down step by step:
  1. Brain senses emotion
  2. Nervous system triggers sweat glands on hands and feet
  3. Hands and feet become slightly wetter
  4. Water conducts electricity
  5. More current gets passed between two electrodes (conductivity)
- Then, technology like the Empatica E4, a wrist worn wearable specifically designed to measure skin conductance, uses two active electrodes on the bottom of the wrist to measure changes in people’s skin conductivity (or electric currents).

And this is a resulting, 1st person POV video annotated with the collected data.
Since I'm not going into detail about my analyses, I wanted to show this as a prime example for the data I have been working with, and continue to work with for publication in CEUS.
We plan to really only use these videos to verify some of my analytics (if needed)
But, these also leads to a very interesting discussion and questions to consider, ones I wanted to pose to you all:
- We always think about big data as big data where we have a global datset for landcover, or an entire NC dataset at 1m resolution.
- But... here in this study, we have 12 people riding a bicycle, and we have terabytes of data.
- Should this be considered as another kind of geospatial big data??
- Its a very different type of data because its geospatial, but you are also getting this street imagery, which, is very different from landsat
- In any case, the data is availabe and there is a lot of potential to do more work on it.

Chapter 3 PoPS: A Spatial Decision-Making Support Framework for Plant & Pest Management Scenarios

Research Objective: Assess the effectiveness and ability of PoPS framework for facilitating plant & pest management decision-making
Design and development work for:
- effective communication of adaptive management through Tangible Landscape & PoPS Dashboard
Two use cases:
- Sudden Oak Death
- Spotted Lantern Fly (PoPS)

SOD: Upcoming Evaluation Efforts

Pre-workshop	Post-workshop	Semi-structured Interview

We have been developing a new geospatial tool to evaluate how SOD management strategies affect disease spread and impacts across the landscape
The goals for this upcoming Oregon workshop are to demonstrate this decision-making tool and work with stakeholders to generate management scenarios
As part of the workshop, participants will be asked to interact with the decision-making tool and with other participants to develop management scenarios for the region
They will also be asked to complete two written questionnaires about sudden oak death management and modeling, and take part in a follow-up semi-structured interview.
These efforts are being carried out under the hopes they will help us further develop the tool with land manager's and stakeholder's interests in mind.

PoPS: User Interaction Design & Development

Landing Page	Welcome Page	Home Page

So the same work is being done with the second use case: Spotted Lantern Fly (PoPS)
However, I've taken on a bit more development heavy responsibilities
And that is, design and development work for:
- not only the effective communication of adaptive management through Tangible Landscape & PoPS Dashboard, but ....
- I've also recently began extensive design and development work with Shannon (front-end developer) and Chris (project lead, back-end developer) on the PoPS dashboard and website.
- I'm now responsible for a page, which we are calling the home page of the website, that highlights all of the pests/pathogens being modeled by PoPS in a visually interactive manner.
- Basically one that pulls in all Pest Information from the database and displays it in an exciting way.
- And so while I don't have any working mockups of it just yet, I wanted to show you the mockups I've been able to draft up, and will begin the development for in the next coming weeks (after SOD and APHIS workshops coming up in September )
The idea behind my involvements on this project is so that I will be able to forfront a more extensive user-testing experiment in the fall, designed to assess the effectiveness and ability of the PoPS framework for facilitating decision-making during plant & pest management scenarios.

Ongoing Analytic Development: Mapping People Toolbox

People & the Environment: Geo-analytical Guidelines and Software Tools for Measuring Environmental Interaction and Experience

Related Publications

Millar, G. C., Mitas, O., Boode, W., Hoeke, L., de Kruijff, J., & Mitasova, H. (in preparation). Space-time Analytics of Human Physiology for Urban Planning., To be published as a Special Issue of: Computers, Environment and Urban Systems (CEUS). Millar, G. C., Tabrizian, P., Petrasova, A., Petras, V., Harmon, B., Mitasova, H., & Meetenmeyer, R. K. (2018). Tangible landscape: A hands-on method for teaching terrain analysis. In Proceedings of the 2018 chi conference on human factors incomputing systems (pp. 380:1–380:12)., New York, NY, USA: ACM. [Winner of the Honorable Mention for Best Paper Award]

Software & Applications

Geospatial Application Stress3d - Uses cyclists' physiological data to see how stress and emotions can be affected by our environment.
Geospatial Application Emotion Musuem - Uses a 3D model of a museum and geotagged emotion data to explore which exhibits in the Vincent Van Gogh Centre visitors find the most exciting.
Serious Game Learning through Play - Computer game which encourages students to design virtual cities and, in the process, think critically about the relationships between numbers.

Extending Dissertation Work

Dissertation Chapter 1.5 Teaching Terrain Analysis with Tangible Landscape

Follow up study to: Tangible Landscape: A Hands-on Method for Teaching Terrain Analysis
Collaborating with Drs. Brendan Harmon & Nicholas Serrano at Louisiana State University
Comparing different user interfaces and teaching methods to substantiate the most effective way for teaching Landscape Architectural grading and site engineering

Dissertation Chapter 2.5

Short paper on geospatial software application Stress3d: Stress3d: An Interactive & 3D Framework for Exploring Human Behavior
Collaborating with Harvard Medical School to develop visualizations for their Nurses' Health Study

Timeline

Fall 2019	Spring 2020	Summer 2020
Complete Research Proposal Preliminary Written & Oral Exams Submit Chapter 2 to Computers, Environment & Urban Systems Special Issue Dissertation-relevant Travel Oregon: Evaluative work for Sudden Oak Death Management Workshop Boston \| Harvard Medical School: Setting goals and deliverables for ongoing development and visualization work for Nurses' Health Study Baton Rouge \| LSU: Running follow-up study to Chapter 1	Finalize work on Chapter 3 Write and submit publication on the effectiveness of PoPS framework for facilitating decision-making during plant & pest management scenarios. Publication on Tangible Teaching follow-up study? Chapter 1.5 Teaching Terrain Analysis with Tangible Landscape Dissertation-relevant Travel: Netherlands: Continuation of geospatial externship with Experience Measurement Lab (CHIPS & CELTH projects)	Submit written dissertation Presentation & oral defense Graduation! 🎓 👨🏼‍🎓 🍻

Presentation Link: gcmillar.github.io/presentations/pre-proposal

Extra Slides

User Experience Survey

(Ras et al., 2012)

Examined how students perceived and interacted with Tangible Landscape, & how they collaborated to solve a problem
Constructs:

Performance expectancy
Pragmatic quality:

physical objects (wooden carving tools, physical landscape model)
visual objects (projection, digital feedback)

Effort expectancy
User experience

Results

Knowledge Building: Tangible Lessons

Individual Scores

Mean Scores

Here we see the results from comparing students' pre- and post-test scores on the Landforms & Cut-Fill Lesson Assessments.
On the left, is the distribution of participants' individual scores (out of 100%), and on the right is the overall mean scores.
After running paired t-tests for each lesson analysis, no significant differences for the Landforms Assessment were shown,
HOWEVER, the analysis did reveal significant differences for the Cut & Fill Assessment, which means participants scored significantly higher on the Cut & Fill assessment after completing the Cut & Fill tangible lesson.

Cut & Fill: (t(2.73), p = .016).
Pre (M = 53.25, SD = 19.27))
Post (M = 59.97, SD = 16.14))

Results

Knowledge Building: TMA

Individual Scores

Mean Scores

Results

User Experience

All items rated above the neutral value of 4 (out of 7)
Most advantageous aspects of Tangible Landscape?

ability to explore various solutions for the given problems (e.g., water flow, landforms, cut and fill)
physical objects allowed students to change parameters (e.g., location of solution points) very quickly
projected visual feedback helped them better understand the effects of changing those parameters

	Tangible Landscape: A Hands-on Method for Teaching Terrain Analysis. 2018 ACM Conference on Human Factors in Computing Systems (CHI), Montreal, Canada, April 2018		Hands-on Methods for Teaching Landscape Form and Processes. US Regional Association of the International Association for Landscape Ecology (USIALE), Chicago, Il, April 8-12 2018
	Increasing Underrepresented High School Students' STEM Career Awareness and Interest: An Informal Geospatial Science Program American Geophysical Union, Fall Meeting 2018, December 12 2018		Mapping the Emotional Dimension: Measuring Human Behavior Across Space & Time to Inform Tourism & Leisure Management. Visiting Scholar Residency at Experience Measurement Lab; Breda University of Applied Sciences, Breda, Netherlands.

Maps & Minds: Geospatial Analytics for Studying Human Cognition

Garrett C. Millar

Preliminary Oral Examination

November 4th, 2020

Pre-CGA Background & Experience

Navigating the Mind: Geospatial Analytic Approaches for Assessing Human Cognition

Chapter 1 Tangible Landscape: A Hands-on Method for Teaching Terrain Analysis

Interaction, Feedback, & Example Solutions

Evaluation Materials

Results & Contributions

Other work Increasing Underrepresented High School Students’ STEM Career Awareness and Interest: An Informal Geospatial Science Program

Developed Activities

Results & Contributions

Chapter 2 Space-time Analytics of Human Physiology for Urban Planning

CHIPS: Cycle Highway Project

Study Area & Cycle Track

Data Collection

Chapter 3 PoPS: A Spatial Decision-Making Support Framework for Plant & Pest Management Scenarios

SOD: Upcoming Evaluation Efforts

PoPS: User Interaction Design & Development

Ongoing Analytic Development: Mapping People Toolbox

Related Publications

Software & Applications

Related Presentations

Extending Dissertation Work

Timeline

Extra Slides

User Experience Survey (Ras et al., 2012)

Results

Knowledge Building: Tangible Lessons

Results

Knowledge Building: TMA

Results

User Experience

Navigating the Mind: Geospatial Analytic Approaches for
Assessing Human Cognition

User Experience Survey

(Ras et al., 2012)