Step Tracker Goal-setting Study
Overview
Context
Step trackers keep track of daily step numbers for people. They also include features that are designed to encourage users to be more active. One example is goal-setting, where a user can set a step number that he/she wants to achieve everyday. If successful, the user will be granted some ‘honor’ by the step tracker.
However, most often, the goal is arbitrarily set by the user (or worse, the step tracker itself). Studies have proved that inappropriate goals can not be effective in encouraging the behaviors as intended. We believe that the goals should be set around the user’s actual active level (i.e. average step number).
Goal
To investigate the effectiveness of different levels of ‘difficulty’ in goals in encouraging users’ walking activities.
Questions
Does goal setting improve people’s physical activity level?
How does difficulty of goals affect people’s physical activity level?
Approach
Field experiment with manipulations in goal difficulty among groups; interview of individual users; comparative statistical analysis and qualitative content analysis
Outcome
Too difficult goals will harm people’s activity level, while easier goals are more effective in encouraging activities.
Study method
Study setting
Participant N = 64
Factorial design: 8 groups, 4 goal level × 2 activeness level
Measure: steps each participant walks every day for 2 weeks
Procedure: as shown in the graph below
We asked the users to choose from the two most popular step-tracking apps and use it throughout the study:
Data analysis
Independent two-sample T test
Two-way ANOVA
Findings
No significant difference in step number before and after goal-setting
Active level does affect the step difference in the two weeks
Thus, we believe it is the effect of the study itself - the users already active are more likely to be encouraged by the study and willing to put in more steps
Post-study interview
Some interesting excerpts from our user interview:
Lessons learned
Details on study design, e.g. time of reporting
Value of user input at the end
An interesting incident in our study:
We ended up having to restart the study after collecting all the data after the first week, because we found that the data of the WeChat users are very different from the data of the iOS Health users. The variance of the WeChat users’ data was so big that we did not feel comfortable with using it. So we tried to figure out why.
So we asked those users to provide step tracking data through that week from other sources - and it turned out to be that the WeChat data were not accurate. However, this only happened to those who do not live in the same time zone as us. Then we realized that the WeChat app did consider time zone issue when displaying the data - which screwed up our study (but luckily we realized it!).
After realizing this, we asked those who use WeChat to send their daily step at the end of their day, instead of us reading their daily step numbers from our friend list at the end of our day, which solved the issue.
Feedback
Please feel free to let me know your thoughts about this study and I would be more than happy to chat with you :D