I have just read a paper from uSamp titled “Are Mobile Insights Better Than Online?” which I have downloaded from http://pages.usamp.com/WP-MobilevsOnline-US300.html. Excellent job from Ben Leet and Edward Appleton focusing on a very relevant topic.
Here I would like to share/discuss a few points which might be useful to improve my understanding – and perhaps that of others:
- On page 2 there is a chart suggesting that (in 2015?) “83% of internet usage [will be] from mobile” That sounds like a quite impressive figure, can anyone support that data? The paper does not specify it but what could it be the source? Are we talking about time units spent connected to internet or about something else?
- On page 5 I read “what was not anticipated was the extent to which mobile methodology experiences higher drop out relative to online.” and “the reason for the high mobile drop-out rates is that mobile respondents are far more impatient.” I wonder, could that have been caused by a substantially different socio-demographic profile between online and mobile users? I understand that “Other factors, such as interview length, locations, and survey experience, were all measured as almost identical across modes, and therefore had no bearing on the drop-off rate” but it is unclear whether socio-demo were also controlled for.
- On page 6 I read “Where were the Lays Cheesy Garlic Bread Chips on the shelf?” followwe by “It is therefore clear that online respondents ‘guessed’ the answer to this question” and I am thinking how much biased must be many of the questionnaires based on recall. Context becomes a must during survey administration and mobile is indeed a very useful instrument in this respect.
- Always on page 6 the authors report quite significantly different answer patterns to the question “From the moment you entered the store, how long did it take you to find the Walkers Prawn Cocktail crisps?” between mobile and online respondents, but again such a difference could be caused by likely differences in the social-demographic profiles of mobile vs online respondents. To further elaborate, my suspicion is that mobile respondents were (on average) probably younger and more dynamic than online respondents, and age is quite likely a determinant of own perception of time-related tasks – that is: the younger one is, the faster one perceive himself/herself to be.
- On page 7 I read “The impact on accuracy of recall is likely negative” and I could not agree more with such a statement. Every minute separating the experience from the recall is filled with thousands if not millions of (unconscious) stimuli our mind elaborates and our memory deals with, so once again context is king.
- Always on page 7 the authors report little differences in sensory evaluation of the crisps by online and mobile respondents. That sounds like good news to me, as it indirectly supports mixed-mode research approaches which – in my opinion – are quite useful and should be further deployed.
- On page 8 I read “With evidence showing that mobile respondents were much closer to the moment of
consumption, it’s very possible that an element of satisficing contained within the online sample” and I wonder if that could also be an effect of different socio-demographic profiles between online and mobile samples. I am guessing that younger, more mobile targets could be less accommodating and satisficing than older, more online respondents.
- On page 9 I read something very revealing to me: “It is a clear sign that the next generation is more comfortable typing a response on a smart phone than on a desktop/laptop keyboard.” It just makes very much sense: users can type on their smartphone wherever they happen to be, and keyboards are at risk to become obsolete (!) especially with younger, on-the-go users. On a side note, I wonder how the likely increased usage of abbreviations – quite often used when typing from smartphones - is going to impact the comprehension and (automatic) decoding/analysis of mobile free text…
- On page 10 I read “Mobile is much closer to the moment and is therefore the obvious methodology for
any in-store research” which I entirely subscribe, and I would like to add that mobile technology such geolocation, foto- & video-cameras can further improve data quality and enrich reporting.
Finally, while I agree with the conclusions of the study on page 11, I would like to see a similar study with an additional control for the socio-demographic profile of respondents as that could even help sort out some of the issues related with timing and costing of mobile market research applications.
To end my commentaries, a big thanks to Ben Leet and Edward Appleton for sharing their research-on-research about this very interesting topic.
Of course, I would welcome a continued discussion on the topic.