Survey design: Mobile User Segmentation

 

Challenge

How do we capture basic user information in a non-intrusive way in a mobile app?

We were having difficulty identifying the type of users downloading Intuit Snap Payroll, our 5-star iOS app that helped small business owners calculate simple paychecks. How could we identify whether these users were small business employers, employees, or accountants? Who exactly are the users of Snap Payroll and how did they create paychecks before discovering our app? My team and I embarked on an effort to re-think our survey.

 

Our Process

  1. Using the data we had collected up to that point, we paid special care and sensitivity to user patience in the first-time use of the app by re-writing and re-arranging our questions to elicit the clearest answers with the least user confusion. 
  2. We then put a re-designed flow through a simulated user test we rapidly tested internally with Intuit employees.
  3. We ran a few simulated rapid experiments with targeted groups of small business owners who responded to our online ads.

Old flow

New and improved flow!

Old flow

  • Didn't clearly capture the nuances and characteristics of users
  • Brought confusion to our team in determining who these folks were
  • Our older assumptions of our users were not holding up with time as we collected more user data

Misconceptions our team learned through the old flow included the following:

  • Small business owners could also be employees themselves of other companies
  • Self-employed small business owners have dual roles as an employer and employee of the business
  • Accountants are often small business owners themselves
  • A subset of users were confused about what a W-2 employee was and how it affected their payroll taxes

New flow

With 3 questions maximum in our newly-improved flow, our team was better able to understand our users. The survey:

  • was not a barrier to entry - the value of the app was compelling enough to motivate their usage during these first few questions
  • determined what users were looking for when they downloaded our app
  • gauged users' level of knowledge of managing and hiring their employees
  • identified the employers' processes of how they are paying their workers prior to downloading our app
  • assisted us in grouping our users into categories like: 
  1. accountants
  2. small business owners who have not hired yet
  3. small business owners who have employees
  4. self-employed people
  5. employers who may not be paying their employees correctly

Impact

This flow in combination with our multiple feedback mechanisms we had in place in the app (e.g. feedback email channels) were key points of contact with our users. We also found that users often learned about articulating their own paycheck process to us while going through this process. In turn, our team was able to serve them better while collecting richer and more accurate data about our users.