Product Operations Survey Workflows

1. You are here:
2. Handbook
3. Product Handbook
4. Product Operations
5. Product Operations Surveys
6. Product Operations Survey Workflows

On this page

• Post purchase survey
  • How PPS works
  • Current workflow and owners
    • Automated tasks
    • Manual tasks
    • Deployment
    • Updating PPS data
    • How to update:
• Paid NPS Survey (PNPS)
  • How PNPS works
  • Current workflow and owners
    • Deployment
    • Outreach to PNPS users

Post purchase survey

How PPS works

The post purchase survey (PPS) is launched in Qualtrics via Salesforce. Currently, any time a sales assisted contract is confirmed in Salesforce, Qualtrics is triggered to send out a survey to the customer contact on that subscription. The feature list in Qualtrics is maintained via monthly issues which leverage the feature by tier page. The feature by tier page is auto generated from features.yml. The quarterly scores from the PPS read-outs below flow into features.yml via the process outlined here, which is owned by Product Operations. We are working on flowing web direct and renewal subscriptions into this survey as well in FY22.

Current workflow and owners

Automated tasks

1. SFDC triggers a Qualtrics workflow:
   • Sales-assisted new subscription purchases
   • Qualtrics sends an email with one survey question
   • Survey question is personalized by plan type
   • "What features contributed to your decision to purchase a subscription? You can select up to 5."
2. On the first day of a new quarter, the bot generates an "FYXX QY Post Purchase Survey Analysis" to drive the manual tasks.
3. One month prior to the end of the quarter, the bot generates an "FYXX QY PPS Feature List Update" issue to drive the manual tasks to update the feature list that is shown after the survey question mentioned above.

Manual tasks

1. The manual tasks are described in the Issue templates and are the SSOT for carrying out the PPS
   • PPS Analysis
   • PPS Feature List Update

Deployment

Updating PPS data

It is important we keep pricing related data in features.yml up to date because it affects pricing decisions and auto-generated pages that are referenced across various teams at GitLab, including Product Pricing, Marketing and Sales. Post-purchase survey (PPS) data is taken monthly via Qualtrics. The output should come in the form of a spreadsheet which should look something like this (internal only) and is shared out via the quarterly analysis issues.

In that spreadsheet, the feature column matches to a feature in data/features.yml. That's what needs to get updated. There are two fields which we are updating here:

1. pps_aggregate represents the total of all PPS scores from all time.
2. pps_recent represents the most recent PPS result. If a feature is not listed in the results, the "recent" value should be marked as 0.

Every feature in features.yml should have both entries. If it's a brand-new feature, it hasn't been surveyed in a PPS yet, the values are still zero.

How to update:

1. Open data/features.yml
2. For each row on the "SUMMARY_alltime" sheet, you'll update the pps_aggregate value to match that new value.
3. In features.yml, find-and-replace all the values to zero. In an IDE, you can use a regular expression like pps_recent: \d* and replace it with pps_recent: 0.
4. For each row on the "SUMMARY_Qx" sheet (the one from the current quarter), you'll update the pps_recent value to match that new value.
5. Submit your MR, and merge! To see a sample, check out the diff in the FY22 Q1 update MR here.

Paid NPS Survey (PNPS)

How PNPS works

Current workflow and owners

Deployment

Outreach to PNPS users