Campaign Insights

What is a Campaign?

A Campaign is a particular instance of sending a Script to one or more contacts (using an Import or Audience). In other words, a Script is a template message or conversation that could be used to communicate - but a Campaign is an instance of actually using it to communicate with someone.

The Campaigns page lists all Campaigns that have been sent, or are scheduled to send. This also includes special introductory campaign aggregate reports. For all other Campaigns, clicking into a previously sent Campaign brings up its Insights reporting page.

Across the top, you'll see the name and description of that Campaign. If the Campaign is still accepting responses (that is, its Script has not yet expired) then there will be an Open indicator.

Campaign Report tab

This tab, depicted above, includes information about message delivery and engagement with your Campaign. Note that for a non-interactive (ie, Nudge) Campaign, some sections will be omitted:

Filtering

These results can be filtered by any Audience. This means that the engagement, opt-outs, errors, and responses displayed on the page - as well as the content included in reports - will be filtered to only include recipients that currently meet that Audience's rules. This allows you to see results for a subset of recipients, regardless of what Audience (or import) the Campaign was initially sent to.

Overview

This first section includes the following information:

• Recipients: the Import or Audience that this Campaign was sent to
• Type: Interactive for Scripts with Yes/No, Multi-Choice, or Open-text questions; Nudge for all others (see Scripts)
• Script: name of the Script
• Trigger: the date and time the Campaign was scheduled to send
• Sent: the total number of contacts who Mainstay sent to or attempted to send do
  • Note that this includes contacts Mainstay sent to, but who experienced phone carrier delivery issues.
• Duration: the amount of time before the Script expires
• Responded: for an Interactive Script, this % is calculated as:
  

  • # contacts who responded to 1+ message / # contacts who were sent 1+ message

  • Note that this includes invalid responses and opting out or #pausing during the Campaign.

Note: prior to October 2022, Sent and Responded did not include people who Mainstay sent to but who later experienced delivery failures or opted out. This was corrected for all historical campaigns.

Campaign Engagement

This section indicates which "bucket" the recipients fall into.

• Active: for an Interactive Script, the # of contacts who responded to 1 or more questions
• Passive: the # of contacts who received the Campaign (but did not respond, if Interactive)
• Opt-outs: the # contacts who received the Campaign and then opted out of future messages, with the STOP command
  • This includes contacts who sent in a STOP message up to 48 hours after the Campaign send, as long as they did not receive any other messages in between (such as Live Chat or bot responses).
• Pauses: the # contacts who received the Campaign and then paused future messages, with the #pause command
  • As above, this includes contacts who sent in a #pause message up to 48 hours after the Campaign send, as long as they did not receive any other messages in between (such as Live Chat or bot responses).
• Delivery failures: the # contacts who did not receive 1 or more messages they were meant to receive. (See below for more information.)

A contact may meet multiple categories, but they will only appear in one bucket, based on this flow:

The Sent count above includes every contact who reached the yellow "sending..." box, regardless of where they ended up. Similarly, for an Interactive campaign, the Responded % includes every contact who reached the yellow "sending..." box and then replied at least once.

If there are any Delivery failures, more detail is available by clicking More detail.

There are 4 different delivery failure situations:

1. The contact had previously opted out of or #paused receiving messages.
2. The contact had an invalid phone number, and so messages could not be sent.
3. Mainstay was unable to send the message for some reason, such as a temporary outage.
4. Our SMS provider reported a message failure, such as the phone number being invalid or the message being blocked as spam. Note that an individual may experience multiple message failures.

Clicking Download Report will export a CSV of intended recipients who had 1 or more delivery failures, including their particular delivery failure code and explanation.

Visit Message Delivery Failures & Error Codes for more information.

Survey Results

Finally, the Survey Results section indicates how many contacts responded with each answer for each question, including the raw count and the % of all recipients for that particular question.

Clicking on the Open-text responses will reveal the full responses given for open-text questions:

Clicking on any of these will jump you to that message in the Conversations page.

Download Report

For a non-interactive (Nudge) Campaign, click the Download Report button at the bottom-right.

For an Interactive Campaign, click Export Results to reveal a selection screen:

From this view, check one or more boxes, and then click Download Report at the bottom. This allows you to export a CSV of the selected recipients only.

The download modal will allow you to customize the report name. You can also select which standard  contact fields, custom contact fields, and Campaign fields you want to include in that CSV, and save your selection for future downloads.

Alternately, click download the complete report at the top of this section to export all recipients.

The full list of available fields for the Campaign report:

• standard contact fields & custom contact fields
• Total Messages Sent - # messages attempted to this contact as part of this Campaign
• Total Messages Delivered - # messages successfully received by this contact as part of this Campaign
• Failed Message Count - # attempted messages for this Campaign that this contact did not receive
• Failed Message Codes - see Message Delivery Failures & Error Codes for troubleshooting steps
• Campaign Send Date - date the Campaign was sent to this recipient
• Campaign Send Time - time the Campaign was sent to this recipient
• First Response Date - for an Interactive Campaign, date that this recipient sent in their first response
• First Response Time - for an Interactive Campaign, time that this recipient sent in their first response
• Opted Out During Campaign - the contact sent in STOP or equivalent during the campaign window or within 48 hours (with no other Live Chat or bot messages in between)
• #Paused During Campaign - the contact triggered the #pause command during the campaign window or within 48 hours (with no other Live Chat or bot messages in between)
• Successful Campaign Delivery - true if all attempted messages were successfully received, or else false
• Incomplete Campaign Delivery - true if one or more attempted messages were not successfully delivered, or else false
• Survey Responses - for an Interactive Campaign, all the questions that were asked, and each recipient's response

Saving a spin-off Audience

In addition to downloading the results of a Campaign, you can also select a subset of recipients and create a new Audience.

After selecting 1 or more checkboxes, clicking Save as audience will bring up an Audience naming screen:

Enter a label, which is a text tag that we will attach to all of the contacts based on your checkbox selection. Next, enter an Audience name. Audiences are rule-based filters, so for this, we will automatically build your new Audience so it finds the contacts with your new label.

This Audience will then be available in the Audience Manager. You can also use it to send a follow-up Campaign, or to track this cohort's Conversations or analytics on the Engagement Insights and Knowledge Insights.

Aggregate Report tab

This tab only appears when viewing Recurring Campaigns.

Overview

• If the Recurring Campaign has 1 or more future sends, an info banner will indicate the next scheduled send.
• The Overview section will indicate how often this Recurs, and when it is scheduled to end.
• The Sent and Responded metrics will be aggregated across all instances.
  • For example, if a campaign sent twice, with response rates of 4 / 4 recipients and 8 / 12 recipients, then it would show Sent: 16 and Responded: 75% (12 / 16).

Engagement & Survey Results

• The Engagement graph and numbers, the Delivery Failures pop-up, and the Survey Results (yes/no, multi-choice, and open-text) will similarly aggregate across all sends.
• When downloading the Survey Results or Delivery Failures, you can choose whether to download data across all instances, or only the Last X sends.

History

The Aggregate tab also shows high-level information across all instances. (This corresponds to the information in the Engagement pie chart.) Clicking on one of the timestamps will open that Recurring Campaign instance.

Script tab

This tab shows the Script that was used for this Campaign. Note that this is a read-only view of the Script. To make changes, click View in Scripts.

Recipients tab

Campaigns are sent to a group of contacts. That group is either an Audience - defined by a set of rules which are evaluated at the time of Campaign send - or an Import - a fixed list of records uploaded into the platform.

Audience

If the Campaign was sent using an Audience, this tab will show information about this Audience.

• name - the name of the Audience at the time the Campaign was sent
• View in Audience Manager - a link to view the live Audience, including which contacts currently meet these rules
• Total Contacts - the number of contacts who met the Audience criteria at the time the Campaign was sent
• Opt Outs - the number of contacts who were in the Audience at the time the Campaign was sent, but were already opted out with STOP. This corresponds to the "prior opt-out" delivery failure.
• Pauses - the number of contacts who were in the Audience at the time the Campaign was sent, but were already paused with #pause. This corresponds to the "prior pause" delivery failure.
• Campaigns - the number of Campaigns (including this one) that were sent with this Audience at the time this Campaign went out
• Audience rule description - a read-only snapshot of the Audience criteria at the time the Campaign was sent; note that it may have changed completely since then

Import

If the Campaign was sent to contacts in a previous Import, or a new Import specific to this Campaign, this tab will show information about that Imported CSV.

• name - the name of the import file
• Upload Date - the day the CSV was imported into Mainstay
• User - the name of the platform user who imported the file
• Contacts - the number of contacts in that imported file
• Import History - a link to the imported file in the Import History tool
• Segment Label - the automatically applied contact label specific to this import
• Contact Labels - additional contact labels applied by the user importing these contacts