Every outreach campaign produces a row of figures: how many emails arrived, how many bounced, how many people replied and how many meetings came out of it. Each figure only means something next to its base, the number it is divided by. This exercise practises the formulas, the calculations and the judgement of which figures to trust. All companies and figures are invented.
The metrics used here
Teams do not all use the same bases. This exercise uses the convention below; whatever convention you use, write the base next to the figure.
Metric | Formula | What it shows |
|---|---|---|
Delivery rate | Delivered emails ÷ sent emails | How many emails reached a mail server that accepted them |
Bounce rate | Bounced emails ÷ sent emails | How many could not be delivered (hard and soft) |
Open rate | Unique opens ÷ delivered emails | How many were counted as opened (unreliable, see below) |
Reply rate | Replies ÷ delivered emails | How many people wrote back, whatever they said |
Positive reply rate | Positive replies ÷ delivered emails | How many showed interest |
Meeting rate | Meetings booked ÷ delivered emails | How many led to the actual goal |
Unsubscribe rate | Unsubscribes ÷ delivered emails | How many asked not to be contacted again |
Complaint rate | Spam complaints ÷ delivered emails | How many marked the email as spam |
A rate is the result times 100, in percent. "Unique" opens means each recipient counts once, however often they opened the email. Only human replies count as replies: out-of-office messages and bounce messages do not.
Why open rates are unreliable
An open is counted when a tiny invisible image in the email, a tracking pixel, is loaded from the sender's server. That method has two big errors.
Too many opens. In 2021 Apple introduced Mail Privacy Protection (with iOS 15 and macOS Monterey). For users who turn it on, the Mail app loads remote images, including tracking pixels, in the background, whether or not the person ever reads the email. Many such emails are counted as opened.
Too few opens. Some mail apps and companies block images by default. A person can read an email, even reply to it, without the pixel ever loading.
So an open rate can rise while nobody reads more, and replies can come from emails that were never counted as opened. Replies, meetings, bounces and complaints are counted by what really happened, which makes them much more useful.
The tasks
Teacher notes
How to run it (30 minutes). Go through the table of metrics together (5 minutes) and ask: why is delivered the base for most rates? Then 20 minutes for the four tasks, with calculators. Use the last 5 minutes for the quiz questions about opens, because they change how learners read any marketing report.
Worked solutions.
Fabrikam: delivered 1,250 − 50 = 1,200. Delivery rate 1,200 ÷ 1,250 = 96 percent. Bounce rate 50 ÷ 1,250 = 4 percent; hard bounces 30 ÷ 1,250 = 2.4 percent. Open rate 540 ÷ 1,200 = 45 percent. Reply rate 60 ÷ 1,200 = 5 percent. Positive reply rate 18 ÷ 1,200 = 1.5 percent; 18 ÷ 60 = 30 percent of replies. Meeting rate 9 ÷ 1,200 = 0.75 percent. Complaint rate 3 ÷ 1,200 = 0.25 percent.
Northwind Traders: delivered 640 − 22 = 618. Delivery rate 618 ÷ 640 = 96.5625, so 96.6 percent. Bounce rate 22 ÷ 640 = 3.4375, so 3.4 percent. Open rate 401 ÷ 618 = 64.89, so 64.9 percent. Reply rate 29 ÷ 618 = 4.69, so 4.7 percent; on sent emails 29 ÷ 640 = 4.53, so 4.5 percent. Positive reply rate 11 ÷ 618 = 1.78, so 1.8 percent. Meeting rate 5 ÷ 618 = 0.81, so 0.8 percent. Complaint rate 2 ÷ 618 = 0.32, so 0.3 percent.
Common mistakes.
Using sent emails as the base for every rate. The difference looks small (4.7 against 4.5 percent) but it grows with the bounce rate.
Rounding too early: rounding the delivered count or an intermediate rate changes the last decimal.
Reading a 0.3 percent complaint rate as "tiny". Google asks bulk senders to stay below 0.1 percent and never to reach 0.3 percent, so Northwind Traders has a real problem even though the figure looks small.
Celebrating a high open rate. Without replies it proves very little.
Follow-up questions.
Which single metric would you report to a sales manager whose goal is meetings, and why?
Northwind's open rate is 64.9 percent and its reply rate 4.7 percent. What could explain the gap?
Continue with the outreach campaign dataset and calculate the same metrics from 238 rows of raw data.