Agile Education

Agile Marketing Case #1


Agile Marketing Case Content

  1. Agile Marketing Learning Objectives
  2. Applying Agility in Experiments
  3. Hypothesis Testing
  4. Case Data

Agile Significance Case Learning Objectives

The Learning Objectives of this Agile Marketing Education Case have been specifically selected and designed (as Agile Educators we apply and use Bloom’s Taxonomy) to help your Marketing Students:

  • Understand by classifying Statistical Significance results in order to Determine the Validity of a Marketing Activity.
  • Analyse by differentiating Performance levels of a Call to Action based on Statistical Significance Data in order to enhance Decision Making for Marketing.

Prior Learning:

As you apply Agile Marketing Education you will want to ensure you tap into and build from prior learning concepts such as Statistics to provide your Students with the best Marketing Outcomes. Help Marketing Students by Getting them to Think in terms of the Pillars of Empiricism (evidence discovered in experiments and validated measurement).


Provide Marketing Learners with a Quick example of Major Discoveries that occurred as a result of Experimentation.

These Pillars are often present during Experiments:


Challenges in Learning about Statistical Significance for Marketing:

Practicians of Agile Marketing are encouraged to engage in many small experiments over a few large bets (Reminder: this is one of the Agile Marketing Manifesto Values).

Challenges Marketing Learners face is with regards to conducting many small experiments without knowing how to apply Statistical Hypothesis Testing.

This is an area where an Agile Marketing Educator can contribute significantly on the Learner’s Marketing Experimentation Success rate and Insightful Tests by sharing practical theory and reliable Steps to ensure Marketers conduct Valid Experiments.


As stated above, conducting many experiments is highly valued in Agile Marketing but how can you help a Learner continuously Conduct Valid Experiments?

You may use any method you wish to conduct an A/B Test so long as it follows a Process guided by genuine Hypothesis Testing. However, for this Case we suggest the following Mnemonic-inspired Test alternative to ensure Students remember the Key Test Phases as it clearly spells out the word HOME:

  • H – Hypothesis that can be tested based on observable Data
  • O – Outcome that objectively suggests Success or Failure
  • M – Marketing Activity Application
  • E – Ensuring Statistical Significance of Observed Results


During an Agile Stand-Up Meeting colleagues from Consumer Behaviour suggested that Product Attributes such as Fast Delivery are very powerful. The Social Media Department also indicated they received two positive Comments (out of 100) regarding Fast Delivery on Facebook. Analytics and Web Design people in the Company are more than happy to change your Call to Action in order to emphasize Fast Delivery.

Based on the above information, you decide to run an A/B Test using two nearly identical Pages on your company website but with different Call to Action.

How will you know if the results of your Test is Valid and legitimately merit to influence Changes on your Website and your Marketing Content?


H – Hypothesis

We will Test if “Fast Delivery” as a Call to Action will yield better Clickthrough Rates

O – Outcome

We will observe response between Call to Action and consider one as Better only if there is clear evidence from the Data gathered for CTA, CTR as well as Statistical Significance between the two samples.

M – Marketing Application

Create two Pages with Different Call to Action, let generic traffic run for 7 Days

E – Ensuring Statistical Significance

We will use a Statistical Significance Calculator for A/B Testing in order to analyse the two samples


As an Agile Marketer pondering the Results of an A/B Test, you find that the Clickthrough Rate (CTR) for one Call to Action (CTA) is 34% versus another at 33%, should you automatically conclude that the 34% is statistically better and will you Direct Marketing Activities based on that Observation alone?

Results of the Test also revealed that each Page did not get equal Traffic (observed 700 Visits on one page while the other Page received 1000), could there be another Variable that influenced the Performance and can we still Test Significance?

Clickthrough Rate (CTR) for “Fast Delivery” was better by 1%.





CTA Clicks: 325

CTA CTR: 33%

CTA2 (“Fast Delivery”)


CTA Clicks: 235

CTA CTR: 34%

At first glance, CTA2 (“Fast Delivery”) appears to perform 4% better than CTA1 with an Overall CTR of 34%. Since 34% is better than 33%, we therefore should choose CTA2 and redesign our website, right?

How do you determine if the CTA had a real Impact?


Use an A/B Test Calculator Online in order to correctly interpret CTR and improve Decisions based on Statistical Significance.

1) Take a Moment to go Online and Select an A/B Test Calculator

2) Input the Visits and CTA Clicks for both Samples

3) Show the Certainty Calculation Results to the Students directly from the screen

Results of CTA2 VS CTA1 Certainty Calculation: 68%

Consider what it means when Statistical Significance is 68%, how certain are we that CTA2 is better than CTA1? What would you do?

Agile Marketing Education A/B Test Calculator
Source: A/B Test Calculator


As an Agile Marketer do you Test Further or simply make your final Marketing Decision based on the 68% Certainty you observed in the Case above?

Duration: 5 Minutes


Statistical Significance in Marketing can be controversial, some proponents of Opportunity Cost in Marketing (i.e. if Marketing waits for 90% Certainty, it makes us slow therefore we may lose Time and Opportunities) suggest that a Minimum Certainty can be established below 90% perhaps even near 70% is sufficient.

Although it may be tempting to discuss the issue solely with Marketing Management and establish your own Marketing threshold, don’t forget the Client.

The best course of action with regards to Marketing KPI level to use is to ask that Performance Question Differently – always embrace the perspective of the Client or even better, ask the them directly!

Consider the following Agile Marketing Value “Customer focused collaboration over silos and hierarchy”.

A Quick Way, and often effective way, of deciding if a Certainty of 68% is adequate would be to honestly determine if we would be comfortable revealing to a Marketing Client that we could be wrong 32% of the time.


Hypothesis Testing is a simple way to help make Marketing Decisions.

It takes a few seconds to check and confirm Statistical Significance using an A/B Test Calculator Online!

Never forget the Client’s role in guiding your Marketing Decisions.

Are you a Marketing Educator?

Do you have Comments, Advice or Experiences to Share with regards to how you Teach Decision Making in Marketing? Contact Us!

Agile Education


How this Agile Marketing Education Community Started.

Two Agile Marketing Educators Join Forces in 2020

It was early 2020 when Thomas Hormaza and Michael Seaton began planning a way to actively proclaim the Benefits of Agile Marketing Education in Canada – and possibly the World.

They regularly deliver Agile Marketing Courses, Workshops and frequent Instructional Advice as Teachers and Guest Speakers.

With the lofty ambition of uniting Agile Marketing and Agile Education in Canada’s University and College curriculum they set out to develop one of Canada’s first Agile Marketing Master Class for Marketing Instructors.

Get Involved in the Agile Marketing Education Community

Agile Marketing Educators now have a place to call Home.

We invite all Teachers, Instructors and Professors in Marketing at a College or University to reach out to us in order to share Best Practices in the Profession.

We’d love to hear from you should you have an Agile Marketing or an Agile Education Experience to Share with the Community. We will be happy to Feature your Work in our Retrospective Blog Section for Educators.

Professionals in the field of Agile Marketing Training are welcome to contact us but you will be featured in our Corporate Training Section only unless you are a Guest Speaker in College or University.

Agile Education Ethics

As Educators we can have a positive Role in helping Learners realise the importance they play and will play as Marketing Professionals.

Beyond upholding and respecting the Charter of Human Rights through our Marketing Education Careers, the three statements below should be considered as a collective Commitment to Responsible Conduct as we engage in Marketing:

  • Ensure equal representation such as gender and orientation, race, religion, disability in Marketing Campaigns and Creatives
  • Ensure equal opportunity in Marketing Hiring Practices
  • Act responsibly as Guardians of Consumer Data, specifically the concept of Data Dignity through our Marketing Activities

This is by no means an exhaustive list, there are many more points we should discuss and embrace of course!

Since the Marketing Profession does not yet have a commonly accepted Code of Ethics it is imperative to Teach in a manner that further prepares Individuals to be productive and respectful members in Society.

Please write to Us to share Ethics Experiences and Best Practices!

Embracing the Agile Education Manifesto

We invite all Marketing Educators to Read the Manifesto for Teaching and Learning, it stands as a Jewel in the Agile Education Profession.

“We are uncovering better ways of teaching and learning by doing it and helping others do it. Through this work we have come to value:

Adaptability over prescriptive teaching methods

Collaboration over individual accomplishment

Achievement of learning outcomes over student testing and assessment

Student-driven inquiry over classroom lecturing

Demonstration and application over accumulation of information

Continuous improvement over the maintenance of current practices

While we believe there is value in the items on the right, we value the items on the left more.”

Agile Manifesto for Teaching and Learning
The Journal of Effective Teaching, Vol. 17, No.2, 2017, 90-111