Media Studies · Media Studies: Reading, Questioning and Creating Media

Evaluating Effectiveness: Does This Media Text Achieve Its Purpose?

1.3About 70 minutes

Learning goals

I can evaluate how effectively a media text communicates information, ideas, issues and opinions, using criteria and evidence.
I can identify ethos, pathos, logos and kairos and judge how well they are balanced.
I can decide whether a text achieves its intended purpose and suggest how it could be improved and measured.

Words to know

evaluate
To judge the quality or effectiveness of something using criteria and evidence.

criteria
The standards you use to judge something (singular: criterion).

PSA (public service announcement)
A media message made to inform or persuade the public for the public good, such as a safety or health campaign.

ethos
Persuasion through the credibility or trustworthiness of the speaker or source.

pathos
Persuasion through emotion.

logos
Persuasion through facts, logic and reasons.

kairos
Persuasion through good timing and context.

vanity metric
A number such as views or likes that looks impressive but doesn't show whether a text achieved its purpose.

Hook: a million views, zero change?

An invented public service announcement (PSA) about texting while walking goes viral: 1.2 million views and thousands of laughing emojis. But a survey at the school it was made for finds that the number of students crossing the street while texting did not change.

Was the PSA effective? It depends what you mean by effective. This lesson gives you a way to judge.

What does it mean to evaluate?

To interpret a text, you explain what it means. To evaluate it, you judge how well it works, using criteria and evidence.

The key question in expectation 1.3 is: Does the text achieve its intended purpose? To answer, first identify the purpose and audience, then judge the text against criteria like these:

Purpose: Will it actually make the audience think, feel or do what the creator wants?
Audience fit: Are the form, language, style and platform right for the target audience?
Clarity: Is the main message easy to understand quickly?
Evidence and accuracy: Are facts true, current and sourced? Are statistics used fairly?
Emotional impact: Does it make people care, without manipulating them unfairly?
Call to action: Is it clear, realistic and easy to do?
Ethics: Is it honest? Does it respect the people it shows and the audience it addresses?

The rhetorical appeals

Many media texts persuade. Since ancient Greece, people have described three appeals (ways to persuade):

Ethos: the credibility of the speaker or source. A paramedic explains what she sees on winter roads.
Pathos: an appeal to emotion. A slow-motion shot of a phone falling from a hand, with sad piano music.
Logos: an appeal to logic, facts and reasons. A statistic from a named, reliable source.

A fourth idea, kairos, means timing and context: a winter-driving PSA works best in November, not July.

Texts that rely on only one appeal are often weaker. All pathos can feel manipulative; all logos can feel boring and be forgotten.

The rhetorical triangle: ethos (why we trust the speaker), pathos (emotion) and logos (facts and reasons). Kairos means the timing and context, like running a winter-driving PSA in November.

Views are not the same as success

Online, it's tempting to judge texts by numbers: views, likes, shares. Marketers sometimes call these vanity metrics when they look impressive but don't show whether the purpose was achieved.

A better measure links to the purpose:

An ad's purpose is sales, so ask whether people bought the product.
A PSA's purpose is behaviour change, so ask whether people changed what they do.
A news report's purpose is to inform, so ask whether audiences understood accurate information.

You often won't have this data. Then you predict effectiveness using criteria and evidence from the text, and say what data would confirm your judgment.

Worked example

Let's evaluate a PSA

Evaluate this invented 30-second PSA, Eyes Up, made by a student club for students at their high school. A teen walks toward a crosswalk, texting, while upbeat music plays. Cartoon sound effects play as she nearly bumps into a pole, a snowbank and a friend. She looks up just before the street, laughs, and puts the phone away. Text on screen: Eyes up. Phone down. Every crossing. No facts are given. Does it achieve its purpose?

Purpose and audience: to persuade students at the school to stop texting while crossing streets. Audience: students aged 14 to 18.
Strengths: the message is clear and the slogan is short and memorable (Eyes up. Phone down.) The humour and music suit a teen audience, and the setting (snowbanks, a familiar crosswalk) feels real. The call to action is easy to do.
Weakness 1, pathos without stakes: the cartoon effects make near-misses funny, which may tell viewers that distracted walking is harmless. The PSA never shows a real consequence.
Weakness 2, no logos or ethos: there are no facts or credible voices, so viewers have no reason to believe the risk is real.
Kairos: posting it in winter, when snowbanks make crossings harder to see, is a smart choice.
Judgment: it is likely effective at getting attention and making the slogan memorable, but less likely to change behaviour, which is its purpose. This matches the hook: lots of views, little change.
Improvement: keep the humour for the first half, then add a serious turn (a car stopping suddenly), a short fact with a source, and a respected voice, such as a crossing guard. Then measure success by observing crossings before and after.

A full evaluation names the purpose and audience, judges the text against criteria with evidence, gives an overall judgment about whether it achieves its purpose, and suggests improvements. Eyes Up is clear and appealing but probably doesn't achieve its purpose of changing behaviour.

An evaluation profile. The PSA is clear and suits teens, but scores low on evidence and on achieving its purpose (changing behaviour). Scores alone aren't enough: each needs a reason.

Evaluation frame

Use this sentence frame: "[Text] is mostly / partly / not effective at [purpose] for [audience] because [criterion + evidence]. However, [weakness + evidence]. It would be more effective if [improvement]."

Just for fun

The world's shortest movie review: "Nope." It's a judgment, but it fails as an evaluation because it has no criteria and no evidence. Yours should have both.

Hands-on activity

Two-ad showdown

You need: two ads or PSAs on a similar topic (for example two ads for winter tires, two ads for phones, or two public-health messages), paper, a pen. Use TV, radio, print, or online videos with a grown-up's OK.

Identify each text's purpose and target audience.
Draw a scoring chart with the six criteria from the evaluation profile (purpose achieved, audience fit, clarity, evidence and accuracy, call to action, ethics). Score each text from 1 to 5 on each criterion, and write a reason for every score.
Label the ethos, pathos and logos in each text.
Decide which text is more likely to achieve its purpose. Write a paragraph using the evaluation frame.
Explain what data you would need to prove your judgment (sales, sign-ups, a survey, behaviour observation).

What to look for: scores backed by specific evidence, correctly labelled appeals, a clear judgment, and a sensible way to measure success.

Your turn

What is the difference between interpreting and evaluating a media text?

Name three criteria for evaluating a media text.

Define ethos, pathos and logos.

Why might the Eyes Up PSA's cartoon sound effects work against its purpose?

What is a vanity metric? Give an example.

How could the school club measure whether Eyes Up worked?

What does kairos mean? Give a media example.

More practice: warm-up

Which appeal is a statistic from Statistics Canada?

Which appeal is sad music in a charity ad?

Which appeal is a doctor speaking in a health PSA?

What is the first step in evaluating a media text?

Fill in: An evaluation needs criteria and ______.

More practice: on your own

A charity ad uses only heartbreaking images of animals and sad music, with no information about what donations pay for. Evaluate it briefly.

Why is a text that is effective for one audience sometimes ineffective for another?

A video ad for a new game gets 3 million views but the game sells poorly. What might explain this?

Evaluate this slogan for a library's teen program: Enhance your literacy outcomes.

Is it ever ethical for a PSA to use fear? Explain.

Challenge

Choose a real ad or PSA. Write a full evaluation paragraph using the evaluation frame, with at least three criteria.

Redesign Eyes Up so it balances ethos, pathos and logos without losing its appeal to teens. Describe your new version shot by shot (3 to 5 shots).

Argue for or against this claim: A media text is effective if people remember it.

Check your understanding

Quiz

To evaluate a media text means to...

A firefighter speaking in a fire-safety ad mainly adds...

Which is the best measure of a behaviour-change PSA's success?

Why was Eyes Up probably not effective at its purpose?

Running an ad for snow shovels in late November uses...

Which statement is a full evaluation, not just an opinion?

For parents and teachers

What this lesson teaches: expectation 1.3: evaluating how effectively information, ideas, issues and opinions are communicated in media texts, and deciding whether texts achieve their intended purpose. Students use criteria, the rhetorical appeals (ethos, pathos, logos, kairos), and the difference between popularity and purpose.

How to help at home:

After an ad, ask: "Would that make you do what they want? Why or why not?"
Compare two charity or public-health ads and decide which is more convincing, with reasons.
Talk about viral videos: "Lots of views, but did it change anything?"

Common difficulties:

Opinions without criteria ("it was good"). Ask: "Good at what? For whom? What's your evidence?"
Judging by popularity. Views are not the same as achieving a purpose.
Forgetting improvements. A full evaluation suggests how the text could work better.

Facts used: the Eyes Up PSA and its view counts are invented examples. The three appeals come from ancient Greek rhetoric (Aristotle described them).