ICE Framework: How (NOT) to Score/Prioritize Growth Experiments

What is the ICE Framework & how does it work?

The ICE Framework is a simple, 3-question prioritization method for growth ideas. It scores ideas between 0-10 for three factors:

• Impact – “If successful, how big will the impact be on our focus?”
• Confidence – “How much evidence do we have that this will work?”
• Ease – “How easy is it to execute (incl. resources, budget & effort)?”

Multiply these scores and the total number will help you to objectively compare and rank different growth opportunities, leading to more effective decision-making and resource allocation.

When NOT to use ICE Method:

If you’re new to the ICE Framework and am considering if it is right for you, here are some reasons when you should or shouldn’t use it:

Pros of ICE Scoring:

• ✅ Simple, quick way to score a lot of ideas
• ✅ More data-driven than having a team discussion/gutfeeling
• ✅ Clear, objective decision making to communicate choices to stakeholders

Cons of ICE Scoring:

• ❌ Not meant for long-term project planning (Consider OKR framework, Growth Mapping or SWOT analyses)
• ❌ May kill big ideas, if hard to make easy (Instead put more weight on ‘Impact’ and less on ‘Ease’, because you can always make a big idea easy to test, and you don’t just want to make little tweaks.
• ❌ Still a bit subjective (“Is it a 6 or an 8?” or “But why is that idea a 5, while the other is a 4? I would swap those!”, so just use it to shift the best ideas from the worst ideas and don’t be too defensive about scoring)

Remember, ICE is a tool, not a rule. Just adapt it to your needs or combine it with other methods for best results. This ICE-framework was invented by Sean Ellis, the pioneer of Growth Hacking, but there are countless other options like RICE, BRASS or PXL that are all just as good: in the end, they all come down to ‘What is the biggest idea, with the most certainty for the least amount of effort?’, because we all want a good ROI 🤷‍♂️

Now, let’s break down this cool (pun intended 🧊) framework and see how it can help you pick your best idea:

Step-by-Step: How to score growth ideas using the ICE Framework?

Alright, growth hackers, it’s time to flex those scoring muscles! Here’s a step-by-step guide to applying the ICE Framework:

• List your ideas: Get all those brilliant (and not-so-brilliant) ideas from the brainstorm and into a list.
• Score Impact (1-10): Ask yourself, “How big of a difference will this make to our key metric at the moment?” Consider things like potential reach and size of the difference.
• Score Confidence (1-10): How sure are you that this will work? Look at available evidence, past experiences, what competitors are doing or industry best practices. (Interesting read on this: The Confidence Meter by Itamar Gilad, but don’t overdo it)
• Score Ease (1-10): How quickly and cheaply can you implement this? Factor in time, resources, and potential obstacles. A good idea should be doable in 2 working days total.
• Calculate the ICE score: Multiply the three scores together (Impact x Confidence x Ease). Tip: Multiplying scores gives a clearer outcome versus adding up the scores.
• Rank your ideas: Sort your list based on the total score; highest to lowest.

Pro tip: To make your scoring more objective, create specific sub-questions for each factor. For instance, when scoring Impact, you might ask: “How many visitors will this affect?” and “How big is the difference versus the original?”. Here’s a growth experiment spreadsheet with sub-question examples.

Here’s a real-world example to illustrate:

In this case, the exit-intent popup wins out, despite having a lower potential impact than the referral program. Its high ease of implementation and confidence boost its overall score.

Remember, the ICE Framework isn’t a science; if in the end you or your team feels better about the second-highest scored idea versus the highest-scored one, then simply go with it. Team enthusiasm and stakeholder management are also important!

Now that we’ve got our ideas scored and ranked, let’s talk about some common pitfalls to avoid. Trust me, there are more nuances than you might expect!

Big mistakes to avoid in ICE Scoring (nuances)

Each factor gets a score from 1-10, and then you multiply them together for a final ICE score. Easy peasy, lemon-squeezy, right? 🍋

Well there are a few nuances in how you approach prioritizing your growth backlog, so here are some tips from my 11 years and counting as a growth consultant…

‘Confidence’ should be ‘Evidence’ to avoid gut feeling

Instead of just asking “How confident are you?” ask, “How much evidence do you have that makes you confident?” This slight tweak makes the whole difference in people scoring their own ideas highly or based on personal preferences, versus just being real with yourself and actually testing things.

Pro tip: If you’re struggling with confidence scores for novel ideas, break them down into smaller, testable hypotheses. This approach can lay out hidden assumptions and help refine your overall confidence score.

‘Impact’ needs context

A score of 8 for Impact might mean different things for different ideas: adding a certain CTA in a single page might have a high impact on that page’s conversion rate, but if your next focus is on increasing overall conversion rate, then other ideas might be better.

So; either group/tag your ideas based on the context/goal/project, or update idea scores down the line.

Overthinking or going for ‘mathematical perfection‘

The goal isn’t to achieve perfect scores, but to create a clear distinction between your best and worst ideas. So don’t get too hung up on whether something’s a 6 or a 7 – it’s about getting a starting point and continuing from there!

Also, different team members might want to give different scores, so letting everyone give their own personal scores might be a better way to not influence others and avoid unnecessary discussions. You could use this ICE Scoring Spreadsheet.

Getting out of balance in your total experiment portfolio

Sometimes you should go for a big swing (an idea that is hard to test small, but might have a huge impact), since that is expected from you as growth team to hit 10x growth, and just going for the ideas that are big-but-only-if-it’s-easy will limit your overall potential. I would always advise to keep an absolute minimum of 20% of your experiments dedicated on big swings, while the other ideas might be safer bets.

Now that we’ve navigated the potential pitfalls, it’s time to talk about how to sell these prioritized ideas to your team and stakeholders. Because even the coolest ideas need a little heat to get moving! 🔥

Communicating ICE Results to stakeholders 😬

You’ve crunched the numbers, you’ve ranked the ideas, and now it’s time to get your team and stakeholders on board. Here’s how to communicate your ICE results like a pro:

• Start with the benefits: Explain the benefits of using a structured framework like ICE for prioritization. (Faster, more reliable than gut feeling, and fits company strategy)
• Show your process: Explain how ICE works, but don’t go in-depth on each score. This builds trust in the process, but keeps the door shut for too much discussion.
• Tell a story: Don’t just present numbers. Weave a narrative around your top ideas and how they align with overall goals – explain the hypothesis behind your main ideas and how those learnings will help you to achieve your goal.
• Invite discussion: Encourage dialog and be open to adjusting scores based on additional insights. Also, open up the conversation with your stakeholders on how risky they want to be to put a weight on your different scores.

Pro tip: When presenting to stakeholders, focus on the top 3-5 ideas rather than the entire list. This keeps the discussion focused and actionable. If you show more, you open up the discussion on whether the scoring was done correctly.

Remember, the goal isn’t just to show off your cool ICE scores – it’s to build consensus and excitement around your growth strategy moving forward.

Conclusion

So, are you ready to put the ICE Framework into action? Here’s your first challenge: Take your top 5 growth ideas and score them using the ICE Framework today. You might be surprised by what rises to the top!

And hey, if you’ve got any questions or doubts, feel free to drop them in the comments below. I’m always happy to help!

FAQs on ICE Framework

It’s best to reassess ICE scores regularly, ideally after every experiment, you update related ideas on the backlog, or at the end of each growth sprint/quarter. This allows you to incorporate new data and insights, ensuring your prioritization remains relevant and effective.

Absolutely! The ICE Framework can be adapted for various growth initiatives, including product development, customer retention, and even internal process improvements. Just adjust the scoring criteria to fit your specific context and goals.

A tie really doesn’t matter: ICE scoring should be used to shift the best ideas from the worst ideas. Then ideally you start with the highest-scored idea, but if you feel better about your second or third highest idea… just go for it!

The ICE Framework can work for teams of any size, from solo entrepreneurs to large growth departments. For larger teams, consider breaking into smaller groups for scoring, then compare and discuss results to reach a consensus.