One of the main reasons why projects fail is not defining clear project success criteria. A project can only be successful if the success is defined. And this ideally upfront. Unfortunately, we are still seeing many projects that skip this part completely.

When starting a project, it’s essential to work actively with the organisation that owns the project to define success criteria across three levels:

1. Project delivery

2. Product or service

3. Business

Project Delivery Success

Project delivery success is about defining the criteria by which the process of delivering the project is successful.

Essentially this addresses the classic triangle “scope, time, budget”. It is limited to the duration of the project and success can be measured as soon as the project is officially completed (with intermediary measures being taken of course as part of project control processes). 

Product or Service Success

Product or service success is about defining the criteria by which the product or service delivered is deemed successful.

For example, the system is used by all users in scope, system reliability is 99.99%, user satisfaction has increased by 25%, operational costs have decreased by 15%, and so on.

These criteria need to be measured once the product/service is implemented and over a defined period of time. This means it cannot  be measured immediately at the end of the project itself.

Business Success

Business success is about defining the criteria by which the product or service delivered brings value to the overall organisation, and how it contributes financially and/or strategically to the business.

For example, financial value contribution (increased turnover, profit, etc.) or competitive advantage (market share won, technology advantage).

Overall Project Success

These levels combined will determine your overall project success. You can be successful on one level but not others. This sounds all good in theory, but in practice, it is not so easy to define the criteria for levels 2 and 3. This is one of the main reasons why so many organisations only look at level 1: scope, time and budget. They are easy to define and  a bit easier to measure.

Personally, I think level 1 matters very little if levels 2 and 3 are not met. So in spite of the “project management triangle”, the fact is that delivering a project on time, in scope and in budget is not enough. The project must be delivered successfully – meaning that the objective(s) or the “Real Jobs To Be Done” that motivated the project in the first place have to be reached.

This is where “Objectives and Key Results,” or OKRs come into the game.

What are OKRs?

OKR is an abbreviation for Objective and Key Result. The concept was invented at the Intel Corporation and is widely used amongst the biggest technology companies in the world.

OKRs are originally meant to set strategy and goals over a specified amount of time for an organisation and teams. At the end of a work period, your OKRs provide a reference to evaluate how well you did in executing your objectives.

You can use the same concept for defining project success. Spending a concerted effort in identifying your project strategy and goals, and laying it out in a digestible way with OKRs can truly help your project team and stakeholders see how they are contributing to the big picture and align with other teams.

Objectives

Any project has one or more objectives. The goal of setting an objective is to write out what you hope to accomplish such that at a later time you can easily tell if you have reached, or have a clear path to reaching, that objective. Choosing the right objectives is one of the hardest things to do and requires a great deal of thinking and courage to do well.

Key Results

Assuming your Objectives are well thought through, Key Results are the secret sauce to using OKRs. Key Results are numerically-based expressions of success or progress towards an Objective. All Key Results have to be quantitative and measurable. 

“If it does not have a number, it is not a Key Result.”                                   

The important element here is measuring success. It’s not good enough to make broad statements about improvement (that are subjectively evaluated). We need to know how well we are succeeding. Qualitative goals tend to under-represent our capabilities because the solution tends to be the lowest common denominator.

For example, if I create a goal to “launch new training for the trading team” I might do that for one trading member. If I alternatively make a Key Result of “train 50 trading team members” and only train 10, I’ve still 10x-ed my original goal.

Don’t Turn OKRs Into a Project Task List

Do you measure effort or results? Are your OKRs focused on your objective or on the means to get there? As I mentioned before, when used correctly, OKRs define success criteria for a project. OKRs should determine whether a project achieved success. But to do that, OKRs cannot be based on activities for three main reasons:

1) We want a results-focused culture, and not one focused on tasks.

2) If you did all your tasks and nothing improved, that is not a success. Success is improving something: users are more satisfied, efficiency has improved, costs have been reduced.

3) Your project is just a series of hypotheses. An idea is just a non-validated hypothesis. In the same way, we don’t know if our project will improve our results or add value to the organisation. The project is just a hypothesis so you cannot attach your OKRs to a non-validated bet. 

Nobody works on projects as a hobby. Behind every project is a desire to improve one or more metrics. So, instead of just tracking the delivery of a project, we should measure the indicators that motivated it in the first place.

Use Value-based Key Results

There are two basic types of Key Results:

1) Activity-based Key Results: These measure the completion of tasks and activities or the delivery of project milestones or deliverables. Some examples of Activity-based Key Results are:

– Release a beta version of the product

– Launch a new feature

– Create a new training program

– Write a solution design document.

Activity-based Key Results usually start with verbs such as launch, create, develop, deliver, build, make, implement, define, release, test, prepare and plan.

2) Value-based Key Results: These measure the delivery of value to the organisation. Value-based Key Results measure the outcomes of successful activities. Some examples of value-based Key Results:

– Reduce Average Cost per Trade from X to Y.

– Increase System Usability Score from from X to Y.

– Maintain Customer Acquisition cost under Y.

– Reduce Source-to-deliver cycle time (the time from sourcing raw materials to delivery of finished goods) from X to Y.

The typical structure of a Value-based Key Result is:

Increase/Reduce Metric M from X to Y                                                     

Where X is the baseline (where we begin) and Y is the target (what we want to achieve).

Using the “from X to Y” model is better than writing a change in percentages because it conveys more information. Compare the two options below:

1) Increase the number of new users by 20%.

2) Increase the number of new users from 400 to 480.

Option 1 can be confusing since it’s hard to tell how ambitious the target is. Are we talking about increasing the number of new users from 50 to 60 or 400 to 480?

When project teams start with Value-based OKRs, it is common for them to get stuck listing activities as Key Results. To convert those activities into value, think about what would be the consequences of being successful with this task. What would be the desired outcomes?

Example 

If we successfully migrate to the new CTRM platform, we will:

– Reduce infrastructure costs from X to Y.

– Maintain availability during and after migration in 99,99%.

– Maintain revenue of $ X.

OKRs as a Communication Tool

Effective OKRs are widely shared and meant to be understood by project teams, related teams, and stakeholders. In that regard, they can serve as a communication tool for directing teams to solve complex challenges with constraints.

As a communication tool, OKRs bring two key things to an organisation:

1) Easily digestible direction such that every project member/stakeholder in the organisation understands how they contribute to the mission; aka focus.

2) Expectations amongst teams and their individual members; aka accountability.

Final Thoughts

Defining measurable results becomes easier as you learn what you should be measuring and what ultimately matters for your project and business. In my work with large ERP and CTRM project teams, I find that the quality of OKRs has a good correlation with the understanding of the project. Just going through the exercise of either defining OKRs, or reworking current project plans into OKRs is a highly effective evaluation tool.

How can we help?

Talk to us about how we can help your organisation focus on using “Objectives and Key Results” to define your CTRM Project Success Criteria. Making it real is what we do.

How can you connect with us?

We’d love to hear about your journey. Did reflecting on OKRs help?

About the Author

Jason Novobranec is Implementary’s Chief Operating Officer.

With over 20 years of Consulting, Program Management & Senior Leadership experience, Jason has delivered initiatives for large multi-national / multi-regional organisations as well as SME’s and is an expert in shaping solutions to fit a customer’s project needs.