"Measuring" Kaizen Event Team Effectiveness: 7 Criteria

Every once and a while someone will ask me to (discretely) evaluate a kaizen event team's effectiveness. I don't necessarily relish doing that when it is intended for the purpose of team comparisons, but it's not an unfair request from a senior leader.

Someday, I should probably try to pull the mystical sensei thing and ask them first what they think...and why.

The criteria that I apply is less than scientific. I don't apply weighting between the criteria and I simply use a 1 to 5 score for each one, with 5 the highest. Really, the important thing is reflecting upon the meaningful stuff, learning and then improving.

My measurement criteria, in no particular order, with links to a handful of relevant prior posts:

  1. Waste elimination effectiveness. The notion here is about how well the team identified, acknowledge and then eliminated the waste within their target process. "W.E.E." is driven as much by team aggressiveness as technical acumen. Lean Metric: Waste Elimination Effectiveness
  2. Projected sustainability. PDCA is one thing and SDCA (standardize-do-check-act) is another. There's nothing as painful as unsustained kaizen gains. They will sap the lifeblood out of a fledgling lean transformation. Gains must be "locked in" with standard work. Lean management systems are needed to drive process adherence and process performance and help facilitate further improvements. Leader Standard Work – Chock that PDCA Wheel
  3. Degree of difficulty. Not much explanation needed here. Scope, technical complexity and change management challenges run the gamut. Some events are easier than others.
  4. Kaizen rigor. Effective teams generally apply rigor around:1) pre-event planning (including linkage to strategy deployment, value stream improvement plans and the like, team selection, appropriate pre-work, etc.), 2) event execution (including event kick-off, team leader meetings, effective work strategies, and the PDCA-driven "kaizen storyline"), and 3) event follow-through. Show Your Work, How to Avoid Kaizen Event Malpractice
  5. Demonstrated application of lean principles, systems and tools. It's a wonderful thing to see the simple elegance of well applied (and validated) standard work...and other lean tools, for that matter. System-level  (or sub-system-level...hey, there's only so much that can be done during an event) application is even more impressive, for example pull systems, lean management systems, etc. Still more "transformative" is something that goes beyond just the "know-how" of tools and systems. Principles encompass not only know-how, but the "know why." Teams that enter that realm are effective during the event...and well beyond. Everyone Is Special, But Lean Principles Are Universal!
  6. Value stream/business impact. Kaizen events are often more about kaikaku than kaizen (small incremental improvement). While we would be mistaken to believe that this is and should always be the case, value stream/business impact should be considered when considering kaizen event team effectiveness.
  7. Learning and development. Kaizen events are excellent and intense laboratories for individual, team and organizational growth. Growth opportunities extend to the technical, teaming, leadership and change management areas and serve as a training ground for daily kaizen. And a final point as we recall Taiichi Ohno's insight that, "Learning comes through difficulties," the lack of gaudy event results does not mean a lack of development! Bridging to Daily Kaizen – 15 (or so) Questions

So, what did I miss?

 

There are 9 Comments

Quixperito.com's picture

Measuring Kaizen Event Team Effectiveness: 7 Criteria...

7 criteria for evaluating kaizen event team effectiveness, including waste elimination effectiveness, business impact, gain sustainability, learning and development, etc....

markrhamel's picture

Hi Sanjib,

Thanks for the comment and the kind words. Be mindful as you develop your evaluation method for kaizens "coming in." The criteria should largely be driven by your value stream improvements plan(s). This is the beginning of system-driven kaizen. As you develop a kaizen culture and apply true daily kaizen, along with system-driven kaizen, you'll begin to enter the realm of principles-driven kaizen. The point here is to kaizen what matters and to develop the organization's problem-solving muscles.

I would try to resist scoring degree of difficulty too formally. The stuff that drives difficulty includes scope, change management challenges, whether the kaizen target has been subjected to a kaizen before or if it is untouched, etc. Kaizen rigor reflects how well/deeply the team applied PDCA throughout the event. There is a certain standard work that should be followed during an event.

Feel free to email me (mark@kaizenfieldbook.com) if you wish to discuss further.

Best regards,
Mark

Andrew Bishop's picture

Mark:

Adding to your list...

Your observation on how "do not know what we do not know" leads me to another (soft) measure: How many new problems did the team process expose?

I would not acutally suggest anybody keep score on this(!), but it highlights some of core of lean - making problems visible - and it certainly corresponds with my experience of good changes. We discover problems that we had no awareness of before. It happens several ways: good changes make problems more apparent; taking away big problems makes us more sensitive to little ones; and we've engaged a group of observers through the team process which means more eyes and insights.

-ALB

markrhamel's picture

Andrew,

Great point - How many new problems did the team process expose? As the team develops their expertise at observation and the scientific method, they should invariably be identifying (and acknowledging) additional "opportunities." If well disciplined and experienced, they will then determine what they SHOULD and CAN attack within their immediate kaizen activity. And you're right, we should not be formally keeping score on this.

Best regards,
Mark

Sanjib's picture

Hi Mark

Thanks for your valuable input.
To start with we are focussing on the following attributes and a scoring matrix.

A1: Innovation & Creativity:
- Doing things differently - 1.00
- Improvement in quality/environment/safety - 0.75
- To reduce cost of production & others - 0.50

A2: Originality
- Concept brought from outside plant - 1.00
- Concept brought from other area/ section of plant - 0.75
- Concept repeated with in the same area - 0.50

A3: Type of Saving
- Recurring - 1.00
- Non-recurring - 0.70
- Not-tangible - 0.50

B. Factor
This factor considers individual/group of the employee. Sphere of Employment Team / Group Individual
B Factor 1/ 1.05

Finally the effectivness would be a function of (A1+A2+A3) X B.

Additionally and referring to your input on how to understand whether and to what extent PDCA has been applied to the kaizen, I was thinking of considering another attribute linked to PDCA by giving weightages on Structured Problem Solving process, Use of Tools, etc.

markrhamel's picture

Hi Sanjib,

I appreciate your diligence. However, I believe that you may be over-thinking things here.

The meaningful "score" is really whether the kaizen activity is going to help you to improve your value stream and processes - making them easier, better, faster and cheaper. The improvements should be PULLED by your value stream improvement plan, strategy deployment imperatives, problems to be solved, engaged and empowered workers, etc. Within the context of the value stream, there are often improvements that may be enablers, without perhaps huge quantifiable benefits. For example, we often first attack the pacemaker process - sometimes just "how" things are scheduled. Furthermore, we regularly do not know what we do not know. With each improvement/attempt at improvement, we learn, we "see" more and we make adjustments.

We should not treat lean transformations and the application of kaizen as a six sigma exercise. We need to make sure that we put substance over form. I am fearful otherwise that we will get into the lean equivalent of, "How many angels can dance on the head of a pin?"

Best regards,
Mark

Andrew Bishop's picture

Mark:

A good go at this! I have to say that I was put off by the heading but after reading I think we see pretty much eye-to-eye. In my experience there are so many intangibles that they usually outweigh "hard" metrics (e.g., R.O.I., productivity improvement, etc.). In the end, it all goes to the bottom line. You just might not see it this quarter or next or be able to make a direct linkage.

I'd add a couple of things:

First, customer impact - how has the customer experience changed as a result of our internal changes?

My second addition to your list may actually be buried in your criteria #7 and #2: the light in the eyes of the team members when they begin to see that the promise is real - they can change their lives at work, they can improve their working conditions, they can do better for customers, and management is there to support them. These are giant steps in development and in sustainability of change.

Thanks for keeping the conversation going!

-ALB

markrhamel's picture

Hello there Mr. Bishop,

As usual, thanks for the insightful comments. Sorry for the off-putting title - a little squeamish myself with it, hence the quotation marks.

Absolutely agree with your additions/distinctions.

Thanks for sharing,
Mark

Sanjib's picture

Dear Mark

Wonderful reading and information on how to evaluate kaizen effectiveness. In my organisation, which is a energy utility, we have just started the Kaizen movement amongst our workforce. To start with, we are trying to develop a evaluation method for kaizens that are coming in. From your article, I was wondering how to develop a scoring matrix by amalagamating two of the Criteria namely, "Degree of Difficulty" & "Kaizen Rigor".
Would appreciate your advice.