In This Chapter
▶ Understanding the difference between good data and bad data
▶ Deciding how to collect your data
Managing by Fact
Whether you manage a day-to-day or lead an improvement project, you need accurate data to help you make the right decisions. The following quote summarizes the importance of facts:
"Unless one can obtain facts and accurate data about the workplace, there can be no control or improvement. It is the task of the middle management and managers below them to ensure the accuracy of their data which enables the company to know the true facts".
-Kaoru Ishikawa, What is Total Quality Tool? The Japanese Way-
Realizing the importance of good data
When you undertake an improvement project, you need to analyze the cause of the problem you are tackling - 'Good data helps you quantify and verify those possible causes'. In developing solutions to address root causes, you need good data to help you determine the most effective approach.
Reviewing what you currently measure
Many organizations have data coming out of their ears! Unfortunately, the data isn't always the right data. Sometimes organizations measure things because they can measure them. But those things are unnecessarily right things to be measured and the resulting data doesn't help you manage your business and its processes.
Sometimes data isn't accurate - intentionally or not - and even if the data is accurate, it may be presented in a way that makes interpretation difficult. Managers often present data as a page full of numbers to encourage comparisons with last week's results or even the results for this week last year. This situation is compounded further if the result shows only averages or percentages and you can't understand the range of the performance or the variation in your process performance. This range and variation in performance are what your customer will be experiencing.
The below figure provides an effective format to help you review your measures.
Deciding what to measure
Lean Six Sigma requires you to manage by fact and have good data - but that doesn't mean you need more data than you currently produce. It means you have the right data. So, too is deciding what not to measure.
You need to review the data you currently have and decide whether it really is helping you manage your process. Does the data add value or is it a waste? Who uses the data? How and why is the data used? The CTQs provide the basis for your process measures.
Developing a data collection plan
Your measurement and data will be only as good as the process that collects it. Enough variation is likely to exist in the operational process itself, without compounding the situation by variation in the measurement.
Data collection involves five steps, which begin with determining the output measures for your processes:
- Agree on the objectives and goals linking to the key outputs from your process that seek to meet the CTQs.
- Develop operational definitions and procedures that help ensure everyone is clear about what's being measured and why.
- Agree on ground rules that ensure you collect valid and consistent data.
- Collect the data.
- Carry on collecting the data and identify ways to improve your approach.
Beginning with output measures
In this first step, we begin with the end in mind by considering the output measures. By agreeing on the end goals for data collection, and linking the data to your key outputs, everyone in the team understands why they're measuring what they're measuring. After the output measures have been agreed upon, you need to develop some additional measures to help you understand how the inputs to your process and the various activities in the process are influencing the output results.
Agreeing on goals and outputs is usually straightforward if you've described the CTQs customer requirements in a clearly measurable way. Use our suggested symbols in the above Figure to check whether you have the appropriate set of measures. You need at least one strong measure for each CTQ.
Cycle time is the most important data. If you simply measure whether or not each item meets the service standard, you don't know the range of performance being delivered. For example, you may see the organization processes 80 percent of orders within the service standard of five hours, but you may not able to see that some orders take an hour, some take two or three hours, and the 20 percent that fails to take at least ten hours. With cycle time data you will understand fully what happens.
In this example, we use average cycle time to present 'the voice of the process'. Doing so isn't actually a good idea, as average cycle times can be misleading. The average performance in the above figure is six days, so the process doesn't meet the customer's requirement of five days or less.
But even if the average performance had been five days, the process wouldn't have been good enough. The customer sees every 'cycle time', not just the average.
Creating clear definitions
Describing your measures in a way that removes any ambiguity about what's being measured is the second plan in your data collection plan. This description is called an operational definition.
When you know what to plan to measure, you need to provide clear, unambiguous operational definitions. These operational definitions help everyone in the team to understand the who, what, when, where, and how of the measurement process, which in turn helps you produce consistent data. For example, if you measure cycle time, you define when the clock starts and finishes; which clock you use; whether you measure in seconds, minutes, or hours; and whether you round up or down.
The 1999 launch of NASA's Mars Lander is a famous example of murky definitions. This 125 Million USD rocket was designed to investigate if water had existed on the red planet. Unfortunately, the rocket disappeared, never to be seen again. The cause was rather embarrassing: the team that built the spacecraft and managed its launch worked in inches and feet... but the team responsible for landing the craft on Mars worked in metric - and no one had thought to convert the data. As a result, the angle of entry into Mars was too sharp and the rocket burned up.
Agreeing on rules to ensure valid and consistent data
Having an effective operational definition is important, but you also need to be able to validate the results. Asking yourself if the data look sensible is the third step in the data collection plan.
Measurement System Analysis(MSA) describes the overall approach to ensuring the validity of your measures. Gauge R & R and Attribute Agreement Analysis are the techniques used for assessing continuous and discrete data, respectively. Gauge R & R is a technique for assessing the repeatability and reproducibility of the measuring system. It confirms how much the measurement system contributes to process deviation.
Repeatability is a measure of the variation seen when one operator uses the same system to measure the same thing. So, imagine you've asked someone to measure a batch of items to determine the time it took to process them. You'd then ask her to measure the same batch again to see whether she gets the same results. If she doesn't get the same results, you need to decide whether the difference is important.
Reproducibility is a measure of the variation seen when different operators use the same system to measure the same thing.
To check reproducibility, you ask someone else to measure the same batch of items and see if his results are different from those of the first person. The person doing measuring must not know the previous results. Again, if a difference does exist, you need to decide whether it's important and if action is needed to improve the measurement system.
In the example below, two people - Timekeeper A and Timekeeper B - check the same batch of products in a random sequence. By averaging the difference of the two readings over the number of products in the batch, we can determine the gauge R & R.
In the above figure, gauge R & R is good for a total time 0.42 percent, but is less accurate for the sub-processes. Overall these results are very good, but we could try to improve 'Vet form' if we really have nothing else to do.
The calculation from the table has been made as follows: take the difference between the two 'times', and divide this by the mean average of the two times, expressing the result as a percentage. So, for example, if we look at 'Vet form', the difference between timekeeper A and B is 4 seconds, the mean average is 43 seconds, and the resulting tolerance is 9.30% (4/43 x 100%).
Determining what's good in gauge R & R terms is somewhat subjective and no truly right answer exists. We can offer some broad guidelines but when you decide whether to take action, much depends on the process and the consequences of inaccurate data. We need to think about the severity of the result out of gauge R & R when we decide the action.
Generally, if gauge R & R exceeds 10 percent you should look to improve the measurement system, perhaps focusing on a better operational definition, for example, or using more accurate measuring equipment. If gauge R & R exceeds 25 percent, change the measurement system.
The above figure covers continuous data - that which can be measured on a continuous scale, such as processing time. Attribute data includes whether or not something is present, or is right or wrong, and categories of items, such as types of compensation claim, complaint, and financial standing.
To check the accuracy and consistency of attribute data, we use attribute agreement analysis. So, for example, you ask a number of people in the process team to classify the items in a batch into various categories. You can then compare their assessments both with one another and with an expert's assessments. Doing so ensures consistent classification by the process team and sometimes highlights training needs, too.
In the below figure, you can see how assessors Ann and Brian classify claims consistently between them but aren't in line with the expert's assessment. This finding indicates the need to improve the quality of the training given to the assessors so that their classification is in line with the expert's view.
This is 'a good case' we can take training for improvement action.
Collecting the data
Data collection sheets make the process straightforward and ensure consistency. A data collection sheet can be as simple as a check sheet that you use to record the number of times something occurs.
The checksheet is best completed in time sequence, as shown in the below figure. This real example shows data from the new business of an insurance company processing personal pension applications from individual clients. It captures the main reasons why applications can't be processed immediately; daily recording the number of times these different issues occur. On a daily basis, you can see the number of errors and the number of application forms, and in the below figure we've recorded the proportion of errors to forms. By adding stratification in this way, we may be able to gain some additional insights into the potential causes of the issues.
Looking across the check sheet from left to right, you can see that we've recorded the total errors by type and have determined their percentage in relation to the whole. This check sheet links neatly to a Pareto analysis, which you can find in the below figure. Here, the 80:20 Pareto rule means that generally 80 percent of the errors are caused by 20 percent of the error types. Your analysis won't always result in precisely 80:20 and, in our example, the main causes of the problem, C and E, account for almost 75 percent of the errors.
The Pareto chart in the above figure highlights this fact. The cumulative percentage line helps you decide which errors to focus on. If you tackle type C errors, you'll address 39.4 percent of the problem, but if you also address the type E errors, you'll cover 73.9 percent. You can tackle the smaller errors, A, B and D, later on. That said, check to see whether any of the error types cost more to resolve than others. For example, you might find that type C and E errors are quite cheap to resolve, whereas type A errors prove to be very expensive. Recasting the Pareto diagram by cost may give you a different picture. So, when you look at the Pareto diagram, please take a different view and angle to understand it.
A concentration diagram is another form of data collection sheet. This technique is good for identifying damage to goods in transit, for example, recording where on the product or packaging marks and holes occur. Car hire companies often ask to complete concentration diagrams. You can see the below figure to understand it.
Real Story] An American colleague recently rented a car and, when filling out the form identifying existing damage, inquired about a small indentation. The agent replied 'Buck and a quarter. When we asked what that meant, she explained that if the indentation was larger than a quarter, it was a dent; otherwise, it didn't matter. It wasn't considered a scratch unless it was longer than a one-dollar bill. So here we can see operational definitions in practice, with readily available references for both the customer and the agent. This story also explains the importance of criteria for mutual understanding.
Identifying ways to improve your approach
Even after you find your initial data, carry on collecting more to identify ways in which to improve your approach. You've determined the data showing your output performance. Now identify and measure the upstream variables that influence the output performance of your process. Typical variables include volumes of work, supply accuracy, supplier timelines, available resources and in-process cycle times. Measure these upstream variables daily basis.
The below figure provides a data collection summary. Use it to ensure you've covered all aspects of your data collection plan; doing so should lead you to collecting data that is accurate, consistent, and valid.
'Lean Six Sigma' 카테고리의 다른 글
| #12 데이터 제공하기 - Presenting Your Data (0) | 2021.04.04 |
|---|---|
| #11 샘플링은 어떻게 해야 할까? - Introducing Sampling (0) | 2021.04.01 |
| #9 Takt time, Cycle time, Processing time 구별하기 - Defining the different times (0) | 2021.03.27 |
| #8 VSM(Value Stream Map) 만들기 - Constructing a value stream map (0) | 2021.03.27 |
| #7 업무의 흐름 파악하기 Part 2 - Determining the Chain of Events (0) | 2021.03.27 |
