Testing Racing Results - Design Of Experiment

If you cannot express what you know in numbers, your knowledge is of a meager and unsatisfactory kind. -Lord Kelvin

While I agree with our buddy Lord Kelvin, there are many who would find this comment somewhat trite and lacking any sort of understanding for our brothers who base their inference space on feeling and emotions rather than hard empirical data. Please do not confuse this with gained or experiential knowledge. When it comes to testing, a number is the best way to define gains and losses. This mindset may make the legal and ecumenical community wince, but the cold, hard truth is that we need to make decisions based on numbers and our total empirical data set. If you cannot convert the data you gather into a winning combination, or at the very least an improvement in vehicle performance, you have just been driving the car around in circles.

This may be a good time to briefly talk about data and what constitutes data. Contrary to popular belief, data comes in two forms: attribute and continuous. Attribute data is the data that is presented or defined by a characteristic such as good, bad, pretty, or ugly. Continuous data is presented in a number. It's just that simple.

Racers excel at taking strings of continuous data and presenting them as attribute data. We may characterize a lap or a string of laps as good or bad and simply just ignore any other information that may be buried in the data. The problem with this activity is that we lose mountains of information when we just throw up our hands and dismiss a string of laps as bad instead of mining the data for information we can use. The use of attribute data has a place-beauty contests, flower judging, dog shows, and anywhere else where an attribute better describes the activity. Racing has no place for attribute data unless you are describing the new sponsor package. Even then you will probably be more interested in the continuous data point-the dollar value of the package rather than the cool paint scheme you will now be using. Tuning is not a place for the use of attribute data. There are ways to turn some attribute data into meaningful numbers; we will save that for a later discourse.

Last month, we examined the current "process" in vogue with racers for testing. As we discussed then, the Scientific Method is the standard for experimentation, not just for the racing community but for the scientific community as a whole. We also examined what we would like to be able to accomplish with any new testing methodology. It is my belief that we can accomplish all of our goals by not replacing the Scientific Method but augmenting it with Design of Experiment (DOE). Just as a reminder, these were our demands on a new test methodology:

· The ability to test multiple factors at the same time. · The ability to compress a large experiment into a much smaller experiment. · Show the results in an easily understood graphical format. · No need for expensive computer-based data acquisition systems. We want to use the tools we already have. · We want to use simple tools for data acquisition, such as pencils and pens, paper, a stopwatch, and a calculator. · Our testing method should lead to a better understanding or characterization of the race car, not just knowledge about a specific adjustment or component(s) · We need to gain knowledge about our race car in a very expedient manner. · After testing, we want to be able to predict what an adjustment change or component will do to the performance of the car. We want to predict performance. No more guessing. · We want to use the Y= (f)X equation to drive our performance. · At the conclusion of our test, we will know if a component or adjustment has a positive effect, a negative effect, or no effect on the performance of the race car. · Separate the insignificant many from the significant few. · With the new method, it should cost less to gather more data.

It sounds like a big order. I agree-this is a tall order. Change is never an easy thing to deal with, but in this case, the end result is more than an adequate reward for the pains associated with the change process. Design of Experiment can deliver on all of these requirements. That said, there will be some pain. We will have to change the fundamental methodology or process utilized to tune our race cars. We will have to plan our experiments in much greater detail, and we will have to follow the experimental design and not deviate from it during the execution of the experiment. There will be exceptions, but for the most part, the design will be the design. We will have to change our tuning paradigm. What we need to see is fewer cowboys at the track testing and more research scientists performing experiments.

Last month, we started to go through the process of defining what we already know about the car. This is the first step in the discovery process, because you don't know what you don't know! We were defining our current knowledge and were starting to drill down to defining and asking the right questions about the things we do not know but have a need to know. The use of a Thought Process Map as an organizing tool is the first step.

We need to avoid what I call the "Macho Tuner Attitude." This is the racer who thinks he is above having to follow any known process. This may be characterized by the driver jumping out of the car after a practice session and complaining about a loose condition and the driver and/or the crewchief start barking orders to the crew about the things that "have to be changed" to fix the condition. Prior to trying to defend this type of activity, let's employ some critical thinking to this situation and see if we can see any problems with this type of activity.

· If this was a "have-to-be-changed" component or adjustment, why wasn't it accomplished prior to the past run? · What makes us so sure this will fix the problem? · What data point supports this change? Is it just a feeling (not always a bad thing)? · If we are changing more than one thing, how do we know that the thing or factor we changed was really contributory to "fixing" the condition? What factors were non-contributory? · How do we know we are not going to make the car even worse?

Our goal is to avoid this type of behavior by defining what combination of components and adjustments will yield a car that is as good as it can be as quickly as possible without massive changes once you arrive at the track.

With any new process, there is usually some special vocabulary or phrases that need to be learned.

· Factor: The thing you are going to change is called the factor. If you are working with springs, the springs would be a factor within the design.

· Level setting: This is the position or level at which you will be setting the factor within the design. Within the spring, the level settings may be a 200-pound spring and a 325-pound spring. Level setting within the design is usually delineated by a "+" or a "-" symbol.

· Experimental run: The actual test run.

· Response or dependent variable: This is what you are trying to change or affect in the design. It may be lap times, tire temperatures, shock travel, interval times-what you are going to measure within each experimental run. You are not limited to one response variable within the design. You may be measuring multiple responses within the experiment. This is a real plus.

· Factor effect: The effect an individual factor has on the response variable.

· Interactions: A level of measure between two factors on the response variable. We may see a positive or a negative effect between responses.

· Boldness: How aggressive you are in setting factor levels. You need to be aggressive or bold in establishing factor settings. If the factors are set too narrow, you may not affect the response variable. Please remember that you are dealing with an experiment that has multiple human beings involved. Do not set the level for any factor that may endanger the safety of the person in the car or persons outside of the car. This is not an industrial experiment where the response variable is how long a drill bit stays sharp, where the only thing at risk is a part. Safety first.