· Full Factorial: 24 is the notation for a full factorial. A full factorial would require 16 runs with this notation (24 would be: 2x2x2x2=16)
· Screening or Fractional Design: A method of reducing the number of runs required to accomplish a DOE. In the course of this series of articles, we will be using screening or fractional designs for the most part. The notation for a fractional design may look like this: 24-1, 2x2x2=8. This would be a fractional factorial.
Design of Experiment is a testing process that will allow us to change multiple factors during the course of the experiment. For example, we could change all of the springs on the car and all of the shocks each time the car went out to the track. Post experiment, we can separate the effect of each change and determine the magnitude of the effect the change has on the dependent variable, which in our case may be lap times. The design will further define the response or dependent variable, the factors, level settings for the factors, and number of experimental runs. What else would we know? Depending on the size of the experiment, we would know the effects of the individual factors and the interactions of the factors with other factors, such as shocks and sway bars if they were in the experiment. We could develop a predictive equation based on the design that would allow us to predict the effects of the factors on the response variable.
Let's design an experimentA little history first. Our team has seen the highs and lows so often associated with any racing endeavor. Some days our team has done well, sitting on the pole and leading the most laps, followed by a very high finish. Other days we could barely qualify for the last row and stay on the lead lap. So the performance has been hit or miss. Based on our conversations with the crew, our thought process map, and our goal definition, we have decided on a rather large design with seven factors. The factor selection was based on the factors we change the most on any given race day. The dependent variable for this experiment is going to be lap times. The team decided that qualifying was not a real issue, as they had not missed any shows due to not qualifying. They still felt they were not completely up to speed on qualifying, but they needed to concentrate on race setups. (Remember the matrix the team created that was in the last article defined the adjustments they made on a more frequent basis. The factors were selected from that matrix.)
Our factors are1. RF spring2. LF spring3. RR spring4. LR spring5. LF shock6. RF shock7. RR shock
A full factorial design would have the numeric designation of 27, which would require 128 experimental runs (2x2x2x2x2x2x2=128). Remember, we have yet to define the factor level settings. Just for fun, how long would this design take to accomplish as a full factorial? If we do just a bit of math, under the best of conditions, each run would take 20 minutes (including the time required to change the factor settings). That said, this test would take 46-plus hours to complete. That is just a bit more time than even the most ardent of teams are willing to take to run an experiment. If we fractionalize this test, we can still learn a great deal. We will lose some data, and we may not be able to define the interactions between factors, but we will be able to define the individual factor effects and possibly identify the adjustments that have a much smaller effect on our list of response variables. We could separate the significant few from the insignificant many. This design can be fractionalized to eight runs, 27-4 (2x2x2=8). We lose some data but, using the same 20 minutes per run, we could have this experiment completed in just over 211/42 hours. This was agreeable to the team, so our next step was to further define the level settings of the seven identified factors.
A review of what the team has in their possession, as far as equipment, yielded that they had a good number of springs and shocks. This meant we would be able to run this test without having to buy any more equipment. Springs for the front ranged from 175 pounds to 420 pounds while the rear springs ranged from 200 pounds to 450 pounds. In the shock department, they had a good number of shocks. They could develop two different shock combinations for the three corners of the car they deemed important. In this test, the team was more concerned with the rebound characteristics of the shocks, so that was the direction we went with the design. So we had two different springs for each corner of the car and a total of six shocks, two for each corner, to be tested. The level settings are defined below in a factor matrix we developed for the test.
Next, we'll be defining test order and the structure of the test. We now have the factors and the level settings defined. This would be accomplished by defining the level setting for each factor in each of the eight experimental runs. We will build a matrix that will define the combinations for each factor on each run. This matrix will also serve as the test plan and data collection format for each run. Since this is a very simple DOE, we will only collect one response variable. Each run will consist of multiple laps, but only lap times will be recorded. Next month, in the analysis phase, we will look at ways to discover more information from a simple string of numbers.
DOE test methodology is non-intuitive, especially as the majority of tuners out there will only have experience with the scientific method or O-FAT tuning methodologies. Part of the planning process will include developing some aids that will allow you to not only plan the structure of the test from a setup perspective, but also track the test at the track. We will try to not have a pile of papers that tell you how to run and record the test. The goal will be to have only one sheet of paper to define the setup and the data collection. That will be the test plan matrix. It will include all of the setup instructions and the blanks where the data will be recorded.
The test matrix shows how each of the factors will be set for each run. The "+" or "-" within each cell of the matrix links back to our factor matrix. Let's go through run one together and define the setup. Run #1 will see the car set up "- - + + - - +." Just what does that mean to the guys at the track working the wrenches? Not much! The test matrix does not mean much without the factor matrix.
While the test matrix is a great tool for the design phase of the experiment, it does not help you with the execution of the test. It is very easy to take the test matrix and convert it to a test plan and recording document that will help your crew and those performing the test make sure the correct change is made at the correct time and in the correct sequence.
Let's review run #1. The car will have a 425-pound RF spring, a 425-pound LF spring, a 275-pound RR spring, and a 275-pound LR spring. The LF shock will be set with the rebound in the "D" position, the RF shock set with the rebound in the "D" position, and the RR shock set with the rebound in the "A" position. As you see, it is all spelled out in the test plan matrix.
The process will be to follow the test order outlined on the test plan matrix. Do not deviate, do not change the plan, and do not analyze as you progress through the test. What you will do is make sure that you do not, through the various factor combinations, make the car unsafe to drive. If the driver is reporting the car is behaving in an unsafe way, stop the test. At this point in time, you may have to re-evaluate your level settings. If the car is working safely, you may proceed.
It may be a good time to talk about the driver and his contribution to the test as more than just a throttle jockey. The driver input is invaluable, and the data point is one you do not want to lose. Just as with any other data, a set number is more valuable than a feeling.
We do not need a column that says "good," "bad," and "OK." We want a number. So we will develop a Likert Scale that we can use to debrief the driver while the crew is changing the setup. Try to capture the driver's feeling about the feel and performance of the variable settings. It's not so much as a comparison between each setup but a comparison to what the driver would consider an ideal car.
A 1-5 scale, with 5 being the best, would be a good place to start. Adding more numbers does not necessarily make the scale more accurate. We are looking for discrimination, not resolution; the two are very different. Add this data point next to the lap times.
You will note that this test methodology takes a great deal of planning. If we assigned percentage values to the various components of the DOE process, planning would represent 70 to 80 percent of the total process. The execution of the on-track test is a much smaller component than one would expect.
Next time, we will do the analysis of this DOE and look at some other ways we can get a greater amount of learning out of this data. We will be looking for more response variables within the data set we will have collected. I will give you a template for an easy three-factor, eight-run DOE. The DOE's of this size need not be fractionalized, but they can be done if you are willing to lose some data depth.
There are some points that we need to remember about any test. Safety is our first concern. We do not want to hurt or injure our driver, crew, or any innocent bystanders through any neglect or poor planning on our part. Double check and always think safety.