Stomper Storm Uses Kowalick Arrays
Here is an article discussing the flaws in using Kowalick Arrays for web conversion optimization.
Dr. Kowalich modified some of the Taguchi arrays in order to give more factors for testing without regard to factor interactions. To this end, he used the L12(211) and L18(21 X 37) due to these arrays having the interactions distributed evenly across the array columns. Here is a list of the Kowalich arrays.
| Kowalick | Levels | Notes | ||
|---|---|---|---|---|
| Name | 2 | 3 | 6 | |
| KQ | 11 | 0 | 0 | This is a standard Taguchi L12 array that does not allow for study of interactions. |
| K1 | 1 | 7 | 0 | This is a standard Taguchi L18 array that only allows interactions between columns 1 and 2 |
| K2 | 3 | 6 | 0 | Not an orthogonal array |
| K3 | 5 | 5 | 0 | Not an orthogonal array |
| K4 | 7 | 4 | 0 | Not an orthogonal array |
| K5 | 9 | 3 | 0 | Not an orthogonal array |
| K6 | 11 | 2 | 0 | Not an orthogonal array |
| K7 | 13 | 1 | 0 | Not an orthogonal array |
| K8 | 15 | 0 | 0 | Not an orthogonal array. Is more efficiently done using the L16 |
| K9 | 0 | 6 | 1 | Only proper modification that preserves orthogonal array |
| K10 | 2 | 5 | 1 | Not an orthogonal array |
| K11 | 4 | 4 | 1 | Not an orthogonal array |
| K12 | 6 | 3 | 1 | Not an orthogonal array |
| K13 | 8 | 2 | 1 | Not an orthogonal array |
| K14 | 10 | 1 | 1 | Not an orthogonal array |
| K15 | 12 | 0 | 1 | Not an orthogonal array |
From this list, you can see why I said they result in Taguchi testing sometimes. Any test done with the arrays labeled as “Not an orthogonal array” is not a Taguchi test. In fact, the columns of those arrays are not statistically independent and result in a very small fraction of the test space being covered. Therefore use of those arrays will not result in the optimal test results comparable to full factorial test like a true Taguchi test can do.
Let’s look at how these arrays are created and why they are not orthogonal array. The array Dr. Kowalick used to create his K2 through K15 arrays was the L18(21 X 37). Here is that array.
| Experiment | Columns | |||||||
|---|---|---|---|---|---|---|---|---|
| Number | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 2 | 1 | 1 | 2 | 2 | 2 | 2 | 2 | 2 |
| 3 | 1 | 1 | 3 | 3 | 3 | 3 | 3 | 3 |
| 4 | 1 | 2 | 1 | 1 | 2 | 2 | 3 | 3 |
| 5 | 1 | 2 | 2 | 2 | 3 | 3 | 1 | 1 |
| 6 | 1 | 2 | 3 | 3 | 1 | 1 | 2 | 2 |
| 7 | 1 | 3 | 1 | 2 | 1 | 3 | 2 | 3 |
| 8 | 1 | 3 | 2 | 3 | 2 | 1 | 3 | 1 |
| 9 | 1 | 3 | 3 | 1 | 3 | 2 | 1 | 2 |
| 10 | 2 | 1 | 1 | 3 | 3 | 2 | 2 | 1 |
| 11 | 2 | 1 | 2 | 1 | 1 | 3 | 3 | 2 |
| 12 | 2 | 1 | 3 | 2 | 2 | 1 | 1 | 3 |
| 13 | 2 | 2 | 1 | 2 | 3 | 1 | 3 | 2 |
| 14 | 2 | 2 | 2 | 3 | 1 | 2 | 1 | 3 |
| 15 | 2 | 2 | 3 | 1 | 2 | 3 | 2 | 1 |
| 16 | 2 | 3 | 1 | 3 | 2 | 3 | 1 | 2 |
| 17 | 2 | 3 | 2 | 1 | 3 | 1 | 2 | 3 |
| 18 | 2 | 3 | 3 | 2 | 1 | 2 | 3 | 1 |
The K2 through K8 array use a 3 level column to create two 2 level columns. Below is a table showing how this transformation is done.
| 3 Level | 2 Level | ||
|---|---|---|---|
| Value | Value | Value | |
| 1 | 1 | 1 | |
| 2 | 1 | 2 | |
| 3 | 2 | 1 | |
Applying this transformation to column 2 of the L18(21 X 37) results in the Kowalick K2 array shown below.
| Experiment | Columns | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Number | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 2 | 1 | 1 | 1 | 2 | 2 | 2 | 2 | 2 | 2 |
| 3 | 1 | 1 | 1 | 3 | 3 | 3 | 3 | 3 | 3 |
| 4 | 1 | 1 | 2 | 1 | 1 | 2 | 2 | 3 | 3 |
| 5 | 1 | 1 | 2 | 2 | 2 | 3 | 3 | 1 | 1 |
| 6 | 1 | 1 | 2 | 3 | 3 | 1 | 1 | 2 | 2 |
| 7 | 1 | 2 | 1 | 1 | 2 | 1 | 3 | 2 | 3 |
| 8 | 1 | 2 | 1 | 2 | 3 | 2 | 1 | 3 | 1 |
| 9 | 1 | 2 | 1 | 3 | 1 | 3 | 2 | 1 | 2 |
| 10 | 2 | 1 | 1 | 1 | 3 | 3 | 2 | 2 | 1 |
| 11 | 2 | 1 | 1 | 2 | 1 | 1 | 3 | 3 | 2 |
| 12 | 2 | 1 | 1 | 3 | 2 | 2 | 1 | 1 | 3 |
| 13 | 2 | 1 | 2 | 1 | 2 | 3 | 1 | 3 | 2 |
| 14 | 2 | 1 | 2 | 2 | 3 | 1 | 2 | 1 | 3 |
| 15 | 2 | 1 | 2 | 3 | 1 | 2 | 3 | 2 | 1 |
| 16 | 2 | 2 | 1 | 1 | 3 | 2 | 3 | 1 | 2 |
| 17 | 2 | 2 | 1 | 2 | 1 | 3 | 1 | 2 | 3 |
| 18 | 2 | 2 | 1 | 3 | 2 | 1 | 2 | 3 | 1 |
If you do the dot products as described in Why Use Taguchi Methods? where a level of 1 is assign value of -1, level of 2 is 1 and level of 3 is 0, you will find that the array is not orthogonal. For example the dot product on column 2 and 3 in the K2 array is -6 and not zero as required to be orthogonal. From a different view point, note that the level of 1 is twice as frequent as level 2 in columns 2 and 3. This destroys the balance and makes the columns statistically dependent. It also destroys the even distribution of interactions across all columns of the array. Due to these issues, using the K2 through K8 and K10 through K15 arrays is not Taguchi testing.
It is true that using these arrays can result in improvement of your conversion or what ever action you are testing. However you must remember that finding the optimal result using the Kowalich arrays is just pure chance. To improve the odds, you need to have factors you know are statistically dependent and do not interact. That is a rare circumstance.
Check out this video to see what Stomper Storm could have been if not for Brad Fallon messing up things. This software has been called "...a category killer" and once you see what it does you'll agree.
You can get it at TaguchiSplitTest.com
