Chapter Six
Evaluation of the Structured and Object Orientated approaches
6.0 Introduction
This chapter explains the range of experience and expertise of the 6 software engineers who answered the questionnaire, and then uses statistical methods to show the ease of understandability, maintainability and code reuse of both programming languages.
6.1 Background to the tests
The Software Engineers were chosen because they had a varying background of languages, paradigms and experience, this is listed below:
| Name | Paradigm | Language | Experience |
| Chris | Object Based | Visual Basic | 1.5 years |
| Jerry | OO | Php3 | 2 years |
| Steve | Object Based | Delphi | 3 years |
| Gavin | Structured | C | 3.5 years |
| Jon | OO | C++ | 3.5 years |
| David | OO | Java | 3.5 years |
The engineers were given two program listings, which were a subsection of the whole link, one of Java and the other of QuickBasic. They looked at the code and then answered a questionnaire for each language regarding the following questions:
| Easy | Hard | ||||
| Question | 1 | 2 | 3 | 4 | 5 |
| How did you find following logical flow? | |||||
| How would you find maintaining this code? | |||||
| How did you find understanding variable names? | |||||
| How did you find understanding the comments? | |||||
| How did you find understanding what the code does? | |||||
| How would you find reusing the code? |
These questions were the same for both languages to enable a comparison to be made between the two paradigms and languages.
6.2 The Tests
The tests were completed and the data grouped for each language and two sets of weighting were created.
One weighted dataset was created by having all the answers equating to a positive number, as the questionnaire above, which allows easy use of the Wilcoxon Signed-Ratings test. The Wilcoxon Signed-Ratings test is used because there is a two condition related design, this is due to the same questions being applied to both programming languages. Therefore the Wilcoxon Signed-Ratings test is used to evaluate the significance of this data.
The other dataset used a -2 to +2 rating instead of 1 to 5, which enables an easier graphical comparison to take place.
6.3 The Graphical Test
These are the combined results:
Java Results:
| Easy | Hard | ||||
| Question | -2 | -1 | 0 | 1 | 2 |
| How did you find following logical flow? | 1 | 2 | 3 | ||
| How would you find maintaining this code? | 2 | 3 | 1 | ||
| How did you find understanding variable names? | 2 | 3 | 1 | ||
| How did you find understanding the comments? | 5 | 1 | |||
| How did you find understanding what the code does? | 4 | 2 | |||
| How would you find reusing the code? | 1 | 3 | 1 | 1 |
QuickBasic Results:
| Easy | Hard | ||||
| Question | -2 | -1 | 0 | 1 | 2 |
| How did you find following logical flow? | 2 | 4 | |||
| How would you find maintaining this code? | 6 | ||||
| How did you find understanding variable names? | 2 | 4 | |||
| How did you find understanding the comments? | 2 | 4 | |||
| How did you find understanding what the code does? | 1 | 5 | |||
| How would you find reusing the code? | 1 | 5 |
This is the graph of the range of the data with the mean for each question marked with an X.
As can be seen from the graph, the answers for the Java program all are within the easy range with an overall mean of -0.95. This places Java in the middle of the neutral to the easiest range, whereas the answers for the QuickBasic questions are all within the hard range. The mean of the QuickBasic questions is 1.78, which places it to the extreme of neutral to hardest range.
6.4 The Wilcoxon Signed-Ratings test
This uses the positive only weighting:
Java Results:
| Easy | Hard | ||||
| Question | 1 | 2 | 3 | 4 | 5 |
| How did you find following logical flow? | 1 | 2 | 3 | ||
| How would you find maintaining this code? | 2 | 3 | 1 | ||
| How did you find understanding variable names? | 2 | 3 | 1 | ||
| How did you find understanding the comments? | 5 | 1 | |||
| How did you find understanding what the code does? | 4 | 2 | |||
| How would you find reusing the code? | 1 | 3 | 1 | 1 |
QuickBasic Results:
| Easy | Hard | ||||
| Question | 1 | 2 | 3 | 4 | 5 |
| How did you find following logical flow? | 2 | 4 | |||
| How would you find maintaining this code? | 6 | ||||
| How did you find understanding variable names? | 2 | 4 | |||
| How did you find understanding the comments? | 2 | 4 | |||
| How did you find understanding what the code does? | 1 | 5 | |||
| How would you find reusing the code? | 1 | 5 |
The totals for each question will be calculated by multiplying each quantity against the value scored and summing these:
| Question | Java | QuickBasic |
| How did you find following logical flow? | 1 + 4 + 9 = 14 | 8 + 20 = 28 |
| How would you find maintaining this code? | 2 + 6 + 5 = 13 | 30 |
| How did you find understanding variable names? | 2 + 6 + 3 = 11 | 8 + 20 = 28 |
| How did you find understanding the comments? | 5 + 2 = 7 | 8 + 20 = 28 |
| How did you find understanding what the code does? | 8 + 6 = 14 | 4 + 25 = 29 |
| How would you find reusing the code? | 1 + 6 + 3 + 5 = 15 | 4 + 25 = 29 |
These values will be placed into a table and the differences calculated and ranked:
| Question | Condition A (Java) | Condition B (QuickBasic) | d (A - B) | Rank of d | Rank of plus differences | Rank of minus differences |
| How did you find following logical flow? | 14 | 28 | -14 | 1.5(-) | 1.5 | |
| How would you find maintaining this code? | 13 | 30 | -17 | 4.5(-) | 4.5 | |
| How did you find understanding variable names? | 11 | 28 | -17 | 4.5(-) | 4.5 | |
| How did you find understanding the comments | 7 | 28 | -21 | 6(-) | 6 | |
| How did you find understanding what the code does? | 14 | 29 | -15 | 3(-) | 3 | |
| How would you find reusing the code? | 15 | 29 | -14 | 1.5(-) | 1.5 | |
| Total | 74 | 172 | 0 | 21 | ||
| Mean | 12.33 | 28.67 |
A two-tailed test has to be carried out on the smaller of the rank totals because a prediction was not made in one particular direction, so W = 0. The number of questions is 6 so N = 6.
If W is greater than the critical value for N = 6, then there is no significant difference where as if W is less than the critical value for N = 6 then there is a significant difference.
For p < .05 (two-tailed) critical value = 1 and because W = 0 and therefore less than 1 there is a significant difference between the two scores for the two groups of matched subjects.
6.5 Conclusions
The graphical test and the Wilcoxon Signed-Ratings test both show that there is a significant difference between the ease of understandability, maintainability and code reuse of both programming languages.
Java appears to be the better language and thus OO appears to be the better paradigm, these findings are not necessarily correct due to the bias inherent within the code shown to the 6 engineers.
The QuickBasic code was very hard to follow and was very old. This code will have been altered and amended over the 10 years since it was written and therefore may have followed the structured programming paradigm well when it was written.
A fairer test may have been to write the Till_Link software using both Java and QuickBasic, rather than using Source Computing Ltd.'s QuickBasic code and Stephen Wilkinson's Java code in the tests.
Back to Contents
