From the 255 United States universities e-mailed, 29 survey responses were collected from at least 17 different universities. In Australia, from the 34 universities e-mailed, 35 responses were collected. 15 of the responses in the Australian survey were from Monash University, the remainder were spread around Australia. The breakdown of the Australian responses by university can be seen in Figure E.2 in the extended results.

Both samples were self selecting.

1.1 Survey population

The sample may reflect a bias from:

• Some characteristic of those departments or heads of department who forwarded the e-mail on to their staff
• Those academic staff members with the inclination and time to fill in the survey (the two may be related as those with a keen interest in the area are more likely to make time)
• As Monash university makes up a large percent of the Australian response, any local bias at Monash may show up in the overall result.

The US e-mail inviting participation and the US survey preamble referred to"Software Engineering" a number of times. This resulted in a response from one Head of Department saying they would forward it on to their software engineering staff. This resulted in a concern about bias in the forwarding process and led to an adaptation of the Australian e-mail and survey so \Software Engineering" did not appear as prominently and the general nature of the audience (within the field of computer science) was stressed.

Prof Waite (Colorado University) replied that "The major problem is that research projects tend to be opportunistic rather than planned" (Waite, 2002a). Prof Schreiner (Rochester Institute of Technology) agreed and added that "the questions do not fit how I work at all" (Schreiner, 2002). Both Prof Waite and Prof Schreiner were speaking with a background of over 20 years in computer science research. Their feedback was seriously considered and changes were made and discussed with them via e-mail. Prior to the release of the Australian survey, both were satisfied that issues they had raised had been addressed.

I received 11 acknowledgement e-mails from Heads of Department in the US willing to forward the survey on. Of those 11 universities, four are represented in the survey. One university took the time to inform me they did not have time to forward on the survey. The Australian survey did not result in any replies. However, judging by the results (almost all of which were received prior to IT Announce being released) the e-mails were received and forwarded on to staff.

The job descriptions of survey participants were divided as expected, with the majority classifying themselves primarily as lecturers. The next largest group were honours students. staff primarily doing Research, PhD candidates and Post-Doctoral researchers were also represented. A breakdown can be seen in Figure E.4 in the extended results.

There was a danger that a disproportionate number of Software Engineering lecturers might add bias to the sample. Those who have taught software engineering made up about 44% of the sample. The breakdown can be seen in Figure E.5 in the extended results.

1.2 The effect of Software Engineering on research outcomes

Click the image for an enlarged view Figure 5.1: Willingness to use more software engineering earlier and significant correlations.

Projects where analysis and design were started later (for example, during the coding stage) were correlated with those run informally, where each team member set their own schedule and delivering goods as they completed them. A late start on analysis and design was correlated with low usage of many software engineering methods. The most significantly correlated software engineering method was UML, with a correlation of -0.571 and significance of 0.01 from a sample of 25. More details in Figure E.6 in the extended results.

The number of higher degree by research students, that is Masters by research and PhD candidates, was correlated with a choice by the survey participant not to use UML or class diagrams. This can be seen in Figure 2 and Table 1.2 below.

Click the image for an enlarged view Figure 5.2: Number of higher degree by research students involved in the project and significant correlations. The red colouring, for example on UML, indicates a negative correlation.

Click the image for an enlarged view Figure 5.3: Average (related) publications per higher degree by research students involved in the project and significant correlations

It is worth noting that the relative satisfaction with the outcome of a project is correlated primarily to the frustration during the project (a large, significant and negative correlation). The size of a project in terms of time and people are also negatively correlated to satisfaction. Satisfaction is positively correlated to funding. These results can all be seen in Figure E.19 in the extended results. The correlations with frustration can be seen in Figure E.8 in the extended results. Despite the negative correlation between satisfaction and frustration, both had large positive correlations with funding.