https://web.archive.org/web/20090401003425/http://www.cs.mdx.ac.uk/research/PhDArea/saeed/paper1.pdf
This is the draft of a research paper from 2006 that explores a common problem encountered by programming teachers: there appears to be two very distinct groups of people, those who can into programming and those who absolutely cannot.
ADHD zoomers need not worry. This paper only has 21 pages, 2.5 of which are references, and it is rather amusing instead of dry and boring. I will still write some highlights that I think are important for the conclusion I will draw at the end.
Sections 1 and 2 lay some context and previous work related to this issue:
>most people can’t learn to program: between 30% and 60% of every university computer science department’s intake fail the first programming course. Experienced teachers are weary but never oblivious of this fact; bright-eyed beginners who believe that the old ones must have been doing it wrong learn the truth from bitter experience
>A study undertaken by nine institutions in six countries [22] looked at the programming skills of university computer science students at the end of their first year of study. All the participants were shocked by the results, which showed that at the conclusion of their introductory courses many students still did not know how to program.
>In [5] the same authors put forward evidence to support the startling theory that prior knowledge of one programming language has a negative impact on novices’ attempts to program in a second language. Soloway and Spohrer [28] looked at novices’ backgrounds, and opined that skill in natural language seemed to have a great deal of impact on their conceptions and misconceptions of programming.
>Many of their subjects tried to use meaningful names for their variables, apparently hoping that the machine would be able to read and understand those names and so perceive their intentions.
>Van Someren looked at novices learning Prolog [29]. He came to the conclusion that those who were successful had a mechanical understanding of the way that the language implementation – the Prolog virtual machine – worked.
>Mayer expressed the opinion that the formal nature of programming (he called it syntactic knowledge) is the difficult bit [20]. He believed that experts were able to think semantically.
>Thomas Green put forward the notion of cognitive dimensions to characterise programming languages and programming problems [12]. Like Detienne in [11], he rejects the notion that programming languages can be ‘natural’: they are necessarily artificial, formal and meaningless.
On section 3 they present the very simple assignment test they administer on students. It may look trivial, but this was meant to evaluate what sort of mental model the students had of how assignment works. The authors wrote a table listing the models they expected to encounter. Furthermore:
>In a bizarre event which one of the authors insists was planned, and the other maintains was a really stupid idea that just happened to work, the test was first administered to about 30 students on a further-education programming course at Barnet College before they had received any programming teaching whatsoever – that is, in week 0 of the course. Those students had no particular pattern of age, sex and educational background.
>The same test was then administered to about 30 students in the first-year programming course at Middlesex University, once again before they had received any programming teaching. They were mostly male, aged about 18-20, from the middle range of educational attainment and from families towards the lower end of income distribution.
>It seems absurd to administer a test to subjects who can have no idea what the questions mean. Humans are clever animals, though, and most of our subjects seem to have guessed, perhaps from the use of ‘new’ in the heading of the multiple choice column, that the questions were about change rather than mathematical equality, and so they answered using a variety of models.
As their courses progressed, the students were given more elaborate questions, samples of which you can see in the PDF.
On section 4 they talk about their results:
>We could hardly expect that students would choose the Java model of assignment (model 2 in table 1), but it rapidly became clear that despite their various choices of model, in the first administration they divided into three distinct groups with no overlap at all:
> 44% used the same model for all, or almost all, of the questions. We call this the consistent group.
> 39% used different models for different questions. We call this the inconsistent group.
> The remaining 8% refused to answer all or almost all of the questions. We call this the blank group.
The consistent group didn't necessarily have the correct model of how the computer works, but their answers showed that they were using only one mental model rather than assuming the computer would somehow function differently based on context.
>Table 2 shows that there is very little movement from consistent to inconsistent between the two test administrations: the two people who did switch appear not to have attended the course at all, and may not have taken the second administration seriously. There is some movement – almost half – from inconsistent or blank to consistent. Everyone became either consistent or inconsistent: there were no blank returns in the second test.
>Table 3 shows the numbers of passes and fails in the end-of-course examination for the consistent group in the first test and the rest. Despite the stark difference in success between the groups, there are only two data points per column. In the rest of the discussion we shall therefore consider only the in-course exam results.
The table shows that those who already had consistent models before being taught were very likely to pass the course (21 passes vs. 8 fails) while the others were very likely to fail (8 passes vs 26 fails).
>Figure 10 compares the three populations. The switchers (grey) are a little bit farther to the right – that is, get slightly higher marks – than the non-switchers (white), but they are clearly distinguishable from the consistent group (black). The only noticeable difference between figures 9 and 10 is that the non-switchers form the left half of the 30-49 hump and the switchers form the right half, a considerably underwhelming result.
That is to say, people who had to be trained to be consistent thinkers aren't as good as those who already were consistent thinkers from the start.
On section 5 they try to make sense of the results:
>It has taken us some time to dare to believe in our own results. It now seems to us, although we are aware that at this point we do not have sufficient data, and so it must remain a speculation, that what distinguishes the three groups in the first test is their different attitudes to meaninglessness.
>Formal logical proofs, and therefore programs – formal logical proofs that particular computations are possible, expressed in a formal system called a programming language – are utterly meaningless. To write a computer program you have to come to terms with this, to accept that whatever you might want the program to mean, the machine will blindly follow its meaningless rules and come to some meaningless conclusion. In the test the consistent group showed a pre-acceptance of this fact: they are capable of seeing mathematical calculation problems in terms of rules, and can follow those rules wheresoever they may lead. The inconsistent group, on the other hand, looks for meaning where it is not. The blank group knows that it is looking at meaninglessness, and refuses to deal with it.
>Figure 13 suggests that it is extremely difficult to teach programming to the inconsistent and blank groups. It might be possible to teach them, if we concentrated on trying to persuade them to see a programming language as a system of rules (though the evidence in section 6 below seems to suggest otherwise).
On section 6 he talks about the similarities between teaching programming and teaching formal logic. Despite natural deduction being much simpler to define than any bloated PL spec, the results of their tests show a similar double-hump.
Now let me share are my thoughts on all of this.
The paper suggests the existence of 2 main groups. It doesn't call them as such, but I believe it is referring to shape rotators and wordcels. I already had this suspicion, but the way they phrase it on section 5 makes it clear to me. Wordcels' inability to think in abstract terms makes it difficult for them to comprehend that things like the problem's context or a variable's name hold no special meaning from the point of view of a computer. The only thing it sees are objects whose properties are defined by the relationships they have with other objects. Merely giving the name "mark" to a variable won't prevent it from having a negative value, despite the fact that intuitively you can't get a mark lower than 0 in a test.
There is evidence that some wordcels can "learn" some degree of programming competency. I assume this is due to acquiring new jargon that they can wordcel with, rather than learning abstract conceptualization - and some of them can't seem to get to that point at all and can never hope to program:
>The second-test inconsistent group, on the other hand, can be almost entirely written off.
That is to say, hardcore wordcels who can't stop wordcelling in human language - even after being shown that this is not how computation works - are probably entirely incapable of acquiring new types of symbols to think with.
Last edited by sturgeon