Jamaican teachers and deductive logic
E.P. Brandon
Caribbean Journal of Education, 19, 113-130 (1997) - written 1993
Levels of performance in typical Jamaican schools constitute a chronic problem. Although one can point to cases of excellent achievement throughout the system, from all-age schools to traditional high schools, the dominant impression is of poor performance on almost all standard indicators: 'O' level or CXC performance at one end, levels of literacy at the other. While overcrowding and underfunding are equally obvious, one major contributory factor is the achievement level of typical teachers. Teachers who believe themselves to be using standard English are oblivious of the non-standard forms they use. Many teachers on the recent upgrading programme are still unable to pass an examination based on the content of the primary school mathematics curriculum. No doubt, analogous inadequacies with respect to the content of the curriculum could be uncovered in subjects thought less central than English and Mathematics. This paper, however, looks at an area of cognitive skill that is paradoxically central to most subjects but universally ignored in their teaching and in the preparation of teachers to teach them (see Brandon [1]): elementary deductive logic.
Before proceeding it is worth stressing that it is not only in Jamaica that deductive logic is effectively ignored by the school system. Unfortunately no data are known to me on the performance of teachers outside the Caribbean on deductive reasoning tests, but the data that exist for school children and unspecified adults show very clearly the damaging effects of pedagogical neglect. Nowhere is ability as high as one presumes it is for simple addition and subtraction; everywhere people are generally taken in by elementary forms of invalid argument. So any judgements of competence expressed in this paper must be understood in a global context of poor performance.
Deductive logic studies those situations, typically but not only in a context of argument, in which a given set of statements (possibly empty), the premises, necessitates another set, the conclusion, that is to say, if the premises were true then the conclusion would have to be true too, or putting it another way, it is impossible that the premises are true and the conclusion false. An argument in which this happens (or more generally, the inference in any such case) is labelled valid. Very many such situations depend only on the structure of the statements involved and not upon their semantic content, so that deductive logic can be studied as a matter of form: the argument every A is a B, x is an A, so x is a B is deductively valid, no matter what one is talking about (humans, mortality and Socrates in the stock example, or equally well whales, mammals, and the whale stranded on your local beach). Just as one can identify valid forms of argument so one can pick out argument structures that do not fit the criterion (in which the premises could all be true while the conclusion is false), although they might seem to be valid. Such structures are deductively invalid arguments; many of what are called fallacies exemplify such deductively invalid principles of reasoning.
Putting it metaphorically but, one hopes, illuminatingly, deductive logic is a matter of the routes that exist or do not exist between distinct items of information. Given some beliefs or suggestions, some other claims will be entailed; they will follow from what we have already got. But other claims will not be thus tied up with our given starting points; we will have to go beyond what we are given if we want to assert them. What we have may constitute evidence for such claims, but it does not guarantee them in the way that deductive conclusions are guaranteed by what they follow from. (It is perhaps worth reiterating that this guarantee is crucially conditional: the conclusion cannot be false provided that the premises are all true.)
Some deductive relations are elementary and, for many people, intuitively obvious. Perhaps this is one reason we typically pay no explicit attention to them in school. But unfortunately, what we often think is intuitively obvious turns out on examination to be, not merely not obviously true, but simply false. We are, in other words, prone to accept fallacious inferences. We think we can get from here to there by a logically guaranteed route, when there is in fact no such connection.
To illustrate: if we are given that all female teachers have been trained and that all the teachers at the local primary school are female, then we do not need to do anything more to be able to claim that all the teachers at the local primary school have been trained. What we are given entails that extra item. What we are given may turn out to be false, so our conclusion may also turn out to be false; the guarantee we have is conditional: if our initial claims are true then our conclusion must be true too. Such a guarantee clearly does not extend to the actual truth of our starting points, but it is the best we can do. It is enough to underwrite the move we made. And it is enough to allow us, if we discover that our conclusion is false, to conclude that somewhere among our starting points there must be at least one more falsehood. But if instead we had been told merely that most female teachers have been trained, everything else remaining as before, then we would not have been in a position to move to our earlier conclusion. If we had, we would have inferred invalidly; and the move would have been equally invalid if we were lucky and found that all the teachers at the local primary school had in fact been trained. (Of course, we might perhaps risk a non-deductive inference to that conclusion; the point is that it would be a gamble, unlike the guaranteed move we began with.)1
A grasp of deductive relations is then a grasp of where you can and where you cannot go, just using the information to hand. It is crucial to most problem-solving, though that is not to say that is the only thing necessary for solving problems. It is often required to evaluate explanations and justifications, though once again it is not the only thing required. Indeed, it is needed to give an explanation in the first place even when the links are left implicit most explanations involve placing something to be explained in a web of other things, and these connections are very often deductive in nature. In brief, deductive relations constitute the scaffolding for much of our ordinary and all our more sophisticated theorizing about the world.
Given this rootedness in our cognitive endeavours, it may perhaps be worth while to discover how well Jamaican teachers grasp such matters. The data to be presented come from a short test of deductive relations that has formed part of one of the three papers taken as an entrance examination by most applicants for the UWI, Mona, B.Ed. and Cert. Ed. programmes, including those offered on UWIDITE to Jamaica and much of the Eastern Caribbean. (The main sources of data in this paper are the examinations of 1990 to 1992, though occasional reference will be made to results for earlier years.) This is obviously not an ideal way to investigate our topic, but it has the advantage of providing a large number of non-graduate practising teachers. Most of these teachers are employed in the primary/all-age sector with some from new secondary schools. The data therefore cannot tell us about performance in the high schools or in the small number of expensive preparatory schools; but it might not be unreasonable to suppose that, as far as the average young Jamaican child goes, we are getting close to his or her typical teacher. Caution is, however, required since applicants are in fact skewed towards males and almost certainly towards those living near Kingston; one might also presume that applicants are distinguished from the rest by a greater drive for achievement or dissatisfaction with their current position. Whether these putative differences would affect their general cognitive functioning, I do not know; but at all events my guess is that they would not diminish it, so that the results obtained ought not to be seriously underestimating the deductive logical competencies with which the bulk of Jamaican school children have to interact.
Applicants for Mona programmes come also from the rest of the English-speaking Caribbean, but in far smaller numbers and in proportions that differ from Jamaican applicants (presumably reflecting in part opportunities available at the other campuses of the UWI). Such applicants have similar educational qualifications and occupy similar situations in their school systems to the Jamaican candidates, but they should not be taken to be representative of their territories, not even in the somewhat dubious way in which I have argued that the Jamaicans we are focussing upon can be. They will, however, be used to give an idea of the relative position of the Jamaican applicants. Table 1 summarises the geographical origins, sex, and programme orientations of the candidates from 1990 to 1992.
Table 1: Applicants, by programme, sex, and origin, 1990-92
| 1990 | 1991 | 1992 | ||||||||||
| N | 537 | 478 | 445 | |||||||||
| B.Ed. | Cert. Ed. | B.Ed. | Cert. Ed. | B.Ed. | Cert. Ed. | |||||||
| N | 341 | 196 | 321 | 157 | 313 | 132 | ||||||
| M | F | M | F | M | F | M | F | M | F | M | F | |
| N | 81 | 260 | 33 | 163 | 78 | 243 | 24 | 133 | 66 | 247 | 37 | 95 |
| Jamaica | 62 | 222 | 13 | 82 | 59 | 204 | 9 | 64 | 50 | 183 | 8 | 28 |
| Non-Jamaica | 19 | 38 | 20 | 81 | 19 | 39 | 15 | 69 | 16 | 64 | 29 | 67 |
The investigation described here is part of a broader survey, inspired by work undertaken by Robert Ennis and others in the USA in the 1960s (Ennis and Paulus [7]). In Jamaica, the first investigation was of conditional reasoning among comprehensive school students, using one of Ennis' tests (Nolan [11], Nolan and Brandon [12]). Subsequently, the main source of data on deductive reasoning has been a part of a paper in the Faculty of Education's entrance examination which has used the same format to investigate a number of forms of reasoning among applicants from Jamaica and the rest of the English-speaking Caribbean (Brandon [4] reports these data for the period from 1985 to 1989 but without separating out the Jamaican subjects). The examination paper consists of three parts. The first is a test of 'verbal' ability, though the tests used since 1986 involve a grasp of relations between pairs of items, and so are more than tests of vocabulary. The second part was until 1988 a test of 'spatial ability,' since 1989 a test of everyday and elementary mathematics. The third part is devoted to logical reasoning. It consists of 36 questions, six assigned to each of six formally different patterns or principles of inference. The format remains the same as Ennis used: suppose you know that .... then would it be true that ...? Three possible answers are offered: Yes, glossed as 'it must be true, given what you are told'; No, 'it can't be true, given what you are told'; and Maybe, 'it may be true or it may be false, you haven't been told enough to be certain whether it is Yes or No'. In 1991 the prompt 'Maybe' was changed to 'Not Necessarily', with the same gloss. A likely consequence of this change will be noted below.
In Ennis' tests, and in these investigations, respondents are held to have mastered a principle if they get at least five of the six questions using that pattern right; they are considered on the borderline if they get exactly four of them right. A consequence of this approach is that a person may be considered to have mastered a valid principle, say, even though he or she seems not to have grasped how it differs from a similar but invalid one. As has been noted elsewhere (Brandon [2]), a stricter view would drastically cut the number of persons who could be regarded as having mastered any logical principles at all.2
Logical principles can be grouped in various ways. The first and crucial criterion is validity or invalidity. The other main way, when they are comparatively simple, is by reference to their major structural components. Thus one can distinguish various principles focussing on conditional statements (such as all the principles used in Nolan's pioneering work in Jamaica) or quantifiers (words for how many, all, some, most, etc.) or disjunctive statements, and so on.
An interesting and unusual feature of the investigations reported here is that principles involving the plurative or pleonetetic (Geach's terms [8]) quantifier most have often been used. This quantifier is not normally studied in elementary formal logic, even though its meaning is perhaps closer to what we often intend in using plurals or the universal quantifier all than how that universal quantifier is itself construed in formal logic (cf. Hodges [9], p. 196).
An important finding of empirical research on logical reasoning is that people do not appear to respond directly to the structures that formal logic would use to characterise an inference.3 It is, of course, a controversial matter how this finding should be understood theoretically (cf. Norris [13]) but for the present it helps to explain the often wide range of facility indices for items that belong to the same formal pattern, and prompts a qualification of the earlier claim that people do worse on invalid patterns. Some invalid arguments are so obviously bad that few people will be taken in. In earlier years, the logic test included some sets of items whose logical status depended much more on their content (numerical or relational) than their form, and with these items it was often found that candidates did better on the invalid items than on the valid ones.
To estimate how well a group performs, one important result from these tests is the percentage mastery of each of the logical principles. Taking valid patterns first, Tables 2-4 give the main findings for each principle tested, first for the Jamaican and then for the non-Jamaican subjects: percentage of candidates who have mastered the principle; are on the borderline; have not mastered the principle; facility indices for the easiest and most difficult item testing the principle; and average facility for the items. (The Appendix gives the actual items that were easiest and most difficult in 1990 so that readers may get a better idea of the kinds of questions used.) These findings indicate the comparative position of Jamaican non-graduate teachers with respect to the rest of the English-speaking Caribbean and also reveal something about the intrinsic nature of the principles themselves.
Table 2: DISSYL
| Jamaican | Non-Jamaican | |||||||||||
| Year | %Master | %Border | %Fail | Facility | %Master | %Border | %Fail | Facility | ||||
| High | Low | Av. | High | Low | Av. | |||||||
| 1990 | 74 | 10 | 16 | 0.91 | 0.71 | 0.83 | 84 | 6 | 10 | 0.96 | 0.78 | 0.88 |
| 1991 | 76 | 12 | 12 | 0.92 | 0.75 | 0.84 | 83 | 5 | 12 | 0.95 | 0.76 | 0.88 |
| 1992 | 78 | 13 | 9 | 0.95 | 0.76 | 0.85 | 88 | 9 | 4 | 0.99 | 0.88 | 0.92 |
Note: Formally the inference pattern: A is either F or G, A is not F, so A is G, often called "disjunctive syllogism."
Table 3: MODPON
| Jamaican | Non-Jamaican | |||||||||||
| Year | %Master | %Border | %Fail | Facility | %Master | %Border | %Fail | Facility | ||||
| High | Low | Av. | High | Low | Av. | |||||||
| 1990 | 69 | 17 | 14 | 0.94 | 0.74 | 0.81 | 78 | 9 | 13 | 0.96 | 0.81 | 0.86 |
| 1991 | 72 | 14 | 14 | 0.96 | 0.73 | 0.83 | 79 | 11 | 10 | 0.96 | 0.81 | 0.87 |
| 1992 | 73 | 14 | 13 | 0.97 | 0.72 | 0.83 | 81 | 13 | 6 | 0.98 | 0.83 | 0.89 |
Note: formally the inference pattern: if p then q, p, so q, traditionally called "modus ponens."
Table 4: MODTOL
| Jamaican | Non-Jamaican | |||||||||||
| Year | %Master | %Border | %Fail | Facility | %Master | %Border | %Fail | Facility | ||||
| High | Low | Av. | High | Low | Av. | |||||||
| 1990 | 44 | 21 | 35 | 0.78 | 0.48 | 0.67 | 54 | 22 | 24 | 0.79 | 0.53 | 0.72 |
| 1991 | 41 | 19 | 40 | 0.75 | 0.46 | 0.65 | 48 | 22 | 30 | 0.79 | 0.49 | 0.69 |
| 1992 | 40 | 23 | 37 | 0.72 | 0.43 | 0.64 | 53 | 18 | 28 | 0.82 | 0.52 | 0.72 |
Note: formally the inference pattern: if p then q, not q, so not p, traditionally called "modus tollens."
Overall performance is fairly stable, with a slight tendency to improvement over the years on most principles (more noticeable when one adds in earlier years and the invalid principles), which is not surprising given both the feedback effect from earlier candidates, the effects of teaching (a very elementary introduction to logical matters is included in one of the Foundation courses for the Certificate) on Certificate students who take the examination again for entrance to the B.Ed., and the small amount of repetition by unsuccessful candidates.
We may note that two of the valid principles, DISSYL (the inference pattern: A is either F or G, A is not F, so A is G) and MODPON (the inference pattern: if p then q, p, so q), prove to be fairly easy; they are both among the most basic patterns at this level of formal structure, indeed MODPON is often taken as the fundamental rule of inference in formal logical systems. (It may be of some small interest to those deviant or non-classical logicians who reject the rule, DISSYL, that our subjects accept it a little more readily than MODPON itself.) The items testing MODPON have been kept identical with those in Ennis' tests throughout the Jamaican investigations, so that it is possible to compare several different groups on this principle (data here are extracted from Brandon and Sirbratthie [6]). Ennis found, among grade 11 high school students in the USA (N = 78), 62% mastery, 16% borderline, and 22% failure. The 1989 male Faculty applicants (N = 117) gave very similar figures: 63%; 18%; 19%, respectively. A group of teacher trainees in St Lucia (N = 100) yielded 70%; 14%; 16%, which is very close to our recent Jamaican applicants. Our teachers are performing somewhat better than Jamaican high school students: Nolan found over 30% failure at grade 11 in a comprehensive school in western Jamaica, and in one grade 10 Kingston high school for boys failure was 36% (N = 61) though in another only 12% (N = 34). But their achievement palls when we compare it with a private Kingston high school's grade 9: 86% mastery; 14% borderline; no failures (N = 21).
It may be of some interest to enquire what exactly people are doing who get these two easy principles wrong. The item analyses presented in Brandon [4] show that in most such cases the answer given is 'Maybe'. John is either rich or famous. John is not rich. So is he famous? Those who answer 'No' here can only be understood as careless in some way; but 'Maybe' could be a result of not entering fully in to the spirit of the test. Subjects are asked to suppose that they know that John is either rich or famous; but of course an arbitrarily selected John might be neither. When you add that he isn't rich, it may seem appealing to hedge one's bets on his fame as well. When we cast the question in the MODPON form, changing the first premise to 'if John is not rich, then he is famous', the 'Maybe' answer may be reflecting a tendency to interpret statements of the form 'if p then q' as the weaker 'if p, then it's more likely than not that q'. With such an interpretation one would not wish to detach the consequent of the conditional absolutely as the question format requires. I do not know if some such construal of the standard conditional is plausible it would run counter to the orthodox view in philosophy that the inferential powers of 'if ... then ...' are central to its meaning but I cannot work out what else sensible people might be doing in offering the 'Maybe' answer here.
While these two principles are easy, they are not in fact the easiest principles that have been empirically examined. An earlier group of candidates did better on examples of hypothetical syllogism: all A are B, all B are C, so all A are C; and students in the USA and Jamaica also found a principle with 'only if' easier: p only if q, p, so q. These two instances exemplify general findings that people perform slightly better on quantificational reasoning than on corresponding conditional principles and also that the word only tends to make inferences easier to appreciate.
While a large majority of the teachers get correct answers on DISSYL and MODPON, things are not so good when it comes to MODTOL (the inference pattern: if p then q, not q, so not p). Roughly 35% of the applicants have not mastered this principle, which is a fundamental reasoning tool in our investigation of the world: a long tradition, stretching at least from Lord Bacon to its most notorious contemporary exponent, Sir Karl Popper, has seen in the falsification of predictions the most powerful means we have of revising our view of the world (cf. Brandon [3], ch. 3). Failure to grasp this mode of reasoning must conduce to woolliness in one's thinking.
Ennis focussed attention on percentage mastery of the principles, but given that the data reported here have been gathered under stressful examination conditions with a strict time limit, perhaps a fairer measure of typical achievement is given by the facility indices for the different items and principles. As can be seen there is a wide range of facility indices for items belonging to the same formal principle (often as much as 0.2); detailed examination of item statistics shows that this range is not primarily a matter of people failing to complete the test but should presumably be explained by the well-known effects of differing content on the perception of logical relations. In the case of the valid principles, average facility indices tell pretty much the same story as percentage mastery; when we turn to the invalid principles to be discussed shortly, the facility indices suggest somewhat better performance than the percentage mastery figures indicate: the contrast betrays the instability of most people's grasp of the logical relations in such cases of invalid inference.
When we turn to invalid principles (Tables 5-7), the picture presented is one of extreme weakness. This was expected on the basis of Ennis' original findings that only a small minority of students correctly diagnose invalid patterns of argument. While performance on valid principles is not spectacular (as seen here with MODTOL and elsewhere with somewhat more complex patterns), one major reason for demanding urgent attention to logical reasoning in the curriculum lies in the amazing prevalence of fallacious inference in the population at large, and teachers in particular.
Table 5: UM
| Jamaican | Non-Jamaican | |||||||||||
| Year | %Master | %Border | %Fail | Facility | %Master | %Border | %Fail | Facility | ||||
| High | Low | Av. | High | Low | Av. | |||||||
| 1990 | 16 | 12 | 72 | 0.51 | 0.32 | 0.38 | 22 | 12 | 66 | 0.50 | 0.37 | 0.44 |
| 1991 | 22 | 18 | 60 | 0.60 | 0.35 | 0.49 | 29 | 15 | 56 | 0.60 | 0.38 | 0.52 |
| 1992 | 20 | 20 | 59 | 0.58 | 0.38 | 0.49 | 33 | 12 | 55 | 0.57 | 0.42 | 0.53 |
Note: formally the principle: most A are B, most B are C, so most A are C.
Table 6: MM
| Jamaican | Non-Jamaican | |||||||||||
| Year | %Master | %Border | %Fail | Facility | %Master | %Border | %Fail | Facility | ||||
| High | Low | Av. | High | Low | Av. | |||||||
| 1990 | 7 | 15 | 78 | 0.47 | 0.14 | 0.35 | 8 | 16 | 76 | 0.49 | 0.11 | 0.35 |
| 1991 | 5 | 11 | 84 | 0.43 | 0.08 | 0.32 | 8 | 14 | 78 | 0.47 | 0.08 | 0.33 |
| 1992 | 6 | 10 | 85 | 0.40 | 0.10 | 0.30 | 12 | 7 | 81 | 0.53 | 0.11 | 0.32 |
Note: formally the principle: all A are B, most B are C, so some A are C.
Table 7: NN
| Jamaican | Non-Jamaican | |||||||||||
| Year | %Master | %Border | %Fail | Facility | %Master | %Border | %Fail | Facility | ||||
| High | Low | Av. | High | Low | Av. | |||||||
| 1990 | 5 | 7 | 88 | 0.26 | 0.20 | 0.23 | 11 | 3 | 86 | 0.37 | 0.22 | 0.30 |
| 1991 | 6 | 9 | 85 | 0.35 | 0.25 | 0.31 | 16 | 12 | 72 | 0.46 | 0.29 | 0.40 |
| 1992 | 7 | 13 | 80 | 0.41 | 0.29 | 0.33 | 9 | 10 | 81 | 0.44 | 0.23 | 0.34 |
Note: formally the principle: no A are B, no B are C, so no A are C.
It was suspected that some part of the problem might have been due to the format of Ennis' tests, so the prompt 'Maybe' was changed to the colloquially more appropriate 'Not necessarily' in 1991. A comparison between overall results for 1990 and 1991 revealed that the mean facility of two of the three invalid types of item increased markedly and that in 13 of the 18 invalid items the distribution of answers was significantly different, while only 1 of the 18 valid items displayed a significantly different distribution of answers and none showed marked differences in facility (for more details of the 1990/1991 comparison, see Brandon [5]). These results seem to show that the change from 'Maybe' to 'Not necessarily' has a marked effect in many of those cases where it is the salient and correct answer. In most cases, more respondents give the correct answer with 'Not necessarily' as the prompt. This result also raises the possibility that the low scores found by Ennis on invalid items might to some small extent be a product of the test format. While performance seems to improve on the invalid items taken individually, subjects do not improve quite so much when one looks at the consistency of their performance on groups of items, as the Tables 5-7 indicate. And, whatever the prompt, most candidates still give mistaken answers to these items.
We can only speculate as to the reasons that lead most candidates to offer mistaken answers to questions embodying invalid inferences (v. Brandon [1]). One general tendency in our interpretation of each other's language is a kind of charitableness: interpret so that what is said comes out true. The items used for invalid principles are such that it is certainly possible that the conclusion is true when the premises are; what makes for invalidity is that it is not necessary. Suppose you know that some teachers are women and that some women like children; does it follow that some teachers like children? It is certainly possible that some teachers like children (and we may take it that we know that this is in fact how things are), but the two facts given as premises do not guarantee it they allow there may be women who don't like children and that only those women are teachers. Recognizing that this possibility is allowed by the premises is tantamount to constructing a model, a counter-example, in which the premises are true while the conclusion is false. We may suppose then that error here is often a matter of not properly grasping the rules of the game: taking the question, not as 'given what you have been told, must it be true...?' but rather 'given what you have been told, might it be true...?' All one can say about such an account is that the difference between must and might is one we should expect teachers not to slur over or ignore.
Another gloss on this excessive charity is to see it as a failure of imagination. An inference is offered; the basic test for invalidity requires the candidate to construct in thought a counter-example: a case where the premises are true but the conclusion is false. This involves aiming to show the person offering the inference to be wrong; it is a challenge and one that must be worked out in imagination. School systems, especially the more traditional and less resourced, do not encourage much use of the imagination and even less of a faculty of critical challenge. They thrive on mindless repetition rather than reflective attention, on automatic response rather than considered judgement.4 Products who are producers in such a system are not likely to do well when faced with these anti-authoritarian cognitive demands. The generally dispiriting fact is that no one else does much better.
While the data are not intended to reveal the actual psychological mechanisms candidates use, we may surmise that many people treat both the valid and the invalid inferences in the same way as forms that 'feel' right rather than actually testing them for validity by searching for counter-examples. When one is confronted by a valid argument pattern, searching for counter-examples is never going to succeed, but it is notorious in logical theory that the mere failure to discover a counter-example cannot in general be taken as a proof that there are none to be found. Validity must ultimately be simply intuited. The suggestion is that many people treat all the inference patterns used in these tests as equally intuitively correct and do not bother to even begin to reflect and test them. This would at least explain the unexpected difficulty of the principle NN (no A are B, no B are C, so no A are C), which is so clearly invalid that it was not even given a name by mediaeval logicians.5 The suggestion is that since people are rightly happy with All A are B, all B are C, so all A are C, they assume that No A are B, no B are C, so no A are C is equally good, without even trying to investigate.
The data already presented have given some indication of the relative position of Jamaican applicants with respect to the rest of the Caribbean. Data from 1990 suggest that when candidates are allocated to individual territories Jamaica goes along with the Bahamas, as against the other territories represented in the sample, in performance on logical reasoning. But while such differences can be found, it is worth noting that they are generally smaller than those revealed on those tests that tap more directly 'school' subjects such as English ('verbal' ability) and Mathematics. In 1990, for example, the mean for the Trinidad applicants on the verbal test (39.96) was almost a standard deviation (overall standard deviation = 7.93) above that for the Jamaicans (33.03), and similarly for the test of everyday mathematics (22.36 against 18.95, standard deviation = 4.44), whereas for logical reasoning (21.84 versus 19.63) it is slightly less than half a standard deviation (overall 5.07) above. What one may presume to be the generally superior formal schooling of most non-Jamaican applicants has a greater effect on their ability in school subjects than in simple logical reasoning, which as we have noted is ignored in school, but it does seem to spill over into it, as one would expect.
One can observe a little of this spill-over by contrasting applicants for different specializations. Not surprisingly, teachers of mathematics and integrated science are at some advantage on the logical reasoning test (and the mathematical one) but once again the superior background of the non-Jamaican applicants seems to come through, since the differences between subject groups are more marked among them than among the Jamaican applicants. (But the small and adventitious samples prevent us from trying to generalize findings here are no more than suggestive when they confirm intuitive expectations.)
In conclusion we can reiterate the main pedagogical interest of these data: performance on everyday reasoning tasks is grossly inadequate. It is bad wherever it is tested, but the standards revealed among the Jamaican applicants are hardly better than Ennis' results from secondary schools in the USA. The little evidence that exists (summarized in Brandon and Sirbratthie [6]) suggests that this disadvantage is not confined to the test format employed here, but extends also to more discursive tests of aspects of critical thinking. Woolliness and sloppiness (or to be blunt, incoherence and contradiction) characterize far too much of our teachers' use of their fundamental cognitive equipment.
What matters most is to find ways to improve that performance. Traditionally, this has been supposed to be a welcome bonus from the teaching of mathematics or Latin or what-have-you. My own inclination is to think that one might be better advised to tackle the problem explicitly and directly. This may well, as it usually does elsewhere, involve the teaching of new techniques, new ways of handling the information given, to replace whatever rules and strategies people currently use when faced with inferential issues.
This paper is not, however, intended to set out such remedial action. Its point is simply to record the situation with respect to a few aspects of logical reasoning as it is in many Jamaican schools at present.
1. We might also note that when one is confronted with an invalid pattern of inference it is no longer possible to move validly backwards from the falsity of the conclusion to the falsity of at least one of the premises. Suppose one of our local teachers has not been trained. That is quite compatible with most teachers, though of course not with all teachers, having been trained. Return to text.
2. Thus, to take but a few examples, if it were required that mastery of the valid principle to be labelled MODTOL (if p then q, not q, so not p) were to involve recognizing also that the structurally similar principle DENANT (if p then q, not p, so not q) is invalid, the percentage mastery in 1989 for all candidates would drop from 44% to 2.3%; with the easier principle MODPON (if p then q, p, so q) there would be a drop from 69% to 4%.
In 1990, 136 of the Jamaican candidates mastered all three valid principles, while only three mastered the three invalid principles. Of these three, one failed all the valid principles, leaving only two candidates with a mastery of all six principles. For the non-Jamaican candidates, the figures are 57 for mastery of the valid principles, 2 for the invalid, and 1 for both. Return to text
3. See Norris [13] for some careful discussion of the problems involved in characterizing the processes actually used by people in performing reasoning tasks, Johnson-Laird [10] for a stimulating psychological review that argues against the use of logical principles or rules, and Brandon [2] for a few comments on this matter in the context of the present investigations.
Readers familiar with typical investigations in educational psychology might have missed any discussion of validity and reliability. The fundamental reason for this is that the tests used are normative; they are not seeking to measure processes people may be using. Items are chosen because they exemplify certain patterns of inference, according to standard logic; people may deal with these problems in any number of disparate ways, or they may just offer answers at random. Validity is, then, given unless the researcher made logical mistakes; reliability, or internal consistency, is irrelevant. Having said that, Cronbach's alpha was calculated for the whole 36-item test in 1990 and a value of 0.77 was found. Return to text
4. I owe to Phillip Nissen (private communication) the plausible thought that one reason for many people making mistakes on some of the everyday mathematics items is that they have swallowed rules such as 'whenever you see from, use subtraction.' If you describe a signpost announcing that A is 50 miles to the north and B is 40 miles to the south, and then ask how far it is from A to B, many people will tell you 10 miles, instead of 90. Return to text
5. Mediaeval scholars, who had at least the benefit of studying a logical theory with a simple set of rules, may not have needed a name for this principle since it breaks one of the most easily checked of the rules: a syllogism must not have more than one negative premise. Return to text
[1] BRANDON, E.P., 1985, "On What isn't Learned in School," Thinking, 5, 22-28.
[2] -------, 1987a, "Deductive Reasoning Ability, Error, and Education," in F.H. van Eemeren, R. Grootendorst, J.A. Blair, and C.A. Willard, (eds.), Proceedings of the First International Conference on Argumentation 3A Argumentation: Perspectives and Approaches, Dordrecht: Foris, pp. 151-161.
[3] -------, 1987b, Do Teachers Care About Truth? London: Allen and Unwin.
[4] -------, 1990, "The Deductive Logical Competence of Non-graduate Caribbean Teachers," ERIC Document Reproduction Service No. ED 315 330.
[5] -------, 1992, "A Note on the Format of Ennis' Multiple-Choice Tests of Deductive Reasoning Competence," ERIC Document Reproduction Service No. TM 019 276.
[6] BRANDON, E.P. and SIRBRATTHIE, N., to appear, "Logical Reasoning as a Curriculum Area in Schools" in D.R. Craig, (ed.). Education in the West Indies: Developments and Perspectives, 1948-1988, Mona: ISER.
[7] ENNIS, R.H. and PAULUS, D.H., 1965, Critical Thinking Readiness in Grades 1-12 (Phase I, Deductive Reasoning in Adolescence). Cornell Critical Thinking Project (ERIC Document Reproduction Service No. ED 003 818).
[8] GEACH, P.T., 1976, Reason and Argument. Oxford: Basil Blackwell.
[9] HODGES, W., 1977, Logic. Harmondsworth: Penguin.
[10] JOHNSON-LAIRD, P.N., 1983, Mental Models. Cambridge: Cambridge University Press.
[11] NOLAN, C.A., 1984, An Investigation into the Logical Reasoning Competence of Selected Groups of Students in a Comprehensive High School in Jamaica. Unpublished B.Ed. study, University of the West Indies, Mona.
[12] NOLAN, C.A. and BRANDON, E.P., 1986, "Conditional reasoning in Jamaica," paper given to the Conference on Thinking, Harvard, 1984 (ERIC Document Reproduction Service No. SO 016 755).
[13] NORRIS, S.P., 1985, "Review article - the choice of standard conditions in defining critical thinking competence," Educational Theory, 35, 97-107.
This appendix gives the items from each logical principle with the highest and lowest facility indices in 1990.
DISSYL: 0.91 (q. 21, correct answer Yes)
DISSYL: 0.71 (q. 1, correct answer Yes)
[It is likely that the unusually low facility index for this item reflects the fact that it comes immediately after the instructions for the test, which conclude with an example where a 'Maybe' answer is unexpectedly correct. Candidates start off over-cautious.]
MODPON: 0.94 (q. 16, correct answer Yes)
MODPON: 0.74 (q. 32, correct answer Yes)
MODTOL: 0.78 (q. 29, correct answer Yes)
MODTOL: 0.48 (q. 5, correct answer Yes)
UM: 0.51 (q. 18, correct answer Maybe)
UM: 0.32 (q. 6, correct answer Maybe)
MM: 0. 47 (q. 17, correct answer Maybe)
MM: 0.14 (q. 28, correct answer Maybe)
[This is an example of one type of what Ennis called 'suggestive' items, where a true conclusion is inferred - in this case invalidly - from directly counter-intuitive premises.]
NN: 0.26 (q. 35, correct answer Maybe)
[Another kind of 'suggestive' item, this time with nonsense words.]
NN: 0.20 (q. 24, correct answer Maybe)
It should be noted that the correct answer for 7 out of the 18 valid items is 'No'.
URL http://www.uwichill.edu.bb/bnccde/epb/CJE.html
HTML revised January 23rd, 2000. Return to Publications; Return to Home.