Writing Skill: Its Peripheral Motor and Central Cognitive Components

Stevan Harnad

Dan Sperber makes the very reasonable prediction that a good deal of our writing will eventually be done by dictation, once automatic speech-transcription becomes sufficiently reliable, but that we will continue to read. For whereas we speak much faster than we can write, we speak (and listen) much slower than we can read.

Dan is also quite right that much of this is because we have evolved a brain that is specifically speaking/listening-prepared rather than reading/writing-prepared (although the fact that language originated in an organic adaptation for an oral/aural "culture" makes our latter-day reading alacrity a rather remarkable epiphenomenon!).

But I am not so convinced by his predictions about the future of writing, and instruction in it. I of course don't mean calligraphy or typing skills. I mean the exercise of the act of creating lapidary written text, by whatever motor organ one may use, be it hand, foot or mouth. The essence of the skill of writing no more resides in the muscular adeptness of a motor endorgan than oratory skill resides in the adroitness of the lips.

An early clue to this mismatch of cognitive skill levels comes here:

Dan Sperber: "Is it that writing systems are more complex than languages? Quite the opposite.... [L]inguists have not yet succeeded in providing a fully explicit grammar of any language, whereas writing systems are based on fully explicit rules."

But that is comparing apples and fruit! The right "system" to compare with the writing system for complexity is not "language," but speech. In other words, grapheme systems must be compared with phoneme systems. The two will be found to be of approximately the same level of complexity (although herein might lie some of the clues to the remarkable speed-advantage of the visuospatial [graphemic] mode over the audiotemporal [phonemic] mode for input [but not for output]). Yet it is LANGUAGE that both modes encode. And language is the system with which grammarians are still struggling, not writing or speech.

So what has gone wrong? Dan notes that "Writers can write, correct, rewrite." Indeed. And what are they correcting? They are not correcting their penmanship or their typing skills. They are correcting their WRITING, and by writing skill I mean something similar to what we mean when we refer to a speaker's "oratory" skill (how well he expresses himself in words orally, in real time).

Writing is not an online, real-time, do-or-die skill like oration. Let us not forget that until the very recent advent of audio recording, speech left no permanent residue (apart from the impressions in auditors' minds). Writing, in contrast, is an off-line, doing and redoing skill, in which the output undergoes a variety of dynamic transformations (each of them in principle preservable) before it is finally etched in stone. And writing's essence is that it does preserve a residue: a plastic, manipulable one.

And that, entirely independent of the peripheral input/output sensorimotor modality, is the central essence of writing skill -- what it is that a writer must LEARN, through trial, error and feedback, rather than already KNOWING, ex officio, so to speak, by virtue of already knowing how to speak.

I don't want to be too apodictic about this. Some writers are "naturals," writing lapidary texts literally "as they speak." But I believe such gifts are a rarity, even among gifted writers, whose skills lie as much in the editing and rewriting as in the spontaneous generation (dictation, if you like) of the first draft.

What resources does Dan envision for the dictaphone version of this skill?

Dan Sperber: "As soon as technology will make it possible to see one's speech properly transcribed as it unfolds, and to modify the transcription by means of oral instructions (and also, probably, of pointing and highlighting hand movements), writing will present no advantage that is sufficient to justify its cost."

If we are agreed that using a computer, keyboard, screen and mouse to edit text -- to erase, write over, cut/paste, move, transpose, search/replace, compare, etc.) is indeed old-style writing (closer to calligraphy than oratory), then the real empirical question here concerns how useful people will find it to inject an oral/aural element into this interaction with their graphemes.

Dan may be speculating here that there will be a more convenient way of cutting and pasting using oral commands rather than reaching out and grabbing text with a mouse. (The "highlighting hand movements" are equivocal, if they are merely shadowing what one does with a mouse!)

Perhaps he is right. Or, equally possible, once the first draft has been generated by dictation, the rest of the writing will be more effective with mouse, keyboard and screen, or still-to-be-discovered sensorimotor peripherals, with the mouth reserved only for relatively long bursts of inserting new text. It is hard to second-guess this in advance, before the hybrid vocal/manual dictation/manipulation capability is actually implemented, and then tested for its usability and user-friendliness by "human factors" specialists.

But what seems clear is that this new hybrid oral/manual skill will indeed be a new (peripheral) motor skill, one that users will have to master! and having mastered it, they will have to turn to mastering the higher level (central) cognitive skill that this technology was devised to serve. And that higher level skill will be WRITING, not SPEAKING.

And both skills will probably have to be taught.

Dan Sperber: "Once writing isn't practised anymore (except by calligraphists), what will happen to its teaching?... [An] economy of effort... would result from teaching children just to read [rather than] teaching both skills [reading and writing]."

I can't quite see the economy. Calligraphy (penmanship) is certainly gone. Possibly (though not surely) keyboarding too. But the bimodal motor skill underlying this new, hybrid oral/manual manipulation of graphemes and lexemes? Will that not have to be taught and learnt too? And only then begins the instruction in the USE of these bimodal text-manipulating motor skills in the service of the real higher-order cognitive skill in question, namely, writing.

Dan Sperber: "One may assume that the teaching of writing will long outlive its obsolescence."

This often happens, out of habit and superstition, sometimes until an entire generation dies off. But is that what is at issue here? Will the counterparts of both the peripheral motor skill and the central cognitive one underlying writing -- the latter always overlapping with the cognitive skill of speech, but never coextensive with it -- not both continue to exist, as skills to be taught and mastered, rather than being supplanted, as Dan suggests, by the extant skills of reading and speaking?

Let us not, by the way, underestimate the importance of the exercise of motor skills, both peripheral and central. What Held & Hein's (1963) experiments (with the active and passive kittens) showed us was that the passive "reading" of visual patterns is not even enough to allow us to learn how to see: Active, dynamic sensorimotor interactions between our bodies and the shapes and locations of objects in space -- trial-and-error, fumble-and-find practice in navigating, reaching, manipulating, using our muscles, guided by feedback from our mistakes -- are necessary, in order to develop normal vision.

Now speaking is "virtual" writing, just as writing is virtual speech. So it is unlikely that we would become either aphasic or alexic, like the passive kittens, if all we were allowed to do was to speak, read, and dictate text. But without the further dynamic interaction with text that is involved in real writing, and editing, and re-writing, I doubt that we would ever generate some of the great prose stylists our species has produced so far. (I make no case for teaching, because I am not convinced that the higher-order skill of writing can be taught; it is learned, yes, from reading and speaking, but also from practice in writing and re-writing text, and not just dictating it -- at least by those of us not among the lucky few who can spontaneously "write as they speak" and "speak as they write.")

Nor should we forget the homology with calculation, a skill of which the computer has relieved our brains -- to the great cost of the numeracy of a subsequent generation. (Dan's suggestion that speaking and reading skills are all you need for writing seems rather like suggesting that seeing and walking skills are all you need for dancing!)

Dan Sperber: "It is all too easy to speak of a return to orality. The most profound effect that writing has had on human civilizations has been to allow them to become truly cumulative instead of evolving forever within the limits of human long-term memory. Far from reversing these effects, the new technologies allow new forms of cultural accumulation as well as new ways of mining the accumulated information."

Agreed. But the question is: What will those new ways be? Dan thinks instantaneous speech transcripts (lexemes, really: one wonders about non-alphabetic languages) will be the "killer" application, with reading & dictating then replacing reading & writing.

In contrast, I think "skywriting" (the rapid online manipulation of, and communication using, digital text) will become faster and faster and more and more powerful, no doubt accelerated by vocal input in place of keyboard input, but that the navigation and manipulation of the visuospatial graphemes is unlikely to be optimal by voice alone, nor even predominantly.

The real revolution, in my view, lies in having accelerated the speed, scope, and interactiveness of writing (sic) to something closer to the speed of thought (whose tempo is of the same order as the speed of speech, for which our brains were specifically "co-prepared"), rather than in its replacement by transcribed speech (Harnad 1991, 1995, 2001).

Harnad, S. (1991) Post-Gutenberg Galaxy: The Fourth Revolution in the Means of Production of Knowledge. Public-Access Computer Systems Review 2 (1): 39 - 53 http://www.cogsci.soton.ac.uk/~harnad/Papers/Harnad/harnad91.postgutenberg.html

Harnad, S. (1995) Interactive Cognition: Exploring the Potential of Electronic Quote/Commenting. In: B. Gorayska & J.L. Mey (Eds.) Cognitive Technology: In Search of a Humane Interface. Elsevier. Pp. 397-414. http://www.cogsci.soton.ac.uk/~harnad/Papers/Harnad/harnad95.interactive.cogniti on.html

Harnad, S. (2001) Beyond Access and Impact: The Ultimate Benefit of Skyreading/writing. http://makeashorterlink.com/?D28851D5 http://www.text-e.org/debats/LeftFrame/printthreads.cfm?ConfText_ID=7&Parent=0&T op_ID=253&Intervention_ID=253

Held, R., & Hein, A. (1963). Movement produced stimulation in the development of visually guided behavior. Journal of Comparative and Physiological Psychology, 56, 872-876 http://www.sscl.uwo.ca/psychology/faculty/goodale/research/active.pdf

Dear Stevan, I concur with most of your points (excepting the premature obituary for calligraphy, which is closer to poetry than to penmanship). The central point is what goes into what you call the "lapidary" aspect of seriously crafted writing, and if/how it differs from casual speech and seriously crafted speech. I refer you to a paper from our Dyslexia Project lab (done almost entirely by Linda Davenport) on the differences in normals' and dyslexics' spontaneous narrative speech. It brings out various components of the planning, self-monitoring, and correcting (editing) process, which clearly apply to both speech and writing. Davenport, L., Yingling, C.D., Fein, G.F., Galin, D., Johnstone, J.: Narrative speech deficits in dyslexics. J. Clin. Exp. Neuropsychol., 8, 347-361, 1986. I think you will also find much wisdom in Jerome Bruner's essay, "On teaching a native language" in his great book Toward a theory of education, Harvard Univ. Press, 1966. Here is the first paragraph:

"I have often thought that I should do more for my students by teaching them to write and think in English than teaching them my own subject. It is not so much that I value discourse to others that is right and clear and graceful --- be it spoken or written --- as that practice in such discourse is the only way of assuring that one says things right and courteously and powerfully TO ONESELF. For it is extraordinarily difficult to say foolishness clearly without exposing it for what it is, whether you recognize it yourself or have the favor done you. So let me explore, then, what is involved in the relation between language and thinking, or, better, between writing and thinking. Or perhaps it would be even better to speak of how the use of language affects the use of mind."

All good wishes (and keep in touch David Galin M.D. Assoc. Prof. Dept. of Psychiatry Director, Laboratory for Deep Questions Langley Porter Neuropsychiatric Institute University of California San Francisco E-mail: DGALIN@ITSA.UCSF.EDU Best for Snail-mail: 5 Mt. Hood Ct., San Rafael, CA. 94903-1018 Please do not snail-mail to the University.

Calligraphy, CyberDactylography, Dictascript, Oratorship, Authorship

Stevan Harnad

David Galin: "I concur with most of your points (excepting the premature obituary for calligraphy, which is closer to poetry than to penmanship)."

Well said! I was not wishing it dead; just acknowledging it's an endangered species. And that was probably already true in the age of cyberdactylography. I think Dan is right that the dictatorship of the dictascript will alas deal it the coup de grace (unless it survives as a zoo/museum art, as reading has in the age of TV/movies/cybersurfing, or classical music, in the age of pop).

David Galin: "a paper from our Dyslexia Project lab... on the differences in normals' and dyslexics' spontaneous narrative speech... brings out various components of the planning, self-monitoring, and correcting (editing) process, which clearly apply to both speech and writing."

The neurological roots of oral and written self-editing are undoubtedly the same (and fundamentally oral), but only in the sense that the neurological roots of locomotion and airplane pilotry (or of reaching and painting) are the same: Their respective cultivable skills, and the requisite practice and instructions for cultivating them, differ.

I am also grateful to David for citing Jerom Brunere, that master prose stylist, on the art of writing:

Jerome Bruner (1966): "I have often thought that I should do more for my students by teaching them to write and think in English than teaching them my own subject... practice in such discourse is the only way of assuring that one says things right... it is extraordinarily difficult to say foolishness clearly without exposing it for what it is. "

This theme has already been broached in this symposium under the thread: "You Can't Kid Kid-Sib: Making the Implicit Explicit in Text, Discourse and Abstract" http://makeashorterlink.com/?Q1B112E5 http://www.text-e.org/debats/index.cfm?ShowIntervention_ID=369&ConfText_ID=9&Par ent=0&Top_ID=0

It is the lapidary, immanent, offline medium (of writing and re-writing), rather than the labile, volatile, online one (of speech) that allows us to practise and perfect this Brunerian art and science of prose (though, paradoxically, it is the feel of elegant, spontaneous conversation that the writer tries to emulate, just as musical instruments aspire to the expressivity of the human voice).

Jerome Bruner (1966): "[This is part of] the relation between language and thinking, or, better, between writing and thinking.... how the use of language affects the use of mind."

Yes, there is an element of the Whorf/Sapir hypothesis in all this: http://www.aber.ac.uk/media/Documents/short/whorf.html just as there is also a note of the McLuhanesque: http://www.mcluhan.utoronto.ca/mm.html

“Writing” is ambiguous. In its strict sense, it refers to a motor activity. By synecdochic extension, it has come also to refer to the complex intellectual and motor activity of creating texts, that is traditionally achieved by means of the motor activity of writing, but that could as well be achieved by other means, such as dictation. The old Milton could not write in the first, narrow sense of the term, but he could superbly write in the second wide sense, and of course, there are many people who write beautifully in the narrow sense and who have never written anything in the wider sense. I thought it was clear that I was speaking about writing in the strict sense, and nothing else. I therefore agree with almost everything Stevan Harnad and David Galin say. Of course, writing in the sense of creating text will deserve being taught just as much when it will be achieved through dictation as it is now, when it is achieved through handwriting or typing.

Harnad and Galin may feel that the teaching of writing in the narrow sense is of little interest. However, it takes a good part of the energy of primary school teachers (and of their pupils) and it is what is primarily being taught (together with reading) in literacy campaign around the world. (See also my response to Thierry Soubrié, Patrick Altman et Gloria Origgi in the thread “Sur l’enseignement de l’écriture”). Moreover, changes at this lowly level of writing may have effects on writing in the higher sense, and I wanted at least to evoke this possibility.

Three minor points in response to Stevan:

1) Stevan Harnad: “The right "system" to compare with the writing system for complexity is not "language," but speech. In other words, grapheme systems must be compared with phoneme systems. The two will be found to be of approximately the same level of complexity.”

DS: Right, and this makes it all the more remarkable that the acquisition of language as a whole is achieved and universally, spontaneously and much more reliably than the teaching of the mere graphic encoding of words and sentences.

2) SH: “Equally possible, once the first draft has been generated by dictation, the rest of the writing will be more effective with mouse, keyboard and screen, or still-to-be-discovered sensorimotor peripherals, with the mouth reserved only for relatively long bursts of inserting new text. It is hard to second-guess this in advance, before the hybrid vocal/manual dictation/manipulation capability is actually implemented, and then tested for its usability and user-friendliness by "human factors" specialists.”

DS: Possible of course, but if typing is of use only in the process of correction, and is just convenient, not necessary, this may not provide a sufficient incentive for people to learn how to type. People like us, who already do type, would of course readily resort to it, as Stevan envisages.

3) SH: "Dan thinks instantaneous speech transcripts ... will be the "killer" application, with reading & dictating then replacing reading & writing. ... The real revolution, in my view, lies in having accelerated the speed, scope, and interactiveness of writing (sic) to something closer to the speed of thought (whose tempo is of the same order as the speed of speech, for which our brains were specifically "co-prepared"), rather than in its replacement by transcribed speech."

DS: I don’t disagree. I did not choose to write about the coming obsolescence of handwriting and typing because I think that this will be the major revolutionary development to come from the new technologies. I chose the topic because it seemed to allow an illustration of a kind of informed reasoning one may engage in about the future effects of the new technologies, and because it fell within the wider topic of this web conference.

Handwriting vs. Mouthwriting

Stevan Harnad

Dan Sperber: "Harnad and Galin may feel that the teaching of writing in the narrow sense is of little interest. However, it takes a good part of the energy of primary school teachers (and of their pupils) and it is what is primarily being taught (together with reading) in literacy campaign[s] around the world."

Okay, I'll bite. I will defend the continued teaching/learning of writing in the form of mouthwriting (dictascript), from an early age, in association with the teaching/learning of reading (but I do this while confessing complete ignorance as to what it is that primary school teachers actually expend their energy in doing, in teaching reading and writing).

Just as spoken language has both a perception and a production side -- in performance and pedagogy as well as in the brain -- so written language does too. They each have their respective neurological disorders (Wernicke's receptive aphasia and Broca's productive aphasia for speech, dyslexia and dysgraphia for writing). Now I know that there there are modality-independent or supramodal aspects of the neurological representation of "speech," for monoglot "speakers" of gestural language can suffer from Broca's and Wernicke's aphasia too. My guess is that there are modality-independent aspects of dyslexia and dysgraphia too (and there would be even more if writing had been taught purely through dactylography rather than calligraphy).

These perception/production correlations are quite important, and can be quite deep. According to Liberman's "motor theory of speech perception," the PERCEPTION of speech is mediated by the PRODUCTION of speech: We recognize speech sounds by means of a template or filter tuned by the movements required in order to produce them. This is why someone who has not learned to pronounce a certain sound will tend to mis-hear it when spoken. This is why a "foreign accent" is not just a problem with forming the proper sounds: it is also a problem in hearing them. (This is in fact another instance of the Held & Hein active/passive effect I cited in my earlier comment.) It is also why Russian immigrants find it much easier to understand French when it is spoken by other Russian immigrants than by native francophones (and not just because of speaking speed).

So one might say that the body's capacity to perceive something is often mediated by its capacity to produce (and manipulate) it. Not always: We can perceive colors without having to be painters, let alone chameleons. But the chameleon question is relevant, for in the case of speech it is not just an arbitrary production modality that is involved, it is an ANALOG modality. Speech input and speech output are isomorphic; the stimuli and the movements have the same "shape." (The same is true of the gestural language of the deaf, and of many other domains, including facial expression, music and dance.)

Now I am not conjecturing that an analog congruity between the shape of writing gestures and the shape of graphemes is necessary in order to learn to read (although it may help!). But I do think Dan (and those contemplating dispensing with the teaching of handwriting) need to consider the possibility that at least an arbitrary if no longer an analog perception/production correspondence is still important for mastering reading itself, especially its compositional aspects (combining and recombining graphemes): Some or even many of the output stages and exercises of the teaching of handwriting (apart from the penmanship itself) in conjunction with the teaching of reading might still have to be taught in the new era of "mouthwriting" (Dan's dictascript), from the single-grapheme stage onward.

So the pedagogical "savings" in the transition to the teaching of writing (sic) in the form of dictascript in place of the traditional digiscript (an ironic barbarism, that, because of the etymology of the analog finger in the digital pie) may not be as large as Dan imagines (just as they would not have been if the pedagogic transition had been merely to the intermediate art of dactylography -- or even fully digital Morse Code).

One closing thought. For a recombinatory sensorimotor skill such as speech or writing -- where there is a relatively small repertoire of atomic elements (phonemes, graphemes) to be recombined into higher-order elements, lexemes (which are in turn also recombined into phrases and sentences) -- the critical property, for both perception and production, is the ability to individuate and actively manipulate those atoms. That is why I had asked about non-alphabetic languages in my earlier comment.

If I were Dan, my reply to the present critique -- that, even divested of the motor chore of mastering the strictly analog perception/production isomorphisms of handwriting, mouthwriting will still inherit many of the "digital" aspects of the teaching and learning of the PRODUCTION side of reading -- would draw upon the teaching and learning of reading and writing (not calligraphy!) with ideographic languages such as Mandarin. Could it be that the atomisation that has already been performed for writing by speech (the categorical perception of phonemes) is atomisation enough, and that graphemes can be "virtual," without needing to have an explicit representation, either sensory or motor, in order to subserve reading and writing?

My own guess, though, is that, even inheriting this implicit atomisation, mouthwriting will still have many production aspects that need to be taught, in the service of, and in association with, reading.

Held, R., & Hein, A. (1963). Movement produced stimulation in the development of visually guided behavior. Journal of Comparative and Physiological Psychology, 56, 872-876 http://www.sscl.uwo.ca/psychology/faculty/goodale/research/active.pdf

Liberman, A. M., Cooper, F. S., Shankweiler, D. P., & Studdert-Kennedy, M. (1967). Perception of the speech code. Psychological Review, 74, 431-461. http://www.haskins.yale.edu/Haskins/STAFF/amlbl.html

Stevan writes: “Some or even many of the output stages and exercises of the teaching of handwriting (apart from the penmanship itself) in conjunction with the teaching of reading might still have to be taught in the new era of "mouthwriting"... So the pedagogical "savings" … may not be as large as Dan imagines”

Stevan’s suggestion is interesting and it might even be right. I am skeptical though, if only because children start reading before writing, and with greater spontaneity. In any case, I was merely asserting that the teaching of reading could not be harder and is likely to be easier than the teaching of both writing and reading. This would be true even if the best way to teach reading involved the teaching of some writing skills, as Stevan suggests. I was also pointing out that we don’t know how much easier it might be. Sure, if the advantage were modest, this would be a strong argument for going on fully teaching both skills together. But at this stage, I just want to raise the issue (and shake a bit the conservatism which often goes with such topics). I therefore welcome Stevan’s cognitively informed discussion. Too much discussion, I feel, of new technology practices present and future lacks this much needed cognitive dimension.

I imagine that with some exercise I could master a dictascript and speechwrite with it, although I guess I would not be so much at ease in the beginning. But what about speechwriting mathematics? It seems to me that the practice of mathematics is essentially written, as opposed to oral. Why is that? Is it something linked to short term memory management or is it something linked to mathematics themselves? In fact, I am just not at ease with saying such a simple formula as: (p->q)->((p&r)->(q&r)) Also, I feel I gain in understanding when writing it.

There are cases where the set of written meaningful terms exceeds the set of oral meaningful terms, i.e some concepts are encoded by written symbols that have no oral equivalent symbol. It seems it is the case with mathematics and physics where specific notations are so important and only awkwardly rendered by oral utterances (take a matrix for instance, with its rows and columns). Think also of the written (iconic?) language of the electrician or of the architect.

If these are cases where the mastering of the pen (or equivalent) is essential, then it lessens the 'pedagogical savings'. Of course, there is such a lessening only for those happy few who learn mathematics or other written subjects.

Christophe Heintz is right : even if the ordinary way of producing texts will be through dictation to a machine, written symbols and figures of various sorts may still be needed, for instance in mathematics. Where in mathematics? Not so much in the kind of elementary calculus for everyday usage (here, for many people, the use of pocket calculator has already replaced pen and paper, and dictating the problem to the machines should be, if anything, easier than pressing keys) but in more advanced areas of mathematics. Anyhow, mathematical symbols, including numerals, are taught over and above the teaching of writing proper. If their teaching endures, how much would this lessen the pedagogical savings made by not teaching writing proper? Not much, I guess.

Christophe writes : “It seems to me that the practice of mathematics is essentially written, as opposed to oral”; I would say that his point is that this practice, like other practices associated to semi-iconic languages, is essentially HAND-written. In this sense, problems with dictation will not be worse than those with specific software designed for writing mathematics. Also, as it is already the case for some areas of mathematics, the introduction of a new technological writing tool will partly modify the practice.

But I do not think that the effects of this transformation are similar to those of the massive introduction of the use of pocket calculators, as Dan seems too suggest in his reply. Pocket calculators are “cognitive artefacts” which compute for us algorithms that we may don’t know anymore (how many people still remember the algorithm for calculating square roots?). From a pedagogical point of view they radically change our knowledge of a domain: children who have not been taught the addition algorithm won’t be able to calculate it without a machine. Whereas in the case of “oral” computations, we can imagine that children are still taught the methods for computing, but they do not need anymore to write them down.

Spelling it out in Reverse Polish Logic and TeX

Stevan Harnad

Although I too believe that the motor successor of the ACT of writing will survive and continue to be necessary if and when speechwriting prevails, I disagree with Christophe Heintz about the non-dictability of mathematics. The formula he gives as his example -- (p->q)->((p&r)->(q&r)) -- with its nested parentheses and its serial order is definitely one that mathematicians could learn to dictate, and dictate serially, with no problems.

Mathematicians can and do overlearn their notational systems until they can "think" in them: As an alternative, the same formula could have been written (and dictated ) in Reverse Polish Logic (RPL), with the parentheses omitted and the connectives at the end (although some notational systems may be more congenial orally than others -- as we know from Chomsky's famous left-branching constructions, which are forbidden by Universal Grammar, at least orally). (I am not adept in RPL, but I think it would have looked something like &rq->&rp->->qp though I'm sure that's full of errors....)

Let us not forget that writing and typing are serial data-entry modes too. It is only after the symbols appear on the page that the larger spatial context, visualizability, and manipulability enter the picture -- but these are precisely the motor-interactive aspects of writing that I think might not be best done orally: a hybrid vocal-manual modality will be the optimal one, but it will still be motor (productive), i.e., writing, not merely sensory (receptive), like reading, and it will be interactive/manipulative, not seamless read-out and read-in.

The real reason the oral medium may not be the right one for the initial dictation of the mathematical formulas is that there are no "words" in mathematics: Writing a formula is essentially analogous to SPELLING a sentence, rather than saying it. It is not at all clear that the oral medium is faster for spelling, in the way it is for speaking. (And we return again to the question of holographic languages.)

The problem is not the lack of names for special symbols, however, as mathematicians' adeptness in learning (and reading, writing and thinking) in terms of the special symbols of TeX and LaTeX has demonstrated. They, like us, are masters at re-chunking, and at overlearning new and more congenial codes.

Miller, G. A. (1956) The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review 63: 81 - 97. http://cogprints.soton.ac.uk/documents/disk0/00/00/07/30/index.html

In my previous intervention, I tried to raise two different points, on which I would like to come back:

1. The first point was the rather trivial claim that icons are widely used and often have no equivalent spoken symbol. I believe D. Sperber's answer bears on this point. And I think it may be on this point that Harnad disagrees with me. However, I am indeed convinced that we can express with spoken words what we normally or traditionally express with non-spoken symbols, but such a translation has a cost. Even if the RPL notation is logically equivalent to the classical one, it may not be cognitively equivalent. There are of course habits of thought to change for passing from one notation to the other, but there is also (more arguably) more ease to deal with the classical notation than with the RPL. There might be something else than just a question of habit. 2. My second point was trying to take on the stream of thought started by Harnad concerning the cognitive benefits that learning to hand write may have. I wanted to illustrate his point with mathematics. My example was the understanding I feel I gain when (hand) writing the formula (pq)((p&r)(q&r)). Gloria Orrigi's example is far better: learning to add is not, even in the new era, learning to type on a pocket calculator, but it partly consists in learning to write the numbers to add in columns with the unities in the same column, the decimals in the same column, writing the decimal number of the result of one column at the top of the column directly at its left and such things. Doing mathematics is not just expressing mathematical thought with symbols, it includes the manipulation of symbols proper. Harnad's "interactive/manipulative, not seamless read-out and read-in" device is therefore essential, and the shape it might take important.

The more general question is: what are the cognitive side effects of the learning of the hand writing skill? D. Sperber showed that there is a cognitive cost, which, he argues, is important enough. We still lack a measure of the cognitive benefits provided by hand writing and what kinds of the manipulative aspects of handwriting have some cognitive effects. My claim is that there are such manipulative aspects that are essential to some scientific practices.

Audiotemporal vs. Visuospatial Modes of Perception, Production and Representation

Stevan Harnad

The differences between speaking and writing are many, and not just restricted to "verba volant, scripta manent." One of the most profound differences is that the acoustic-vocal input/output medium is intrinsically a linear, serial one, whereas the visual-manual one is parallel, with 2-3 spatial dimensions at its disposal. This is probably the reason we can perceive written text so much faster than spoken -- and this is only partly remedied by dictascript: We still can't speak as fast as we can read (as the users of spoken books for the blind remind their readers, with frustration!).

The Church/Turing theory of computation shows that just about anything can be represented (and manipulated) by serial, linear symbol-strings, but that still leaves us in the dark as to what representations are optimal for the human eye/ear/hand/brain, given that time, resources and effort (Dan will be pleased to agree!) are the primary considerations here.

So, in reply to Christophe Heintz: Yes, we agree about the importance of the active, interactive manipulation of symbols (in particular, their spatial configuration: Gloria Origgi), and it is not at all clear that the optimal motor channel for this manipulation is the oral one, whether in the form of text dictation (if, say, spelling-out is required) or of oral "commands" for its spatial manipulation. We have bodies, not just mouths, and as, for input, our parallel-processing, multidimensional eyes seem to be more powerful in all of this than our serial-processing, unidimensional ears, it will probably continue to be more "cost-effective" to manipulate symbols with our full, parallel somatic resources, for output.