Using a Language Generation System for Second Language Learning


Michael Levison

Department of Computing and Information Science

Queen's University, Kingston, Ontario, K7L 3N6, Canada


Greg Lessard

Department of French Studies

Queen's University, Kingston, Ontario, K7L 3N6, Canada




Abstract


VINCI is a language generation system which, given data files describing a natural language and a user's specification, generates utterances of the class the user has specified. The system, which is embedded in an editor environment, has been described in detail elsewhere. It can exercise control over the syntax, lexicon and morphology of the language, and to some extent, over semantics. On a suitable workstation, "utterances" can contain picture and sound/voice output as well as written language.


For its application to CALL, VINCI can generate clusters of related utterances. At the simplest end of the scale, these can be used for drill exercises where the student is given sentences and asked to carry out some operation. More elaborately, the lexicon can be used as a database to create a dialogue for language practice. In this paper, we explore a range of potential applications to second-language learning, including approaches to intelligent detection of learner errors.



VINCI Outline


VINCI is described in detail elsewhere (see Levison and Lessard, 1992). We therefore provide here only a brief outline.


VINCI is a program which accepts a description of some part of a natural language, and generates phrases or sentences in it. It has applications in language modelling and subject testing as well as language learning. Since it uses a formalism quite similar to that used every day by linguists, no particular computational expertise is required of the user. Furthermore, input takes the form of simple text files, produced by any editor program.


A VINCI language description is in three parts: syntax, lexicon and morphology. In the first phase of the generation process, VINCI builds a syntax tree using context-free phrase structure rules, and then manipulates the tree with the help of syntactic transformations. Since alternative sets of transformations can be applied to the same basic tree, VINCI can actually create clusters of related utterances. In a drill exercise environment, for example, it might be made yield a

question and the expected answer.


The nodes of the trees generated by the syntax contain, in addition to the usual syntactic class names, collections of attributes which can be passed down to the nodes' descendants. These attributes may carry grammatical information (masculine/feminine, singular/plural) or semantic information (human/animal/...), and are used to ensure grammatical agreement between different nodes and to provide limited semantic control.


Once syntax trees have been generated, the lexicon is searched and entries are selected to match the leaf-nodes. Typical entries in a simple English lexicon might be:


"cat"|N|Number, >animal||$2|||||||"chat"|

"banana"|N|Number, >edible, >plant||$2|||||||"banane"|

"eat"|V|vtrans, Number, Tense, Person, <animate.subj, <edible.obj|

|$12||"ate"|"eaten"||||"manger"|

"small"|ADJ|||#1||||||syn:"little"; ant:"large", "big"|"petit"|


which give the word, its category, the attributes which apply to it, some additional grammatical information, pointers to related words, and so on. Thus "cat" belongs to category N (presumably the nouns), has both of the Number-attributes (can be either singular or plural), and belongs to the semantic class "animal" or any class which includes it ("animate", "physical_object", etc.) It is inflected using morphology rule 2, which one may guess forms plurals by adding "s". Field 12 contains its equivalent in French ("chat"). The verb "eat" is transitive, has all Tense-, Number-and Person-attributes, needs subject and object in semantic classes "animate" and "edible" respectively, or in any more finely divided classes which these include. (For example, "animate" may include "animal" and "human"; "animal" may include "feline".) It forms its conjugates using rule 12, which in turn calls on the lexicon to supply some irregular tenses. The adjective "small" points at a synonym and two alternative antonyms.


When a word has been selected for a leaf-node, VINCI may be directed to substitute another word pointed at by the original. A command to carry out such substitution is placed on the leaf-node by the syntax.


The words attached to the leaf-nodes are inflected by the morphology component to yield the final utterance(s).



Language-learning Exercises


By exercise we mean a set of individual instances (maybe question/answer pairs) which provide a learner with practice in some aspect of a language. The VINCI system lets an instructor specify an exercise, and generates as many instances at random as one wants. In a typical scenario, VINCI displays each question and retains the expected answer internally. It then waits for a student to enter an actual answer, and compares actual and expected results. More generally there may be several alternative correct answers. Usually an exercise will be accompanied by a rubric telling the student what is expected.


Potential exercises range from simple drill exercises to pieces of (written) dialogue between computer and learner. We illustrate here a selection of exercises which can be created using VINCI. Each group is typical of a wide range of possibilities. In general, we give both the question part of an instance and the anticipated correct response.


Figure 1 shows three exercises in which the student is given nouns and asked to attach a definite article and/or make the noun plural. Obviously (a) and (c) provide practice in noun inflection; (b), knowledge of gender.


--- Figure 1


(a) (English) dog dogs

box boxes

child children


(b) (French) livre le livre

plume la plume


(c) (French) livre les livres

chou les choux

---



Four similar examples appear in Figure 2, where the student is given words, and asked to provide synonyms or antonyms or derived forms or, recalling that a field in a lexicon item may contain a language gloss, even the equivalent in another language.


--- Figure 2


(d) small [synonym] little


(e) small [antonym] large


(f) small [derived noun] smallness


(g) small [in French] petit, petite, petits, petites


---



Figure 3 illustrates Cloze exercises in which the student is given short sentences with a gap, and a word. The proper form of the word is to be placed in the gap. In case (i) a tense is also supplied. The word might be expressed in either the first or second language.


--- Figure 3


(h) The dog ___ a bone. [eat] eats

The cat ___ the hand. [scratch] scratches


(i) The dog ___ a bone. [eat, past] ate


---



Other exercises of this kind can be used to practise verb conjugation or preposition usage (Figure 4). We may note that VINCI itself can generate the whole of the question, including the metalinguistic prompts.


--- Figure 4


(j) Nous ___ des pommes frites. [manger, futur] mangerons


(k) We lived ___ an old house. in

They went ___ the kitchen. into

He came to work ___ bus. by

---



Figure 5 typifies some exercises in which the student is given simple sentences and asked to carry out some transformation. Case (l), for example, calls for each sentence to be made negative-interrogative; case (m), for the noun phrases to be turned into pronouns. In case (n), two sentences are presented which have related subjects or objects, and the student is asked to combine them using relative clauses or embedding.


--- Figure 5


(l) The dog ate the bone. Didn't the dog eat the bone?

He is driving his car to work.

Isn't he driving his car to work?



(m) L'homme a mangé une pomme.

L'homme l'a mangée.

Il a donné le livre à l'étudiante.

Il le lui a donné.


(n) The cat is grey.

The cat chases a mouse.

The cat which chases the mouse is grey.

or The grey cat chases a mouse.

---



Two exercises similar to these deal with specific areas of a language (Figure 6). In case (o), the student is given a clock time in numerical form, and asked to express it in words. Case (p) illustrates an analogous exercise for numbers.


--- Figure 6


(o) 7.20 twenty past seven

7.45 quarter to eight


(p) 101 a hundred and one

1994 one thousand nine hundred and ninety four

---



Figure 7 shows an instance of an exercise in which we introduce some dialogue. Here VINCI generates a cluster of utterances, comprising a statement and several related questions with expected answers. The statement might be displayed for a few seconds and then removed. Each question can then be posed in turn, and an actual student answer awaited before VINCI asks the next question.


--- Figure 7


(q) John saw a grey cat in the barn.


What did John see? A grey cat.

Where did he see it? He saw it in the barn.

(or) In the barn.

What colour was the cat? Grey.

---



In this sample, each piece of the dialogue is merely a syntactic transformation of the initial sentence. By incorporating more semantic information in the lexicon, however, we can produce a greater range of possibilities. For example, with suitable semantic attribute classes, we might have lexical entries of the form:


"cat"|N|animal, small, quadruped, furry|...|food: "mice"|


VINCI might then present the student with the information contained in the entry by generating a paragraph of small sentences.


A cat is a four-legged animal. It is small and furry.

It eats mice.


Subsequently it might ask:


What is a cat?

What does it look like?

How big is it?

What does it eat?


Alteratively we might devise a word-charade game in which VINCI describes traits one at a time until the student correctly names the animal.


Carrying further the idea of semantic information within a lexicon, we see that a VINCI dictionary can function as a simple relational database. A database for English literature might have dictionary entries like:


"Romeo and Juliet"|PLAY|...|author: "Shakespeare"/HUMAN|...


"Shakespeare"|HUMAN|...|birth:"1564"/DATE; death:"1616"/DATE;

birthplace:"Stratford-on-Avon"/PLACE|...


...


From a random choice of entry, VINCI might then enter into a dialogue of the form:


Vinci: Who wrote Romeo and Juliet?

(Vinci - expected reply: It was Shakespeare.)

Student: It was Shakespeare.

V: Good.


V: When was he born?

(V - expected reply: He was born in 1564.)

S: In 1572.

V: It was actually in 1564.


V: Where was his place of birth?

(V - expected reply: He was born in Stratford-on-Avon.)

...


and eventually on to another work or author. The topics can be wide-ranging, and the student can be asked to select some area of interest.


The goal of such an exercise might be either linguistic (teaching language) or encyclopedic (teaching facts about the world). In the language context, the purpose is to engage the student in conversation, and thereby cause him/her to write simple, or more complex, sentences in the second language.


An exercise similar to this in French literature was used by the authors with some success to study errors in the choice of prepositions "dans",

"en" and "à".



Voice Output


The preceding exercises imply the use of text as the medium for communication. With suitable hardware and software interface, however, VINCI could readily generate voice as well as visual output. If a phoneme approach is to be used, for instance, the lexical and morphological components of VINCI could be made to produce the phonemic spelling of words, as well as the stresses. Thus the student might ask to hear the pronounciation of a word, or be posed oral questions in the second language.



Pictures


We have already experimented in a limited way with the use of pictures in VINCI. Some of the nouns in our French lexicon specify small picture files which can be displayed along with the textual output.


This suggests simple exercises in which a student is shown pictures of objects (apple, dog, ...), and is asked to enter their names. One might ask for a noun, or perhaps for a short description (red apple, spotted dog). With suitable pictures of actions, this can be extended to verbs; or one might show several pictures (noun, verb, noun) and ask the student to compose a short sentence.


Developing this concept, one may present the student with a scene. This might be a generated picture, created by juxta-posing and superimposing smaller picture components; or it might be an actual photograph, where a human has noted information about objects in the picture, and has marked objects or zones.


We can now imagine the student asking questions about the scene; for example, he/she may use a mouse to point at a tree and ask what it is. Conversely, the program may highlight some object in a picture and ask the student questions about it. (Compare with Hamburger, 1994.)



Reference Tool


Combining these features we see that, given a sufficiently comprehensive lexicon, VINCI can function not only as a language generator, but also as a useful reference tool. A user might ask for a definition of a word, for its pronunciation, for synonyms or antonyms or hyperonyms, for a picture, and in inflected languages, for a table of its inflected forms. In effect, the system can be used as a thesaurus or even an encyclopedia.



Errors


We have mentioned that VINCI can be asked to create, for each instance of an exercise, not only a "question" but also one or more "expected answers". Since the system is implemented within the framework of an easy-to-use text editor, a common scenario is to have the system generate and display a question, and then to pass control to the student for her to enter/edit a response. When she indicates that this has been done, the system can compare the student response to the expected response(s) to obtain a right/wrong indication. It is then straightforward for the program to present a series of instances in an

adaptive fashion, progressing to greater complexity as a student's performance improves.


A more valuable approach will try to discover the kinds of error a student is making in order to suggest corrections.


Research has shown that learners of a second language rarely make errors at random. Rather, they fill gaps in their linguistic knowledge by following erroneous rules, which they have inferred consciously or unconsciously from their previous experience of the language. For example, many speakers with a core knowledge of French, encountering a noun whose gender they do not know, assume it to be masculine (the "unmarked" form) unless it ends in "e", when they assume feminine gender. Such erroneous rules are referred to as malrules (Sleeman and Brown, 1982).


In recent years, the authors have combined data extracted from a corpus of student essays with information from experiments run under the VINCI system, investigate malrules employed by second-language learners of French (Lessard and Levison, 1992, 1994a and 1994b).


Now suppose we can discover a set of common malrules, and incorporate these into the description of the language. Then an improved scenario for the use of the system not only determines that a student's response is wrong, but also tries to discover a combination of malrules which yield what the student has produced. The system is then in a position to suggest what the student is doing wrong, and advise her accordingly.


In general, we might expect the system to discover several alternative explanations for an error. It might then try to eliminate some of the alternatives by setting further instances of the exercise, perhaps designed to discriminate between them.


This is the area of our current research.


***


References


Hamburger, Henry. (1994) "Foreign Language Immersion: Science, Practice, and a System." Journal of Artificial Intelligence in Education 5: 429-453.


Lessard, Greg, M. Levison, E. Girard, D. Maher (1992) "Form, Frequency, Markedness and Strategies in Second Language Performance Modelling." In Intelligent Tutoring Systems, (C. Frasson, G. Gauthier, G.I. McCalla, eds.), Berlin: Springer-Verlag, pp 360-371.


Lessard, Greg and Michael Levison. (1994a) "Prepositional Usage in L2 French". Joint Conference of ALLC/ACH, Paris, April 1994.


Lessard, Greg and Michael Levison (1994b) "Computer Elicitation of L2 Performance Errors: Theory and Practice". Second Language Research Forum, Montreal, 1994.


Levison, Michael and Greg Lessard. (1992) "A System for Natural Language Generation." Computers and the Humanities 26: 43-58.


Sleeman, D. and S. Brown. (1982) Intelligent Tutoring Systems. New York: Academic Press.