Lexical Creativity in L2 French


Greg Lessard, French Studies, Queen’s University

Michael Levison, Computing and Information Science, Queen’s University



1. Introduction


One of the defining characteristics of knowledge of a language is access to linguistic creativity. Native speakers can produce and understand new forms and structures at the phonological, lexical, syntactic and semantic levels. In principle, coming to learn a second language also involves acquiring similar skills, at least at the level beyond simple repetition of learned items.


However, measurement of lexical creativity poses a number of thorny problems. In earlier research (Lessard, Levison, Maher & Tomek 1995) we presented examples of derivational productivity found in a corpus of written texts produced by university level Anglophone learners of French. The following table presents data for adjectival, nominal and verbal derivation taken from this corpus. Forms shown here represent those which are unattested by the Petit Robert. Where they exist, already lexicalized variants are shown in parentheses.


Adjectives


-0: grenoble (grenoblois)


-able: controversable, inexpliquable (inexplicable), raisonable (raisonnable), reconnaisable (reconnaissable)


-aire: particulaire (particulier)


-al: critical (critique), physicale (physique)


-é: distorté (distordu), entourné (entouré), révolué (révolu)


-el: externel (externe), traditionel (traditionnel)


-ent: indépendent (indépendant)


-eque: romaneque (romanesque)


-eur: aventureur (aventureux)


-eux: continueux (continuel), superficieux (superficiel), dévasteuse (dévastatrice)


-icque:féericque (féerique)


-if: conservatif (conservateur)


-istique: moralistique (moraliste)


Nouns


-age: témoinage (témoignage)


-an: écrivan (écrivain)


-ence: espérence (espérance)


-eur: écriteur (écrivain)


-ie: affinitie (affinité), poèsie (poésie)


-isme: Canadiennisme (canadianisme), romanticisme (romantisme)


-ité: faussité (fausseté)


-logie:idéalogie (idéologie), généologie (généalogie)


-ment:réfléchissement (réflexion), dévelopement (développement), divorcement (divorce)


Verbs


-er: jeuer (jouer), expériencer (avoir l’expérience de)


Table 1: Adjectival, nominal and verbal derivation in L2 corpus


As the table illustrates, the number of forms is relatively small, considering the size of the corpus (61736 word tokens). In addition, many of the forms represent orthographic variants of existing lexical items; there is little evidence of attempts to produce non­lexicalized items. This is perhaps not surprising, considering the nature of the corpus (essays produced by second and fourth year students in literature courses) which tends to favour normative language. However, it demonstrates that we must move beyond corpus data if we wish to access L2 subjects’ derivational mechanisms.


The two primary alternatives to corpus materials are elicited productions and metalinguistic judgements (see Greenbaum and Quirk 1970 for discussion). With respect to the latter, Aronoff and Schvaneveldt (1978) and later Gorska (1982) used pencil and paper tests to determine native speaker reactions to potential derived forms in English. More recently (Levison and Lessard 1995a) we produced a computational implementation of this test in which subjects were shown on a computer screen a random series of derived forms based on the prefixes non-, semi- and super- and the suffixes -less, -like and -ish as applied to English count nouns from the CUVOALD machine-readable dictionary (Mitton 1992) and required to judge whether each form was or was not acceptable. Results of the testing illustrated that overall, the subjects tested demonstrated a significant preference for forms in -like and a dispreference for forms in -ish and non-. At the same time, we observed that results were complicated by the inability to determine whether subjects in fact knew the base forms used and by the relatively imprecise nature of the question asked.


Since then, we have applied a similar but more precise methodology to L2 derivational mechanisms. In what follows, we will discuss the following aspects of the research.


  1. the methodology used for subject testing and its implementation;

  2. use of L1 corpus data as a frame of reference;

  3. use of L1 subject data as a point of comparison;

  4. L2 subject testing results.


2. Methodology and implementation


Materials for the testing included all regular first conjugation verbs from a machine­readable French dictionary (2800 verbs in all). Since the dictionary was not tagged for part of speech, all forms ending in the string -eraient were extracted from the dictionary and readjusted to produce appropriate infinitive forms. At the same time, five deverbal suffixes were selected:


-able

-age

-ment

-tion

-ure


The first suffix was chosen in part as a test, since in principle it may be applied to any transitive verb in French. We assumed that any test subject not reflecting this would be showing a significant disregard or misunderstanding of French derivation. The other four were chosen as competing nominal suffixes.


A random number generator was applied to sort the verbs into random order. Each of the five suffixes was applied to 20 verbs. No attempt was made to remove items which represented already lexicalized forms. The random number generator was applied again to produce collections of 100 verbs in which suffixes appeared in random order. Each block of 100 verbs and derived forms was stored in a separate computer file. There were 20 files in all. Since more than 20 subjects were tested, the initial 20 files were copied to produce 20 more. The following figure shows part of the contents of a typical file.


\r"paresser"|V|able|"paressable"|$12|||?|?|0.114475|0.00784326|

\r"stipuler"|V|ure|"stipulure"|$12|||?|?|0.255013|0.00793481|

\r"scandaliser"|V|tion|"scandalisation"|$12|||?|?|0.684652|0.00875881|

\r"repasser"|V|able|"repassable"|$12|||?|?|0.115085|0.0168767|

\r"enfiler"|V|age|"enfilage"|$12|||?|?|0.539018|0.0261849|

\r"bénéficier"|V|able|"bénéficiable"|$12|||?|?|0.112308|0.0297555|

\r"voisiner"|V|able|"voisinable"|$12|||?|?|0.113071|0.0479141|

\r"dételler"|V|age|"détellage"|$12|||?|?|0.541581|0.0724204|

\r"deviser"|V|age|"devisage"|$12|||?|?|0.540696|0.0743431|

\r"demander"|V|tion|"demandation"|$12|||?|?|0.686666|0.0804773|

\r"accabler"|V|age|"accablage"|$12|||?|?|0.535386|0.0885037|

\r"déballer"|V|ment|"déballement"|$12|||?|?|0.394482|0.0903348|

\r"solliciter"|V|ment|"sollicitement"|$12|||?|?|0.39671|0.105014|


Figure 1: Contents of typical test file


Note that the vertical bar functions as a field separator. Fields are, from left to right, the verb base, the part of speech, the suffix used, the derived form, the morphology rule in the VINCI formalism, two empty fields, two question marks, the random number used to sort the verbs and the random number used to sort the suffixed forms. The two question mark fields are used to capture subjects’ responses. During testing, data files are presented using the IVI/VINCI environment which includes a record editor. Subjects saw a screen such as the following:


-----------------------------------------------------------------------------------------------------------


Dans vos réponses, utilisez votre intuition. Ne passez pas beaucoup

de temps à réfléchir. Ne revenez pas sur vos réponses antérieures.


Connaissez-vous le verbe suivant? “fraterniser”


Répondez y (oui) ou n (non): ?



Pourrait-on en faire le mot suivant? “fraternisement”


Répondez y (oui) ou n (non): ?


Appuyez sur ESC puis 1 pour avancer.


-----------------------------------------------------------------------------------------------------------


Figure 2: A typical test screen


Typing esc 1 placed the cursor on each question mark in turn, allowing the subject to indicate first whether the base verb was known, and second whether the proposed derived form was seen as acceptable. Subjects indicated their responses by typing y or n which were recorded in the file for later analysis. No backtracking was allowed, but subjects were allowed as much time as they wished to complete the 100 forms. However, it was indicated to them that it was appropriate to use intuition in responding rather than an attempt to determine a rule.


3. L1 corpus data as a frame of reference


Earlier work had demonstrated that native speakers of French are capable of judging the relative frequency of lexical items with a relatively high degree of accuracy (Oléron 1966). On this basis, we postulated that it would be possible to use corpus material from French as a frame of reference within which to evaluate subjects’ attempts to characterize derivational productivity. We used as a source of data the ARTFL textbase, which is based on the corpus for the Trésor de la langue française, including literary and specialized texts. The World Wide Web version of ARTFL allows a user to extract all lexical items matching a particular string pattern, together with their absolute frequency, from some subcorpus of the entire textbase. We selected as subcorpus all texts for the period 1900 onward, which contains a total of 31,870,355 occurrences. From this, we extracted all forms matching the patterns .*able, .*age, .*ment, .*tion, .*ure.


Output of the query was in the form of alphabetically sorted lists of all forms, together with the absolute frequency of each form. This output was hand-edited to remove all forms which did not meet the following criteria: verbal base, and derived form based on one of the five suffixes. In the case of -ure, -ment, -tion, -age, only nominal derived forms were retained. In addition, forms which represented spelling errors (mostly missing accents) were also removed. This output was then used to determine the relative productivity of the various suffixes. It was decided not to use dictionary information as a point of comparison to determine whether forms showed evidence of productivity, since the treatment of suffixed forms by dictionaries is notoriously variable. Rather, we used the criterion proposed by Baayen and Lieber (1991). In brief, they propose that an appropriate measure of productivity for an affix is provided by the ratio of hapax forms to the total number of forms containing the affix. In essence, they claim that the more productive an affix, the more likely it is to give rise to hapax forms in a given corpus, whereas lexicalized forms will tend to have greater frequency of occurrence.


The following table presents the hapax forms found for each of the five suffixes in the ARTFL data, together with the total number of wordforms containing each suffix and the total of their occurrences. The last column provides the ration of hapax to total forms, in other words, the relative productivity according to the Baayen and Lieber model.


Suffix Hapax Forms Occur. Hapax/

Forms With With Total

Suffix Suffix Forms


-age 205 546 14945 0.3755

-able 223 626 25941 0.3562

-ment 310 1120 61967 0.2768

-ure 28 120 9401 0.2333

-tion 227 1250 133826 0.1816



Table 2: Hapax forms in the ARTFL textbase, 1900-1999 (31,870,355 occ.)


Note first that in terms of absolute frequency, -ure is by far the least frequent producer of hapax forms, while -tion produces the most forms, followed closely by -able and -age. In fact, because of the smaller number of total occurrences found for -age, its ratio of hapax forms to overall forms is actually higher than that of -able. This difference should however be put in perspective. As Dubois and Dubois 1971 have shown, -age appears to be losing its productivity in recent years; in fact, the examples found in the ARTFL data are often from marginal domains like agriculture (ex. binage, sarclage). As a result, the productivity of the suffix seems prone to variation according to the base forms chosen.

On the other hand, -able attaches in principle to any transitive verbal base and has almost unrestricted productivity (***ref***).


Paragraph*** on ment, tion,


4. Use of L1 subject data as a point of comparison


As an alternative to corpus data, we tested a number of native speakers of French to determine their metalinguistic evaluation of the productivity of the five suffixes. We hypothesized that if in fact subjects are capable of evaluating relative productivity, we would expect some degree of convergence among the results produced by native speakers, as well as some degree of similarity between corpus measures of productivity and metalinguistic judgements. Accordingly, we applied the tests described above to 6 native speakers of French. Subjects varied in age from 20 to 55, and in education from undergraduate university students to university professors. There were 4 females and 2 males. A detailed questionnaire was used to determine the linguistic antecedents of subjects, and only those of exclusively Francophone background were included here. (Thus, 2 subjects declaring alsatian and serbo-croatian as early first languages were excluded.) In addition, the rate of recognition of base forms was used as an additional test of language proficiency.


The following table shows the judgements of the Francophone subjects, sorted by preference for suffix.





Subject Suffix Items Known Accepted %

Verbs Derived Accepted

Forms


f21 ure 20 20 0 0.00

f21 tion 20 20 3 15.00

f21 age 20 20 6 30.00

f21 ment 20 20 8 40.00

f21 able 20 20 13 65.00

TOTAL 100 100


f22 ure 20 19 1 5.26

f22 ment 18 18 4 22.22

f22 tion 20 20 6 30.00

f22 age 20 20 7 35.00

f22 able 20 20 12 60.00

TOTAL 98 97


f19 ure 20 19 4 21.05

f19 ment 20 19 11 57.89

f19 tion 20 20 12 60.00

f19 able 20 20 18 90.00

f19 age 20 20 20 100.00

TOTAL 100 98


f23 ure 20 19 2 10.53

f23 tion 20 20 5 25.00

f23 age 20 20 8 40.00

f23 ment 20 19 8 42.10

f23 able 20 19 13 68.42

TOTAL 100 97


f20 ure 20 19 0 0.00

f20 tion 20 20 1 5.00

f20 ment 20 19 2 10.53

f20 age 20 17 4 23.53

f20 able 20 19 15 78.94

TOTAL 100 94


f24 ure 20 20 5 25.00

f24 tion 19 18 7 38.89

f24 ment 20 19 12 63.15

f24 age 20 17 14 82.35

f24 able 20 19 17 89.47

TOTAL 99 94


Table 3: L1 subjects’ evaluation of derived forms (by subject)


By and large, subjects agree in favouring forms in -able and in dispreferring forms in -ure. The other three suffixes are scattered between the two extremes. Note however two points. First, we are measuring relative rather than absolute values of acceptance. In fact, subjects vary in their mean rate of acceptance, some being more rigorous than others. Second, we do find differences with respect to individual suffixes, in particular -age. Whereas subject f20 accepts only 23.5% of forms in -age, subject f19 accepts 100%. This difference may be due to the interplay between normative and dialectal factors. Both subjects are from Montréal and have approximately the same age and education. It seems clear that -age has a high productivity in the spoken form of Montreal French. However, subject f20, with an average acceptance rate of 23.4% over the 100 forms, is far more normative than subject f19, with an average acceptance rate of 66.3%, and this appears to account for the differences in their judgements.


When results are averaged over all subjects, the order of preference is in broad agreement with ARTFL data, as the following table illustrates:


Subject Suffix Items Known Accepted %

Verbs Derived Accepted

Forms


f22 able 20 20 12 60.00

f21 able 20 20 13 65.00

f23 able 20 19 13 68.42

f20 able 20 19 15 78.94

f24 able 20 19 17 89.47

f19 able 20 20 18 90.00

MEAN 76.97

RANGE (MIN-MAX) 30.00


f20 age 20 17 4 23.53

f21 age 20 20 6 30.00

f22 age 20 20 7 35.00

f23 age 20 20 8 40.00

f24 age 20 17 14 82.35

f19 age 20 20 20 100.00

MEAN 51.81

RANGE (MIN-MAX) 76.47


f20 ment 20 19 2 10.53

f22 ment 18 18 4 22.22

f21 ment 20 20 8 40.00

f23 ment 20 19 8 42.10

f19 ment 20 19 11 57.89

f24 ment 20 19 12 63.15

MEAN 39.32

RANGE (MIN-MAX) 52.62


f20 tion 20 20 1 5.00

f21 tion 20 20 3 15.00

f23 tion 20 20 5 25.00

f22 tion 20 20 6 30.00

f24 tion 19 18 7 38.89

f19 tion 20 20 12 60.00

MEAN 29.98

RANGE (MIN-MAX) 55.00


f20 ure 20 19 0 0.00

f21 ure 20 20 0 0.00

f22 ure 20 19 1 5.26

f23 ure 20 19 2 10.53

f19 ure 20 19 4 21.05

f24 ure 20 20 5 25.00

MEAN 10.31

RANGE (MIN-MAX) 25.00


Table 4: L1 subjects’s evaluation of derived forms (by suffix)


This now provides us with a point of comparison with which to evaluate L2 data.


5. L2 subject testing results


The same methodology was applied to L2 testing. All subjects filled out a detailed linguistic questionnaire and only those having English as a native language were retained. Data collected on 3 subjects declaring themselves to be bilingual in English and French were included here. However, data from 4 subjects having a native language other than English were not included.


Initially, it was intended that subjects should be ranked based on the number of years of study of French. Exposure to French varied from a low of only several years of study at the high school level, to a high of graduate studies in French. However, there are a number of factors which make this measure inherently unstable. First, some subjects had spent time in a Francophone environment, without having advanced formal studies in French. Second, some subjects had completed their studies in French some time in the past and were potentially `rusty’. Consequently, success in recognizing base verb forms in the experiment was used to distinguish three classes of subjects.


The first class includes all subjects having at least a 90% success rate in recognizing the 100 verb forms presented. All 6 Francophone subjects fell within this class, as well as 4 of the Anglophones. However, none of the three self-declared bilinguals fell within the class (although one was at 89%).


Results for these subjects are similar to those obtained for Francophones.


Subject Suffix Items Known Accepted %

Verbs Derived Accepted

Forms


a04 ure 20 19 0 0.00

a04 tion 20 18 9 50.00

a04 age 20 20 12 60.00

a04 ment 20 20 14 70.00

a04 able 20 20 19 95.00

TOTAL 100 97


a25 ure 20 20 4 20.00

a25 tion 20 20 7 35.00

a25 ment 20 20 8 40.00

a25 age 20 19 8 42.11

a25 able 20 18 13 72.22

TOTAL 100 97


a15 ure 20 19 1 5.26

a15 tion 20 20 3 15.00

a15 age 20 19 7 36.84

a15 ment 20 18 9 50.00

a15 able 20 19 11 57.89

TOTAL 100 95


a14 ure 20 19 2 10.53

a14 tion 20 18 4 22.22

a14 ment 20 18 8 44.44

a14 age 20 18 9 50.00

a14 able 20 18 12 66.66

TOTAL 100 91


The second class included all subjects recognizing 70-89% of the verb forms. This group included 8 Anglophones and 3 bilinguals, but no Francophones. Results from this group are still similar to those shown by Francophones. However, in some instances (a07, b10, a30, for example), differences between values are too small to be significant. Toward the bottom of the scale, we begin to observe some discrepancies; thus a05 prefers -ure to both -age and -ment, although the difference is small.


Subject Suffix Items Known Accepted %

Verbs Derived Accepted

Forms


b12 age 19 17 3 17.65

b12 ure 20 16 4 25.00

b12 ment 20 18 5 27.78

b12 able 19 18 7 38.89

b12 tion 20 20 8 40.00

TOTAL 98 89


a07 ure 20 18 1 5.56

a07 tion 20 18 7 38.89

a07 ment 20 18 8 44.44

a07 age 20 17 8 47.06

a07 able 20 17 14 82.35

TOTAL 100 88


b10 ure 20 19 3 15.79

b10 ment 20 17 5 29.41

b10 tion 20 16 5 31.25

b10 age 20 16 6 37.50

b10 able 20 18 16 88.89

TOTAL 100 86


a17 age 18 15 2 13.33

a17 ure 20 17 4 23.53

a17 ment 19 18 8 44.44

a17 tion 20 18 9 50.00

a17 able 19 16 12 75.00

TOTAL 96 84


a06 ure 20 17 1 5.88

a06 ment 20 16 5 31.25

a06 tion 20 15 6 40.00

a06 age 20 14 8 57.14

a06 able 20 18 18 99.99

TOTAL 100 80


a30 ure 20 15 3 20.00

a30 tion 20 16 10 62.50

a30 ment 20 16 11 68.75

a30 age 20 15 11 73.33

a30 able 20 18 14 77.78

TOTAL 100 80


a28 ure 20 16 0 0.00

a28 ment 20 17 3 17.65

a28 age 20 16 5 31.25

a28 tion 20 13 10 76.92

a28 able 20 15 14 93.33

TOTAL 100 77


b08 ure 20 17 4 23.53

b08 age 20 14 4 28.57

b08 ment 20 16 7 43.75

b08 tion 20 13 8 61.54

b08 able 20 15 13 86.67

TOTAL 100 75


a09 ure 20 16 3 18.75

a09 ment 20 15 4 26.66

a09 age 20 15 10 66.66

a09 tion 20 14 10 71.42

a09 able 20 15 12 79.99

TOTAL 100 75


a05 age 20 16 3 18.75

a05 ment 20 17 5 29.41

a05 ure 20 15 7 46.66

a05 tion 20 13 8 61.53

a05 able 19 13 9 69.23

TOTAL 99 74


a02 ure 20 16 2 12.50

a02 age 20 14 5 35.71

a02 tion 20 15 8 53.33

a02 ment 20 15 9 60.00

a02 able 20 13 8 61.53

TOTAL 100 73


The final class included all subjects scoring under 70% in verb recognition. It included 4 subjects, including 3 (a26, a29, a27) with only high school training in French. It is in the final three subjects that we begin to see a significant breakdown of the system, with the order of suffixes being radically altered. The case of subject a03 is more complex. This subject completed the test in less than half the average time. It may be that this factor should be controlled in future.


Subject Suffix Items Known Accepted %

Verbs Derived Accepted

Forms


a03 tion 20 15 5 33.33

a03 ure 20 12 4 33.33

a03 ment 20 17 11 64.70

a03 age 20 13 11 84.61

a03 able 19 11 10 90.90

TOTAL 99 68


a26 age 20 11 5 45.45

a26 tion 20 12 7 58.33

a26 able 20 16 10 62.50

a26 ure 20 14 9 64.29

a26 ment 20 14 11 78.57

TOTAL 100 67


a29 ure 20 11 0 0.00

a29 able 20 9 2 22.22

a29 tion 20 9 3 33.33

a29 age 20 9 4 66.67

a29 ment 20 9 9 100.00

TOTAL 100 47


a27 tion 20 8 1 12.50

a27 ment 20 6 2 33.33

a27 ure 20 12 8 66.67

a27 age 20 8 6 75.00

a27 able 20 7 6 85.71

TOTAL 100 41


6. Conclusions


Although the results presented here should be seen as preliminary, given the relatively small number of subjects tested, it is possible to draw a number of tentative conclusions.


First, the testing methodology and its implementation appear to be fairly robust. They lend themselves to easy use by experimenters and subjects, both locally and potentially for remote testing, and the data obtained is in a format which allows easy manipulation by statistical and search software.


Second, the use of a double question based on recognition of the base form and then on acceptance of the derived form appears to have removed at least some of the ambiguity we found in our earlier testing. In addition, the rate of recognition of base forms appears to provide a better measure of linguistic (or at least lexical) competence than the linguistic questionnaires used.


Third, we found that Anglophone L2 speakers of French appear to develop very early an accurate subjective image of the relative productivity of suffixes. In fact, only the weakest of the subjects showed results which differed radically from L1 data. In fact, however, this conclusion is subject to two caveats. On the one hand, we have not measured the degree of agreement between L1 and L2 subjects on the same lexical items. On the other, we have not succeeded in removing the potentially confounding factor of the knowledge of English suffixation. Both will require significant additional work.


Finally, we saw at the outset that corpus materials have not provided a rich source of data for evaluating L2 linguistic creativity. In the framework proposed by Greenbaum and Quirk 1970, we have explored L2 subjects’ metalinguistic judgements, but there still remains the problem of their linguistic performance. The next step in the project will involve using the VINCI generation environment to dynamically produce cloze and other production exercises in an attempt to elicit L1 and L2 production in the area of derivation. (See Levison and Lessard 1995b for details of the formalism and software.)


Notes


1. This research was funded by a Social Sciences and Humanities Research Council of Canada grant to the two authors. We would also like to thank our research assistant Isabelle Gélineau, who performed some of the subject testing, as well as the various subjects themselves.



References


Aronoff, M. and R. Schvaneveldt (1978), Testing morphological productivity, Annals of the New York Academy of Sciences 318: 106-114.


Baayen, H. and R. Lieber (1991), Productivity and English derivation: a corpus-based study, Linguistics 29: 801-843.


Dubois, J., Cl. Dubois (1971) Introduction à la lexicographie, le dictionnaire. Paris: Larousse.


Greenbaum, S., R. Quirk (1970) Elicitation Experiments in English: Linguistic Studies in Use and Attitude. London: Longman.


Lessard, G., M. Levison, D. Maher, I.V. Tomek (1995) Modelling Second Language Learner Creativity. Journal of Artificial Intelligence in Education 5:4:455-480.


Levison, M., G. Lessard (1995a) Experiments in Word Creation. ACH/ALLC, Santa Barbara. Conference extended abstracts, pp. 74-77.


Levison, M., G. Lessard (1995b) New Words from Old: A Formalism for Word Formation. Computing and the Humanities 29:463-479.


Gorska, E. (1982), A way of testing the productivity of word formation rules (WFRs)?, Studia Anglica Posaniensa 14/1: 169-174.


Mitton, R. (1992), A description of a computer-usable dictionary file based on the Oxford Advanced Learner's dictionary of current English. Computer-readable file, Oxford Text Archive (ota.ox.ac.uk).


Oléron, P. (1966) Estimation de mots français sur des échelles de fréquence et d’abstraction. Bulletin de psychologie 247, XIX, 603-610.