Romance language stemmers


 

Links to resources

Snowball main page
French stemmer
Spanish stemmer
Portuguese stemmer
Italian stemmer


The Romance languages have a wealth of different i-suffixes (*) among the verb forms, and relatively few for the other parts of speech. In addition to this, many verbs exhibit irregularities. Many also have short stems, leading to dangers of over-stemming. The verb, therefore, tends to dominate initial thinking about stemming in these languages.

An algorithmic stemmer can usually reduce the multiple forms of a verb to at most two or three, and often just one. This is probably adequate for standard IR use, where the verb is used rather less than other parts of speech in short queries.

In French the verb endings ent and ons cannot be removed without unacceptable overstemming. The ons form is rarer, but ent forms are quite common, and will appear regularly throughout a stemmed vocabulary.

In Italian, the final vowel of nouns and adjectives indicates number and gender (amico is male friend, amica is female friend) and its removal is a necessary part of stemming, but the final vowel sometimes separates words of different meanings (banco is bench, banca is bank), which leads to some over-stemming.

The d-suffixes of all four languages follow a similar pattern. They can be tabulated as follows,
French Spanish Portug. Italian
noun ANCE ance anza eza anza
adjective IC ique ico ico ico
noun ISM isme ismo ismo ismo
adjective ABLE able able ável abile
adjective IBLE - ible ível ibile
noun IST iste ista ista ista
adjective OUS eux oso oso oso
noun MENT ment amiento amento mente
noun ATOR ateur ador ador attore
noun ATRESS atrice - - atrice
noun ATION ation ación ação azione
noun LOGY logie logía logía logia
noun USION usion ución ución uzione
noun ENCE ence encia ência enza
adjective ENT ent ente ente ente
noun ANCE ance ancia ância anza
noun ANT ant ante ante ante
adverb LY (e)ment (a)mente (a)mente (a)mente
noun ITY ité idad idade ità
adjective IVE if ive ivo ivo
verb ATE at at at at
Equivalent English forms are shown in upper case. In English, ATE is a valid ending, but in the Romance languages it only exists in combinations. The endings can appear in a number of styles. In Italian, oso can also be osa, osi or ose, French ique becomes ic in combinations.

The important combining forms are summarised in the following picture:
                               OUS

                                |
                                |
                                v

         .----------------->    LY   <---------  ANT
         |                           <-----.
         |                      ^          |
         |                      |          |
         |                      |          |

        IVE   <--   ATE   <--   IC        ABLE

         |                      |          |
         |                      |          |
         |                      v          |
         |                                 |
         `----------------->   ITY   <-----'

In English, ABLE combines with LY to form ABLY. So in French, for example, able combines with (e)ment to form ablement. In some languages particular combinations are rare. In Italian, for example, ANT + LY, which would be the ending antemente, is so rare that it does not figure in the stemming algorithm. According to the picture, we should encounter the forms ICATIVELY and ICATIVITY, and dictionaries instance a few English words with these endings (communicatively for example). But in practice three is the maximum number of derivational suffixes that one need consider in combination.