Xiao,
I am glad you sorted it out.
I will need to return to Snowball at some point, when I have extended some
other software to handle Unicode, so that might be a good time to address
the UTF8 encoding business. I do not think it will be until next year however.
It would be possible to put UTF-8 Unicode straight in, if Snowball had not
introduced these things called 'classes'. For example, the line
define v 'aeiou{a'}{e'}{i'}{o'}{u'}{u"}'
defines v as one of the bytes in the succeeding string. If instead one had
define v as among ('a' 'e' 'i' 'o' 'u'
'{a'}' '{e'}' '{i'}' '{o'}' '{u'}' '{u"}')
it would not matter whether the macros {a'} etc were made up of 1, 2 or 3
bytes. The Snowball scripts could be rewritten to avoid the use of classes.
Retrospectively, one can say that the idea of character classes is a
mistake: the increased speed by which they are implemented should be made an
optimisation feature of among expressions of a certain shape.
Martin
> Martin,
>
>Thanks for your help, I can compile the snowball for stemming UCS2-based
russian text.
>
>It is better that snowball can stem UTF8-based text, Could you have any
plan to modify the snowball to support stemming UTF8-based text?
>
>Xiao Shibin
>TRS Ltd.
>
This archive was generated by hypermail 2.1.3 : Thu Sep 20 2007 - 12:02:46 BST