Hi Gerd My description didn't quite mean what I hoped it did - sorry. I was thinking that there would be a single attempt at encoding the whole string, and if that fails, start again but char-by-char. But, assuming result.length() works and charBuffer.get() and outBuff.put() maintain positions used by main encoder, within the loop the failed component needs to be processed input char-by-char, transliterated (if no change replaced by "?") and encoded with another encoder. Any variable length nature of the output charset shouldn't be a problem. The variable length input UTF-16 will need care. Ticker On Wed, 2021-11-17 at 11:16 +0000, Gerd Petermann wrote:
Hi Ticker,
remember that cs932 is a double-byte character set. With your code only a few unmappable utf-16 characters are replaced, for the rest one of cs932 is used, but without any good reason. The result is typically garbage.
I've modified the patch to replace any unmappable character that was not transliterated by '?' . I've also attached a debug version that shows what goes on. A possible change in SparseTransliterator would be to add a mapping for the MATH MINUS, the other FULLWIDTH digits are supported in cs932.
Gerd