Package opennlp.tools.util.normalizer

# Class AccentFoldCharSequenceNormalizer

java.lang.Object
opennlp.tools.util.normalizer.AccentFoldCharSequenceNormalizer

All Implemented Interfaces:
  `Serializable`, `opennlp.tools.util.normalizer.CharSequenceNormalizer`

public class AccentFoldCharSequenceNormalizer extends Object implements opennlp.tools.util.normalizer.CharSequenceNormalizer
A `CharSequenceNormalizer` that folds diacritics for matching, the multilingual-safe counterpart to a Latin-only ASCII folding filter.

Folding decomposes the text (NFD) and drops nonspacing combining marks, but only for base characters whose script is in `foldScripts` (Latin, Greek, and Cyrillic by default). Marks on other scripts are left untouched, because there they are essential orthography rather than decoration: stripping an Indic vowel sign or a virama, an Arabic harakat, a Hebrew point, or a Thai vowel changes the word. This script gating is the key correctness rule; never strip all nonspacing marks globally.

Many "accented" Latin letters are atomic and do not decompose (`o` with stroke, the `ae`/`oe` ligatures, eszett, thorn, and so on). When `foldStrokeLetters` is enabled (the default) these are mapped to an ASCII approximation. Folding is a recall optimization, not a linguistically correct transform, so it is intended for a matching token rather than for display or language-specific analysis.

Scanning is a single cursor pass over the decomposed text; no regular expression is used, and no global `\p{Mn}` strip is performed.

See Also:
  - Serialized Form

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `AccentFoldCharSequenceNormalizer(Set<Character.UnicodeScript> foldScripts, boolean foldStrokeLetters)`
  Creates an accent-folding normalizer.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `static AccentFoldCharSequenceNormalizer`
  `getInstance()`
  Returns the shared instance with the safe defaults: Latin, Greek, and Cyrillic plus the stroke-letter map.
  `CharSequence`
  `normalize(CharSequence text)`

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### AccentFoldCharSequenceNormalizer

    public AccentFoldCharSequenceNormalizer(Set<Character.UnicodeScript> foldScripts, boolean foldStrokeLetters)
    Creates an accent-folding normalizer.
    Parameters:
      `foldScripts` - The scripts whose base characters' diacritics are folded; marks on every other script are preserved.
      `foldStrokeLetters` - Whether atomic Latin letters such as the stroke letters and ligatures are mapped to an ASCII approximation.

- ## Method Details

  - ### getInstance

    public static AccentFoldCharSequenceNormalizer getInstance()
    Returns the shared instance with the safe defaults: Latin, Greek, and Cyrillic plus the stroke-letter map.
    Returns:
      the shared instance with the safe defaults: Latin, Greek, and Cyrillic plus the stroke-letter map

  - ### normalize

    public CharSequence normalize(CharSequence text)
    Specified by:
      `normalize` in interface `opennlp.tools.util.normalizer.CharSequenceNormalizer`