Package opennlp.tools.util.normalizer

# Class InvisibleCharSequenceNormalizer

java.lang.Object
opennlp.tools.util.normalizer.InvisibleCharSequenceNormalizer

All Implemented Interfaces:
  `Serializable`, `opennlp.tools.util.normalizer.CharSequenceNormalizer`, `opennlp.tools.util.normalizer.OffsetAwareNormalizer`

public class InvisibleCharSequenceNormalizer extends Object implements opennlp.tools.util.normalizer.OffsetAwareNormalizer
A `CharSequenceNormalizer` that removes invisible format and bidirectional control characters that add no textual content and are a common source of noise and spoofing (the byte-order mark, zero width space, word joiner, bidi marks/embeddings/overrides/isolates, the invisible math operators, soft hyphen, and the Arabic letter mark).

Membership is an O(1) `CharClass` lookup and removal is a single cursor pass with no regular expression. The zero width joiner (`U+200D`) and non-joiner (`U+200C`) are deliberately kept, because they carry meaning in Persian, Indic scripts, and emoji sequences; so are variation selectors. Use this only for a matching/search form, not for display.

See Also:
  - Serialized Form

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `InvisibleCharSequenceNormalizer()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `static InvisibleCharSequenceNormalizer`
  `getInstance()`
  Returns the shared, stateless instance.
  `CharSequence`
  `normalize(CharSequence text)`
  `opennlp.tools.util.normalizer.AlignedText`
  `normalizeAligned(CharSequence text)`

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### InvisibleCharSequenceNormalizer

    public InvisibleCharSequenceNormalizer()

- ## Method Details

  - ### getInstance

    public static InvisibleCharSequenceNormalizer getInstance()
    Returns the shared, stateless instance.
    Returns:
      the shared, stateless instance

  - ### normalize

    public CharSequence normalize(CharSequence text)
    Specified by:
      `normalize` in interface `opennlp.tools.util.normalizer.CharSequenceNormalizer`

  - ### normalizeAligned

    public opennlp.tools.util.normalizer.AlignedText normalizeAligned(CharSequence text)
    Specified by:
      `normalizeAligned` in interface `opennlp.tools.util.normalizer.OffsetAwareNormalizer`