Package opennlp.tools.util.normalizer

# Class CharClass

java.lang.Object
opennlp.tools.util.normalizer.CharClass

public final class CharClass extends Object
A configurable class of Unicode code points and the cursor based operations over it.

A `CharClass` pairs a `CodePointSet` of member code points with a single canonical ASCII `replacement` code point. Whitespace and dashes are the two built-in presets (`whitespace()`, `dashes()`); any other class is one more configured instance with no new engine code.

Every operation is a single forward cursor pass over the text: no regular expression and no per-call allocation beyond the result.

Instances are immutable and thread-safe.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `String`
  `collapse(CharSequence text)`
  Collapses each maximal run of member code points to a single replacement.
  `AlignedText`
  `collapseAligned(CharSequence text)`
  Like `collapse(CharSequence)` but also produces the `Alignment` back to the original text.
  `String`
  `collapsePreserving(CharSequence text, CodePointSet keep, int keepReplacement)`
  Collapses runs of members like `collapse(CharSequence)`, but emits `keepReplacement` instead of the usual replacement for any run that contains a code point in `keep`.
  `AlignedText`
  `collapsePreservingAligned(CharSequence text, CodePointSet keep, int keepReplacement)`
  Like `collapsePreserving(CharSequence, CodePointSet, int)` but also produces the `Alignment` back to the original text.
  `boolean`
  `contains(int codePoint)`
  Tests membership.
  `static CharClass`
  `dashes()`
  Returns the dash preset: the Unicode `Dash` set excluding the mathematical minus signs, replacement `U+002D`.
  `CodePointSet`
  `members()`
  Returns the member code points of this class.
  `String`
  `normalize(CharSequence text)`
  Replaces each member code point with the replacement, one for one.
  `AlignedText`
  `normalizeAligned(CharSequence text)`
  Like `normalize(CharSequence)` but also produces the `Alignment` back to the original text.
  `static CharClass`
  `of(CodePointSet members, int replacement)`
  Creates a class from a member set and a replacement code point.
  `String`
  `removeAll(CharSequence text)`
  Removes every member code point.
  `AlignedText`
  `removeAllAligned(CharSequence text)`
  Like `removeAll(CharSequence)` but also produces the `Alignment` back to the original text.
  `int`
  `replacement()`
  Returns the canonical replacement code point.
  `String[]`
  `split(CharSequence text)`
  Splits text into the maximal runs of non-member code points.
  `List<Span>`
  `splitSpans(CharSequence text)`
  Splits text into the maximal runs of non-member code points, as character spans into the original text.
  `static String`
  `substitute(CharSequence text, IntFunction<String> substitution)`
  Applies a per-code-point substitution: each code point for which `substitution` returns a non-null string is replaced by that string, and the rest are copied through.
  `static AlignedText`
  `substituteAligned(CharSequence text, IntFunction<String> substitution)`
  Like `substitute(CharSequence, IntFunction)` but also produces the `Alignment` back to the original text.
  `String`
  `trim(CharSequence text)`
  Removes leading and trailing member code points.
  `AlignedText`
  `trimAligned(CharSequence text)`
  Like `trim(CharSequence)` but also produces the `Alignment` back to the original text.
  `static CharClass`
  `whitespace()`
  Returns the whitespace preset: the Unicode `White_Space` set, replacement `U+0020`.
  `CharClass`
  `withAdditional(CodePointSet extra)`
  Returns a copy of this class whose member set is extended with `extra` (for example, user-defined code points loaded from `CodePointSet.fromFile(java.nio.file.Path, java.lang.String)`).

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Method Details

  - ### of

    public static CharClass of(CodePointSet members, int replacement)
    Creates a class from a member set and a replacement code point.
    Parameters:
      `members` - The member code points.
      `replacement` - The canonical code point used by `normalize(CharSequence)` and `collapse(CharSequence)`.
    Returns:
      The class.
    Throws:
      `IllegalArgumentException` - Thrown if `members` is `null` or `replacement` is not a valid code point.

  - ### whitespace

    public static CharClass whitespace()
    Returns the whitespace preset: the Unicode `White_Space` set, replacement `U+0020`.
    Returns:
      the whitespace preset: the Unicode `White_Space` set, replacement `U+0020`

  - ### dashes

    public static CharClass dashes()
    Returns the dash preset: the Unicode `Dash` set excluding the mathematical minus signs, replacement `U+002D`.
    Returns:
      the dash preset: the Unicode `Dash` set excluding the mathematical minus signs, replacement `U+002D`

  - ### withAdditional

    public CharClass withAdditional(CodePointSet extra)
    Returns a copy of this class whose member set is extended with `extra` (for example, user-defined code points loaded from `CodePointSet.fromFile(java.nio.file.Path, java.lang.String)`).
    Parameters:
      `extra` - The additional member code points.
    Returns:
      A new `CharClass`; this instance is unchanged.
    Throws:
      `IllegalArgumentException` - Thrown if `extra` is `null`.

  - ### members

    public CodePointSet members()
    Returns the member code points of this class.
    Returns:
      the member code points of this class

  - ### replacement

    public int replacement()
    Returns the canonical replacement code point.
    Returns:
      the canonical replacement code point

  - ### contains

    public boolean contains(int codePoint)
    Tests membership.
    Parameters:
      `codePoint` - The code point to test.
    Returns:
      `true` if the code point is a member of this class.

  - ### splitSpans

    public List<Span> splitSpans(CharSequence text)
    Splits text into the maximal runs of non-member code points, as character spans into the original text. Runs of members are delimiters and produce no empty spans.
    Parameters:
      `text` - The text to split.
    Returns:
      The token spans, in order.
    Throws:
      `IllegalArgumentException` - Thrown if `text` is `null`.

  - ### split

    public String[] split(CharSequence text)
    Splits text into the maximal runs of non-member code points.
    Parameters:
      `text` - The text to split.
    Returns:
      The tokens, in order, with no empty entries.
    Throws:
      `IllegalArgumentException` - Thrown if `text` is `null`.

  - ### normalize

    public String normalize(CharSequence text)
    Replaces each member code point with the replacement, one for one.

    When no code point of `text` is a member, the text is returned unchanged (as its `string form`) without copying.

    Parameters:
      `text` - The text to normalize.
    Returns:
      The normalized text.
    Throws:
      `IllegalArgumentException` - Thrown if `text` is `null`.

  - ### collapse

    public String collapse(CharSequence text)
    Collapses each maximal run of member code points to a single replacement.

    Edges are not trimmed: a leading or trailing run becomes a single replacement, and a string made up entirely of members collapses to one replacement (for whitespace, a single space, not the empty string). Use `trim(CharSequence)` to drop edge members, or collapse and then trim to do both.

    When no code point of `text` is a member, the text is returned unchanged (as its `string form`) without copying.

    Parameters:
      `text` - The text to collapse.
    Returns:
      The collapsed text.
    Throws:
      `IllegalArgumentException` - Thrown if `text` is `null`.

  - ### collapsePreserving

    public String collapsePreserving(CharSequence text, CodePointSet keep, int keepReplacement)
    Collapses runs of members like `collapse(CharSequence)`, but emits `keepReplacement` instead of the usual replacement for any run that contains a code point in `keep`. The whitespace "squish" that preserves a line break uses this with the line-break code points as `keep` and `'\n'` as `keepReplacement`.
    Parameters:
      `text` - The text to collapse.
      `keep` - The member code points whose presence in a run preserves structure.
      `keepReplacement` - The replacement emitted for a run that contains a `keep` member.
    Returns:
      The collapsed text.
    Throws:
      `IllegalArgumentException` - Thrown if `text` or `keep` is `null`, or `keepReplacement` is not a valid code point.

  - ### trim

    public String trim(CharSequence text)
    Removes leading and trailing member code points.
    Parameters:
      `text` - The text to trim.
    Returns:
      The trimmed text.
    Throws:
      `IllegalArgumentException` - Thrown if `text` is `null`.

  - ### removeAll

    public String removeAll(CharSequence text)
    Removes every member code point.

    When no code point of `text` is a member, the text is returned unchanged (as its `string form`) without copying.

    Parameters:
      `text` - The text to filter.
    Returns:
      The text with all members removed.
    Throws:
      `IllegalArgumentException` - Thrown if `text` is `null`.

  - ### normalizeAligned

    public AlignedText normalizeAligned(CharSequence text)
    Like `normalize(CharSequence)` but also produces the `Alignment` back to the original text.
    Parameters:
      `text` - The text to normalize.
    Returns:
      The normalized text and its alignment.
    Throws:
      `IllegalArgumentException` - Thrown if `text` is `null`.

  - ### collapseAligned

    public AlignedText collapseAligned(CharSequence text)
    Like `collapse(CharSequence)` but also produces the `Alignment` back to the original text. Each collapsed run maps to the run's whole original extent.
    Parameters:
      `text` - The text to collapse.
    Returns:
      The collapsed text and its alignment.
    Throws:
      `IllegalArgumentException` - Thrown if `text` is `null`.

  - ### collapsePreservingAligned

    public AlignedText collapsePreservingAligned(CharSequence text, CodePointSet keep, int keepReplacement)
    Like `collapsePreserving(CharSequence, CodePointSet, int)` but also produces the `Alignment` back to the original text.
    Parameters:
      `text` - The text to collapse.
      `keep` - The member code points whose presence in a run preserves structure.
      `keepReplacement` - The replacement emitted for a run that contains a `keep` member.
    Returns:
      The collapsed text and its alignment.
    Throws:
      `IllegalArgumentException` - Thrown if `text` or `keep` is `null`, or `keepReplacement` is not a valid code point.

  - ### trimAligned

    public AlignedText trimAligned(CharSequence text)
    Like `trim(CharSequence)` but also produces the `Alignment` back to the original text. The trimmed leading and trailing members appear as deletions, so a span never reports through them.
    Parameters:
      `text` - The text to trim.
    Returns:
      The trimmed text and its alignment.
    Throws:
      `IllegalArgumentException` - Thrown if `text` is `null`.

  - ### removeAllAligned

    public AlignedText removeAllAligned(CharSequence text)
    Like `removeAll(CharSequence)` but also produces the `Alignment` back to the original text. Every removed member appears as a deletion, so a span never reports through one.
    Parameters:
      `text` - The text to filter.
    Returns:
      The filtered text and its alignment.
    Throws:
      `IllegalArgumentException` - Thrown if `text` is `null`.

  - ### substitute

    public static String substitute(CharSequence text, IntFunction<String> substitution)
    Applies a per-code-point substitution: each code point for which `substitution` returns a non-null string is replaced by that string, and the rest are copied through. This is the shared cursor pass behind the expanding folds, with no regular expression.

    When `substitution` returns `null` for every code point of `text`, the text is returned unchanged (as its `string form`) without copying. The mapper is still applied exactly once per code point.

    Parameters:
      `text` - The text to transform.
      `substitution` - The replacement for a code point, or `null` to copy it through.
    Returns:
      The transformed text.
    Throws:
      `IllegalArgumentException` - Thrown if `text` or `substitution` is `null`.

  - ### substituteAligned

    public static AlignedText substituteAligned(CharSequence text, IntFunction<String> substitution)
    Like `substitute(CharSequence, IntFunction)` but also produces the `Alignment` back to the original text. Each replaced code point maps to its replacement string as one block.
    Parameters:
      `text` - The text to transform.
      `substitution` - The replacement for a code point, or `null` to copy it through.
    Returns:
      The transformed text and its alignment.
    Throws:
      `IllegalArgumentException` - Thrown if `text` or `substitution` is `null`.