Package opennlp.spellcheck.normalizer

# Class SpellCheckingCharSequenceNormalizer

java.lang.Object
opennlp.spellcheck.normalizer.SpellCheckingCharSequenceNormalizer

All Implemented Interfaces:
  `Serializable`, `opennlp.tools.util.normalizer.CharSequenceNormalizer`

public class SpellCheckingCharSequenceNormalizer extends Object implements opennlp.tools.util.normalizer.CharSequenceNormalizer
A `CharSequenceNormalizer` that corrects spelling in text using a `SpellChecker` (typically a SymSpell engine).

The normalizer works in one of two modes:

- `PER_TOKEN` (default) – the input is split into whitespace-delimited tokens (whitespace being the Unicode `White_Space` set, `StringUtil.isUnicodeWhitespace(int)`) and each token is corrected independently with `SpellChecker.lookup(java.lang.String, opennlp.spellcheck.Verbosity, int)`. The original whitespace runs between tokens are preserved verbatim, so the shape of the line is kept. Tokens the dictionary already contains (best suggestion at edit distance `0`) are left untouched.
- `COMPOUND` – the whole input is passed to `SpellChecker.lookupCompound(java.lang.String, int)`, which additionally repairs wrongly inserted or omitted spaces (word splits and merges). This collapses runs of whitespace to single spaces, as the compound corrector re-tokenizes the input.

Several guards, all configurable through the `SpellCheckingCharSequenceNormalizer.Builder`, keep the corrector from "fixing" tokens that should be left as they are: tokens shorter than `minTokenLength` are skipped, numeric tokens are skipped when `skipNumbers` is set (on by default), URL- and email-like tokens are skipped when `skipUrls` is set (on by default), and a token whose lower-cased form is already in the dictionary is never changed.

Casing. Lookups are performed on the lower-cased token and the original casing pattern is re-applied to the correction: an all-upper token yields an all-upper correction, a leading-capital token yields a leading-capital correction, otherwise the suggestion's own casing is used. When no correction applies, the original token is emitted unchanged.

This normalizer composes cleanly inside an `AggregateCharSequenceNormalizer`; place it after noise-removing normalizers (URL, emoji, shrink) so it sees clean tokens.

Serialization. The settings fields are serialized, but the backing `SpellChecker` is held in a `transient` field and is `null` after Java deserialization; re-attach a checker with `withSpellChecker(SpellChecker)` to obtain a working copy with the same settings. Calling `normalize(java.lang.CharSequence)` on an instance with no checker throws `IllegalStateException`.

See Also:
  - Serialized Form

- ## Nested Class Summary

  Nested Classes
  Modifier and Type
  Class
  Description
  `static final class`
  `SpellCheckingCharSequenceNormalizer.Builder`
  A mutable builder for `SpellCheckingCharSequenceNormalizer`.
  `static enum`
  `SpellCheckingCharSequenceNormalizer.Mode`
  The correction mode.

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final int`
  `DEFAULT_MIN_TOKEN_LENGTH`
  The default minimum token length below which tokens are left untouched.

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `SpellCheckingCharSequenceNormalizer(SymSpellModel model)`
  Creates a normalizer in `SpellCheckingCharSequenceNormalizer.Mode.PER_TOKEN` mode with default guards from a loaded `SymSpellModel` (uses the model's `engine`).
  `SpellCheckingCharSequenceNormalizer(SpellChecker spellChecker)`
  Creates a normalizer in `SpellCheckingCharSequenceNormalizer.Mode.PER_TOKEN` mode with default guards from a `SpellChecker`.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `static SpellCheckingCharSequenceNormalizer.Builder`
  `builder(SymSpellModel model)`
  `static SpellCheckingCharSequenceNormalizer.Builder`
  `builder(SpellChecker spellChecker)`
  `CharSequence`
  `normalize(CharSequence text)`
  `SpellCheckingCharSequenceNormalizer`
  `withSpellChecker(SpellChecker checker)`
  Returns a copy of this normalizer carrying the same settings but backed by the given checker.

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### DEFAULT_MIN_TOKEN_LENGTH

    public static final int DEFAULT_MIN_TOKEN_LENGTH
    The default minimum token length below which tokens are left untouched.
    See Also:
      - Constant Field Values

- ## Constructor Details

  - ### SpellCheckingCharSequenceNormalizer

    public SpellCheckingCharSequenceNormalizer(SpellChecker spellChecker)
    Creates a normalizer in `SpellCheckingCharSequenceNormalizer.Mode.PER_TOKEN` mode with default guards from a `SpellChecker`.
    Parameters:
      `spellChecker` - the engine used to correct tokens; must not be `null`
    Throws:
      `IllegalArgumentException` - if `spellChecker` is `null`

  - ### SpellCheckingCharSequenceNormalizer

    public SpellCheckingCharSequenceNormalizer(SymSpellModel model)
    Creates a normalizer in `SpellCheckingCharSequenceNormalizer.Mode.PER_TOKEN` mode with default guards from a loaded `SymSpellModel` (uses the model's `engine`).
    Parameters:
      `model` - the loaded model whose engine is used; must not be `null`
    Throws:
      `IllegalArgumentException` - if `model` is `null`

- ## Method Details

  - ### builder

    public static SpellCheckingCharSequenceNormalizer.Builder builder(SpellChecker spellChecker)
    Parameters:
      `spellChecker` - the engine to wrap; must not be `null`
    Returns:
      a new `SpellCheckingCharSequenceNormalizer.Builder` seeded with sensible defaults
    Throws:
      `IllegalArgumentException` - if `spellChecker` is `null`

  - ### builder

    public static SpellCheckingCharSequenceNormalizer.Builder builder(SymSpellModel model)
    Parameters:
      `model` - the loaded model whose engine to wrap; must not be `null`
    Returns:
      a new `SpellCheckingCharSequenceNormalizer.Builder` seeded with sensible defaults
    Throws:
      `IllegalArgumentException` - if `model` is `null`

  - ### withSpellChecker

    public SpellCheckingCharSequenceNormalizer withSpellChecker(SpellChecker checker)
    Returns a copy of this normalizer carrying the same settings but backed by the given checker. This is the supported way to re-attach an engine to an instance restored by Java deserialization (whose `transient` checker is `null`).
    Parameters:
      `checker` - the engine to attach; must not be `null`
    Returns:
      a new, ready-to-use normalizer with this instance's settings
    Throws:
      `IllegalArgumentException` - if `checker` is `null`

  - ### normalize

    public CharSequence normalize(CharSequence text)
    Specified by:
      `normalize` in interface `opennlp.tools.util.normalizer.CharSequenceNormalizer`