Package opennlp.tools.util

# Class StringUtil

java.lang.Object
opennlp.tools.util.StringUtil

public class StringUtil extends Object

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final List<String>`
  `ASCII_DIGIT_STRINGS`
  The ten ASCII digit strings `"0"` to `"9"`, indexed by digit value.

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `StringUtil()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `static void`
  `computeShortestEditScript(String wordForm, String lemma, int[][] distance, StringBuffer permutations)`
  Computes the Shortest Edit Script (SES) to convert a word into its lemma.
  `static String`
  `decodeShortestEditScript(String wordForm, String permutations)`
  Reads the predicted Shortest Edit Script (SES) by a lemmatizer model and applies the permutations to obtain the lemma from the `wordForm`.
  `static String`
  `getShortestEditScript(String wordForm, String lemma)`
  `static boolean`
  `isEmpty(CharSequence theString)`
  `static boolean`
  `isUnicodeBlank(CharSequence input)`
  `true` when `input` is `null`, empty, or consists only of Unicode `White_Space` code points.
  `static boolean`
  `isUnicodeWhitespace(char charCode)`
  Determines if the specified `Character` is a whitespace under the Unicode `White_Space` property; delegates to `UnicodeWhitespace.isWhitespace(int)`.
  `static boolean`
  `isUnicodeWhitespace(int charCode)`
  Determines if the specified code point is a whitespace under the Unicode `White_Space` property; delegates to `UnicodeWhitespace.isWhitespace(int)`.
  `static boolean`
  `isWhitespace(char charCode)`
  Determines if the specified `Character` is a whitespace under the active `WhitespaceMode`: the Unicode `White_Space` property by default, or, when the "opennlp.whitespace.mode" system property selects `WhitespaceMode.LEGACY`, the union of `Character.isWhitespace(int)` and the Unicode `Zs` category (`Character.SPACE_SEPARATOR`) that OpenNLP 1.x and 2.x used.
  `static boolean`
  `isWhitespace(int charCode)`
  Determines if the specified code point is a whitespace under the active `WhitespaceMode`; see `isWhitespace(char)` for details.
  `static int[][]`
  `levenshteinDistance(String wordForm, String lemma)`
  Computes the Levenshtein distance of two strings in a matrix.
  `static String[]`
  `splitOnUnicodeWhitespace(CharSequence input)`
  Splits `input` on runs of Unicode `White_Space`.
  `static String`
  `toLowerCase(CharSequence string)`
  Converts a `CharSequence` to lower case, independent of the current `Locale` via `Character.toLowerCase(int)` which uses mapping information from the UnicodeData file.
  `static CharBuffer`
  `toLowerCaseCharBuffer(CharSequence sequence)`
  `static String`
  `toUpperCase(CharSequence string)`
  Converts a `CharSequence` to upper case, independent of the current `Locale` via `Character.toUpperCase(char)` which uses mapping information from the UnicodeData file.
  `static String`
  `trimUnicodeWhitespace(CharSequence input)`
  Trims leading and trailing runs of Unicode `White_Space`, the same set `splitOnUnicodeWhitespace(CharSequence)` breaks terms on.

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### ASCII_DIGIT_STRINGS

    public static final List<String> ASCII_DIGIT_STRINGS
    The ten ASCII digit strings `"0"` to `"9"`, indexed by digit value. Precomputed so code folding digits to ASCII does not allocate a new single-character string per digit; the list is immutable and safe to share.

- ## Constructor Details

  - ### StringUtil

    public StringUtil()

- ## Method Details

  - ### isWhitespace

    public static boolean isWhitespace(char charCode)
    Determines if the specified `Character` is a whitespace under the active `WhitespaceMode`: the Unicode `White_Space` property by default, or, when the "opennlp.whitespace.mode" system property selects `WhitespaceMode.LEGACY`, the union of `Character.isWhitespace(int)` and the Unicode `Zs` category (`Character.SPACE_SEPARATOR`) that OpenNLP 1.x and 2.x used.

    Tokenization, corpus format parsing, and feature generation all resolve whitespace through this method, so they share one `WhitespaceMode` for the life of the process; use `isUnicodeWhitespace(char)` directly where the Unicode definition should apply unconditionally, such as user-text normalization with no trained-model dependency.

    Parameters:
      `charCode` - The character to check.
    Returns:
      `true` if `charCode` represents a white space under the active `WhitespaceMode`, `false` otherwise.

  - ### isWhitespace

    public static boolean isWhitespace(int charCode)
    Determines if the specified code point is a whitespace under the active `WhitespaceMode`; see `isWhitespace(char)` for details.
    Parameters:
      `charCode` - An int representation of a character to check.
    Returns:
      `true` if `charCode` represents a white space under the active `WhitespaceMode`, `false` otherwise.

  - ### isUnicodeWhitespace

    public static boolean isUnicodeWhitespace(char charCode)
    Determines if the specified `Character` is a whitespace under the Unicode `White_Space` property; delegates to `UnicodeWhitespace.isWhitespace(int)`. This is also what `isWhitespace(char)` resolves to under the default `WhitespaceMode`.
    Parameters:
      `charCode` - The character to check.
    Returns:
      `true` if `charCode` has the `White_Space` property, `false` otherwise.

  - ### isUnicodeWhitespace

    public static boolean isUnicodeWhitespace(int charCode)
    Determines if the specified code point is a whitespace under the Unicode `White_Space` property; delegates to `UnicodeWhitespace.isWhitespace(int)`. This is also what `isWhitespace(int)` resolves to under the default `WhitespaceMode`.
    Parameters:
      `charCode` - An int representation of a character to check.
    Returns:
      `true` if `charCode` has the `White_Space` property, `false` otherwise.

  - ### splitOnUnicodeWhitespace

    public static String[] splitOnUnicodeWhitespace(CharSequence input)
    Splits `input` on runs of Unicode `White_Space`. Leading and trailing runs are ignored, so whitespace-only input yields an empty array. This is a code-point scan, not a regular expression.
    Parameters:
      `input` - The text to split. Must not be `null`.
    Returns:
      The non-whitespace terms in order.
    Throws:
      `IllegalArgumentException` - If `input` is `null`.

  - ### trimUnicodeWhitespace

    public static String trimUnicodeWhitespace(CharSequence input)
    Trims leading and trailing runs of Unicode `White_Space`, the same set `splitOnUnicodeWhitespace(CharSequence)` breaks terms on.
    Parameters:
      `input` - The text to trim. Must not be `null`.
    Returns:
      The trimmed string; may be empty when `input` is whitespace-only.
    Throws:
      `IllegalArgumentException` - If `input` is `null`.

  - ### isUnicodeBlank

    public static boolean isUnicodeBlank(CharSequence input)
    `true` when `input` is `null`, empty, or consists only of Unicode `White_Space` code points.
    Parameters:
      `input` - The text to test; `null` is treated as blank.
    Returns:
      `true` if there is no non-whitespace code point.

  - ### toLowerCase

    public static String toLowerCase(CharSequence string)
    Converts a `CharSequence` to lower case, independent of the current `Locale` via `Character.toLowerCase(int)` which uses mapping information from the UnicodeData file.
    Parameters:
      `string` - The `CharSequence` to transform.
    Returns:
      The lower-cased String.

  - ### toLowerCaseCharBuffer

    public static CharBuffer toLowerCaseCharBuffer(CharSequence sequence)

  - ### toUpperCase

    public static String toUpperCase(CharSequence string)
    Converts a `CharSequence` to upper case, independent of the current `Locale` via `Character.toUpperCase(char)` which uses mapping information from the UnicodeData file.
    Parameters:
      `string` - The `CharSequence` to transform.
    Returns:
      The upper-cased String

  - ### isEmpty

    public static boolean isEmpty(CharSequence theString)
    Returns:
      `true` if `CharSequence.length()` is `0` or `null`, otherwise `false`
    Since:
      1.5.1

  - ### levenshteinDistance

    public static int[][] levenshteinDistance(String wordForm, String lemma)
    Computes the Levenshtein distance of two strings in a matrix.

    Based on this pseudo-code which in turn is based on the paper Wagner, Robert A.; Fischer, Michael J. (1974), "The String-to-String Correction Problem", Journal of the ACM 21 (1): 168-173

    Parameters:
      `wordForm` - The form as input.
      `lemma` - The target lemma.
    Returns:
      A 2-dimensional Levenshtein distance matrix.

  - ### computeShortestEditScript

    public static void computeShortestEditScript(String wordForm, String lemma, int[][] distance, StringBuffer permutations)
    Computes the Shortest Edit Script (SES) to convert a word into its lemma. This is based on Chrupala's PhD thesis (2008).
    Parameters:
      `wordForm` - The token.
      `lemma` - The target lemma.
      `distance` - A 2-dimensional Levenshtein distance matrix.
      `permutations` - The number of permutations.

  - ### decodeShortestEditScript

    public static String decodeShortestEditScript(String wordForm, String permutations)
    Reads the predicted Shortest Edit Script (SES) by a lemmatizer model and applies the permutations to obtain the lemma from the `wordForm`.
    Parameters:
      `wordForm` - The wordForm as input.
      `permutations` - The permutations predicted by the lemmatizer model.
    Returns:
      The decoded lemma.

  - ### getShortestEditScript

    public static String getShortestEditScript(String wordForm, String lemma)
    Parameters:
      `wordForm` - The word as input.
      `lemma` - The target lemma.
    Returns:
      Retrieves the Shortest Edit Script (SES) required to go from a word to a lemma.