StringTokenizer in Java with Examples

StringTokenizer class in Java lets you split a String into multiple tokens.

In this tutorial, we will take a look at ways in which you can make use of this class to split your String.

Constructors:

• StringTokenizerExample(String str)
• StringTokenizerExample(String str, String delim)
• StringTokenizer(String str, String delim, boolean returnDelims)

StringTokenizerExample(String str)

Example 1: Space-separated String

import java.util.StringTokenizer;

/**
 * Example: StringTokenizer
 * Author: Code2care.org
 * Date: 03-Apr-2022
 * 
 */

public class StringTokenizerExample {

    public static void main(String[] args) {

        String myStr = "a b c d e f g";
        StringTokenizer stringTokenizer = new StringTokenizer(myStr);

        while (stringTokenizer.hasMoreTokens()) {
            System.out.println(stringTokenizer.nextToken());
        }
    }
}

Output:

a
b
c
d
e
f
g

Example 2: Tab Separated String

        String myStr = "1   2   3   4   5   6   7";
        StringTokenizer stringTokenizer = new StringTokenizer(myStr);

        while (stringTokenizer.hasMoreTokens()) {
            System.out.println(stringTokenizer.nextToken());
        }

Output:

1
2
3
4
5
6
7

Example 3: \r \n \t space all combiled:

        String myStr = "Hello\nThere\rHow   are you";
        StringTokenizer stringTokenizer = new StringTokenizer(myStr);

        while (stringTokenizer.hasMoreTokens()) {
            System.out.println(stringTokenizer.nextToken());
        }

Output:

Hello
There
How
are
you

Construtor Syntax:

public StringTokenizer(String str) {
    this(str, " \t\n\r\f", false);
}

Note that the StringTokenizer methods do not distinguish between identifiers, numbers, and quoted strings, nor do they recognize and skip comments.

StringTokenizerExample(String str, String delim)

If you want to use a specific delimiter to tokenize your String, you can make use of the constructor with two arguments

public StringTokenizer(String str, String delim)

Example 4:

import java.util.StringTokenizer;

/**
 * Example: StringTokenizer with Delimiter
 * Author: Code2care.org
 * Date: 03-Apr-2022
 *
 */

public class StringTokenizerExampleWithDelimiter {

    public static void main(String[] args) {

        String myStr = "1;2;2,2;3;4;5";
        StringTokenizer stringTokenizer = new StringTokenizer(myStr,";");

        while (stringTokenizer.hasMoreTokens()) {
            System.out.println(stringTokenizer.nextToken());
        }
    }
}

Output:

1
2
2,2
3
4
5

StringTokenizer(String str, String delim, boolean returnDelims)

If you also want to print the delimiter out you can make use of the this constructor,

Example 5:

String myStr = "1;2;2,2;3;4;5";
StringTokenizer stringTokenizer = new StringTokenizer(myStr,";",true);

while (stringTokenizer.hasMoreTokens()) {
    System.out.println(stringTokenizer.nextToken());
}

Output:

1
;
2
;
2,2
;
3
;
4
;
5