Table Of Contents

Parsing a String into Smaller Pieces

Last Modified: January 10, 2020

Parsing a string into smaller pieces allows you to perform operations on individual words or groups of characters in the string. These words or groups of characters are often referred to as tokens. A token is defined as either the next set of characters that appears before a separator character, called a delimiter, or one of a specified set of operators.

What to Use

What to Do

Create the following diagram to parse a string into smaller pieces.

Customize the gray sections for your unique programming goals.

In order to identify all the tokens in a string, you must use a While Loop. Inside a While Loop, Scan String for Tokens scans the entire input string, returning all tokens it identifies until it reaches the end of the string. If you do not use Scan String for Tokens inside a While Loop, the node stops scanning as soon as it identifies the first token in a string and returns only that token.

The input string input of Scan String for Tokens contains the string to scan for tokens.

To start each scan at the location within the string where the preceding scan ended, pass the offset past token output of Scan String for Tokens through a shift register and back into the offset input of the same node.

Initialize this shift register with the location within the string at which you want to begin parsing. Use 0 if you want Scan String for Tokens to begin its operation at the beginning of the string each time the program runs.

Add any strings that you want Scan String for Tokens to identify as tokens to the operators input array. The node identifies these strings as tokens even if they are not surrounded by any delimiters.

Detect the end of the input string by comparing the token index output of Scan String for Tokens to -2.

Use the auto-indexing output tunnel of the While Loop to collect the individual tokens in an array. This array contains all text that appears between delimiters as well as any strings specified in the operators input array that are found in the input string.

Scan String for Tokens does not return delimiters as tokens but instead uses them to determine where tokens begin and end.


  • If Scan String for Tokens does not identify a token that you expect it to identify, check to make sure the operators input array does not contain regular expression notation or any invisible characters. Scan String for Tokens does not process regular expressions. Also check for correct capitalization of items in the operators input array, as scanning is case-sensitive.


input string operators delimiters token string Comments
4>=0 [>, =, >=] \s, \t, \r, \n (default) [4, >=, 0]

If a portion of the input string matches more than one defined operator, Scan String for Tokens chooses the longest match as a token.



[==, !=] \s, \t, \r, \n (default) [a, ==, b, c, !=, d]
G2 X0.5Y1.0 i0.5j0 z-0.05 [X, Y, Z, i, j, z] \s, \t, \r, \n (default) [G2, X, 0.5, Y, 1.0, i, 0.5, j, 0, z, -0.05]

This is an example of a string of G-code, a language commonly used for machine control. This string describes a circle.

C1_1.11C2_2.22C3_3.33 None C, _ (add to delimiters array)

\s, \t, \r, \n (default)

[1, 1.11, 2, 2.22, 3, 3.33]

This is an example of a string from a DAQ log with three channels.

Recently Viewed Topics