Transformer is designed to handle the more common character sets used in delimited text data sources. However, consider the following when working with delimiters.
Standard delimiters such as the comma, semicolon, or space character remain the same across character sets.
To use the tab character as a delimiter, you must type \t (backslash t).
Only the first byte in double-byte or multi-byte characters is examined to confirm that a character in the input stream matches the delimiter character that you specify in Transformer.
With DOS outputs, the source data is rendered in the DOS Code Page (OEM) character set, which maps to the active DOS code page. This is typically Code Page 437, also known as the IBM PC character set, instead of the default for Windows (ANSI or Latin 1).
Regardless of the data source, Transformer converts the delimiter character to the OEM character set.
For more information about character sets and code pages, see your Microsoft Windows documentation.