LaTeX ReferenceDownload TeX64

Encoding & newlines

A .tex file is ultimately bytes, and the engine must read them in the right character encoding to interpret them correctly. Newline codes matter too, for editing and diffs. Japanese has historically juggled several encodings, which is where people trip. The short answer: today, standardize on UTF-8 + LF. This page covers the encodings, the newline codes, and the nkf conversion tool.

Encodings — UTF-8 today

A character encoding is the rule mapping bytes to characters. Japanese has historically used Shift_JIS (older Windows), EUC-JP (older Unix), and ISO-2022-JP (email’s “JIS code”); today Unicode’s UTF-8 is the universal standard. Recent TeX Live and LaTeX default to UTF-8, so saving your .tex as UTF-8 avoids most garbling. A mismatch in encoding produces the classic garbled text (*mojibake*).

EncodingWhere it was used
UTF-8Today’s standard (Unicode); the only choice for new work
Shift_JISOlder Windows environments
EUC-JPOlder Unix environments
ISO-2022-JPEmail, “JIS code”

For (u)pLaTeX, you can state the input encoding explicitly with -kanji=utf8 (see “Compile commands”).

Newline codes — LF / CRLF / CR

Lines end with invisible control bytes: LF (\n, Unix/macOS), CRLF (\r\n, Windows), and CR (\r, classic Mac). LaTeX itself tolerates all of them, but mixed newlines within a file or across a team make diffs messy and trip up some tools. Standardize on LF, and normalize with .gitattributes under Git.

terminal
# .gitattributes — 改行を正規化 / normalize newlines
*.tex text eol=lf

Convert with nkf

nkf (Network Kanji Filter) is the standard tool to detect and convert Japanese encodings and newlines. First inspect with nkf -g file; then convert to UTF-8 with -w, Shift_JIS with -s, EUC-JP with -e, or ISO-2022-JP with -j. Newlines use -Lu/-Lw/-Lm, and --overwrite rewrites the file in place.

OptionWhat it does
-gDetect the current encoding and newline (no conversion)
-wConvert to UTF-8 (no BOM)
-s / -e / -jConvert to Shift_JIS / EUC-JP / ISO-2022-JP
-Lu / -Lw / -LmConvert newlines to LF / CRLF / CR
--overwriteOverwrite the input file in place
terminal
nkf -g old.tex                      # 文字コードを判定 / detect the encoding
nkf -w -Lu --overwrite old.tex      # UTF-8 + LF に変換して上書き / to UTF-8 + LF, in place

Recommended in practice

  • Standardize everything on UTF-8 + LF, new or old.
  • Batch-convert legacy Shift_JIS/EUC sources with nkf -w -Lu --overwrite *.tex.
  • Force UTF-8 input for (u)pLaTeX with -kanji=utf8 when needed.
  • Normalize newlines via .gitattributes under Git; check with nkf -g when unsure.

Read next