Understanding Text Encoding: Unicode, UTF-8, and Garbled Text
Understand the difference between Unicode and UTF-8, why text becomes garbled, and how to check encoding before importing or publishing text.
Text can look correct in one application and appear as strange symbols in another. This often happens because the applications interpret the same bytes using different character encodings. Understanding the difference between characters and their encoded bytes helps diagnose the problem without randomly changing settings.
Unicode and UTF-8 are not the same thing
Unicode assigns code points to characters used by many writing systems and symbols. UTF-8 is a way to encode those code points as bytes. A character such as a Latin letter may use one byte in UTF-8, while many other characters use two, three, or four bytes. Unicode describes the character repertoire; UTF-8 describes a common byte representation.
Why text becomes garbled
Garbled text, sometimes called mojibake, can occur when bytes written using one encoding are decoded using another. It can also happen when a file is opened with the wrong encoding, when a conversion step runs more than once, or when a system mishandles a byte-order mark. Replacing punctuation or copying the text through several apps may hide symptoms without fixing the underlying issue.
A safer troubleshooting workflow
- Keep an untouched copy of the original file before converting it.
- Check the source application or export settings to learn which encoding was used.
- For modern web text, prefer UTF-8 unless the receiving system requires something else.
- Inspect a small sample containing accented letters, non-Latin scripts, and symbols.
- Convert once, then reopen the result in the target application and compare it with the original.
- Verify the file format and delimiter separately; encoding and CSV structure are different issues.
Common mistakes to avoid
- Assuming every text file is UTF-8 just because it ends in .txt or .csv.
- Confusing a font problem with an encoding problem. A missing glyph can display as a box even when the underlying text is valid.
- Saving over the original before confirming the converted result.
- Using URL encoding or HTML entities as a general fix for file encoding. Those formats solve different problems.
Useful next steps
When working with text snippets, use the Character Encoding Inspector to examine characters and their code points. For content that needs to be moved between spreadsheet applications, also review the CSV vs TSV conversion guide. Always verify the final file in the application that will consume it.
Limitations
A browser-based inspection can explain characters in a pasted sample, but it cannot always identify the original encoding of an arbitrary file from a few characters alone. Preserve the original bytes and check the export settings when the source is available.