Why did ALTER TABLE CONVERT TO utf8mb4 make my Ã© worse?

Because the column was labelled latin1 while actually holding UTF-8 bytes, and CONVERT TO translates every byte as if it were Latin-1, encoding é a second time. Relabel instead of converting: `ALTER TABLE t MODIFY body BLOB;` drops the charset, then `ALTER TABLE t MODIFY body TEXT CHARACTER SET utf8mb4;` reads the same bytes back as UTF-8. Back up the table first.

How do I fix ÃƒÂ© when it should be é?

That's é mangled twice: its UTF-8 bytes were read as Windows-1252, saved as UTF-8, then read wrong again. The ftfy library works out how many layers there are and undoes them, so `ftfy.fix_text('ÃƒÂ©')` returns é. Then find the step that encodes twice, such as a UTF-8 connection writing into a latin1 column, or it'll keep happening.

Does dropping the tilde from Ã change a Portuguese word?

It can. The tilde marks a nasal vowel, and some words differ by it alone: manhã is morning, while manha is a tantrum. Capitals keep it for the same reason, so a sign reading MANHA names the wrong word. In names like SÃO PAULO the tilde is part of the spelling, not decoration.

Why does my console print ├ before accented letters?

The program writes UTF-8 and the console reads code page 437, which draws C3, the lead byte of letters like é and ñ, as ├. So é becomes ├⌐ and ñ becomes ├▒. In C or C++, call `SetConsoleOutputCP(CP_UTF8);` once at startup and the console reads the bytes as UTF-8.

ASCII 195

Uppercase A with tildeBox T-junction pointing right

ASCII 195 is Ã on Windows and ├ in old DOS.

Ã (Capital A with Tilde) is byte 195 in Windows-1252, Unicode U+00C3. Portuguese needs it in capitals like SÃO PAULO and NÃO. It's also the first character of mangled accented letters. UTF-8 writes the characters from U+00C0 to U+00FF with C3 as the first byte, and C3 in Windows-1252 is Ã, so é, ü and ñ come out as Ã©, Ã¼ and Ã±. You can decode one by hand: take the Windows-1252 byte of the character after Ã and add 0x40. © is A9, plus 0x40 is E9, which is é. In UTF-8, Ã itself is C3 83, misread as Ãƒ, the pair that shows up once text has been mangled twice. The HTML entity is Ã.

├ (Light Vertical and Right) is byte 195 in code page 437, the original IBM PC character set, and maps to Unicode U+251C. It's the tee on the left edge of a single-line box, and the branch in directory trees: ├── src. If you print trees yourself, the prefix for the lines under an entry depends on this character. Children of a ├ entry get │ plus three spaces, so the trunk keeps going down to the next sibling. Children of the last entry, the └ one, get four spaces, because nothing follows it. Mixing the two leaves a trunk hanging below the last item. In UTF-8, ├ is E2 94 9C, and the HTML entity is &boxvr;.

Latin LetterBox Drawing

Technical Details

Code Example

</>

#include <stdio.h>

int main(void) {
    /* On Windows (code page 1252) byte 195 is Ã.
       Modern terminals expect UTF-8, so print the code point, not the byte. */
    printf("\u00C3\n");   /* UTF-8: C3 83 */

    unsigned char b = 195;         /* the raw Windows-1252 byte */
    printf("%d 0x%02X\n", b, b);    /* 195 0xC3 */
    return 0;
}

Code Example

</>

#include <stdio.h>

int main(void) {
    /* Byte 195 is ├ only on a console using code page 437
       (chcp 437 on Windows, or DOSBox). On a UTF-8 terminal the
       lone byte 0xC3 is invalid and prints as garbage, often �. */
    putchar(195);

    /* char is signed on x86, so a plain char holding 0xC3 is -61.
       Use unsigned char when you compare or index by byte value. */
    char c = (char)195;
    unsigned char u = 195;
    printf("\n%d %d\n", c, u);   /* -61 195 */

    /* Portable: print the Unicode character as UTF-8 instead. */
    printf("\u251C\n");          /* E2 94 9C */
    return 0;
}

How to Type It

Windows: Hold Alt and type 0195 on the numeric keypad, with Num Lock on. The leading zero picks the Windows code page. No numeric keypad? In Word, type 00C3 and press Alt+X.
Mac: Press Option+N, then Shift+A.
Linux: In GNOME, or any desktop running IBus, press Ctrl+Shift+U, type c3, then Space. With a Compose key set up, press Compose, tilde, A.
Phone: Tap Shift, then long-press A and pick Ã.

How to Type It

Windows: Hold Alt and type 195 on the numeric keypad, with no leading zero. Windows reads it as a DOS code and inserts ├. No numeric keypad? In Word, type 251C and press Alt+X.
Mac: Press Control+Command+Space to open the Character Viewer, then search for "box drawings light vertical and right".
Linux: In GNOME, or any desktop running IBus, press Ctrl+Shift+U, type 251c, then Space.
Phone: Phone keyboards have no key for it. Tap the character at the top of this page to copy it.

Frequently Asked Questions

Because the column was labelled latin1 while actually holding UTF-8 bytes, and CONVERT TO translates every byte as if it were Latin-1, encoding é a second time. Relabel instead of converting: `ALTER TABLE t MODIFY body BLOB;` drops the charset, then `ALTER TABLE t MODIFY body TEXT CHARACTER SET utf8mb4;` reads the same bytes back as UTF-8. Back up the table first.
That's é mangled twice: its UTF-8 bytes were read as Windows-1252, saved as UTF-8, then read wrong again. The ftfy library works out how many layers there are and undoes them, so `ftfy.fix_text('ÃƒÂ©')` returns é. Then find the step that encodes twice, such as a UTF-8 connection writing into a latin1 column, or it'll keep happening.
It can. The tilde marks a nasal vowel, and some words differ by it alone: manhã is morning, while manha is a tantrum. Capitals keep it for the same reason, so a sign reading MANHA names the wrong word. In names like SÃO PAULO the tilde is part of the spelling, not decoration.
The program writes UTF-8 and the console reads code page 437, which draws C3, the lead byte of letters like é and ñ, as ├. So é becomes ├⌐ and ñ becomes ├▒. In C or C++, call `SetConsoleOutputCP(CP_UTF8);` once at startup and the console reads the bytes as UTF-8.

Frequently Asked Questions

The program writes UTF-8 and the console reads code page 437, which draws C3, the lead byte of letters like é and ñ, as ├. So é becomes ├⌐ and ñ becomes ├▒. In C or C++, call `SetConsoleOutputCP(CP_UTF8);` once at startup and the console reads the bytes as UTF-8.

ASCII 195

Uppercase A with tildeBox T-junction pointing right

ASCII 195 is Ã on Windows and ├ in old DOS.

Latin Letter

Box Drawing

#include <stdio.h>

int main(void) {

/* On Windows (code page 1252) byte 195 is Ã.

Modern terminals expect UTF-8, so print the code point, not the byte. */

printf("\u00C3\n"); /* UTF-8: C3 83 */

unsigned char b = 195; /* the raw Windows-1252 byte */

printf("%d 0x%02X\n", b, b); /* 195 0xC3 */

return 0;

}

#include <stdio.h>

int main(void) {

/* Byte 195 is ├ only on a console using code page 437

(chcp 437 on Windows, or DOSBox). On a UTF-8 terminal the

lone byte 0xC3 is invalid and prints as garbage, often �. */

putchar(195);

/* char is signed on x86, so a plain char holding 0xC3 is -61.

Use unsigned char when you compare or index by byte value. */

char c = (char)195;

unsigned char u = 195;

printf("\n%d %d\n", c, u); /* -61 195 */

/* Portable: print the Unicode character as UTF-8 instead. */

printf("\u251C\n"); /* E2 94 9C */

return 0;

}