Why do Italian words lose their last letter, like così becoming cos?

The missing letter is ì, which DOS stores as byte 141, and Windows-1252 has no character there, so it vanishes or turns into an invisible control code. Italian puts ì at the end of words like così and lunedì, so that's where the gaps show. Don't save from the program showing it wrong, since it may write the damage back. Convert from code page 850 first.

UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d

Python opened a UTF-8 file as Windows-1252, the default text encoding on most Western Windows setups, and 0x8D is one of the five bytes Windows-1252 leaves empty. In UTF-8 it turns up inside emoji sequences, because the zero-width joiner that glues 👨👩👧 together is E2 80 8D, and in Russian text as the second byte of э. Name the encoding when you open the file: `open(path, encoding='utf-8')`.

ASCII 141

Empty on WindowsLowercase i with grave accent

ASCII 141 is empty on Windows and ì in old DOS.

Byte 141 (0x8D) is another unassigned slot in Windows-1252, so a strict decoder has nothing to map it to. If one chokes on it, the text may be UTF-8, where 8D shows up inside several characters. It's the last byte of the zero width joiner (E2 80 8D), the invisible character that glues emoji sequences like 👨‍💻 together, so a message with emoji can contain 0x8D even if every letter in it is plain ASCII. It also ends capital Í (C3 8D), as in Ísland, the Icelandic name for Iceland.

ì (Small I with Grave) is byte 141 in code page 437, the original IBM PC character set, and maps to Unicode U+00EC. Windows-1252 leaves byte 141 unassigned, so Italian DOS text read as Windows-1252 drops the letter or turns it into an invisible control code, and così comes out as cos. Code page 850 kept ì at 141 too.

UnassignedLatin Letter

Technical Details

Code Example

</>

/* Byte 141 (0x8D) has no character in Windows-1252.
   MultiByteToWideChar passes it through as the control code U+008D,
   so treat it as a sign that the text is not really Windows-1252. */
unsigned char b = 141;
char c = (char)141;   /* -115 on x86, where char is signed */

Code Example

</>

#include <stdio.h>

int main(void) {
    /* Byte 141 is ì only on a console using code page 437
       (chcp 437 on Windows, or DOSBox). On a UTF-8 terminal the
       lone byte 0x8D is invalid and prints as garbage, often �. */
    putchar(141);

    /* char is signed on x86, so a plain char holding 0x8D is -115.
       Use unsigned char when you compare or index by byte value. */
    char c = (char)141;
    unsigned char u = 141;
    printf("\n%d %d\n", c, u);   /* -115 141 */

    /* Portable: print the Unicode character as UTF-8 instead. */
    printf("\u00EC\n");          /* C3 AC */
    return 0;
}

How to Type It

Windows: Hold Alt and type 141 on the numeric keypad, with no leading zero. Windows reads it as a DOS code and inserts ì.

ì is also ASCII 236 on Windows, which has the full guide.

Frequently Asked Questions

Python opened a UTF-8 file as Windows-1252, the default text encoding on most Western Windows setups, and 0x8D is one of the five bytes Windows-1252 leaves empty. In UTF-8 it turns up inside emoji sequences, because the zero-width joiner that glues 👨‍👩‍👧 together is E2 80 8D, and in Russian text as the second byte of э. Name the encoding when you open the file: `open(path, encoding='utf-8')`.

Frequently Asked Questions

The missing letter is ì, which DOS stores as byte 141, and Windows-1252 has no character there, so it vanishes or turns into an invisible control code. Italian puts ì at the end of words like così and lunedì, so that's where the gaps show. Don't save from the program showing it wrong, since it may write the damage back. Convert from code page 850 first.

ASCII 141

Empty on WindowsLowercase i with grave accent

ASCII 141 is empty on Windows and ì in old DOS.

Unassigned

Latin Letter

/* Byte 141 (0x8D) has no character in Windows-1252.

MultiByteToWideChar passes it through as the control code U+008D,

so treat it as a sign that the text is not really Windows-1252. */

unsigned char b = 141;

char c = (char)141; /* -115 on x86, where char is signed */

#include <stdio.h>

int main(void) {

/* Byte 141 is ì only on a console using code page 437

(chcp 437 on Windows, or DOSBox). On a UTF-8 terminal the

lone byte 0x8D is invalid and prints as garbage, often �. */

putchar(141);

/* char is signed on x86, so a plain char holding 0x8D is -115.

Use unsigned char when you compare or index by byte value. */

char c = (char)141;

unsigned char u = 141;

printf("\n%d %d\n", c, u); /* -115 141 */

/* Portable: print the Unicode character as UTF-8 instead. */

printf("\u00EC\n"); /* C3 AC */

return 0;

}