Messages in this thread | | | Date | Mon, 20 Jan 2020 16:20:09 +0100 | From | Pali Rohár <> | Subject | Re: vfat: Broken case-insensitive support for UTF-8 |
| |
On Monday 20 January 2020 15:07:20 David Laight wrote: > From: Pali Rohár > > Sent: 20 January 2020 11:05 > > On Monday 20 January 2020 13:04:42 OGAWA Hirofumi wrote: > > > Pali Rohár <pali.rohar@gmail.com> writes: > > > > > > > Which means that fat_name_match(), vfat_hashi() and vfat_cmpi() are > > > > broken for vfat in UTF-8 mode. > > > > > > Right. It is a known issue. > > > > Could be this issue better documented? E.g. in mount(8) manpage where > > are written mount options for vfat? I think that people should be aware > > of this issue when they use "utf8=1" mount option. > > What happens if the filesystem has filenames that invalid UTF8 sequences
Could you please describe what you mean by this question?
VFAT filesystem stores file names in UTF-16. Therefore you cannot have UTF-8 on FS (and therefore also you cannot have invalid UTF-8).
Ehm... UTF-16 is not fully truth, MS FAT32 implementations allows half of UTF-16 surrogate pair stored in FS.
Therefore practically, on VFAT you can store any uint16_t[] sequence as filename, there is no invalid sequence (except those characters like :<>?... which are invalid in MS-DOS).
If by "the filesystem has filenames" you do not mean filesystem file names, but rather Linux VFS file names (e.g. you call creat() call with invalid UTF-8 sequence) then function utf8s_to_utf16s() (called in namei_vfat.c) fails and returns error. Which should be propagated to open() / creat() call that it is not possible to create filename with such UTF-8 sequence.
> or multiple filenames that decode from UTF8 to the same 'wchar' value.
This is not possible. There is 1:1 mapping between UTF-8 sequence and Unicode code point. wchar_t in kernel represent either one Unicode code point (limited up to U+FFFF in NLS framework functions) or 2bytes in UTF-16 sequence (only in utf8s_to_utf16s() and utf16s_to_utf8s() functions).
> Never mind ones that are just case-differences for the same filename. > > UTF8 is just so broken it should never have been allowed to become > a standard.
Well, UTF-16 is worse then UTF-8... incompatible with ASCII, variable length and space consuming.
-- Pali Rohár pali.rohar@gmail.com
| |