musl/src/multibyte/internal.h

#define bittab __fsmu8

#include <stdint.h>

#ifdef __PIC__
__attribute__((__visibility__("hidden")))
#endif
extern const uint32_t bittab[];

/* Upper 6 state bits are a negative integer offset to bound-check next byte */
/*    equivalent to: ( (b-0x80) | (b+offset) ) & ~0x3f      */
#define OOB(c,b) (((((b)>>3)-0x10)|(((b)>>3)+((int32_t)(c)>>26))) & ~7)

/* Interval [a,b). Either a must be 80 or b must be c0, lower 3 bits clear. */
#define R(a,b) ((uint32_t)((a==0x80 ? 0x40u-b : 0u-a) << 23))
#define FAILSTATE R(0x80,0x80)

#define SA 0xc2u
#define SB 0xf4u

/* Arbitrary encoding for representing code units instead of characters. */
#define CODEUNIT(c) (0xdfff & (signed char)(c))
#define IS_CODEUNIT(c) ((unsigned)(c)-0xdf80 < 0x80)

/* Get inline definition of MB_CUR_MAX. */
#include "locale_impl.h"
initial check-in, version 0.5.0 2011-02-12 00:22:29 -05:00			`#define bittab __fsmu8`

include cleanups: remove unused headers and add feature test macros 2013-12-12 05:09:18 +00:00			`#include <stdint.h>`
cleanup and work around visibility bug in gcc 3 that affects x86_64 in gcc 3, the visibility attribute must be placed on both the declaration and on the definition. if it's omitted from the definition, the compiler fails to emit the ".hidden" directive in the assembly, and the linker will either generate textrels (if supported, such as on i386) or refuse to link (on targets where certain types of textrels are forbidden or impossible without further assumptions about memory layout, such as on x86_64). this patch also unifies the decision about when to use visibility into libc.h and makes the visibility in the utf-8 state machine tables based on libc.h rather than a duplicate test. 2012-02-23 21:24:56 -05:00
remove libc.h dependency from otherwise-independent multibyte code 2015-04-22 02:34:16 -04:00			`#ifdef __PIC__`
			`__attribute__((__visibility__("hidden")))`
			`#endif`
			`extern const uint32_t bittab[];`
initial check-in, version 0.5.0 2011-02-12 00:22:29 -05:00
			`/* Upper 6 state bits are a negative integer offset to bound-check next byte */`
			`/* equivalent to: ( (b-0x80) \| (b+offset) ) & ~0x3f */`
			`#define OOB(c,b) (((((b)>>3)-0x10)\|(((b)>>3)+((int32_t)(c)>>26))) & ~7)`

			`/* Interval [a,b). Either a must be 80 or b must be c0, lower 3 bits clear. */`
fix undefined left-shift of negative values in utf-8 state table 2015-07-25 03:15:45 +00:00			`#define R(a,b) ((uint32_t)((a==0x80 ? 0x40u-b : 0u-a) << 23))`
initial check-in, version 0.5.0 2011-02-12 00:22:29 -05:00			`#define FAILSTATE R(0x80,0x80)`

cleanup multibyte stuff to remove ugly casts, sanitize the ptr align casts 2011-02-13 23:08:18 -05:00			`#define SA 0xc2u`
fix out-of-bounds access in UTF-8 decoding SA and SB are used as the lowest and highest valid starter bytes, but the value of SB was one-past the last valid starter. this caused access past the end of the state table when the illegal byte '\xf5' was encountered in a starter position. the error did not show up in full-character decoding tests, since the bogus state read from just past the table was unlikely to admit any continuation bytes as valid, but would have shown up had we tested feeding '\xf5' to the byte-at-a-time decoding in mbrtowc: it would cause the funtion to wrongly return -2 rather than -1. I may eventually go back and remove all references to SA and SB, replacing them with the values; this would make the code more transparent, I think. the original motivation for using macros was to allow misguided users of the code to redefine them for the purpose of enlarging the set of accepted sequences past the end of Unicode... 2013-04-08 22:29:46 -04:00			`#define SB 0xf4u`
byte-based C locale, phase 1: multibyte character handling functions this patch makes the functions which work directly on multibyte characters treat the high bytes as individual abstract code units rather than as multibyte sequences when MB_CUR_MAX is 1. since MB_CUR_MAX is presently defined as a constant 4, all of the new code added is dead code, and optimizing compilers' code generation should not be affected at all. a future commit will activate the new code. as abstract code units, bytes 0x80 to 0xff are represented by wchar_t values 0xdf80 to 0xdfff, at the end of the surrogates range. this ensures that they will never be misinterpreted as Unicode characters, and that all wctype functions return false for these "characters" without needing locale-specific logic. a high range outside of Unicode such as 0x7fffff80 to 0x7fffffff was also considered, but since C11's char16_t also needs to be able to represent conversions of these bytes, the surrogate range was the natural choice. 2015-06-16 04:44:17 +00:00
			`/* Arbitrary encoding for representing code units instead of characters. */`
			`#define CODEUNIT(c) (0xdfff & (signed char)(c))`
			`#define IS_CODEUNIT(c) ((unsigned)(c)-0xdf80 < 0x80)`

			`/* Get inline definition of MB_CUR_MAX. */`
			`#include "locale_impl.h"`