From: Donat Enikeev <donat@enikeev.net>
Subject: locale: [1/1] patch - 0x10000000 flag is NORM_STOP_ON_NULL
Message-Id: <6771071428181747@web1m.yandex.ru>
Date: Sun, 05 Apr 2015 00:09:07 +0300

Hi all,

While investigating reasons of one application not running with Wine, I met again:

fixme:nls:CompareStringEx semi-stub behavor for flag(s) 0x10000000

Quite common message affecting a lot of applications. Decided to get to bottom of this. 

Found no information about this flag on MSDN. While it was said in comment at locale.c about some diacritic language specifics I had been digging in wrong direction for a while, and checking .NET equivalents without any success though. Then started to search by 0x10000000 on different codes hosting and found 

const DWORD NORM_STOP_ON_NULL = 0x10000000;

At http://andrew-grechkin.googlecode.com/svn-history/r250/trunk/libbase/include/libbase/pcstr.hpp

So, I assumed that idea of this flag is to stop comparing whenever null is met even if length is defined explicitly. I've checked and confirmed this version on free Windows 10 TechPreview+Visual studio by some tests (sample code and output attached as well) for: CompareStringEx, CompareStringW, CompareStringA

Please have a look, one can imagine case when incorrect processing of this flag could lead to completely unexpected behavior.

P.s. Confirmed correctness of some presenting tests in wine.git

--de

C:\Users\Don\Documents\Visual Studio 2013\Projects\ConsoleApplication1\Release>ConsoleApplication1.exe
Hello world!

===== CompareStringEx =====

Equal is expected as length to compare is calculated for 0 terminated (assume flag is useless): Equal
Equal is expected as length to compare is calculated for 0 terminated (flag is not used): Equal
NOT Equal expected as compared by length passed defined explicitely: Less
Equal is expected with flag and length passed: Equal

===== CompareStringW =====

Equal is expected as length to compare is calculated for 0 terminated (assume flag is useless): Equal
Equal is expected as length to compare is calculated for 0 terminated (flag is not used): Equal
NOT Equal expected as compared by length passed defined explicitely: Less
Equal is expected with flag and length passed: Equal

===== CompareStringA =====

Equal is expected as length to compare is calculated for 0 terminated (assume flag is useless): Equal
Equal is expected as length to compare is calculated for 0 terminated (flag is not used): Equal
NOT Equal expected as compared by length passed defined explicitely: Greater
Equal is expected with flag and length passed: Equal

===== Tests from wine.git =====

aLuZkUtZ vs aLuZkUtZ\0A expected CSTR_LESS_THAN or CSTR_EQUAL
Less
aLu\0ZkUtZ vs aLu\0ZkUtZ\0A expected CSTR_LESS_THAN or CSTR_EQUAL
Less
a vs a expected CSTR_EQUAL
Greater
a\0b vs a expected CSTR_EQUAL or CSTR_GREATER_THAN
Greater

// ConsoleApplication1.cpp : Defines the entry point for the console application.
//
#include "stdafx.h"
#include <iostream>

void print_compare_result(int i) {
	switch (i) {
	case CSTR_EQUAL:
		printf("Equal\n");
		break;
	case CSTR_GREATER_THAN:
		printf("Greater\n");
		break;
	case CSTR_LESS_THAN:
		printf("Less\n");
		break;
	}
}

int _tmain(int argc, _TCHAR* argv[])
{
	printf("Hello world!\n");

	#define NORM_STOP_ON_NULL 0x10000000

	wchar_t * a = L" Donat Enikeev \x0000  ";
	wchar_t * b = L" Donat Enikeev \x0000 !";
	int allocated = 18; // chars allocated

	printf("\n===== CompareStringEx =====\n\n");

	printf("Equal is expected as length to compare is calculated for 0 terminated (assume flag is useless): ");
	print_compare_result(CompareStringEx(LOCALE_NAME_SYSTEM_DEFAULT, NORM_STOP_ON_NULL, a, -1, b, -1, NULL, NULL, NULL));

	printf("Equal is expected as length to compare is calculated for 0 terminated (flag is not used): ");
	print_compare_result(CompareStringEx(LOCALE_NAME_SYSTEM_DEFAULT, 0x0, a, -1, b, -1, NULL, NULL, NULL));

	printf("NOT Equal expected as compared by length passed defined explicitely: ");
	print_compare_result(CompareStringEx(LOCALE_NAME_SYSTEM_DEFAULT, 0x0, a, allocated, b, allocated, NULL, NULL, NULL));

	printf("Equal is expected with flag and length passed: ");
	print_compare_result(CompareStringEx(LOCALE_NAME_SYSTEM_DEFAULT, NORM_STOP_ON_NULL, a, allocated, b, allocated, NULL, NULL, NULL));

	printf("\n===== CompareStringW =====\n\n");

	printf("Equal is expected as length to compare is calculated for 0 terminated (assume flag is useless): ");
	print_compare_result(CompareStringW(LOCALE_SYSTEM_DEFAULT, 0x10000000, a, -1, b, -1));

	printf("Equal is expected as length to compare is calculated for 0 terminated (flag is not used): ");
	print_compare_result(CompareStringW(LOCALE_SYSTEM_DEFAULT, 0x0, a, -1, b, -1));

	printf("NOT Equal expected as compared by length passed defined explicitely: ");
	print_compare_result(CompareStringW(LOCALE_SYSTEM_DEFAULT, 0x0, a, allocated, b, allocated));

	printf("Equal is expected with flag and length passed: ");
	print_compare_result(CompareStringW(LOCALE_SYSTEM_DEFAULT, 0x10000000, a, allocated, b, allocated));

	char * aa = " Donat Enikeev \x0000  ";
	char * bb = " Donat Enikeev \x0000 !";

	printf("\n===== CompareStringA =====\n\n");

	printf("Equal is expected as length to compare is calculated for 0 terminated (assume flag is useless): ");
	print_compare_result(CompareStringA(LOCALE_SYSTEM_DEFAULT, 0x10000000, aa, -1, bb, -1));

	printf("Equal is expected as length to compare is calculated for 0 terminated (flag is not used): ");
	print_compare_result(CompareStringA(LOCALE_SYSTEM_DEFAULT, 0x0, aa, -1, bb, -1));

	printf("NOT Equal expected as compared by length passed defined explicitely: ");
	print_compare_result(CompareStringA(LOCALE_SYSTEM_DEFAULT, 0x0, aa, allocated, bb, allocated));

	printf("Equal is expected with flag and length passed: ");
	print_compare_result(CompareStringA(LOCALE_SYSTEM_DEFAULT, 0x10000000, aa, allocated, bb, allocated));
	//std::cin.ignore();
	return 0;
}

From 08c03088e6837419798f5b43cb500a535ad38ccf Mon Sep 17 00:00:00 2001
From: Donat Enikeev <donat@enikeev.net>
Date: Sat, 4 Apr 2015 23:11:49 +0300
Subject: NORM_STOP_ON_NULL flag

---
 dlls/kernel32/locale.c       | 12 ++++-----
 dlls/kernel32/tests/locale.c | 59 ++++++++++++++++++++++++++++++++++++++++++++
 include/winnls.h             |  1 +
 3 files changed, 66 insertions(+), 6 deletions(-)

diff --git a/dlls/kernel32/locale.c b/dlls/kernel32/locale.c
index afc5b9d..913565f 100644
--- a/dlls/kernel32/locale.c
+++ b/dlls/kernel32/locale.c
@@ -3235,9 +3235,9 @@ INT WINAPI CompareStringEx(LPCWSTR locale, DWORD flags, LPCWSTR str1, INT len1,
                            LPCWSTR str2, INT len2, LPNLSVERSIONINFO version, LPVOID reserved, LPARAM lParam)
 {
     DWORD supported_flags = NORM_IGNORECASE|NORM_IGNORENONSPACE|NORM_IGNORESYMBOLS|SORT_STRINGSORT
-                           |NORM_IGNOREKANATYPE|NORM_IGNOREWIDTH|LOCALE_USE_CP_ACP;
-    DWORD semistub_flags = NORM_LINGUISTIC_CASING|LINGUISTIC_IGNORECASE|0x10000000;
-    /* 0x10000000 is related to diacritics in Arabic, Japanese, and Hebrew */
+                           |NORM_IGNOREKANATYPE|NORM_IGNOREWIDTH|LOCALE_USE_CP_ACP|NORM_STOP_ON_NULL;
+    DWORD semistub_flags = NORM_LINGUISTIC_CASING|LINGUISTIC_IGNORECASE;
+
     INT ret;
 
     if (version) FIXME("unexpected version parameter\n");
@@ -3257,10 +3257,10 @@ INT WINAPI CompareStringEx(LPCWSTR locale, DWORD flags, LPCWSTR str1, INT len1,
     }
 
     if (flags & semistub_flags)
-        FIXME("semi-stub behavor for flag(s) 0x%x\n", flags & semistub_flags);
+        FIXME("semi-stub behaviour for flag(s) 0x%x\n", flags & semistub_flags);
 
-    if (len1 < 0) len1 = strlenW(str1);
-    if (len2 < 0) len2 = strlenW(str2);
+    if (len1 < 0 || (flags & NORM_STOP_ON_NULL) ) len1 = strlenW(str1);
+    if (len2 < 0 || (flags & NORM_STOP_ON_NULL) ) len2 = strlenW(str2);
 
     ret = wine_compare_string(flags, str1, len1, str2, len2);
 
diff --git a/dlls/kernel32/tests/locale.c b/dlls/kernel32/tests/locale.c
index 95e0ac6..02296ec 100644
--- a/dlls/kernel32/tests/locale.c
+++ b/dlls/kernel32/tests/locale.c
@@ -1742,19 +1742,44 @@ static void test_CompareStringA(void)
 
     /* WinXP handles embedded NULLs differently than earlier versions */
     ret = CompareStringA(LOCALE_USER_DEFAULT, 0, "aLuZkUtZ", 8, "aLuZkUtZ\0A", 10);
+    /* Windows 10 - LESS */
     ok(ret == CSTR_LESS_THAN || ret == CSTR_EQUAL, "aLuZkUtZ vs aLuZkUtZ\\0A expected CSTR_LESS_THAN or CSTR_EQUAL, got %d\n", ret);
 
     ret = CompareStringA(LOCALE_USER_DEFAULT, 0, "aLu\0ZkUtZ", 8, "aLu\0ZkUtZ\0A", 10);
+    /* Windows 10 - LESS */
     ok(ret == CSTR_LESS_THAN || ret == CSTR_EQUAL, "aLu\\0ZkUtZ vs aLu\\0ZkUtZ\\0A expected CSTR_LESS_THAN or CSTR_EQUAL, got %d\n", ret);
 
     ret = CompareStringA(lcid, 0, "a\0b", -1, "a", -1);
+    /* Windows 10 - EQUAL */
     ok(ret == CSTR_EQUAL, "a vs a expected CSTR_EQUAL, got %d\n", ret);
 
     ret = CompareStringA(lcid, 0, "a\0b", 4, "a", 2);
+    /* Windows 10 - GREATER */
     ok(ret == CSTR_EQUAL || /* win2k */
        ret == CSTR_GREATER_THAN,
        "a\\0b vs a expected CSTR_EQUAL or CSTR_GREATER_THAN, got %d\n", ret);
 
+    /* Testing NORM_STOP_ON_NULL flag*/
+    ret = CompareStringA(lcid, NORM_STOP_ON_NULL, "don\0a", -1, "don\0b", -1);
+    ok(ret == CSTR_EQUAL,
+       "With str length autodetect (-1) and NORM_STOP_ON_NULL flag - strings `don\\0a` and `don\\0b` expected CSTR_EQUAL , got %d\n", ret);
+
+    ret = CompareStringA(lcid, NORM_STOP_ON_NULL, "don\0a", -1, "don\0b", 5);
+    ok(ret == CSTR_EQUAL,
+       "With NORM_STOP_ON_NULL flag - strings `don\\0a` (-1) and `don\\0b` (5) expected CSTR_EQUAL , got %d\n", ret);
+
+    ret = CompareStringA(lcid, NORM_STOP_ON_NULL, "don\0a", 5, "don\0b", 5);
+    ok(ret == CSTR_EQUAL,
+       "With NORM_STOP_ON_NULL flag - strings `don\\0a` (-1) and `don\\0b` (5) expected CSTR_EQUAL , got %d\n", ret);
+
+    ret = CompareStringA(lcid, 0 , "don\0a", 5, "don\0b", 5);
+    ok(ret == CSTR_LESS_THAN,
+       "Without NORM_STOP_ON_NULL flag - strings `don\\0a` (5) and `don\\0b` (5) expected CSTR_LESS_THAN, got %d\n", ret);
+
+    ret = CompareStringA(lcid, 0 , "don\0a", 5, "don\0b", -1);
+    ok(ret == CSTR_GREATER_THAN,
+       "Without NORM_STOP_ON_NULL flag - strings `don\\0a` (5) and `don\\0b` (-1) expected CSTR_GREATER_THAN, got %d\n", ret);
+
     ret = CompareStringA(lcid, 0, "\2", 2, "\1", 2);
     todo_wine ok(ret != CSTR_EQUAL, "\\2 vs \\1 expected unequal\n");
 
@@ -1951,6 +1976,39 @@ static void test_CompareStringEx(void)
     WCHAR locale[6];
     INT ret, i;
 
+    /* NORM_STOP_ON_NULL flag tests*/
+
+    const WCHAR a[] = {'d','o','n',0,'a','t',0}; // don\0at
+    const WCHAR b[] = {'d','o','n',0,'_','_',0}; // don\0__
+
+    ret = CompareStringEx(LOCALE_NAME_SYSTEM_DEFAULT, NORM_STOP_ON_NULL, a, -1, b, -1, NULL, NULL, 0);
+    ok(ret == CSTR_EQUAL,
+       "NORM_STOP_ON_NULL passed, though its equal to length autodetect (-1) - strings `don\\0at` (-1) and `don\\__` (-1) expected CSTR_EQUAL, got %d\n", ret);
+
+    ret = CompareStringEx(LOCALE_NAME_SYSTEM_DEFAULT, 0, a, -1, b, -1, NULL, NULL, 0);
+    ok(ret == CSTR_EQUAL,
+       "NORM_STOP_ON_NULL is not set, strings considered as null terminated - so strings `don\\0at` (-1) and `don\\__` (-1) expected CSTR_EQUAL, got %d\n", ret);
+
+    ret = CompareStringEx(LOCALE_NAME_SYSTEM_DEFAULT, NORM_STOP_ON_NULL, a, 6, b, 6, NULL, NULL, 0);
+    ok(ret == CSTR_EQUAL,
+       "NORM_STOP_ON_NULL passed, and lengths are defined exlicitly - so strings `don\\0at` (6) and `don\\__` (6) are expected CSTR_EQUAL due to flag, got %d\n", ret);
+
+    ret = CompareStringEx(LOCALE_NAME_SYSTEM_DEFAULT, NORM_STOP_ON_NULL, a, 6, b, -1, NULL, NULL, 0);
+    ok(ret == CSTR_EQUAL,
+       "NORM_STOP_ON_NULL passed, and length is defined exlicitly for one string - anyway strings `don\\0at` (6) and `don\\__` (-1) are expected CSTR_EQUAL due to flag, got %d\n", ret);
+
+    const WCHAR c[] = {'d','O','n',0,'A','t',0}; // dOn\0At
+
+    ret = CompareStringEx(LOCALE_NAME_SYSTEM_DEFAULT, NORM_STOP_ON_NULL | NORM_IGNORECASE, a, -1, c, -1, NULL, NULL, 0);
+    ok(ret == CSTR_EQUAL,
+       "NORM_STOP_ON_NULL | NORM_IGNORECASE passed, though its equal to length autodetect (-1) - so strings `don\\0at` (-1) and `dOn\\At` (-1) expected CSTR_EQUAL, got %d\n", ret);
+
+    ret = CompareStringEx(LOCALE_NAME_SYSTEM_DEFAULT, NORM_STOP_ON_NULL | NORM_IGNORECASE, a,  6, c, 6, NULL, NULL, 0);
+    ok(ret == CSTR_EQUAL,
+       "NORM_STOP_ON_NULL | NORM_IGNORECASE passed, and lengths are defined explicitely - so strings `don\\0at` (-1) and `dOn\\At` (-1) expected CSTR_EQUAL, got %d\n", ret);
+
+    /* end of NORM_STOP_ON_NULL test */
+
     /* CompareStringEx is only available on Vista+ */
     if (!pCompareStringEx)
     {
@@ -1976,6 +2034,7 @@ static void test_CompareStringEx(void)
         }
     }
 
+
 }
 
 static void test_LCMapStringA(void)
diff --git a/include/winnls.h b/include/winnls.h
index 3dacc09..3b5ecb0 100644
--- a/include/winnls.h
+++ b/include/winnls.h
@@ -352,6 +352,7 @@ static const WCHAR LOCALE_NAME_SYSTEM_DEFAULT[] = {'!','s','y','s','-','d','e','
 #define NORM_IGNOREKANATYPE        0x00010000
 #define NORM_IGNOREWIDTH           0x00020000
 #define NORM_LINGUISTIC_CASING     0x08000000
+#define NORM_STOP_ON_NULL          0x10000000
 
 #define CP_ACP        0
 #define CP_OEMCP      1

-- 
2.1.0