[PATCH] Fixed some bugs due to pointer overflow. In the four cases fixed, the developer wrote a check to see if the sum of a pointer and a positive number was less than the value of the pointer, intending to check for pointer overflow. Unfortunately, pointer overflow is undefined behavior in c, so the compiler is free to optimize out this overflow check. I replaced these checks with safer checks that comply with the C standard and won't get optimized out

[Eric Lubin]

# HG changeset patch
# User Eric Lubin <eric at lubin.us>
# Date 1386282190 28800
# Node ID 005a0b96477ed6cde4e880861c9e7a2239ed7770
# Parent  1c92524c37cdd251c1a36b2da0fb4148b0e6ba09
Fixed some bugs due to pointer overflow. In the four cases fixed, the developer wrote a check to see if the sum of a pointer and a positive number was less than the value of the pointer, intending to check for pointer overflow. Unfortunately, pointer overflow is undefined behavior in c, so the compiler is free to optimize out this overflow check. I replaced these checks with safer checks that comply with the C standard and won't get optimized out.

diff -r 1c92524c37cd -r 005a0b96477e mercurial/mpatch.c
--- a/mercurial/mpatch.c	Sun Dec 01 21:24:48 2013 -0600
+++ b/mercurial/mpatch.c	Thu Dec 05 14:23:10 2013 -0800
@@ -218,7 +218,7 @@
 		if (lt->start > lt->end)
 			break; /* sanity check */
 		bin = data + lt->len;
-		if (bin < data)
+		if (data > UINTPTR_MAX - lt->len)
 			break; /* big data + big (bogus) len can wrap around */
 		lt->data = data;
 		data = bin + 12;
@@ -372,7 +372,7 @@
 		if (start > end)
 			break; /* sanity check */
 		bin = data + len;
-		if (bin < data)
+		if (data > UINTPTR_MAX - len)
 			break; /* big data + big (bogus) len can wrap around */
 		data = bin + 12;
 		outlen += start - last;
diff -r 1c92524c37cd -r 005a0b96477e mercurial/parsers.c
--- a/mercurial/parsers.c	Sun Dec 01 21:24:48 2013 -0600
+++ b/mercurial/parsers.c	Thu Dec 05 14:23:10 2013 -0800
@@ -186,7 +186,7 @@
 		mtime = getbe32(cur + 9);
 		flen = getbe32(cur + 13);
 		cur += 17;
-		if (cur + flen > end || cur + flen < cur) {
+		if (cur + flen > end || cur > UINTPTR_MAX - flen) {
 			PyErr_SetString(PyExc_ValueError, "overflow in dirstate");
 			goto quit;
 		}
@@ -1700,8 +1700,8 @@
 		len++;
 		old_data = data;
 		data += incr;
-		if (data <= old_data)
-			break;
+		if (old_data > UINTPTR_MAX - incr)
+			break; /* check for wrap around */
 	}
 
 	if (data != end && data + v1_hdrsize != end) {