D4164: index: move check for too large capacity into nt_init()

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  It's clearer to have the check just before the allocation happens.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4164

AFFECTED FILES
  mercurial/cext/revlog.c

CHANGE DETAILS

diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c
--- a/mercurial/cext/revlog.c
+++ b/mercurial/cext/revlog.c
@@ -1069,6 +1069,10 @@
 	self->capacity = capacity;
 	self->depth = 0;
 	self->splits = 0;
+	if ((size_t)self->capacity > INT_MAX / sizeof(nodetreenode)) {
+		PyErr_SetString(PyExc_ValueError, "overflow in init_nt");
+		return -1;
+	}
 	self->nodes = calloc(self->capacity, sizeof(nodetreenode));
 	if (self->nodes == NULL) {
 		PyErr_NoMemory();
@@ -1133,10 +1137,6 @@
 static int index_init_nt(indexObject *self)
 {
 	if (self->nt == NULL) {
-		if ((size_t)self->raw_length > INT_MAX / sizeof(nodetreenode)) {
-			PyErr_SetString(PyExc_ValueError, "overflow in index_init_nt");
-			return -1;
-		}
 		self->nt = PyMem_Malloc(sizeof(nodetree));
 		if (self->nt == NULL) {
 			PyErr_NoMemory();



To: martinvonz, #hg-reviewers
Cc: mercurial-devel