[PATCH] convert: when getting file from Perforce concatenate data at the end

Eugene Baranov eug.baranov at gmail.com
Wed Jul 29 18:58:55 CDT 2015


# HG changeset patch
# User Eugene Baranov <eug.baranov at gmail.com>
# Date 1438214285 -3600
#      Thu Jul 30 00:58:05 2015 +0100
# Branch stable
# Node ID ced462474db13d175e76168014e94ea7be40b764
# Parent  7d2b42b3a15039ac01c218ce8e9d04854f8a7e02
convert: when getting file from Perforce concatenate data at the end

As it turned out, even when getting relatively small files, concatenating
string data every time when new chunk is received is very inefficient.
Maintaining a string list of data chunks and concatenating everything in one go
at the end seems much more efficient - in my testing it made getting 40 MB file
7 times faster, whilst converting of a particularly big changelist with some big
files went down from 20 hours to 3 hours.

diff -r 7d2b42b3a150 -r ced462474db1 hgext/convert/p4.py
--- a/hgext/convert/p4.py	Fri Jul 24 15:10:18 2015 +0100
+++ b/hgext/convert/p4.py	Thu Jul 30 00:58:05 2015 +0100
@@ -304,7 +304,7 @@
             stdout = self.p4.runcommand(cmd)
 
             mode = None
-            contents = ""
+            contents = []
             keywords = None
 
             for d in loaditer(stdout):
@@ -340,7 +340,7 @@
                             keywords = self.re_keywords
 
                 elif code == "text" or code == "binary":
-                    contents += data
+                    contents.append(data)
 
                 lasterror = None
 
@@ -350,6 +350,8 @@
         if mode is None:
             return None, None
 
+        contents = ''.join(contents)
+
         if keywords:
             contents = keywords.sub("$\\1$", contents)
         if mode == "l" and contents.endswith("\n"):


More information about the Mercurial-devel mailing list