C-hglib - Level 0, API proposal.

[Giovanni Gherdovich]

Hello Iulian,

Iulian wrote:
> b) hg log <- can produce huge output
>
> [...]
> My single thought right now is to set a limit for the huge mass of data.

I am examinig 3 documents, i.e

[1] Matt's code snippet at
http://www.selenic.com/pipermail/mercurial-devel/2013-June/051737.html

[2] IRC logs from last Tue, July 2nd (topic: exit code), which I report at
http://bpaste.net/show/FW3tCYalMJIjNkgRwyOG/

[3] Iulian second iteration of the header, at
http://selenic.com/pipermail/mercurial-devel/2013-July/051862.html

and I have a doubts; from those [2] and [3] it appears
that Iulian did not realize the following:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++
hg_rawread will (should) be called *multiple* times in a while loop,
as Matt shows in [1]. In a snippet like Matt's one, or

while( read( fd, buf, 4096 ) ) { /* do stuff with buf */ }

you are not filling up your RAM, even if the file pointed by fd
was 1 YottaByte big. You're just sliding a 4K window across a
(possibly) huber-huge file.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++

>From what I read in [2], you Iulians write:

"I think that you cannot keep 500G in memory for a single output." (line 72)
and
"I understand that it's impossible to return a message of 500G,
in a single call." (line 82)

and from what I read in [3], Iulian's email reads

-------------
[...]
        case 'o':
            /* Make something with data save it into a data structure*/
            hg_rawread(handle, buffer , size);
            save_into_out_channel(buffer);
[...]
-------------

Are my concerns correct?

Did you, Iulians, overlooked that while loop in [1]?
That's a very basic pattern on how you use the read(2) syscall.
(yes yes read(2) returns -1 in case of error, which would be
bool-evaluated to true, it's just an "almost correct" example).
That's okay, no problem, me too before knowing that, I didn't know it.

Please let me know (mail or IRC) if this was of help and we're one step
closer to sattle this first part.

Cheers,
Giovanni
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20130709/a3e4ae0c/attachment.html>