[PATCH 08 of 10 shelve-ext v3] shelve: add obs-based unshelve functionality

[Sean Farley]

Jun Wu <quark at fb.com> writes:

> Excerpts from Sean Farley's message of 2017-02-06 11:41:55 -0800:
>> Perhaps I'm the only one worried about adding all these markers. To me,
>> it would make more sense to delete these markers since we're never going
>> to push these shelved changesets.
>
> You are not the only one.
>
> While I agree that the perf of obsstore is a pain, all possible fixes that I
> can think of require too much work that should not block this series. Let's
> talk about it during the sprint, or earlier if you want.
>
> I'd like to note that the traditional "strip" is while useful, not friendly
> to a giant repo - it's slow and dangerous.
>
> I personally like to see a data structure similar with "sparkey" [1]
> implemented in core. The data structure has two parts:
>
>   - data:  append-only, include deleting operations (like a journal)
>   - index: NOT append-only; do not include pointers to deleted data
>            could be rebuilt from data
>
> For the index part, sparkey uses a hash-table. We may or may not want a
> git-packfile-index-like thing, depending if the "key"s are all SHA1s or not.
>
> That would be useful to all kinds of "quick-random-lookups" without
> loading/building the whole index problems, although some of them could be
> worked around by the chg repo preloading plan.
>
> [1]: https://github.com/spotify/sparkey

I've seen you mention this before and I really like it.

>> Also, this sounds like we're overloading the concept of markers. In the
>> past, I have wanted a concept of "markers that hide changesets" but to
>> group them into different namespaces. For example, remote branches not
>> checked out locally could be hidden into one such group, shelved changes
>> into another group, etc.
>
> That sounds related to Stanislau's bitmap approach. "hidden" is not
> necessarily implemented as obsmarkers. They can come from other sources:
>
>   obsstore -> obsstore hidden bitmap -+
>                                       |
>               shelve hidden bitmap   -+- (bitmap or) -> final hidden bitmap
>                                       |
>               other hidden bitmap    -+

Yep, exactly what I was thinking. I'll move over to that thread to save
from derailing Kostia's thread.