Subject: [Fwd: Cheaper options for VC import]
From: Benjamin Smedberg <[email-masked]>
To: Boris Zbarsky <[email-masked]>
Cc: Benjamin Smedberg <[email-masked]>
Date: Fri, 20 Apr 2007 16:14:46 -0400

I'm hoping that I can post this thread on the web directly, but here it is.

--BDS

Subject: Cheaper options for VC import
From: Graydon Hoare <[email-masked]>
To: "J. Paul Reed" <[email-masked]>
Cc: Vladimir Vukicevic <[email-masked]>, Mike Schroepfer <[email-masked]>, Brendan Eich <[email-masked]>, [email-masked], Benjamin Smedberg <[email-masked]>, Damon Sicore <[email-masked]>
Date: Tue, 13 Feb 2007 15:01:37 -0800

Hi,

I've been muttering about this for a few days and brendan asked me to 
produce a coherent description of my mutterings.

I think importing the whole CVS history into Hg is not necessary. I'd 
like to suggest simply importing the head state of the 1.9 trunk and 
continuing to make snapshot imports of it as time progresses, then 
forming a 2.0 branch off the snapshot branch and putting both up on a 
public Hg server.

This is what developers are already doing on our workstations (at least 
myself, vlad and dbaron are) and it works well enough. CVS appears as an 
opaque "upstream" and you get to do all your "downstream" work with 
nicer tools. I did the same thing all year for work on the cycle 
collector and valgrind annotation patches, using monotone.

The incoming work from CVS gets reasonably sophisticated *forward* 
merging into the downstream branches (which there are many of). 
Individual changes that you want to backport to CVS/1.9 need to be 
exported from Hg, applied to a CVS workspace and committed a second 
time. You'd have to do this even if you did a full CVS import, so that's 
no loss: backports will hurt so long as you're backporting *to* CVS.

Such a setup will not suffice for archeology: "annotate" for example 
will not show you correct history. You will need to use CVS, either 
through bonsai or a local checkout. You will want to keep the CVS server 
alive indefinitely for this reason.

This scenario also has the advantage that the initial Hg pull will be 
small: developers won't be pulling 10 years of mozilla history (yet). In 
10 more years when we've developed that much Hg state, bandwidth and 
disk will be much cheaper. It also has the advantage that we know it's 
possible. We can do it within an afternoon, rather than fighting with 
cranky CVS importer tools for another few months.

This lowers the activation cost and gets us working with better tools 
*now*. It's a compromise, but I think a good one that several of us have 
already demonstrated the validity of.

-graydon

Subject: Re: Cheaper options for VC import
From: Vladimir Vukicevic <[email-masked]>
To: Graydon Hoare <[email-masked]>
Cc: Mike Schroepfer <[email-masked]>, Brendan Eich <[email-masked]>, [email-masked], Benjamin Smedberg <[email-masked]>, Damon Sicore <[email-masked]>, "J. Paul Reed" <[email-masked]>
Date: Tue, 13 Feb 2007 16:26:44 -0800 (PST)

I suggested this a while ago as well, though I'm not so convinced it's a good idea any more.  I think importing the current state of HEAD would be fine, as long as all future imports preserved individual commits/changesets.  I don't care about having the historical stuff, but I do think that we need to have individual commits, even from CVS, going forward; it will make merging and other work much, much simpler.

   - Vlad

----- "Graydon Hoare" <[email-masked]> wrote:
> Hi,
> 
> I've been muttering about this for a few days and brendan asked me to
> 
> produce a coherent description of my mutterings.
> 
> I think importing the whole CVS history into Hg is not necessary. I'd
> 
> like to suggest simply importing the head state of the 1.9 trunk and 
> continuing to make snapshot imports of it as time progresses, then 
> forming a 2.0 branch off the snapshot branch and putting both up on a
> 
> public Hg server.
> 
> This is what developers are already doing on our workstations (at
> least 
> myself, vlad and dbaron are) and it works well enough. CVS appears as
> an 
> opaque "upstream" and you get to do all your "downstream" work with 
> nicer tools. I did the same thing all year for work on the cycle 
> collector and valgrind annotation patches, using monotone.
> 
> The incoming work from CVS gets reasonably sophisticated *forward* 
> merging into the downstream branches (which there are many of). 
> Individual changes that you want to backport to CVS/1.9 need to be 
> exported from Hg, applied to a CVS workspace and committed a second 
> time. You'd have to do this even if you did a full CVS import, so
> that's 
> no loss: backports will hurt so long as you're backporting *to* CVS.
> 
> Such a setup will not suffice for archeology: "annotate" for example 
> will not show you correct history. You will need to use CVS, either 
> through bonsai or a local checkout. You will want to keep the CVS
> server 
> alive indefinitely for this reason.
> 
> This scenario also has the advantage that the initial Hg pull will be
> 
> small: developers won't be pulling 10 years of mozilla history (yet).
> In 
> 10 more years when we've developed that much Hg state, bandwidth and 
> disk will be much cheaper. It also has the advantage that we know it's
> 
> possible. We can do it within an afternoon, rather than fighting with
> 
> cranky CVS importer tools for another few months.
> 
> This lowers the activation cost and gets us working with better tools
> 
> *now*. It's a compromise, but I think a good one that several of us
> have 
> already demonstrated the validity of.
> 
> -graydon

Subject: Re: Cheaper options for VC import
From: Benjamin Smedberg <[email-masked]>
To: Graydon Hoare <[email-masked]>
Cc: "J. Paul Reed" <[email-masked]>, Vladimir Vukicevic <[email-masked]>, Mike Schroepfer <[email-masked]>, Brendan Eich <[email-masked]>, [email-masked], Damon Sicore <[email-masked]>
Date: Tue, 20 Feb 2007 15:31:38 -0500

Graydon Hoare wrote:
> Hi,
> 
> I've been muttering about this for a few days and brendan asked me to 
> produce a coherent description of my mutterings.
> 
> I think importing the whole CVS history into Hg is not necessary. I'd 
> like to suggest simply importing the head state of the 1.9 trunk and 
> continuing to make snapshot imports of it as time progresses, then 
> forming a 2.0 branch off the snapshot branch and putting both up on a 
> public Hg server.
> 
> This is what developers are already doing on our workstations (at least 
> myself, vlad and dbaron are) and it works well enough. CVS appears as an 
> opaque "upstream" and you get to do all your "downstream" work with 
> nicer tools. I did the same thing all year for work on the cycle 
> collector and valgrind annotation patches, using monotone.
> 
> The incoming work from CVS gets reasonably sophisticated *forward* 
> merging into the downstream branches (which there are many of). 
> Individual changes that you want to backport to CVS/1.9 need to be 
> exported from Hg, applied to a CVS workspace and committed a second 
> time. You'd have to do this even if you did a full CVS import, so that's 
> no loss: backports will hurt so long as you're backporting *to* CVS.
> 
> Such a setup will not suffice for archeology: "annotate" for example 
> will not show you correct history. You will need to use CVS, either 
> through bonsai or a local checkout. You will want to keep the CVS server 
> alive indefinitely for this reason.
> 
> This scenario also has the advantage that the initial Hg pull will be 
> small: developers won't be pulling 10 years of mozilla history (yet). In 
> 10 more years when we've developed that much Hg state, bandwidth and 
> disk will be much cheaper. It also has the advantage that we know it's 
> possible. We can do it within an afternoon, rather than fighting with 
> cranky CVS importer tools for another few months.
> 
> This lowers the activation cost and gets us working with better tools 
> *now*. It's a compromise, but I think a good one that several of us have 
> already demonstrated the validity of.

Preed tells me I'm tilting against the tide here, but I think this is a 
really bad idea. Our log/annotation data is extremely valuable, to new 
hackers and veterans. See, for instance, shaver's talk at Seneca about how 
to use LXR to get familiar with the Mozilla codebase:

http://vocamus.net/dave/?p=6

If we have disjunct history implementations where Hg has an arbitrary start 
point, doing historical archaeology becomes almost impossible (i.e. blame 
info in neither system is correct).

If we follow vlad's proposal and import a flat tree once and then do 
progressive imports of changesets after that time, we'd still need an import 
mechanism for those changesets. And once we have that, why not go back to 
the beginning of time, instead of starting now?

I cannot imagine maintaining rules.mk without reliable blame/log history 
which goes back to 1999.

--BDS

Subject: Re: Cheaper options for VC import
From: Brendan Eich <[email-masked]>
To: Benjamin Smedberg <[email-masked]>
Cc: Graydon Hoare <[email-masked]>, "J. Paul Reed" <[email-masked]>, Vladimir Vukicevic <[email-masked]>, Mike Schroepfer <[email-masked]>, [email-masked], Damon Sicore <[email-masked]>
Date: Tue, 20 Feb 2007 16:39:49 -0800

On Feb 20, 2007, at 12:31 PM, Benjamin Smedberg wrote:

> Graydon Hoare wrote:
>> Such a setup will not suffice for archeology: "annotate" for  
>> example will not show you correct history. You will need to use  
>> CVS, either through bonsai or a local checkout. You will want to  
>> keep the CVS server alive indefinitely for this reason.
>> This scenario also has the advantage that the initial Hg pull will  
>> be small: developers won't be pulling 10 years of mozilla history  
>> (yet). In 10 more years when we've developed that much Hg state,  
>> bandwidth and disk will be much cheaper. It also has the advantage  
>> that we know it's possible. We can do it within an afternoon,  
>> rather than fighting with cranky CVS importer tools for another  
>> few months.
>> This lowers the activation cost and gets us working with better  
>> tools *now*. It's a compromise, but I think a good one that  
>> several of us have already demonstrated the validity of.
>
> Preed tells me I'm tilting against the tide here, but I think this  
> is a really bad idea. Our log/annotation data is extremely  
> valuable, to new hackers and veterans. See, for instance, shaver's  
> talk at Seneca about how to use LXR to get familiar with the  
> Mozilla codebase:
>
> http://vocamus.net/dave/?p=6
>
> If we have disjunct history implementations where Hg has an  
> arbitrary start point, doing historical archaeology becomes almost  
> impossible (i.e. blame info in neither system is correct).

It's not impossible, it's just harder.  I've had to do blame chasing  
"up" a CVS branch or two.

> If we follow vlad's proposal and import a flat tree once and then  
> do progressive imports of changesets after that time, we'd still  
> need an import mechanism for those changesets. And once we have  
> that, why not go back to the beginning of time, instead of starting  
> now?

Because we can't get hg to import everything, and the initial pull is  
huge.

> I cannot imagine maintaining rules.mk without reliable blame/log  
> history which goes back to 1999.

I do archeology all the time too.  But as Graydon noted,  
cvs.mozilla.org will still be there. We are talking about a cost  
increase, a tool tax: not a cliff that can't be scaled. The win is  
that we actually get going on Mozilla 2 this quarter, which is past  
the half-way point with nothing yet to show, at this point for want  
of a repository. That's a very bad condition.

/be

>
> --BDS
>