DarcsTwo

Darcs 2

Darcs-2 was released in early April 2008.

Darcs-2 features (apart from numerous bug and performance regression fixes) a completely rewritten rollback command and new progress-reporting functionality. If darcs takes more than a couple of seconds for a given operation and provides you with no feedback as to what it's up to, let us know, so we can fine-tune the progress-reporting output!

Darcs 2 features user-visible changes in two broad categories, and several under-the-hood improvements designed to improve code stability and safety. The user-visible changes are a new "hashed" repository format, and the new darcs-2 conflict handling. The new "hashed" repository format can be used in a manner that is interchangeable with older darcs--although older versions of darcs cannot read the hashed format, darcs 2 allows you to exchange patches between repositories in new and old formats. The new conflict handling requires a repository conversion that is not backwards-compatible, so projects switching to darcs-2 format will have require that all their users upgrade to darcs 2.

Getting darcs 2

See http://darcs.net/ .

Enabling a global cache

Enabling a global cache saves you from downloading the same patch twice for two different repos you have on the same machine. It is very simple to enable a global cache, and it can save you much time and bandwidth. Simply execute

mkdir -p $HOME/.darcs/cache
echo cache:$HOME/.darcs/cache > $HOME/.darcs/sources

This will cause darcs to store hard links in ~/.darcs/cache. It is always safe to delete this directory.

Darcs-2 the format

The future of darcs is in the darcs-2 repository format, which features a new merge algorithm that introduces two major user-visible changes

  1. It should no longer be possible to confuse darcs or freeze it indefinitely by merging conflicting changes.
  2. Identical primitive changes no longer conflict. This is a long-requested feature, and has far-reaching implications. See below (the section on "new semantics") for a discussion of these implications.

Creating a repository in the darcs-2 format

Converting an existing repository to the darcs-2 format is as easy as

darcs convert oldrepository newrepository

The convert command runs slowly. Perform this command once per project, not once for each repo. The conversion is not reversible and its result is dependent on the order of patches in your repository. Projects should switch to darcs-2 format in unison.

You can also create a fresh repository with

darcs initialize

That is, darcs-2 format is the default format for new repositories.

Changes in semantics

When using the darcs-2 format, darcs treats identical primitive patches as the same patch. This has dramatic implications in how darcs-2 will define dependencies. In particular, dependencies (except those explicitly created by the use with --add-deps) are always dependencies on a given primitive patch, not on a given named patch. This means that the change named "foo" may in effect depend on either the change named "bar" or the change named "baz". This prerelease of darcs 2 has not been fully converted to always take advantage of these new semantics--it will not cause corruption, but under unusual circumstances, could exit with an error. We need to decide how to handle these semantics in the user interface.

A simple example

Let me illustrate what could happen with a story. Steve creates changes "A" and "B":

steve$ echo A > foo
steve$ darcs add foo
steve$ darcs record -m Anote
steve$ echo B > foo
steve$ darcs record -m Bnote

Meanwhile, Monica also decides she'd like a file named foo, and she also wants it to contain A, but she also wants to make some other changes:

monica$ echo A > foo
monica$ darcs add foo
monica$ echo Z > bar
monica$ darcs add bar
monica$ darcs record -m AZnote

At this point, Monica pulls from Steve:

monica$ darcs pull ../steve

but she decides she prefers her AZ change, to Steve's A change, and being a harsh person, she decides to obliterate his change:

monica$ darcs obliterate  --match 'exact Anote' --all

At this point, darcs 1 would complain, pointing out that patch B depends on patch A. However, darcs 2 will happily obliterate patch A, because patch AZ provides the primitive patches that B depends upon. At this point, however, we run into the limitations of this prerelease version of darcs 2: If Steve pulls from Monica, his darcs will fail, because the common set of patches (which is only B-) cannot exist without either A or AZ. I plan to fix this behavior, but the internal API for doing so is not at all clear to me, which is why I'm looking for input from others. But note that this situation can only occur if users take advantage of the new semantics, which I suspect will be relatively seldom, until we give them tools to more easily do so.

Hashed repository format

If you don't want to convert your repositories to the darcs 2 format, you can enjoy many (but not all) of its benefits by adopting the hashed format. Darcs 2 operating on a repository in the hashed format can exchange patches with repositories in darcs 1's old format using push and pull. Though, darcs 1 will be unable to exchange patches with that repository. Hence, the hashed format is always safe on a "client" machine, but if you have other people pulling from your repo, or sending to it, then you have to make sure they have darcs 2 before going hashed.

The hashed repository format has a number of changes that are visible to users.

  1. The hashed format allows for greater atomicity of operations. This makes for greater safety and simultaneously greater efficiency. These benefits, however, have not been fully realized in this release. For instance, with a hashed repository, there is no need for darcs push to require a repository lock, so you could record patches while waiting for a push to finish (for instance, if it's waiting on the test suite).
  2. The _darcs/pristine directory no longer holds the pristine cache. This disallows certain hackish short-cuts, but also dramatically reduces the danger of third-party programs (e.g. !DreamWeaver) recursing into the pristine cache and corrupting darcs repositories.
  3. Darcs get is now much faster, and always operates in a "lazy" fashion, meaning that patches are downloaded only when they are needed. This gives us much of the benefits of --partial repositories, without most of their disadvantages. This approach, however, does have several new dangers. First, some operations may unexpectedly require the download of large numbers of patches, which could be slow (but you could always interrupt with ^C). Secondly, if the source repository disappears, or you lose network connectivity, some operations may fail. I do not believe these dangers will prove particularly problematic, but we may need to fine-tune the user interface to make it more clear what is going on.
  4. Darcs now supports caching of patches and file contents to reduce bandwidth and save disk space. See below for how to enable this. In my opinion, this is actually the most exciting new feature, as it greatly speeds up a number of operations, and is essentially transparent. The only reason we don't enable it by default is because I'm uncomfortable creating a large directory in ~/.darcs/cache without the user's explicit consent.

Creating a repository in the hashed format

Creating a hashed repository is as easy as

darcs get --hashed oldrepository newrepository

or alternatively you could create a fresh repository with

darcs initialize --hashed

You can push, pull and send patches at will between hashed and old-fashioned repositories, so you should be able to experiment with this format even on projects that you do not control.

Upgrading from darcs 1

Three upgrade sizes

You could say that three upgrade "sizes": small, medium and large. The small upgrade consists in using the darcs 2 software without changing a single thing. The medium upgrade consists in converting your repositories to a fully backwards compatible 'halfway' format (hashed repositories). The large upgrade means going all the way and converting to the darcs 2 format.

If you use the darcs 2 software on old darcs 1 repositories, you get

  • greater stability (fewer 'bug in darcs' messages)
  • faster HTTP access (needs --with-curl-pipelining)
  • faster ssh operations (needs darcs 2 on the other end too)

  • more progress reporting! If you upgrade your repositories to the hashed format you also get

  • full compatibility with old darcs 1 repositories
  • more corruption-proofing
  • better handling of case-insensitive file systems

  • (optional) lazy darcs get (a safer alternative to partial repositories) And if you start using the new darcs 2 format, you lose backwards compatibility, but then you also get

  • improved merging, hurrah! (most 'exponential' merges just zip right through)

  • more intuitive handling of duplicate patches (identical patches no longer conflict)

Zooko's recommendations

  1. Upgrade any executables you have to the latest stable version of darcs immediately. The new executables are fully backwards compatible -- you will not have any compatibility problems as a result of upgrading. They are also much less buggy and faster than older darcs executables.
  2. (There are at least two known issues with the darcs-2.0.2 executable which cause performance problems -- one is a issue with ghc v6.8.2 and the other is an issue with the latest version of libcurl -- v7.18.2. Hopefully the darcs core team will release darcs-2.0.3 soon to fix these two issues, but even if darcs-2.0.2 is still the current version, I recommend it over any previous version.) If your use of darcs is currently satisfying (fast enough, no bugs getting in your way), then stop at this step. You are now using the latest stable darcs executable while using old-fashioned darcs-v1 repositories. Other people who are using older versions of darcs can continue to share patches with your repositories just like always.
3. If there are problems with this situation of using darcs-2

executables and darcs-1-format repositories, then report it to the darcs-users mailing list, and then make a new hashed-format repository in parallel with your existing darcs-1-format repository, like this: ``

darcs get --hashed ${repo} ${repo}-hashedformat

Now people who have new darcs-v2 can push and pull to the hashedformat repository and people who have darcs-v1 can continue to push and pull to the original repository. You can transfer patches between those repositories with normal darcs push and pull. For those projects of mine which have a high rate of patches I have gone ahead and added post-apply hooks to automatically do a "push --all" to the other one, like this:

echo "apply posthook darcs push --all --repodir=${repo}-hashedformat ${repo}" >> ${repo}-hashedformat/_darcs/prefs/defaults echo "apply run-posthook" >> ${repo}-hashedformat/_darcs/prefs/defaults echo "apply posthook darcs push --all --repodir=${repo} ${repo}-hashedformat" >> ${repo}/_darcs/prefs/defaults echo "apply run-posthook" >> ${repo}/_darcs/prefs/defaults `` 4. (Note that this is mutually recursive, but it terminates because the post-apply hook doesn't trigger in the case that zero patches were pushed.) If your use of darcs is currently satisfying then stop at this step. You are now using the latest stable darcs executable, and maintaining two darcs repositories -- an old-fashioned darcs-1-format repository for your darcs-1-executable-using friends and a hashed-format repository for your darcs-2-executable using friends. Your post- apply hooks cause these two repositories to get automatically synced with each other.

  1. If there are problems with this situation of using darcs-2 executables and a pair of repos -- one of which is darcs-1-format repo for the use of darcs-1-executable-using users and the other of which is a hashed-format repo for the use of darcs-2-executable-using users -- then report it to the darcs-users mailing list and await further instructions. Note that these instructions do not involve any use of darcs-2-format repositories. Those are only for projects which (a) have no darcs-1-executable-using users, and (b) have no extant branches stored in darcs-1-format or hashed-format repositories. Personally step 2 has satisfied all my needs so I use the step 2 strategy for all my publicly shared repositories, although I do tend to create new private repositories with darcs-2-format.