|
Size: 7129
Comment: tweak that
|
Size: 13941
Comment: Overhaul the configuration sections to help others avoid and understand the nasty gotchas I got bitten by
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 21: | Line 21: |
| As a result of these differences, when people using different operating systems are working on the same repository, there is a real risk that someone's text editor will change all the line endings in the files they edit. If they commit these changes, then unfortunately the diff for their commit will be trashed, and hg annotate/blame for the touched files will be useless. | As a result of these differences, when people using different operating systems are working on the same repository, there is a real risk that someone's text editor will change the line endings in the files that they edit. If they commit these line ending changes, then unfortunately the diff for their commit will be trashed, and hg annotate/blame for the touched files will be useless. |
| Line 23: | Line 23: |
| The eol extension solves this problem by allowing a canonical line ending type to be specified for certain files in a repository, and by converting seamlessly between the canonical type and the user's native type as the user checks out and commits changes to these files. In this way user's get to work with files using their native line ending type while the conversion process makes sure the wrong line endings aren't committed to those files. The extension also allows certain files to have a fixed line ending type both internally in the repository and when checked out, and for others to be treated as binary files. | The eol extension solves this problem by allowing a native (canonical) line ending type to be specified for the repository, and by converting seamlessly between the repository's and the user's native types as the user checks out and commits changes to select files. In this way user's get to work with files using their native line ending type while the conversion process makes sure the wrong line endings aren't committed to those files. The extension also allows certain files to have a fixed line ending type both internally in the repository and when checked out (useful for Makefiles/.vcproj files, for example), and for others to be treated as binary files. |
| Line 25: | Line 25: |
| == Configuration == | == Per-user configuration == |
| Line 27: | Line 27: |
| '''Important:''' before you turn on the eol extension, first make sure that you have no outstanding local changes in your working directory. The (important) steps below to make sure your working directory has files with the correct line endings necessarily throws these away. | Each user needs to enable the eol extension on their system. If the repository uses the eol extension and some users do not have the extension enabled, then those users will have problems - see '''Problem 2''' in the Troubleshooting section below. (And if shared repositories don't guarded against bad line endings being pushed, then those problems will likely be shared with other users - see the hooks section for setting up push guards.) |
| Line 29: | Line 29: |
| Configure your `.hgrc` to enable the extension by adding following lines: | === Enabling and configuring the eol extension === To enable the eol extension add the following lines to an appropriate configuration file: |
| Line 36: | Line 38: |
| '''Important:''' After you've finished your EOL configuration you should make a clean checkout. This is necessary since the EOL extension ''only'' converts line endings when `hg update` is ran (and then only on files that are modified by this command). Line endings are ''not'' converted when committing a file. The easiest way to do a clean checkout is to delete all files from your working copy (use `hg co -C null`, don't use `hg remove` for this) - except the `.hg` directory of course. Then run `hg update -C` to check this files out again - now with the correct line endings. | You should probably add this to $HOME/.hgrc (Mac/Linux) or %USERPROFILE%\.hgrc (Windows) to enable the extension for all repositories with a `.hgeol` file at their root that your user works with, but there are other [[http://www.selenic.com/mercurial/hgrc.5.html#files|Mercurial configuration files]] you can use. (Since only repositories with a `.hgeol` file will be affected, and if you have any such repositories then the eol extension ''really'' should be enabled for them, then you should know what you're doing if you choose a different file.) |
| Line 38: | Line 40: |
| === Per-repository configuration === | The eol extension can be further configured per-user by adding a `[eol]` section like: |
| Line 40: | Line 42: |
| Then create a file called `.hgeol` in the root of your working copy. This file contains patterns that tells the extension how files should be converted back and forth between the working copy and the repository. If this file is missing, the extension will not convert any files. The patterns look like this: | {{{ [eol] native = CRLF only-consistent = False }}} This section only accepts these two values. The `native` value can be assigned either LF or CRLF, and tells the eol extension to use this line ending type as the user's native type instead of their actual operating system's type (probably you should not set this). The `only-consistent` value tells the eol extension whether it should only do line ending conversion for files with consistent line endings (that is, files containing `\r\n` OR `\n` line endings, but not both). The default value is True, on the assumption that the user knows what he or she is doing if a file has mixed line endings. === IMPORTANT - after enabling the eol extension === Do you have any preexisting repositories on your system (with a working directory checked out) that should be affected by the enabling of the eol extension? In other words, do you have any repositories that have a `.hgeol` file at their root and that are affected by the configuration file that you enabled the eol extension in? If you don't yet have any such repositories, then you can skip this section. If you do have such repesitories, then after enabling the eol extension it is important that you remove then re-check out ALL the version controlled files from all affected local repositories (hopefully not many, since you should have enabled the eol extension as soon as you started working with such a repository). If you don't carry out the following steps, then you have a real risk of encountering '''Problem 1''' detailed in the Troubleshooting section below, possibly at some later date when you've forgotten about enabling the eol extension. First an explanation for those that prefer to understand the issue before proceeding: the issue here is that the eol extension only converts the line endings of working directory files to the user's native type when (a different) revision of those files is checked out into the working directory. If you have a working directory that was checked out prior to enabling the eol extension, then the files in your working directory will have the same line endings that the files are stored with internally. If this happens to be different to the line endings the files would be checked out with ''after'' enabling the eol extension, then Mercurial is going to be unhappy. First figure out which of your local repositories are affected. Next, for each affected repository, do the following... If there are local changes in the repository, save the changes (to a patch file, or whatever), then use `hg revert .` to revert them from the working directory. Now delete all version controlled files from your working copy using `hg co -C null` (do not use `hg remove` for this, that is ''not'' what you want). Next check everything out again using something like `hg update -C tip` (replacing "tip" with a different revision if appropriate). The files in your working directory will now have the "correct" line endings. Finally, if you previously saved any local changes, reapply these now. == Configuring a repository using the .hgeol file == The line ending type that the eol extension gives files as they are checked in/out of the repository is configured using a version controlled file called `.hgeol` at the root of the repository (version controlled so everyone uses the same rules!). If this file is missing, the extension will not convert any files. The `.hgeol` file can contain two configuration sections as detailed below, but before we get to that first read this... === IMPORTANT - before adding/changing a .hgeol file === So as to avoid confusion during the checks that you should make AFTER adding/changing a `.hgeol` file (see below), it is important that you remove any local changes from the working directory of your repository BEFORE adding or changing the `.hgeol` file. === The [repository] section === The `.hgeol` file can be given an optional `[repository]` section to set the ''repository's'' native line ending type (its canonical type). The default type is LF, but if you want to change the type to CRLF (the only other supported type) then add the following to your repository's `.hgeol`: {{{ [repository] native = CRLF }}} === The [patterns] section === The meat of the `.hgeol` file is in the `[patterns]` section. This section contains patterns that tell the extension what line ending type should be used internally in the repository vs in the working directory for files that match a given pattern. There are four supported types for patterns. Those types are: * '''native''' - use the repository's native type internally, and use the user's native type (their operating system's native type, or the type they configure for themselves) in the working directory * '''LF''' - use LF both internally and in the working directory, regardless of the user's native type * '''CRLF''' - use CRLF both internally and in the working directory, regardless of the user's native type * '''BIN''' - do not convert line endings (the eol extension will never touch files containing NUL bytes (\0) so creating rules of this type is not usually necessary for genuinely binary file types) An example of a `[patterns]` section is: |
| Line 45: | Line 93: |
| **.txt = native | |
| Line 46: | Line 95: |
| **.txt = native | |
| Line 53: | Line 101: |
| * all Python files (`**.py`, a recursive glob pattern) should be stored in the working copy using the operating system native EOLs (`\r\n` on Windows, `\n` on Mac and Linux). The files are stored in LF format in the repository -- this is configurable using the `[repository]` section described below. * all Visual Studio project files (`**.vcproj`) should be stored in Windows format in both the working copy and the repository. This rule applies equally to all operating systems. * all text files (`**.txt`) should also be stored in native form in the working copy. * the `Makefile` should be stored in Unix format in working copy and repository. * all JPEG images (`**.jpg`) are binary and should be left alone. The extension will never touch files containing NUL bytes (`\0`) so this rule is not strictly necessary. |
* all Python files (`**.py`, a recursive glob pattern) and .txt files (`**.txt`) should be checked out to the working copy using operating system native EOLs (`\r\n` on Windows, `\n` on Mac OS X and Linux). Internally the repository will store these files using LF by default, or the type specified in the `[repository]` section of the `.hgeol` file (see above). * all Visual Studio project files (`**.vcproj`) should be stored internally and checked out in Windows format. This rule applies equally to all operating systems. * the root `Makefile` should be stored internally and checked out in Unix format. Again, this rule applies equally to all operating systems. * all JPEG images (`**.jpg`) should be treated as binary and left alone. The eol extension will never touch files containing NUL bytes (`\0`) so this rule is not strictly necessary for JPEG files, but you may sometimes find it necessary for other file types. |
| Line 59: | Line 106: |
| The `native` format must know how to convert the operating system native encoding back to the repository encoding. The default repository encoding is `LF`, but this is configurable too. Put the following section into `.hgeol` to override the default: {{{ [repository] native = CRLF }}} |
Note that the first matching pattern is used, so put more specific patterns first. |
| Line 65: | Line 108: |
| You should put the `.hgeol` file under version control so that all members of your project use the same rules. | For more information on pattern formats see `hg help patterns` or the [[http://www.selenic.com/mercurial/hg.1.html#file-name-patterns|File Name Patterns]] section of the man hg man page. |
| Line 67: | Line 110: |
| === Per-user configuration === | === IMPORTANT - after adding/changing a .hgeol file === |
| Line 69: | Line 112: |
| The extension will normally not touch files in which the EOLs are not consistent, that is, files containing both `\r\n` and `\n` line endings. This is on the assumption that the user knows what he or she is doing and has put in the mixed line endings on purpose. | After adding or changing a `.hgeol` file it is very important that you check whether Mercurial reports line ending changes to files that you didn't touch, and that you do this BEFORE pushing the `.hgeol` changes out to other users. If you don't resolve any such changes before pushing the new/changed `.hgeol` file out to other users, then those users are going to see the same unexpected line ending changes as ''uncommitted'' changes in their working directories, which is really going to confuse and annoy them. |
| Line 71: | Line 114: |
| If you want to convert such files anyway, then set {{{ [eol] only-consistent = False }}} in your `.hgrc` file. |
Checking for any such changes is as simple as running `hg status` or `hg diff`. However, MAKE SURE THAT THE EOL EXTENSION IS ENABLED for you first (see the per-user configuration section) or you won't be able to see any such changes, whereas users that DO have the extension enabled ''will'' see them after you push (assuming any exist)! If no changed files are reported, then you're good to go. |
| Line 78: | Line 116: |
| Please see `hg help eol` for help matching your version of the extension and see `hg help patterns` for more information on the glob patterns used. | If Mercurial ''does'' report unexpected line ending changes, you may want to understand why; this happens if the repository is storing preexisting files internally with line endings that the new/changed `.hgeol` file now says are the wrong internal line ending type. When working copy files are compared to their internal copy the eol extension converts the line endings to the (new) internal type first, and if the new and old internal line endings are different, then as far as Mercurial is concerned the files have changed, since, well, they have. |
| Line 80: | Line 118: |
| There are two options to clear up such line ending changes; you can either reconsider the changes you made to the `.hgeol` file, or else you can commit the line ending changes that Mercurial is reporting with a message along the lines of "convert line endings to new canonical type". If you do decide to commit such line ending changes then be sure to push those changes at the same time as you push your `.hgeol` changes! == Troubleshooting == '''Problem 1:''' You get the error message 'abort: local changes found' or 'local changed ... which remote deleted' but `hg diff` does NOT show any changes. (`hg status` should still show which files Mercurial considers to have been modified though.) You can get these messages if a file matching a 'native' pattern in the `.hgeol` file contains line endings that are not your operating system's native type. This is likely to happen if you failed to carry out the `hg co -C null; hg co tip` step detailed above when enabling the eol extension. To recover, if you have any local changes, first save them to an mq patch using `hg qnew tmp; hg qpop`. Now do something like `hg co -C null; hg co tip`. This will update all tracked files in your working directory to have the correct line endings. If you had any local changes, then finally reinstate them using `hg qpush; hg qref -X .; hg qpop -f; hg qdel tmp`. '''Problem 2:''' You get the error 'abort: local changes found' and `hg diff` DOES show line ending changes, but you didn't touch the affected files. This will happen if the repository's INTERNAL copy of the file(s) shown in the diff has the WRONG (non-canonical) line endings according to the `.hgeol` file. This can happen if you/someone else did not follow the IMPORTANT section above that should be carried out after adding/changing the `.hgeol` file. It can also happen when someone who hasn't turned on the eol extension commits the wrong line endings to the repository. See the IMPORTANT section mentioned above to understand the issue and your options. '''Problem 3:''' You get the error 'abort: inconsistent newline style in ...' when trying to commit: either you need to make the newlines in the file consistent yourself before committing, or else you should add the 'only-consistent = False' detailed above to your hgrc to have the eol extension do it for you. |
|
| Line 82: | Line 129: |
| TBD | See `hg help eol`. |
| Line 88: | Line 136: |
| == Troubleshooting == 'abort: local changes found' but `hg diff` does NOT show any changes: remove all tracked files from the repository and check them out afresh by running `hg co -C null; hg co tip`. You may still get 'abort: local changes found', but now `hg diff` should show you the changes it's complaining about (see next paragraph). This problem can occur if the working copy of a file and the repository's internal copy both have the same line endings, but they are the WRONG line endings according to the .hgeol file. 'abort: local changes found' and `hg diff` DOES show the changes: assuming you've enabled the eol extension and run the `hg co -C null; hg co tip` step, this is probably because the repository's INTERNAL copy of the file(s) shown in the diff has the WRONG line endings. This can happen if the file was committed to the repository with these line endings by someone who hasn't turned on the eol extension, or by someone before your project started using the eol extension. If the changeset adding the wrong line endings was created by you and it has NOT yet been transferred to a shared repository, then you can use the qimport command to change the changeset to use the correct line endings. If the changeset has been shared, then probably the best thing to do is to commit the changes with a message along the lines of "fix line endings" (or alternatively reevaluate whether the rules in the repository's .hgeol file are even correct). 'abort: inconsistent newline style in ...' when trying to commit: either you need to make the newlines in the file consistent yourself before committing, or else you should add the 'only-consistent = False' detailed above to your hgrc to have the eol extension do it for you. |
EOL Extension
Automatic management of EOL conversion.
Contents
1. Status
This extension has been distributed with Mercurial as of version 1.5.4.
Author: Martin Geisler mg@lazybytes.net
2. Overview
Different platforms have different conventions for representing line endings in text files:
Windows traditionally uses CRLF (\r\n, carriage-return followed by line-feed). The default text editor on Windows, Notepad, only understands CRLF. Command line tools and redirection also use CRLF.
Unix and Linux traditionally uses LF (\n), though many tools can handle CRLF as well.
Older versions of Mac OS used CR (\r), but Mac OS X is Unix and uses LF. This extension does not support the old CR format.
As a result of these differences, when people using different operating systems are working on the same repository, there is a real risk that someone's text editor will change the line endings in the files that they edit. If they commit these line ending changes, then unfortunately the diff for their commit will be trashed, and hg annotate/blame for the touched files will be useless.
The eol extension solves this problem by allowing a native (canonical) line ending type to be specified for the repository, and by converting seamlessly between the repository's and the user's native types as the user checks out and commits changes to select files. In this way user's get to work with files using their native line ending type while the conversion process makes sure the wrong line endings aren't committed to those files. The extension also allows certain files to have a fixed line ending type both internally in the repository and when checked out (useful for Makefiles/.vcproj files, for example), and for others to be treated as binary files.
3. Per-user configuration
Each user needs to enable the eol extension on their system. If the repository uses the eol extension and some users do not have the extension enabled, then those users will have problems - see Problem 2 in the Troubleshooting section below. (And if shared repositories don't guarded against bad line endings being pushed, then those problems will likely be shared with other users - see the hooks section for setting up push guards.)
3.1. Enabling and configuring the eol extension
To enable the eol extension add the following lines to an appropriate configuration file:
[extensions] eol =
You should probably add this to $HOME/.hgrc (Mac/Linux) or %USERPROFILE%\.hgrc (Windows) to enable the extension for all repositories with a .hgeol file at their root that your user works with, but there are other Mercurial configuration files you can use. (Since only repositories with a .hgeol file will be affected, and if you have any such repositories then the eol extension really should be enabled for them, then you should know what you're doing if you choose a different file.)
The eol extension can be further configured per-user by adding a [eol] section like:
[eol] native = CRLF only-consistent = False
This section only accepts these two values. The native value can be assigned either LF or CRLF, and tells the eol extension to use this line ending type as the user's native type instead of their actual operating system's type (probably you should not set this). The only-consistent value tells the eol extension whether it should only do line ending conversion for files with consistent line endings (that is, files containing \r\n OR \n line endings, but not both). The default value is True, on the assumption that the user knows what he or she is doing if a file has mixed line endings.
3.2. IMPORTANT - after enabling the eol extension
Do you have any preexisting repositories on your system (with a working directory checked out) that should be affected by the enabling of the eol extension? In other words, do you have any repositories that have a .hgeol file at their root and that are affected by the configuration file that you enabled the eol extension in? If you don't yet have any such repositories, then you can skip this section.
If you do have such repesitories, then after enabling the eol extension it is important that you remove then re-check out ALL the version controlled files from all affected local repositories (hopefully not many, since you should have enabled the eol extension as soon as you started working with such a repository). If you don't carry out the following steps, then you have a real risk of encountering Problem 1 detailed in the Troubleshooting section below, possibly at some later date when you've forgotten about enabling the eol extension.
First an explanation for those that prefer to understand the issue before proceeding: the issue here is that the eol extension only converts the line endings of working directory files to the user's native type when (a different) revision of those files is checked out into the working directory. If you have a working directory that was checked out prior to enabling the eol extension, then the files in your working directory will have the same line endings that the files are stored with internally. If this happens to be different to the line endings the files would be checked out with after enabling the eol extension, then Mercurial is going to be unhappy.
First figure out which of your local repositories are affected. Next, for each affected repository, do the following...
If there are local changes in the repository, save the changes (to a patch file, or whatever), then use hg revert . to revert them from the working directory. Now delete all version controlled files from your working copy using hg co -C null (do not use hg remove for this, that is not what you want). Next check everything out again using something like hg update -C tip (replacing "tip" with a different revision if appropriate). The files in your working directory will now have the "correct" line endings. Finally, if you previously saved any local changes, reapply these now.
4. Configuring a repository using the .hgeol file
The line ending type that the eol extension gives files as they are checked in/out of the repository is configured using a version controlled file called .hgeol at the root of the repository (version controlled so everyone uses the same rules!). If this file is missing, the extension will not convert any files. The .hgeol file can contain two configuration sections as detailed below, but before we get to that first read this...
4.1. IMPORTANT - before adding/changing a .hgeol file
So as to avoid confusion during the checks that you should make AFTER adding/changing a .hgeol file (see below), it is important that you remove any local changes from the working directory of your repository BEFORE adding or changing the .hgeol file.
4.2. The [repository] section
The .hgeol file can be given an optional [repository] section to set the repository's native line ending type (its canonical type). The default type is LF, but if you want to change the type to CRLF (the only other supported type) then add the following to your repository's .hgeol:
[repository] native = CRLF
4.3. The [patterns] section
The meat of the .hgeol file is in the [patterns] section. This section contains patterns that tell the extension what line ending type should be used internally in the repository vs in the working directory for files that match a given pattern. There are four supported types for patterns. Those types are:
native - use the repository's native type internally, and use the user's native type (their operating system's native type, or the type they configure for themselves) in the working directory
LF - use LF both internally and in the working directory, regardless of the user's native type
CRLF - use CRLF both internally and in the working directory, regardless of the user's native type
BIN - do not convert line endings (the eol extension will never touch files containing NUL bytes (\0) so creating rules of this type is not usually necessary for genuinely binary file types)
An example of a [patterns] section is:
[patterns] **.py = native **.txt = native **.vcproj = CRLF Makefile = LF **.jpg = BIN
This example specifies the following rules:
all Python files (**.py, a recursive glob pattern) and .txt files (**.txt) should be checked out to the working copy using operating system native EOLs (\r\n on Windows, \n on Mac OS X and Linux). Internally the repository will store these files using LF by default, or the type specified in the [repository] section of the .hgeol file (see above).
all Visual Studio project files (**.vcproj) should be stored internally and checked out in Windows format. This rule applies equally to all operating systems.
the root Makefile should be stored internally and checked out in Unix format. Again, this rule applies equally to all operating systems.
all JPEG images (**.jpg) should be treated as binary and left alone. The eol extension will never touch files containing NUL bytes (\0) so this rule is not strictly necessary for JPEG files, but you may sometimes find it necessary for other file types.
Note that the first matching pattern is used, so put more specific patterns first.
For more information on pattern formats see hg help patterns or the File Name Patterns section of the man hg man page.
4.4. IMPORTANT - after adding/changing a .hgeol file
After adding or changing a .hgeol file it is very important that you check whether Mercurial reports line ending changes to files that you didn't touch, and that you do this BEFORE pushing the .hgeol changes out to other users. If you don't resolve any such changes before pushing the new/changed .hgeol file out to other users, then those users are going to see the same unexpected line ending changes as uncommitted changes in their working directories, which is really going to confuse and annoy them.
Checking for any such changes is as simple as running hg status or hg diff. However, MAKE SURE THAT THE EOL EXTENSION IS ENABLED for you first (see the per-user configuration section) or you won't be able to see any such changes, whereas users that DO have the extension enabled will see them after you push (assuming any exist)! If no changed files are reported, then you're good to go.
If Mercurial does report unexpected line ending changes, you may want to understand why; this happens if the repository is storing preexisting files internally with line endings that the new/changed .hgeol file now says are the wrong internal line ending type. When working copy files are compared to their internal copy the eol extension converts the line endings to the (new) internal type first, and if the new and old internal line endings are different, then as far as Mercurial is concerned the files have changed, since, well, they have.
There are two options to clear up such line ending changes; you can either reconsider the changes you made to the .hgeol file, or else you can commit the line ending changes that Mercurial is reporting with a message along the lines of "convert line endings to new canonical type". If you do decide to commit such line ending changes then be sure to push those changes at the same time as you push your .hgeol changes!
5. Troubleshooting
Problem 1: You get the error message 'abort: local changes found' or 'local changed ... which remote deleted' but hg diff does NOT show any changes. (hg status should still show which files Mercurial considers to have been modified though.) You can get these messages if a file matching a 'native' pattern in the .hgeol file contains line endings that are not your operating system's native type. This is likely to happen if you failed to carry out the hg co -C null; hg co tip step detailed above when enabling the eol extension. To recover, if you have any local changes, first save them to an mq patch using hg qnew tmp; hg qpop. Now do something like hg co -C null; hg co tip. This will update all tracked files in your working directory to have the correct line endings. If you had any local changes, then finally reinstate them using hg qpush; hg qref -X .; hg qpop -f; hg qdel tmp.
Problem 2: You get the error 'abort: local changes found' and hg diff DOES show line ending changes, but you didn't touch the affected files. This will happen if the repository's INTERNAL copy of the file(s) shown in the diff has the WRONG (non-canonical) line endings according to the .hgeol file. This can happen if you/someone else did not follow the IMPORTANT section above that should be carried out after adding/changing the .hgeol file. It can also happen when someone who hasn't turned on the eol extension commits the wrong line endings to the repository. See the IMPORTANT section mentioned above to understand the issue and your options.
Problem 3: You get the error 'abort: inconsistent newline style in ...' when trying to commit: either you need to make the newlines in the file consistent yourself before committing, or else you should add the 'only-consistent = False' detailed above to your hgrc to have the eol extension do it for you.
6. Hooks
See hg help eol.
7. Migrating from win32text to eol
In earlier versions of Mercurial, the conversion of line endings was done with the Win32TextExtension. Please see the page MigratingFromWin32TextToEol for a guide for migrating from win32text to eol.