/
githubmirror
/
sed
Обзор
Документация
Войти
/
githubmirror
/
sed
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность
v4.9
NEWS
756 строк
27 KB
Jim Meyering
version 4.9
07 ноя 2022, 00:49
07 ноя 2022, 00:49
7e2e575
Код
Авторство
О чём код?
GNU sed NEWS -*- outline -*- * Noteworthy changes in release 4.9 (2022-11-06) [stable] ** Bug fixes 'sed --follow-symlinks -i' no longer loops forever when its operand is a symbolic link cycle. [bug introduced in sed 4.2] a program with an execution line longer than 2GB can no longer trigger an out-of-bounds memory write. using the R command to read an input line of length longer than 2GB can no longer trigger an out-of-bounds memory read. In locales using UTF-8 encoding, the regular expression '.' no longer sometimes fails to match Unicode characters U+D400 through U+D7FF (some Hangul Syllables, and Hangul Jamo Extended-B) and Unicode characters U+108000 through U+10FFFF (half of Supplemental Private Use Area plane B). [bug introduced in sed 4.8] I/O errors involving temp files no longer confuse sed into using a FILE * pointer after fclosing it, which has undefined behavior in C. ** New Features The 'r' command now accepts address 0, allowing inserting a file before the first line. ** Changes in behavior Sed now prints the less-surprising variant in a corner case of POSIX-unspecified behavior. Before, this would print "n". Now, it prints "X": printf n | sed 'sn\nnXn'; echo * Noteworthy changes in release 4.8 (2020-01-14) [stable] ** Bug fixes "sed -i" now creates temporary files with correct umask (limited to u=rwx). Previously sed would incorrectly set umask on temporary files, resulting in problems under certain fuse-like file systems. [bug introduced in sed 4.2.1] ** Release distribute gzip-compressed tarballs once again ** Improvements a year's worth of gnulib development, including improved DFA performance * Noteworthy changes in release 4.7 (2018-12-20) [stable] ** Bug fixes Some uses of \b in the C locale and with the DFA matcher would fail, e.g., the following would mistakenly print "123-x" instead of "123": echo 123-x|LC_ALL=C sed 's/.\bx//' Using a multibyte locale or certain regexp constructs (some ranges, backreferences) would avoid the bug. [bug introduced in sed 4.6] * Noteworthy changes in release 4.6 (2018-12-19) [stable] ** Improvements sed now prints a clear error message when r/R/w/W (and s///w) commands are missing a filename. Previously, w/W commands would fail with confusing error message, while r/R would be a silent no-op. sed now uses fully-buffered output (instead of line-buffered) when writing to files. This should noticeably improve performance of "sed -i" and other write commands. Buffering can be disabled (as before) with "sed -u". sed in non-cygwin windows environments (e.g. mingw) now properly handles '\n' newlines in -b/--binary mode. ** Bug fixes sed no longer accesses invalid memory (heap overflow) when given invalid backreferences in 's' command [bug#32082, present at least since sed-4.0.6]. sed no longer adds extraneous NUL when given s/$//n command. [related to bug#32271, present since sed-4.0.7] sed no longer accesses invalid memory (heap overflow) with s/$//n regexes. [bug#32271, present since sed-4.3]. ** New Features New option, --debug: print the input sed script in canonical form and annotate program execution. * Noteworthy changes in release 4.5 (2018-03-31) [stable] ** Bug fixes sed now fails when matching very long input lines (>2GB). Before, sed would silently ignore the regex without indicating an error. [Bug present at least since sed-3.02] sed no longer rejects comments and closing braces after y/// commands. [Bug existed at least since sed-3.02] sed -E --posix no longer ignores special meaning of '+','?','|' . [Bug introduced in the original implementation of --posix option in v4.1a-5-gba68fb4] sed -i now creates selinux context based on the context of the symlink instead of the symlink target. [Bug present since at least sed-4.2] sed -i --follow-symlinks remains unchanged. sed now treats the sequence '\x5c' (ASCII 92, backslash) as literal backslash character, not as an escape prefix character. [Bug present since sed-3.02.80] Old behavior: $ echo z | sed -E 's/(z)/\x5c1/' # identical to 's/(z)/\1/' z New behavior: $ echo z | sed -E 's/(z)/\x5c1/' \1 * Noteworthy changes in release 4.4 (2017-02-03) [stable] ** Bug fixes sed could segfault when invoked with specific combination of newlines in the input and regex pattern. [Bug introduced in sed-4.3] * Noteworthy changes in release 4.3 (2016-12-30) [stable] ** Improvements sed's regular expression matching is now typically 10x faster sed now uses unlocked-io where available, resulting in faster I/O operations. ** Bug fixes sed no longer mishandles anchors ^/$ in multiline regex (s///mg) with -z option (NUL terminated lines). [Bug introduced in sed-4.2.2 with the initial implementation of -z] sed no longer accepts a ":" command without a label; before, it would treat that as defining a label whose name is empty, and subsequent label-free "t" and "b" commands would use that label. Now, sed emits a diagnostic and fails for that invalid construct. sed no longer accesses uninitialized memory when processing certain invalid multibyte sequences. Demonstrate with this: echo a | LC_ALL=ja_JP.eucJP valgrind sed/sed 's/a/b\U\xb2c/' The error appears to have been introduced with the sed-4.0a release. The 'y' (transliterate) operator once again works with a NUL byte on the RHS. E.g., sed 'y/b/\x00/' now works like tr b '\0'. GNU sed has never before recognized \x00 in this context. However, sed-3.02 and prior did accept a literal NUL byte in the RHS, which was possible only when reading a script from a file. For example, this: echo abc|sed -f <(printf 'y/b/\x00/\n')|cat -A is what stopped working. [bug introduced some time after sed-3.02 and prior to the first sed-4* test release] When the closed-above line number ranges of N editing commands overlap (N>1), sed would apply commands 2..N to the line just beyond the largest range endpoint. [bug introduced some time after sed-4.09 and prior to release in sed-4.1] Before, this command would mistakenly modify line 5: $ seq 6|sed '2,4d;2,3s/^/x/;3,4s/^/y/' 1 yx5 6 Now, it does not: $ seq 6|sed '2,4d;2,3s/^/x/;3,4s/^/y/' 1 5 6 An erroneous sed invocation like "echo > F; sed -i s//b/ F" no longer leaves behind a temporary file. Before, that command would create a file alongside F with a name matching /^sed......$/ and fail to remove it. sed --follow-symlinks now works again for stdin. [bug introduced in sed-4.2.2] sed no longer elides invalid bytes in a substitution RHS. Now, sed copies such bytes into the output, just as Perl does. [bug introduced in sed-4.1 -- it was also present prior to 4.0.6] sed no longer prints extraneous character when a backslash follows \c. '\c\\' generates control character ^\ (ASCII 0x1C). Other characters after the second backslash are rejected (e.g. '\c\d'). [bug introduced in the sed-4.0.* releases] sed no longer mishandles incomplete multibyte sequences in s,y commands and valid multibyte SHIFT-JIS characters in character classes. Previously, the following commands would fail: LC_ALL=en_US.UTF-8 sed $'s/\316/X/' LC_ALL=ja_JP.shiftjis sed $'/[\203]/]/p' [bug introduced some time after sed-4.1.5 and before sed-4.2.1] ** Feature removal The "L" command (format a paragraph like the fmt(1) command would) has been listed in the documentation as a failed experiment for at least 10 years. That command is now removed. ** Build-related "make dist" now builds .tar.xz files, rather than .tar.gz ones. xz is portable enough and in wide-enough use that distributing only .tar.xz files is enough. It has been fine for coreutils, grep, diffutils and parted for a few years. ** New Features new --sandbox option rejects programs with r/w/e commands. * Noteworthy changes in release 4.2.2 (2012-12-22) [stable] * don't misbehave (truncate input) for lines of length 2^31 and longer * fix endless loop on incomplete multibyte sequences * -u also does unbuffered input, rather than unbuffered output only * New command `F' to print current input file name * sed -i, s///w, and the `w' and `W' commands also obey the --binary option (and create CR/LF-terminated files if the option is absent) * --posix fails for scripts (or fragments as passed to the -e option) that end in a backslash, as they are not portable. * New option -z (--null-data) to separate lines by ASCII NUL characters. * \x26 (and similar escaped sequences) produces a literal & in the replacement argument of the s/// command, rather than including the matched text. ---------------------------------------------------------------------------- Sed 4.2.1 * fix parsing of s/[[[[[[[[[]// * security contexts are preserved by -i too under SELinux * temporary files for sed -i are not made group/world-readable until they are complete ---------------------------------------------------------------------------- Sed 4.2 * now released under GPLv3 * added a new extension `z` to clear pattern space even in the presence of invalid multibyte sequences * a preexisting GNU gettext installation is needed in order to compile GNU sed with NLS support * new option --follow-symlinks, available when editing a file in-place. This option may not be available on some systems (in this case, the option will *not* be a no-op; it will be completely unavailable). In the future, the option may be added as a no-op on systems without symbolic links at all, since in this case a no-op is effectively indistinguishable from a correct implementation. * hold-space is reset between different files in -i and -s modes. * multibyte processing fixed * the following GNU extensions are turned off by --posix: options [iImMsSxX] in the `s' command, address kinds `FIRST~STEP' and `ADDR1,+N' and `ADDR1,~N', line address 0, `e' or `z' commands, text between an `a' or `c' or `i' command and the following backslash, arguments to the `l' command. --posix disables all extensions to regular expressions. * fixed bug in 'i\' giving a segmentation violation if given alone. * much improved portability * much faster in UTF-8 locales * will correctly replace ACLs when using -i * will now accept NUL bytes for `.' ---------------------------------------------------------------------------- Sed 4.1.5 * fix parsing of a negative character class not including a closed bracket, like [^]] or [^]a-z]. * fix parsing of [ inside an y command, like y/[/A/. * output the result of commands a, r, R when a q command is found. ---------------------------------------------------------------------------- Sed 4.1.4 * \B correctly means "not on a word boundary" rather than "inside a word" * bugfixes for platform without internationalization * more thorough testing framework for tarballs (`make full-distcheck') ---------------------------------------------------------------------------- Sed 4.1.3 * regex addresses do not use leftmost-longest matching. In other words, /.\+/ only looks for a single character, and does not try to find as many of them as possible like it used to do. * added a note to BUGS and the manual about changed interpretation of `s|abc\|def||', and about localization issues. * fixed --disable-nls build problems on Solaris. * fixed `make check' in non-English locales. * `make check' tests the regex library by default if the included regex is used (regex tests had to be enabled separately up to now). ---------------------------------------------------------------------------- Sed 4.1.2 * fix bug in 'y' command in multi-byte character sets * fix severe bug in parsing of ranges with an embedded open bracket * fix off-by-one error when printing a "bad command" error ---------------------------------------------------------------------------- Sed 4.1.1 * preserve permissions of in-place edited files * yield an error when running -i on terminals or other non regular files * do not interpret - as stdin when using in-place editing mode * fix bug that prevented 's' command modifiers from working ---------------------------------------------------------------------------- Sed 4.1 * // matches the last regular expression even in POSIXLY_CORRECT mode. * change the way we treat lines which are not terminated by a newline. Such lines are printed without the terminating newline (as before) but as soon as more text is sent to the same output stream, the missing newline is printed, so that the two lines don't concatenate. The behavior is now independent from POSIXLY_CORRECT because POSIX actually has undefined behavior in this case, and the new implementation arguably gives the ``least expected surprise''. Thanks to Stepan Kasal for the implementation. * documentation improvements, with updated references to the POSIX.2 specification * error messages on I/O errors are better, and -i does not leave temporary files around (e.g. when running ``sed -i'' on a directory). * escapes are accepted in the y command (for example: y/o/\n/ transforms o's into newlines) * -i option tries to set the owner and group to the same as the input file * `L' command is deprecated and will be removed in sed 4.2. * line number addresses are processed differently -- this is supposedly conformant to POSIX and surely more idiot-proof. Line number addresses are not affected by jumping around them: they are activated and deactivated exactly where the script says, while previously 5,8b 1,5d would actually delete lines 1,2,3,4 and 9 (!). * multibyte characters are taken in consideration to compute the operands of s and y, provided you set LC_CTYPE correctly. They are also considered by \l, \L, \u, \U, \E. * [\n] matches either backslash or 'n' when POSIXLY_CORRECT. * new option --posix, disables all GNU extensions. POSIXLY_CORRECT only disables GNU extensions that violate the POSIX standard. * options -h and -V are not supported anymore, use --help and --version. * removed documentation for \s and \S which worked incorrectly * restored correct behavior for \w and \W: match [[:alnum:]_] and [^[:alnum:]_] (they used to match [[:alpha:]_] and [^[:alpha:]_] * the special address 0 can only be used in 0,/RE/ or 0~STEP addresses; other cases give an error (you are hindering portability for no reason if specifying 0,N and you are giving a dead command if specifying 0 alone). * when a \ is used to escape the character that would terminate an operand of the s or y commands, the backslash is removed before the regex is compiled. This is left undefined by POSIX; this behavior makes `s+x\+++g' remove occurrences of `x+', consistently with `s/x\///g'. (However, if you enjoy yourself trying `s*x\***g', sed will use the `x*' regex, and you won't be able to pass down `x\*' while using * as the delimiter; ideas on how to simplify the parser in this respect, and/or gain more coherent semantics, are welcome). ---------------------------------------------------------------------------- Sed 4.0.9 * 0 address behaves correctly in single-file (-i and -s) mode. * documentation improvements. * tested with many hosts and compilers. * updated regex matcher from upstream, with many bugfixes and speedups. * the `N' command's feature that is detailed in the BUGS file was disabled by the first change below in sed 4.0.8. The behavior has now been restored, and is only enabled if POSIXLY_CORRECT behavior is not requested. ---------------------------------------------------------------------------- Sed 4.0.8 * fix `sed n' printing the last line twice. * fix incorrect error message for invalid character classes. * fix segmentation violation with repeated empty subexpressions. * fix incorrect parsing of ^ after escaped (. * more comprehensive test suite (and with many expected failures...) ---------------------------------------------------------------------------- Sed 4.0.7 * VPATH builds working on non-glibc machines * fixed bug in s///Np: was printing even if less than N matches were found. * fixed infinite loop on s///N when LHS matched a null string and there were not enough matches in pattern space * behavior of s///N is consistent with s///g when the LHS can match a null string (and the infinite loop did not happen :-) * updated some translations ---------------------------------------------------------------------------- Sed 4.0.6 * added parameter to `v' for the version of sed that is expected. * configure switch --without-included-regex to use the system regex matcher * fix for -i option under Cygwin ---------------------------------------------------------------------------- Sed 4.0.5 * portability fixes * improvements to some error messages (e.g. y/abc/defg/ incorrectly said `excess characters after command' instead of `y arguments have different lengths') * `a', `i', `l', `L', `r' accept two addresses except in POSIXLY_CORRECT mode. Only `q' and `Q' do not accept two addresses in standard (GNU) mode. ---------------------------------------------------------------------------- Sed 4.0.4 * documentation fixes * update regex matcher ---------------------------------------------------------------------------- Sed 4.0.3 * fix packaging problem (two missing translation catalogs) ---------------------------------------------------------------------------- Sed 4.0.2 * more translations * fix build problems (vpath builds and bootstrap builds) ---------------------------------------------------------------------------- Sed 4.0.1 * Remove last vestiges of super-sed * man page automatically built * more translations provided * portability improvements ---------------------------------------------------------------------------- Sed 4.0 * Update regex matcher ---------------------------------------------------------------------------- Sed 3.96 * `y' command supports multibyte character sets * Update regex matcher ---------------------------------------------------------------------------- Sed 3.95 * `R' command reads a single line from a file. * CR-LF pairs are always ignored under Windows, even if (under Cygwin) a disk is mounted as binary. * More attention to errors on stdout * New `W' command to write first line of pattern space to a file * Can customize line wrap width on single `l' commands * `L' command formats and reflows paragraphs like `fmt' does. * The test suite makefiles are better organized (this change is transparent however). * Compiles and bootstraps out-of-the-box under MinGW32 and Cygwin. * Optimizes cases when pattern space is truncated at its start or at its end by `D' or by a substitution command with an empty RHS. For example scripts like this, seq 1 10000 | tr \\n \ | ./sed ':a; s/^[0-9][0-9]* //; ta' whose behavior was quadratic with previous versions of sed, have now linear behavior. * New command `e' to pipe the output of a command into the output of sed. * New option `e' to pass the output of the `s' command through the Bourne shell and get the result into pattern space. * Switched to obstacks in the parser -- less memory-related bugs (there were none AFAIK but you never know) and less memory usage. * New option -i, to support in-place editing a la Perl. Usually one had to use ed or, for more complex tasks, resort to Perl; this is not necessary anymore. * Dumped buffering code. The performance loss is 10%, but it caused bugs in systems with CRLF termination. The current solution is not definitive, though. * Bug fix: Made the behavior of s/A*/x/g (i.e. `s' command with a possibly empty LHS) more consistent: pattern GNU sed 3.x GNU sed 4.x B xBx xBx BC xBxCx xBxCx BAC xBxxCx xBxCx BAAC xBxxCx xBxCx * Bug fix: the // empty regular expressions now refers to the last regular expression that was matched, rather than to the last regular expression that was compiled. This richer behavior seems to be the correct one (albeit neither one is POSIXLY_CORRECT). * Check for invalid backreferences in the RHS of the `s' command (e.g. s/1234/\1/) * Support for \[lLuUE] in the RHS of the `s' command like in Perl. * New regular expression matcher * Bug fix: if a file was redirected to be stdin, sed did not consume it. So (sed d; sed G) < TESTFILE double-spaced TESTFILE, while the equivalent `useless use of cat' cat TESTFILE | (sed d; sed G) printed nothing (which is the correct behavior). A test for this bug was added to the test suite. * The documentation is now much better, with a few examples provided, and a thorough description of regular expressions. The manual often refers to "GNU extensions", but if they are described here they are specific to this version. * Documented command-line option: -r, --regexp-extended Use extended regexps -- e.g. (abc+) instead of \(abc\+\) * Added feature to the `w' command and to the `w' option of the `s' command: if the file name is /dev/stderr, it means the standard error (inspired by awk); and similarly for /dev/stdout. This is disabled if POSIXLY_CORRECT is set. * Added `m' and `M' modifiers to `s' command for multi-line matching (Perl-style); in addresses, only `M' works. * Added `Q' command for `silent quit'; added ability to pass an exit code from a sed script to the caller. * Added `T' command for `branch if failed'. * Added `v' command, which is a do-nothing intended to fail on seds that do not support GNU sed 4.0's extensions. ---------------------------------------------------------------------------- Sed 3.02.80 * Started new version nomenclature for pre-3.03 releases. (I'm being pessimistic in assuming that .90 won't give me enough breathing room.) * Bug fixes: the regncomp()/regnexec() interfaces proved to be inadequate to properly handle expressions such as "s/\</#/g". Re-abstracted the regex code in the sed/ tree, and now use the re_search_2() interface to the GNU regex routines. This change also fixed a bug where /./ did not match the NUL character. Had the glibc folk fix a bug in lib/regex.c where 's/0*\([0-9][0-9]\)/X\1X/' failed to match on input "002". * Added new command-line options: -u, --unbuffered Do not attempt to read-ahead more than required; do not buffer stdout. -l N, --line-length=N Specify the desired line-wrap length for the `l' command. A length of "0" means "never wrap". * New internationalization translations added: fr ru de it el sk pt_BR sv (plus nl from 3.02a). * The s/// command now understands the following escapes (in both halves): \a an "alert" (BEL) \f a form-feed \n a newline \r a carriage-return \t a horizontal tab \v a vertical tab \oNNN a character with the octal value NNN \dNNN a character with the decimal value NNN \xNN a character with the hexadecimal value NN This behavior is disabled if POSIXLY_CORRECT is set, at least for the time being (until I can be convinced that this behavior does not violate the POSIX standard). (Incidentally, \b (backspace) was omitted because of the conflict with the existing "word boundary" meaning. \ooo octal format was omitted because of the conflict with backreference syntax.) * If POSIXLY_CORRECT is set, the empty RE // now is the null match instead of "repeat the last REmatch". As far as I can tell this behavior is mandated by POSIX, but it would break too many legacy sed scripts to blithely change GNU sed's default behavior. ---------------------------------------------------------------------------- Sed 3.02a * Added internationalization support, and an initial (already out of date) set of Dutch message translations (both provided by Erick Branderhorst). * Added support for scripts like: sed -e 1ifoo -e '$abar' (note no need for \ <newline> after a, i, and c commands). Also, conditionally (on NO_INPUT_INDENT) added experimental support for skipping leading whitespace on each {a,i,c} input line. * Added addressing of the form: /foo/,+5 p (print from foo to 5th line following) /foo/,~5 p (print from foo to next line whose line number is a multiple of 5) The first address of these can be any of the previously existing addressing types; the +N and ~N forms are only allowed as the second address of a range. * Added support for pseudo-address "0" as the first address in an address-range, simplifying scripts which happen to match the end address on the first line of input. For example, a script which deletes all lines from the beginning of the file to the first line which contains "foo" is now simply "sed 0,/foo/d", whereas before one had to go through contortions to deal with the possibility that "foo" might appear on the first line of the input. * Made NUL characters in regexps work "correctly" --- i.e., a NUL in a RE matches a NUL; it does not prematurely terminate the RE. (This only works in -f scripts, as the POSIX.1 exec*() interface only passes NUL-terminated strings, and so sed will only be able to see up to the first NUL in any -e scriptlet.) * Wherever a `;' is accepted as a command terminator, also allow a `}' or a `#' to appear. (This allows for less cluttered-looking scripts.) * Lots of internal changes that are only relevant to source junkies and development testing. Some of which might cause imperceptible performance improvements. ---------------------------------------------------------------------------- Sed 3.02 * Fixed a bug in the parsing of character classes (e.g., /[[:space:]]/). Corrected an omission in djgpp/Makefile.am and an improper dependency in testsuite/Makefile.am. ---------------------------------------------------------------------------- Sed 3.01 * This version of sed mainly contains bug fixes and portability enhancements, plus performance enhancements related to sed's handling of input files. Due to excess performance penalties, I have reverted (relative to 3.00) to using regex.c instead of the rx package for regular expression handling, at the expense of losing true POSIX.2 BRE compatibility. However, performance related to regular expression handling *still* needs a fair bit of work. * One new feature has been added: regular expressions may be followed with an "I" directive ("i" was taken [the "i"nsert command]) to indicate that the regexp should be matched in a case-insensitive manner. Also of note are a new organization to the source code, new documentation, and a new maintainer. ---------------------------------------------------------------------------- Sed 3.0 * This version of sed passes the new test-suite donated by Jason Molenda. * Overall performance has been improved in the following sense: Sed 3.0 is often slightly slower than sed 2.05. On a few scripts, though, sed 2.05 was so slow as to be nearly useless or to use up unreasonable amounts of memory. These problems have been fixed and in such cases, sed 3.0 should have acceptable performance.