New features with AN-2020-08-12: This is the first localization step for the schily source consolidation. Many programs now (hopefully) call gettext() for all strings that need localization. - The next step will include dgettext() calls for the libraries and the missing programs - The following step will include the extracted strings - The last step will include German translations and install support for the resulting binary message object files. ----------> Please test and report compilation problems! <--------- ***** NOTE: As mentioned since 2004, frontends to the tools should ***** ***** call all programs in the "C" locale ***** ***** by e.g. calling: LC_ALL=C cdrecord .... ***** ***** unless these frontends support localized strings ***** ***** used by the cdrtools with NLS support. ***** *** WARNING *** *** Need new smake *** *** Due to the fact that schily-tools 2014-04-03 introduced to use new macro *** expansions and a related bug fix in smake, you need a newer smake *** to compile this source. If your smake is too old and aborts, ensure to *** use the recent smake by calling: cd ./psmake ./MAKE-all cd .. psmake/smake psmake/smake install The new smake version mentioned above is smake-1.2.4 The recent smake version is smake-1.3 *** Due to the fact that schily-tools 2018-01-26 introduced *** optimizations for the Schily version of SunPro Make, you *** need at least the dmake version from 2018/01/11 with support *** for the "export" directive to compile with this makefile system. WARNING: the new version of the isoinfo program makes use of the *at() series of functions that have been introduced by Sun in August 2001 and added to POSIX.1-2008. For older platforms, libschily now includes emulations for these functions but these emulations have not yet been tested thoroughly. Please report problems! BUG WARNING: Please never report bugs only to Linux distributions as they usually do not forward these bug reports upstream and as the Linux distributions typically do not let skilled people check the bugs. We did not hear about a FIFO problem in star for a long time. Then a problem on Linux occurred once every 6000-10000 tries but it did not happen on Solaris after even 10 million tries, so it was not known besides Linux. BUG WARNING: *** GNU make *** starts too early with parallel execution (when reading Makefiles and evaluating rules for "include" statements already). Since GNU make does not support a concept for a correct ordering of such actions, you need to be prepared to see gmake fail in parallel mode. If you are interested in reliable parallel execution, it is recommended to use the included "dmake" program with a command line like: dmake -j10 -f SMakefile from the top level directory. Note that if you are on Linux, you need a halfway recent kernel or the compile time will not go down because of the low POSIX semaphore performance in older Linux kernels. The "dmake" program included in the schilytools tarball is the current version of the "new" SunOS make program that has been introduced in January 1986 by Sun Microsystems. It also introduced new features like the "include" directive that 3 years later have been copied by gmake in a partially buggy way. As gmake does not fix showstopper bugs, it cannot be supported. Current showstoppers are: 1) gmake executes "include" related rules in the inverse order, causing rules to fail if they depend on files created by an "earlier" action 2) gmake caches an outdated state of the directory and aborts with a wrong complain about allegedly missing files that in fact exist already. - autoconf: added a new test for the existence of - autoconf: added new tests for Linux getauxval() and FreeBSD elf_aux_info() - libschily: getexecpath.c now uses getauxval() on Linux and elf_aux_info() on FreeBSD. This was needed since readlink("/proc/curproc/file") on FreeBSD returns random values for hardlinked files, making it impossible to use the result in order to find out which behavior variant of a fat binary is requested. - hdump/od: The automatic POSIX/non-POSIX switch based on a hardlink and a path that contains /xpg4/ and another that does not now also works on Linux and FreeBSD. This is a result of using getauxval()/elf_aux_info(). Before, this switch did only work on Solaris. - isaexec: The program now also supports Linux and freeBSD, but we do not yet have a way to read the supported ISA list on the Linux/FreeBSD. Are there ideas to to this? - star: GNU cpio archives may contain longer path names than permitted by the AT&T documentation. This did cause problems with our automated recognition of swapped CPIO headers. We now use a more robust method. - star: GNU cpio archives may contain longer path names than permitted by the AT&T documentation. This could cause star to dump core because the cpio module did not yet call the routine to grow our new dynamic path store. - star: older GNU compilers seem to decide that a division: long long / long results in a long. We now cast the divisor to long long to definitely get the right result for a printf argument. - star: changed a name in a parameter list for lreadlink() to avoid a GCC shadow warning. - star: The variable use_fifo is also needed if the FIFO is not available (e.g. on DOS). The previous code did not compile on platforms that did not support to implement the FIFO. We moved extern BOOL use_fifo out of a #ifdef FIFO Thanks to a hint from Albert Wik. - star: The file README.otherbugs has been changed to match the state of today and a geocrawler based URL now includes an archive.org prefix as geocrawler no longer exists. - cal: cal now works on platforms without i18n support. We now check whether MON_1 and ABMON_1 are defined and otherwise leave the english month names. Thanks to a hint from Albert Wik. - mkisofs: The test if (s_entry->size > (((off_t)190)*(off_t)0x3FFFF800)) in tree.c is now only run if the platform supports largefiles at all. Thanks to a hint from Albert Wik. - patch: A call to signal(SIGCHLD, ...) has been put into #ifdef SIGCHLD for DOS. Thanks to a hint from Albert Wik. - Bourne Shell: Several #ifdef SIG* have been added to make it compile on older UNIX versions. Thanks to a hint from Albert Wik. - smake: Added some code to allow better debugging of MAKEFLAGS related problems. - smake: we now use strdup() to remember the content of the MAKEFLAGS environment as the content of a pointer into the environment array did change unexpectedly on DJGPP. Thanks to a hint from Albert Wik. - SCCS: libmpw added a xpopen() to allow pipes to stdin/stdout/stderr. - SCCS: Fixed a bug in libcombj introduced with last release: the lock update function did not #include and thus did get the SA_RESTART definition only on platforms that pollute the namespace line Linux. - SCCS: All man pages now have a recent (complete) list of SCCS commands in the SEE ALSO section. This adds the man pages that recently have been written. - SCCS: Unit tests: The file sccs/tests/Makefile has been modified to allow the directories /tmp/SCCSTEST and /tmp/sccstests to be tmpfs mountpoints or symlinks that point to a tmpfs mount. Without that trick, the SCCS unit tests take 50 minutes on Linux because of the slow Linux file systems, while they take 3 minutes on Solaris (if run on UFS or ZFS). With tmpfs, Linux is in a similar time range as Solaris. - SCCS: sccslog again allows ovelapping delgets and now allows overlapping delget groups to be correctly detected as commit bundle. Historical SCCS based projects constantly do this...e.g. the CSRG BSD SCCS history from Kirk McKusick, so we need to allow it. - SCCS: The buffer size for setvbuf() may now be modified by a compile time make argument for testing purposes. - SCCS: diff now uses getexecpath() from libschily if getexecname() is not available. On Solaris, this does not require libschily. - SCCS: struct packet now has a new member no_chksum. - SCCS: getline() no longer computes checksums at all in case gpkt->no_chksum has been set. This is used to speed up SCCS in some cases. - SCCS: delta no longer computes the checksum in the first pass if called with -Xprepend. This reduces the run time by 7% in case of updating larger changeset files. - SCCS: "Reading" the SCCS history files now is implemented using mmap() if possible. This speeds up operation by approx. another 15%. - SCCS: the function getline() now supports mmap(). - SCCS: the function putline() now uses fwrite() instead of fputs() when in copy mode. This is needed since fputs() needs a nul byte at the end of the string that cannot be granted when using mmap(). With these changes, we are now close to allow normal line based diffs in SCCS even for binary files. We would need to completely avoid fputs() and this needs an initialized member p_line_length in struct packet. - SCCS: the function sclose() now also unmap()'s the file if it was mmap()ed via sinit(). - SCCS: the function sinit() now also mmap()s the file in case that it is smaller than 1G and the flag SI_NOMAP was not specified. - SCCS: new functions sseek() and stell() for libcomobj, to support the use of mmap(). - SCCS: delta no longer closes and reopens the history file in case that bdiff(1) reports a segmentation problem and needs another try with smaller segments. This change was needed in order to support mmap() on the input file. Delta now instead calls grewind() that knows about mmap(). - SCCS: sccscvt now calls sseek() instead of fseek() and stell() instead of ftell() in order to support mmap(). - SCCS: val now calls sseek() instead of fseek() and stell() instead of ftell() in order to support mmap(). - SCCS: admin now also supports the -Xgpath=name option previously introduced to delta. Thanks to a hint from Albert Wik. - SCCS: val added some more comment. - SCCS: The programs admin and delta removed some org_*hash named variables because they have not been used. - SCCS: sccslog now uses setvbuf() for a larger I/O size - SCCS: sccslog now uses getdelim() to support unlimited line length in the delta section even though this is improbable in the delta table section. - SCCS: sccslog now moved the code to update the changeset file (used for converting to a project oriented SCCS) into a new function commit_changeset(). - SCCS: sccslog now supports a new option usermap=file to specify a project specific mapping file. This helps e.g. to convert the CSRG BSD archive into modern SCCS by specifying a CSRG BSD specific map file. - SCCS: sccslog now supports a new option maxdelta=timespec that allows to configure the timespan for previous "sccs delget" calls that shall be bundled into a virtual commit. The syntax is the same as with -mtime xxx in modern find(1) implementations. - SCCS: sccslog now prevents the same file to appear more than once in the same commit bundle, except when using the new option -multfile. - SCCS: "sccs help log" now includes usermap=file, maxdelta=timespec -multfile and -R - SCCS: sccslog renamed struct xx into struct delt - SCCS: sccslog now correctly passes the GMT offset to the delta(1) program. The previous version did use a double dash by mistake in case that the GMT offset was negative. - SCCS: sccslog now marks removed deltas in file names with an "R" in case that it is used to just list entries. - SCCS: sccslog no longer includes removed deltas into the changeset when converting an old SCCS project into a project oriented bundle that forms a changeset. - SCCS: sccslog now implements a larger cache for user name to email translations. Before, only the last hit has been remembered, but the CSRG project has over 100 users. - SCCS: sccslog now uses the correct combination of SCCS delta comment, programmer name and already seen flag to create bundles of "sccs delget" calls as project bundled commits. This results in a sccslog that now correctly deals with overlapping commits and correctly handles the SCCS history from the CSRG BSD UNIX (1979-1995). The final result is a changeset history with monotone increasing time stamp list. WARNING: the correctness has only be verified when the conversion from the CSRG BSD SCCSv4 history files into SCCSv6 history files was done using TZ=US/Pacific. This could either be a still existing bug or caused by CSRG commits that have been done while switching into Wintertime in Europe. - SCCS: The current idea for converting a historic SCCS project into a project oriented SCCS history bundle is the following: - Create a user map file for "sccslog" by calling: mkdir $HOME/.sccs $EDITOR $HOME/.sccs/usermap Enter the UNIX login names followed by a TAB, followed by an E-mail notation. Use one line per user, e.g. joerg J. Schilling - Create a copy of the whole project to work on for this test. Do not do this conversion on the original project until sccs-6.0 is ready. - chdir to the project home directory of the just created copy. - Call "sccs init -i ." to make the project using an in-tree project oriented repository. - Call: find * -path '*SCCS/s.*' | /opt/schily/ccs/bin/sccscvt -NSCCS/s. -k -ooo -V6 - for the CSRG BSD project use: find * -path '*SCCS/s.*' | TZ=US/Pacific /opt/schily/ccs/bin/sccscvt -NSCCS/s. -k -ooo -V6 - to convert all history files into SCCSv6 history files. The TZ=US/Pacific is important for the UCB conversion since SCCSv6 uses timezones but SCCSv4 does not and we need to have the correct timezone entries in the SCCSv6 history files. For the complete "schilytools" project with 4200 SCCS history files in 55 Mbytes, this takes 12 seconds for the SCCS history from 1984 .. 2020, but note that most of the edits from the 1980s are lost, so there are few entries from the time before 1989. An alternate example: the SCCS history from the BSD-4.4 project from December 1979 up to June 1995 is in 12600 SCCS history files that take up 125 MB. The conversion time to the SCCSv6 history file format is 18 seconds. - Call: find * -path '*SCCS/s.*' | /opt/schily/ccs/bin/sccslog -changeset - to populate the changeset file from the existing deltas. For the complete "schilytools" project with 19600 commits, this takes 8 minutes. The resulting file .sccs/SCCS/s.changeset has a size of approx. 7 MBytes. An alternate example: the SCCS history from the BSD-4.4 project from December 1979 up to June 1995 has approx. 47000 commits. The conversion time is approx. 40 minutes. The size of the resulting changeset file is aprox. 14 MBytes. - convert the in-tree repository into an off-tree repository. This final step is not yet needed and there is currently no code to do that automatically. - If you like to check the resulting changeset file, there is currently only one way to look at it, by calling: sccs -O get -p -A -m .sccs/SCCS/s.changset | more This prints an anannotated version of the changeset file. The next task is to develop an enhancement to "sccs log" that prints the changeset in a way similar to what "hg log -v" prints. - NOTE: Normal filesystems on Linux are slow, it is advised to make the conversions on tmpfs for permformance reasons in case you are using Linux. Please however keep in mind that this is still experimental and there is absolutely no grant that a changelog created with current experimental software will work correctly with the final SCCS version. The procedure is just an example to check how it may look like. The final conversion method will be more automated... most likely by a command similar to "sccs import ..." IMPORTANT: This is not yet the time to finally convert a project into the project mode, because the project would be stuck in the current state. What we need to continue work in that repository state in the project mode is at least a working "sccs commit". Be prepared to remove the changeset history file once "sccs commit" works and to re-create the changeset file for that time. - SCCS TODO: - Implement something that outputs similar information from the changeset file as printed with "hg log -v". This would be the next key feature. - verify whether sccs.c uses -NSCCS in the back end programs correctly, instead of converting g-file names from the command line into s.file names in the frontend in order to forward s.file names to the backend programs. This is neded for an off-tree repository. The related unit tests are already passed. - Add code to to sccs(1) to send a list of files to admin(1) and delta(1) with new or modified files in order to have all important code for a "sccs commit" in a single program that does not need to deal with ARG_MAX limitations. - Add code to admin(1), delta(1), sccs-log(1) and get(1) to maintain/understand the changeset file. This is mainly writing out the sccschangeset(4) entries to an imtermediate store if a single file has been treated succsessfully. For sccs-log(1), see below. - Finish the work to allow normal line based diffs in SCCS even for binary files. This are files that include nul bytes and this needs to completely avoid fputs() and this needs an initialized member p_line_length in struct packet even for all content that does not result from a previous getline() call. - sccs -R tell (and probably other subcommands?) does not yet work in NewMode - Add code to libcomobj to understand the changeset file. This is needed in order to e.g. know the file names and file specific SIDs/state that corresponds to a project global SID. - Find/verify a complete transactional model that allows to repair complex changes to the set of files for a project that have been aborted in the middle. The current idea is to create the file $PROJECTHOME/.sccs/changeset with the deltas to the changeset during a complex update operation. - Find a decision on how to deal with the admin flags that are currently implemented as global flags and thus do not depend on the SID (version) if the history file. - Aborting a transaction via ^C currently requires a manual removal of the global lock file. Find a way to avoid this in case that a commit has been aborted while being prompted for a commit message (which is before any real action happened). - Implement a fully automated method to convert a SCCSv4 based history with unrelated history files into a new SCCSv6 based project mode history with a populated changeset history file. This will most likely be done as a variant of the to be defined new command "sccs sccsimport" that imports a whole existing old SCCS project. - Implement this "sccs sccsimport" based conversion in a way where sccs(1) holds the global changeset lock for the whole time of the conversion. - Bourne Shell Missing features for POSIX compliance: - Support for $'...' quoting (this is not needed for the current version of POSIX but for the next POSIX version that will be named SUSv8). The development of SUSv8 will start in late 2016. We are now expecting the Bourne Shell to be fully POSIX compliant. - Bourne Shell further TODO list: - Finish loadable builtin support. - POSIX does not allow us to implement ". -h", so we will add a "source" builtin to be able to implement "source -h" - The following builtins (that are available in bsh) are still missing in the Bourne Shell: err echo with output going to stderr glob echo with '\0' instead of ' ' between args env a builtin version of /usr/bin/env The following bsh intrinsics are still missing in the Bourne Shell: - the restricted bsh has restriction features that are missing in the Bourne shell. - source -h read file into history but do not execute and probably more features not yet identified to be bsh unique. Author: Joerg Schilling D-13353 Berlin Germany Email: joerg@schily.net, joerg.schilling@fokus.fraunhofer.de Please mail bugs and suggestions to me.