tex-common 2.10 / usr / share / tex-common / tpm2licenses.README

README file for tpm2licenses
*****************************

This README file explains what the script tpm2licenses is meant for,
and how it can be used.

1. Purpose
2. Requirements
3. Usage of the script
4. License verification procedure

1. Purpose
===========

TeXlive is a big collection, itself a distribution put together from
individual packages on CTAN (ftp://cam.ctan.org/tex-archive/).
Upstream (the TeXLive team) do share Debian's view on Free Software
(except maybe for documentation issues), but there might still be
files included that are non-free or are intended to be free, but don't
have a proper license statement.  The reason is mostly historical - in
the old days, both upstream and individual package authors didn't care
about these things as they should have.  And TeX has grown for a long
time...

In the past, the Debian teTeX packages (the predecessor of TeXLive)
have been audited for license freeness at least once.  However, this
has been done on a per-CTAN- package basis (and even that, perhaps,
not too systematically), not on a per-file basis.  Furthermore, the
process of auditing has not been documented.  Therefore, it is not
possible today to check which parts were included and checked back
then.

As a consequence, we have decided to do it systematically this time,
and to keep information about which files are associated with a
particular package, which license they're under, and where this
information can be found.

This information will ultimately end up in debian/copyright in a very
condensed form.  For maintaining the information, we use different,
more structured files at different locations, re-using already
existing infrastructure in the TeXlive package building mechanisms and
the TeX Catalogue.  This script is used to verify whether this
existing information is correct and applies to the current TeXLive
packages, and to merge the information into the copyright file.

2. Requirements
======================

- installed packages

  tpm2licenses is installed with the tex-common package, as an
  executable in /usr/share/tex-common, but since it is not at the core
  of tex-common's functionality, two additional packages are needed
  that tex-common does not depend upon: 

	libxml-dom-perl, libappconfig-perl.

- Available sources

  Obviously, you need the tetex-base[1] source package.  You also need
  the tetex-src package - either the installed binary package or,
  better, the source package.

  Having TeXlive sources installed is not strictly needed, but it
  makes your contribution much more valuable

  Finally, the script can access a local checkout of the TeX Catalogue
  sources, but this is not needed to do the auditing, only for the
  final copyright file generation (and even here an intermediate file
  can be used)

3. Usage of the script
========================

SYNTAX:

  /usr/share/tex-common/tpm2licenses [ options ] [ tpm file(s) ]

The script must be run from the root directory of an unpacked source
package (e.g. tetex-base-3.0/) [FIXME: texlive runs it on installed
files?] and understands the following options:

   --catalogue <directory>
   --catalogue file:<file>

	Path to a local checkout of the TeX Catalogue sources, or in
	the second form the path to a file that contains a precompiled
	list of "license lines" for CTAN packages.

   --nocheckcatalogue

	Do not try to check the catalogue

   --tpmdir

	The directory where tpm files can be found.  Not necessary
	when auditing single tpm files

   --package [tetex-base|tetex-src|texlive-*]

	Assume we are in a source tree of this package (affects
	whether sources or runtime files are searched for, or both)

   --listallfiles

	list the files even in the tpm files for which we do not have
	Catalogue information, even if a Catalogue location is given

   --nocoverage

	Do not list files in the source package that are not listed in
	any of the tpm files.  Useful if you're giving tpm files on
	the command line.

The options can also be written into a configuration file, like this:

Catalogue = /home/frank/src/Upstream-source/texcatalogue
nocoverage

The file should be named .tpm2license.cfg and will be searched for in
the current directory, it's subdirectory debian, the parent directory
and the user's home directory (in this order).  If multiple
configuration files exist, all are read, and later entries override
earlier ones.  Thus, you can e.g. specify "nocheckcatalogue" in your
home directory.

The script will do the following things:

- Extract filenames with path information from a tpm file, one tpm
  file after the other.

- Check whether all files in the tpm do in fact exist in the source
  tree, and warn if some are missing.

- Extract license information from the TeX Catalogue, if available,
  and print it.

- For each directory mentioned in the tpm file, check whether the
  directory is complete with the files from that tpm.  If there are
  additional files in it, only the files listed in the tpm are
  returned.  If the directory is complete (not counting subdirectories
  and their contents), only the directory name with a * appended is
  printed.

If more than one tpm file is given, or the whole tpmdir is acted on,
this sequence is repeated multiple times, separately for each tpm file.

- After that, unless nocoverage is given, the script complains about any
  files that are in the source tree but not in the tpm files used in
  this run.  The coverage should be complete for TeXLive, but it isn't
  for teTeX - therefore tetex-base and tetex-src should have noocoverage
  in their configuration files currently.

4. License verification procedure
===================================

We suggest that everybody follows the procedure outlined below.

a) coordinate on debian-tetex-maint and/or in the Wiki at
   http://wiki.debian.org/LicenseAuditing which parts you are going to
   check, to prevent duplicate work.

b) copy a tpm file from texlive to tetex-base's debian/tpm/ directory.
   Of course, if you've got SVN write access, use "svn copy".  In
   texlive's SVN repository, the tpm files are alltogether in
   LocalTPM/texmf-dist/tpm/.  In the individual source packages, they
   are in texmf-dist/tpm.

   The tpm files consist of some general information about the
   package, and three lists of RunFiles, DocFiles and SourceFiles.
   Any of these may be empty.  Run and Doc files should be in
   tetex-base, Source files in tetex-src.

c) Change directory to the tetex-base source tree and run

/usr/share/tex-common/tpm2licenses debian/tpm/<your_new>.tpm >/dev/null

   The redirection makes sense because this time, you're only
   interested in the error messages about missing files.  There are
   many possible reasons for missing files, for example:

   1. TeXLive has a newer version of the package, and the newer
      version has additional files.  Of course, other files may also
      have vanished, see below.

   2. TeXlive often installs files into the documentation tree
      (DocFiles) that are in tetex-src.  

   3. Different file extensions, e.g. DVI files instead of PDF, or a
      file "packagename.readme" has been renamed to "readme.txt" since
      it's installed in the packagename directory, anyway

   4. Packaging errors by teTeX's upstream, the file *should* be
      there. 

   Therefore, for any file missing in the RunFiles or DocFiles sections,
   don't delete the line in the tpm file.  Instead, move it to
   SourceFiles and change "doc" to "source", "tex" to "source" etc.

   Proceed to the next step if there are no more error messages

d) Change to the tetex-src source tree and run 

/usr/share/tex-common/tpm2licenses path/to/<your_new>.tpm >/dev/null

   In many cases, files moved to SourceFiles from RunFiles or DocFiles
   will be reported as missing again - sometimes because they are
   really missing, sometimes because the paths or names don't fit
   completely. 

e) Now come two technically distinct steps which can conveniently be
   done at once: On the one hand, we have to check whether any files
   exist in the source trees that should be in the tpm file we're
   looking at.  On the other hand, we need to verify whether the
   license the package is under actually covers all the files listed
   in its tpm file.

   1. Adding files

   To check for files you can possibly add to the tpm file, rerun the
   above commands without the redirection.  In the ideal case, you'll
   get a list of directories, like this:

% acronym: lppl (verification data:1.25:1.17:2006-03-07:frank:header)
tex/latex/acronym/*
doc/latex/acronym/*

   This indicates that all files in these two directories are already
   listed in acronym.tpm - no need to add any.  However, if you get a
   list of many individual files in the same directory, chances are
   that one or two are missing:  It might be that the file no longer
   exists in TeXlive's newer version, an error in TeXlive, or
   whatever. 

   Furthermore, in many cases files need to be copied from DocFiles to
   SourceFiles.  This is because in TeXlive, where the tpms have been
   developed, every file exists only once.  In teTeX, on the other
   hand, the source directories in tetex-src just mirror what is on
   CTAN: Some of these files are source files, others are
   documentation which has also been copied to the tetex-base source
   tree by teTeX's upstream. They are therefore in two source trees
   and need to be in two sections in the tpm file.

   2. Checking Licenses

   Of course, we cannot have files in a package's tpm file that in
   fact is not covered by the same license!  Therefore you have to
   carefully check the license.  First of all, find out where the
   license is specified and whether it is free; then find out which
   files are covered.

   Some packages have a file "manifest.txt" or similar which lists all
   files included, and is referenced from the file that specifies the
   license.  Others have the license statement in the header of every
   source file, or also in extracted TeX input files.  In any case,
   there may be files which are not indicated as belonging to the
   package, being licensed under its license, and still belong to the
   tpm.  These fall into two categories:

   - trivial files

     Very simple readme files, simple Makefiles and their TeX
     counterparts, *.ins files, don't need a license statement.  They
     still should have one, so if we're contacting upstream, we can as
     well request to add them to the list of covered files.  But it's
     not required.

     Also, pure license files, including manifest.txt-style files,
     don't need a license statement.

   - generated files

     The files that can be autogenerated from a *.dtx file are usually
     listed in an *.ins file (which might have a different name) in a
     command starting with \generate on a line containing
     \file{<filename>}.  Some distributions also contain the generated
     file <dtx-basename>.drv.  Both from the *.dtx file itself and
     from tex input files generated by the *.ins file, documentation
     in PDF, PS or DVI format can be generated (and in other formats
     using appropriate converters). 

   Most tpm files refer to only one subdirectory in the tetex-src
   tree, and all files in that directory should be covered by the tpm
   file (except any additions by teTeX upstream, usually indicated by
   "tetex" in their name).  In tetex-base, however, files in a
   particular subdirectory *may* also come from a different tpm file.
   If you are unsure, just ignore them, they will be found in the end
   by the coverage check.

f) Reporting success and problems

   If all is well, just submit the following information to
   debian-tetex-maint@lists.debian.org:

   - the working tpm file

   - the license text, or a name if it is LPPL or in
     /usr/share/common-licenses 

   - The name of the file where the license information is given, the
     version in teTeX and, if you also checked that, versions in
     TeXlive and on CTAN.

   If you found incomplete license information, or non-free license
   texts, please report as much info as you have to the mailing list
   and to the Wiki at http://wiki.debian.org/LicenseAuditing.  Steps
   to perform now can include:

   - Check whether problematic files exist elsewhere on CTAN (some
     files are only in zip or tar.gz archives!) and have other,
     hopefully newer license information
 
   - Check whether any license discussion is archived by Google or
     other search engines

   - Find out the responsible upstream author and contact them.

     When contacting upstream, please be careful.  Remind yourself
     that most of the time, upstream authors want their work to be
     free and think they have already done enough.  Explain in some
     detail and with polite words what should be done and why.  If
     upstream has been inactive in the TeX community, it might make
     sense to offer that you do an upload on behalf of them.  Maybe
     not in the initial mail, but later in the conversation.
   
 


[1] Eventually, tetex-bin will have to be checked, too, but currently
we're concentrating on tetex-base.