libcatmandu-perl 1.0700-1 / usr / bin / catmandu

#!/usr/bin/perl 

use Catmandu;
use Catmandu::CLI;
use Cwd ();

Catmandu->default_load_path(Cwd::getcwd);
Catmandu::CLI->run // exit(2);

=head1 NAME

catmandu - LibreCat command line tools

=head1 SYNOPSIS

 catmandu [-D|-D2|-D3] [-I path] [-L path] [COMMAND] [-?hqv] [long options...] [ARGS]

 catmandu --version

 catmandu help

 # convert
 catmandu convert JSON to YAML < file.json
 catmandu convert YAML to JSON < file.yaml
 catmandu convert YAML < file.yaml # 'to JSON' is implicit
 catmandu convert MARC < records.mrc
 catmandu convert MARC --fix 'marc_map(245,title)' < records.mrc
 catmandu convert MARC --fix myfixes.txt < records.mrc
 catmandu convert OAI --url https://biblio.ugent.be/oai
 catmandu convert SRU --base http://www.unicat.be/sru --query dna

 # import
 catmandu import JSON to MongoDB --database_name mydb < records.json
 catmandu import MARC to MongoDB --database_name mydb < records.mrc
 catmandu import MARC to ElasticSearch --index_name mydb < records.mrc
 catmandu import MARC to ElasticSearch --index_name mydb --fix 'marc_map("245a","title")' < records.mrc

 # export
 catmandu export MongoDB --database_name mydb --bag data to JSON
 catmandu export MongoDB --database_name mydb --bag data to JSON --fix 'retain("_id")'
 catmandu export Solr --url http://localhost:8983/solr to JSON
 catmandu export ElasticSearch --index_name mydb to JSON

 # copy
 catmandu copy MongoDB --database_name mydb to ElasticSearch --index_name mydb 

 # show the contents of catmandu.yml
 catmandu config   

 # count items in a store
 catmandu count test1

 # delete items from store
 catmandu delete test1 -q 'title:"My Rabbit"'

 # run a fix script
 $ catmandu run myfixes.fix

 # or, create an executable fix script
 $ cat myfixes.fix
 #!/usr/local/bin/catmandu run
 do importer(OAI,url:"http://biblio.ugent.be/oai")
    retain(_id)
 end
 $ chmod 755 myfixes.fix
 $ ./myfixes.fix
 
 # configuration
 $ cat catmandu.yml
 ---
 store:
  test1:
   package: MongoDB
   options:
    database_name: mydb
 catmandu import JSON to test1 < records.json

 # show installed commands
 $ catmandu info
 $ catmandu info --namespace XML

=head1 COMMANDS

=head2 help COMMAND

Documentation about command line options.

=head2 config

Display the contents of the catmandu.yml files in your project.

=head2 convert IMPORTER [OPTIONS] [[--fix FIX]] to EXPORTER [OPTIONS]

Convert data from one format to another format with help of L<Catmandu::Importer>s and
L<Catmandu::Exporter>s. All options given on the command line will be send to the Importer
or Exporter as initialization parameters. E.g.

 catmandu convert OAI --url BASEURL --metadataPrefx PREFIX \
                  to \
                  CSV --header 1 --sep_char '|'

Look at the documentation of all the importers and exporters to see which options are
available.

All importers accept one or more --fix options with the name of a fix file or file command.

 catmandu convert JSON --fix fixes.txt
 catmandu convert JSON --fix 'remove_field(abc)'

The JSON exporter is implicit and is optional.

 catmandu convert YAML to JSON
 catmandu convert YAML # shorter

=head2 copy STORE [OPTIONS] [-q QUERY] [--limit LIMIT] [[--fix FIX]] to STORE [OPTIONS]

Copy items from one L<Catmandu::Store> to another. All options given
on the command line will be send to the Stores as initialization parameters.
Optionally a QUERY and LIMIT can be provided to selectively export items from the Store.
Optionally a FIX can be provided to be executed on all the items before they are
stored.

=head2 count STORE [OPTIONS]

Count the number of items in a L<Catmandu::Store>. All options given on the command line
will be send to the store as initialization parameters. E.g.

 catmandu count MongoDB --database_name mydb --bag data

=head2 delete STORE [OPTIONS] [-q QUERY]

Delete items from a L<Catmandu::Store>. All options given on the command line
will be send to the store as initialization parameters. Optionally a QUERY can
be provided to select the items to delete. E.g.

 catmandu delete MongoDB --database_name mydb --bag data -q "title:Delete me"

=head2 export STORE [OPTIONS] [-q QUERY] [--limit LIMIT] [[--fix FIX]] to EXPORTER [OPTIONS]

Export items from a L<Catmandu::Store> using a L<Catmandu::Exporter>. All options given
on the command line will be send to the Store or Exporter as initialization parameters.
Optionally a QUERY and LIMIT can be provided to selectively export items from the Store.
Optionally a FIX can be provided to be executed on all the items before they are
exporter by the exporter.

E.g.

 catmandu export MongoDB --database_name mydb --bag data -q "title:Perl"

=head2 import IMPORTER [OPTIONS] [[--fix FIX]] to STORE [OPTIONS]

Import items from a L<Catmandu::Importer> to a L<Catmandu::Store>. All options given
on the command line will be send to the Importer or Store as initialization parameters.
Optionally a FIX can be provided to be executed on all the items before they are
stored into the Store.

 catmandu import JSON to MongoDB --database_name mydb < records.json

=head2 info [OPTIONS]

List installed Catmandu modules (importers, exporters, stores, and fixes).

=head2 run [FIX]

Execute a fix script or run an interactive Fix shell.

=head1 OPTIONS

=over 4

=item -v

Verbose output. This includes progress of operations.

=item -h

=item -?

=item --lib-path PATH

=item -I

Specifies Perl library include paths, like perl's -I option. This option is
repeatable.

=item --load-path PATH

=item -L PATH

Search upwards from this path for configuration files and accompanying lib
directories. This option is repeatable.

=item --var [NAME]=[VALUE]

Catmandu such as convert, import and export accept command line variables
that can be read by any fix script provided. For instance, if provenance
information need to be added to the output, then this can be provided
via the command line:

     $ catmandu convert MARC --fix myfixes.txt --var source=Springer --var date=2016 < data.mrc

with myfixes.txt a file like:

     add_field(source,{{source}})
     add_field(data,{{date}})
     ...(other fixes)...

=item --debug [LEVEL]

=item -D[LEVEL]

Set the debugging level for Catmandu. LEVEL is one of:

    1 - WARN messages
    2 - INFO messages
    3 - DEBUG messages

Requires an installation of L<Log::Log4perl> and L<Log::Any::Adapter::Log4perl>. By
default an internal log4perl configuration file is used that sends all debug messages
to stderr. Optionally in the catmandu.yml the path to a log4perl configuration file
can be set. E.g.

    catmandu.yml:

    log4perl: /etc/log4perl.conf

    or

    log4perl: "
     log4perl.category.Catmandu=$level,$appender

	 log4perl.appender.STDOUT=Log::Log4perl::Appender::Screen
	 log4perl.appender.STDOUT.stderr=0
	 log4perl.appender.STDOUT.utf8=1

	 log4perl.appender.STDOUT.layout=PatternLayout
	 log4perl.appender.STDOUT.layout.ConversionPattern=%d [%P] - %p %l %M time=%r : %m%n

	 log4perl.appender.STDERR=Log::Log4perl::Appender::Screen
	 log4perl.appender.STDERR.stderr=0
	 log4perl.appender.STDERR.utf8=1

	 log4perl.appender.STDERR.layout=PatternLayout
	 log4perl.appender.STDERR.layout.ConversionPattern=%d [%P] - %l : %m%n
    "

See also L<Catmandu::Logger> to enable logging for your own Catmandu packages.

=back

=head1 DATA OPTIONS

=over 4

=item --fix EXPRESSION

=item --fix FILE

When exporting or storing apply a fix EXPRESSION or all the fixes in FILE on
each item. This option is repeatable.

=item --start NUM

When exporting or storing data skip the first NUM items.

=item --total NUM

When exporting of storing data process only a total NUM of items.

=item --replace

When storing data first delete all the items in the Store or Bag.

=item --query QUERY

=item --cql-query QUERY | -q QUERY

When a Store is searchable execute and return the results. Using the --cql-query or -q
option the CQL query language can be used (if supported).

=item --sort SORT

=item --sru-sortkeys SORT

Set the sort order for the results returned from a searchable Store. --sru-sortkeys
is only available if the Store supports CQL.

=item --limit NUM

Limit the number of results returned from the searchable results to NUM.

=item --count

Report the number of items processed.

=item --version

Return the installed version of Catmandu.

=back

=head1 SEE ALSO

L<Catmandu::Importer>
L<Catmandu::Exporter>
L<Catmandu::Store>
L<Catmandu::MARC>
L<Catmandu::OAI>
L<Catmandu::MongoDB>
L<Catmandu::Solr>
L<Catmandu::ElasticSearch>

=cut