pegasus-wms 4.0.1+dfsg-8 / etc / pegasus / basic.properties

# TITLE "BASIC PROPERTIES"
#
# This is the reference guide to the basic properties regarding the
# Pegasus Workflow Planner, and their respective default values. Please refer
# to the advanced properties guide to know about all the properties that
# a user can use to configure the Pegasus Workflow Planner.
# Please note that the values rely on proper capitalization, unless explicitly 
# noted otherwise.
#
# Some properties rely with their default on the value of other
# properties. As a notation, the curly braces refer to the value of the
# named property. For instance, ${pegasus.home} means that the value depends
# on the value of the pegasus.home property plus any noted additions. You
# can use this notation to refer to other properties, though the extent
# of the subsitutions are limited. Usually, you want to refer to a set
# of the standard system properties. Nesting is not allowed.
# Substitutions will only be done once.
#
# There is a priority to the order of reading and evaluating properties.
# Usually one does not need to worry about the priorities. However, it
# is good to know the details of when which property applies, and how
# one property is able to overwrite another. The following is a mutually exclusive
# list ( highest priority first ) of property file locations.  
#
# <orderedlist>
# <listitem>--conf option to the tools. Almost all of the clients that use properties 
#  have a --conf option to specify the property file to pick up. 
# </listitem>
# <listitem> submit-dir/pegasus.xxxxxxx.properties file. All tools that work on the 
# submit directory ( i.e after pegasus has planned a workflow) pick up the 
# pegasus.xxxxx.properties file from the submit directory. The location for the 
# pegasus.xxxxxxx.propertiesis picked up from the braindump file.
# </listitem>
# <listitem>The properties defined in the user property file
# <emphasis>${user.home}/.pegasusrc</emphasis> have lowest priority. 
# </listitem>
# </orderedlist>
# 
# Commandline properties have the highest priority. These override any property loaded 
# from a property file. Each  commandline property is introduced by a -D argument. 
# Note that these arguments  are parsed by the shell wrapper, and thus the -D arguments 
# must be the first arguments to any command. Commandline properties are useful for debugging
# purposes.
#
# From Pegasus 3.1 release onwards, support has been dropped for the following
# properties that were used to signify the location of the properties file
#
# <itemizedlist>
#  <listitem>pegasus.properties</listitem>
#  <listitem>pegasus.user.properties</listitem>
# </itemizedlist>
#
# The following example provides a sensible set of properties to be set
# by the user property file. These properties use mostly non-default
# settings. It is an example only, and will not work for you:
#
# <screen>
# pegasus.catalog.replica              File
# pegasus.catalog.replica.file         ${pegasus.home}/etc/sample.rc.data
# pegasus.catalog.transformation       Text
# pegasus.catalog.transformation.file  ${pegasus.home}/etc/sample.tc.text
# pegasus.catalog.site                 XML3
# pegasus.catalog.site.file            ${pegasus.home}/etc/sample.sites.xml3
# </screen>
#
# If you are in doubt which properties are actually visible, pegasus during the 
# planning of the workflow  dumps all properties after reading and prioritizing 
# in the submit directory in a file with the suffix properties.

# Property : pegasus.home
# Systems  : all
# Type     : directory location string
# Default  : "$PEGASUS_HOME"
#
# The property pegasus.home cannot be set in the property file. This property is 
# automatically set up by the pegasus clients internally by determining the installation
# directory of pegasus. Knowledge about this property is important for developers who 
# want to invoke PEGASUS JAVA classes without the shell wrappers. 
#
# pegasus.home		"$PEGASUS_HOME"


#
# SECTION "CATALOG PROPERTIES"
#

#
# SUBSECTION "REPLICA CATALOG"
#


# Property : pegasus.catalog.replica
# System   : Pegasus
# Since    : 2.0
# Type     : enumeration
# Value[0] : JDBCRC
# Value[1] : File
# Value[2] : MRC
# Default  : File
#
# Pegasus queries a Replica Catalog to discover the physical filenames
# (PFN) for input files specified in the DAX. Pegasus can interface
# with various types of Replica Catalogs. This property specifies
# which type of Replica Catalog to use during the planning process.
#
# <variablelist>
# <varlistentry>
# <term>JDBCRC</term>
# <listitem> 
#      In this mode, Pegasus queries a SQL based replica catalog that
#      is accessed via JDBC. The sql schema's for this catalog can be
#      found at $PEGASUS_HOME/sql directory. 
#      To use JDBCRC, the user additionally needs to set the following
#      properties
#      <orderedlist>
#		<listitem>pegasus.catalog.replica.db.url</listitem>
#		<listitem>pegasus.catalog.replica.db.user</listitem>
#		<listitem>pegasus.catalog.replica.db.password</listitem>
#	   </orderedlist>		
# </listitem>
# </varlistentry>
# <varlistentry>
# <term>File</term>
# <listitem><para>In this mode, Pegasus queries a file based replica catalog. 
#     It is neither transactionally safe, nor advised to use for
#     production purposes in any way. Multiple concurrent access to
#     the File will end up clobbering the contents of the file.  The 
#     site attribute should be specified whenever possible. The attribute 
#     key for the site attribute is "pool".  
#
#     The LFN may or may not be quoted. If it contains linear
#     whitespace, quotes, backslash or an equality sign, it must be
#     quoted and escaped. Ditto for the PFN. The attribute key-value
#     pairs are separated by an equality sign  without any
#     whitespaces. The value may be in quoted. The LFN  sentiments about quoting apply.
#
#     <screen>
#     LFN PFN
#     LFN PFN a=b [..]
#     LFN PFN a="b" [..]
#     "LFN w/LWS" "PFN w/LWS" [..]
#     </screen>
#
#     To use File, the user additionally needs to specify
#     pegasus.catalog.replica.file property to specify the path to the
#     file based RC.
# </para></listitem>
# </varlistentry>
# <varlistentry>
# <term>MRC</term>
# <listitem><para>In this mode, Pegasus queries multiple replica catalogs to
#     discover the file locations on the grid.  To use it set
# 
#     <screen>
#     pegasus.catalog.replica MRC
#     </screen>
#
#     Each associated replica catalog can be configured via properties
#     as follows. 
#
#     The user associates a variable name referred to as [value] for
#     each of the catalogs, where [value] is any legal identifier
#     (concretely [A-Za-z][_A-Za-z0-9]*) For each associated replica
#     catalogs the user specifies the following properties. 
#
#     <screen>	
#      pegasus.catalog.replica.mrc.[value]       specifies the type of replica catalog.
#      pegasus.catalog.replica.mrc.[value].key   specifies a property name key for a
#						 particular catalog
#     </screen>	 
#
#     For example, if a user wants to query two lrc's at the same time
#     he/she can specify as follows 
#
#     <screen>	
#     pegasus.catalog.replica.mrc.lrc1 LRC
#     pegasus.catalog.replica.mrc.lrc2.url rls://sukhna
#
#     pegasus.catalog.replica.mrc.lrc2 LRC
#     pegasus.catalog.replica.mrc.lrc2.url rls://smarty
#     </screen>
# 
#
#     In the above example, lrc1, lrc2 are any valid identifier names
#     and url is the property key that needed to be specified. 
# </para></listitem>
# </varlistentry>
# </variablelist>
#
#
#
# pegasus.catalog.replica			RLS


# Property : pegasus.catalog.replica.url
# System   : Pegasus
# Since    : 2.0
# Type     : URI string
# Default  : (no default)
#
# When using the modern RLS replica catalog, the URI to the Replica
# catalog must be  provided to Pegasus to enable it to look up
# filenames. There is no  default.
#
# pegasus.catalog.replica.url			(no default)





#
# SUBSECTION "SITE CATALOG"
#

# Property : pegasus.catalog.site
# System   : Site Catalog
# Since    : 2.0
# Type     : enumeration
# Value[0] : XML3
# Value[1] : XML
# Default  : XML3
#
# The site catalog file is available in three major flavors: The Text and 
# and XML formats for the site catalog are deprecated.
# Users can use pegasus-sc-converter client to convert their site catalog 
# to the newer XML3 format.
# <orderedlist>
# <listitem> THIS FORMAT IS DEPRECATED. WILL BE REMOVED IN COMING VERSIONS.
#     USE pegasus-sc-converter to convert XML format to XML3 Format.
#      The "XML" format is an XML-based file. The XML format reads site
#      catalog conforming to the old site catalog schema available at
#      http://pegasus.isi.edu/wms/docs/schemas/sc-2.0/sc-2.0.xsd 
# </listitem>      
# <listitem> The "XML3" format is an XML-based file. The XML format reads site
#      catalog conforming to the old site catalog schema available at
#      http://pegasus.isi.edu/wms/docs/schemas/sc-3.0/sc-3.0.xsd 
# </listitem>      
# </orderedlist>
#
# pegasus.catalog.site		XML3



# Property : pegasus.catalog.site.file
# System   : Site Catalog
# Since    : 2.0
# Type     : file location string
# Default  : ${pegasus.home.sysconfdir}/sites.xml3 |${pegasus.home.sysconfdir}/sites.xml
# See also : pegasus.catalog.site
#
# Running things on the grid requires an extensive description of the
# capabilities of each compute cluster, commonly termed "site". This
# property describes the location of the file that contains such a site
# description. As the format is currently in flow, please refer to the
# userguide and Pegasus for details which format is expected.
# The default value is dependant on the value specified for
# the property pegasus.catalog.site . If type of SiteCatalog used is XML3, then sites.xml3
# is picked up from sysconfdir else sites.xml
#
# pegasus.catalog.site.file ${pegasus.home.sysconfdir}/sites.xml3 | ${pegasus.home.sysconfdir}/sites.xml

#
# SUBSECTION "TRANSFORMATION CATALOG"
#


# Property : pegasus.catalog.transformation
# System   : Transformation Catalog
# Since    : 2.0
# Type     : enumeration
# Value[0] : Text
# Value[1] : File
# Default  : Text
# See also : pegasus.catalog.transformation.file
#
# 
# <variablelist>
# <varlistentry><term>Text</term>
# <listitem><para>In this mode, a multiline file based format is understood. The file
#     is read and cached in memory. Any modifications, as adding or
#     deleting, causes an update of the memory and hence to the file
#     underneath. All queries are done against the memory
#     representation. 
#     
#     The file sample.tc.text in the etc directory contains an example
#
#     Here is a sample textual format for transfomation catalog containing
#     one transformation on two sites
#
#     <screen>
#     tr example::keg:1.0 {
#
#       #specify profiles that apply for all the sites for the transformation
#       #in each site entry the profile can be overriden
#       profile env "APP_HOME" "/tmp/karan"
#       profile env "JAVA_HOME" "/bin/app"
# 
#       site isi {
#          profile env "me" "with"
#          profile condor "more" "test"
#          profile env "JAVA_HOME" "/bin/java.1.6"
#          pfn "/path/to/keg"
#          arch  "x86"
#          os    "linux"
#          osrelease "fc"
#          osversion "4"
#          type "INSTALLED"
#       }
#
#       site wind {
#          profile env "me" "with"
#          profile condor "more" "test"
#          pfn "/path/to/keg"
#          arch  "x86"
#          os    "linux"
#          osrelease "fc"
#          osversion "4"
#          type "STAGEABLE"
#       }
#     }
#     </screen>
# </para>
# </listitem></varlistentry>
# <varlistentry><term>File</term>
# <listitem>THIS FORMAT IS DEPRECATED. WILL BE REMOVED IN COMING VERSIONS.
#     USE pegasus-tc-converter to convert File format to Text Format.
#     In this mode, a file format is understood. The file is
#     read and cached in memory. Any modifications, as adding or
#     deleting, causes an update of the memory and hence to the file
#     underneath. All queries are done against the memory
#     representation. The new TC file format uses 6 columns:
#     <orderedlist>
#	<listitem>The resource ID is represented in the first column.</listitem> 
#	<listitem>The logical transformation uses the colonized format
#	    ns::name:vs.</listitem> 
#	<listitem>The path to the application on the system</listitem>
#	<listitem>The installation type is identified by one of the following
#	    keywords - all upper case: INSTALLED, STAGEABLE. 
#	    If not specified, or <command>NULL</command> is used, the type
#	    defaults to INSTALLED.</listitem> 
#	<listitem>The system is of the format ARCH::OS[:VER:GLIBC]. The
#	    following arch types are understood: "INTEL32", "INTEL64",
#	    "SPARCV7", "SPARCV9". 
#	    The following os types are understood: "LINUX", "SUNOS",
#	    "AIX". If unset or <command>NULL</command>, defaults to
#	    INTEL32::LINUX.</listitem> 	
#	<listitem>Profiles are written in the format
#	    NS::KEY=VALUE,KEY2=VALUE;NS2::KEY3=VALUE3 
#	    Multiple key-values for same namespace are seperated by a
#	    comma "," and multiple namespaces are seperated by a
#	    semicolon ";". If any of your profile values contains a
#	    comma  you must not use the namespace abbreviator.</listitem>
#    </orderedlist>
# </listitem></varlistentry>
# </variablelist>
#
#
# pegasus.catalog.transformation		Text


# Property : pegasus.catalog.transformation.file
# Systems  : Transformation Catalog
# Type     : file location string
# Default  : ${pegasus.home.sysconfdir}/tc.text | ${pegasus.home.sysconfdir}/tc.data
# See also : pegasus.catalog.transformation
#
# This property is used to set the path to the textual transformation 
# catalogs of type File or Text. If the transformation catalog is of type Text
# then tc.text file is picked up from sysconfdir, else tc.data
#
#
# pegasus.catalog.transformation.file	 ${pegasus.home.sysconfdir}/tc.text | ${pegasus.home.sysconfdir}/tc.data


#
# SECTION "DATA STAGING CONFIGURATION"
#



# Property : pegasus.data.configuration
# System   : Pegasus
# Since    : 3.1
# Type     : enumeration
# Value[0] : sharedfs
# Value[1] : nonsharedfs
# Value[2] : condorio
# Default  : sharedfs
#
# This property sets up Pegasus to run in different environments.
#
#
# <variablelist>
# <varlistentry><term>sharedfs</term>
# <listitem>If this is set, Pegasus will be setup to execute jobs on the shared
#    filesystem on the execution site. This assumes, that the head node of a cluster
#    and the worker nodes share a filesystem. The staging site in this case is 
#    the same as the execution site. Pegasus adds a create dir job to the executable
#    workflow that creates a workflow specific directory on the shared filesystem . 
#    The data transfer jobs in the executable workflow ( stage_in_ , stage_inter_ ,
#    stage_out_ ) transfer the data to this directory.The compute jobs in the 
#    executable workflow are launched in the directory on the shared  filesystem.
#    Internally, if this is set the following properties are set.
#     <screen>
#     pegasus.execute.*.filesystem.local   false
#     </screen>
# </listitem>
# </varlistentry>
# <varlistentry><term>condorio</term>
# <listitem>If this is set, Pegasus will be setup to run jobs in a pure condor pool, 
#     with the nodes not sharing a filesystem. Data is staged to the compute nodes from
#     the submit host using Condor File IO.
#     The planner is automatically setup to use the submit host ( site local ) as the
#     staging site. All the auxillary jobs added by the planner to the executable
#     workflow ( create dir, data stagein and stage-out, cleanup ) jobs refer to
#     the workflow specific directory on the local site.  The data transfer jobs in 
#     the executable workflow ( stage_in_ , stage_inter_ , stage_out_ ) transfer the 
#     data to this directory. When the compute jobs start, the input data for each 
#     job is shipped from the workflow specific directory on the submit host to
#     compute/worker node using Condor file IO. The output data for each job is 
#     similarly shipped back to the submit host from the compute/worker node.
#     This setup is particularly helpful when running workflows in the cloud 
#     environment where setting up a shared filesystem across the VM's may be
#     tricky. 
#     On loading this property, internally the following properies are set
#     <screen>
#     pegasus.transfer.sls.*.impl          Condor
#     pegasus.execute.*.filesystem.local   true
#     pegasus.gridstart 		   PegasusLite
#     pegasus.transfer.worker.package      true
#     </screen>
# </listitem>
# </varlistentry>
# <varlistentry><term>nonsharedfs</term>
# <listitem>If this is set, Pegasus will be setup to execute jobs on an execution site
#     without relying on a shared filesystem between the head node and the worker nodes.
#     You can specify staging site ( using --staging-site option to pegasus-plan) to
#     indicate the site to use as a central storage location for a workflow. The 
#     staging site is independant of the execution sites on which a workflow executes.
#     All the auxillary jobs added by the planner to the executable
#     workflow ( create dir, data stagein and stage-out, cleanup ) jobs refer to
#     the workflow specific directory on the staging site.  The data transfer jobs in 
#     the executable workflow ( stage_in_ , stage_inter_ , stage_out_ ) transfer the 
#     data to this directory. When the compute jobs start, the input data for each 
#     job is shipped from the workflow specific directory on the submit host to
#     compute/worker node using pegasus-transfer. The output data for each job is 
#     similarly shipped back to the submit host from the compute/worker node.
#     The protocols supported are at this time SRM, GridFTP, iRods, S3.
#     This setup is particularly helpful when running workflows on OSG where
#     most of the execution sites don't have enough data storage. Only a few
#     sites have large amounts of data storage exposed that can be used to place
#     data during a workflow run. This setup is also helpful when running workflows
#     in the cloud environment where setting up a shared filesystem across the VM's may be
#     tricky. 
#     On loading this property, internally the following properies are set
#     <screen>
#     pegasus.execute.*.filesystem.local   true
#     pegasus.gridstart 		   PegasusLite
#     pegasus.transfer.worker.package      true
#     </screen>
# </listitem></varlistentry>
# </variablelist>
#
# 
#
# pegasus.data.configuration		sharedfs