template ======== ``bgqmap template`` is a tool aimed to ease the creation of a :ref:`jobs file` to be used with **bgqmap submit**. Features: - find your current loaded *EasyBuild modules* and adds them to the output as pre-command - find your current *conda environment* and add them to the output as pre-command - the *job parameters* passed through command line are added to the generated jobs_file Generating a file ----------------- By default, the output is printed to the standard output. You can provide a file with the ``-f`` flag or redirect the output to a file (``> file.txt``). Using wildcards --------------- ``bgqmap template`` accepts two types of wildcards: - **user wildcards**: indicated with ``{{...}}`` They can contain: - list of ``,`` separated items: the wildcard is replaced by each item - file name: he wildcard is replaced by each line in the file - **glob wildcards**: if any of ``*``, ``**``, ``?`` and ``[x-y]`` is found, it is assumed to be a *glob* wildcard and therefore expanded using the python glob module. How it works ^^^^^^^^^^^^ The expansion of wildcards is a two step process. First *user wildcards* are expanded and *glob wildcards* are expanded in a second phase. For the latter, any set of characters surrounded by blanks is analysed. If it contains one or more of the mentioned wildcards, a glob search is performed. .. note:: Use ``\`` before glob wildcards to avoid their expansion Named groups ^^^^^^^^^^^^ ``bgqmap template`` contains a special feature that allows the user to replace the values of **any of the wildcards** in different parts of the command. To use this feature, the wildcard needs to be named using ``{{?:}}`` and it can be replaced anywhere using ``{{?=}}``. The *name* can be anything, but a *glob wildcard* character. It cannot start with a number. .. tip:: We recommend to limit the names to characters in a-z, A-Z and 0-9. The *value* can be **anything** in a *user wildcard* or a *glob wildcard*. .. note:: Even if it is possible to use a glob wildcard in a user wildcard (e.g. {{a,*.txt}}) we do not recommend this use for named groups as the result might differ from the expected. .. warning:: As mentioned, in a user group you can place anything that is is a user or glob wildcard. Thus, ``{{?group:*}}.txt`` recognize the glob wildcard and and will do a glob search for all :file:`.txt` files. On the other hand, ``{{?group:*.txt}}`` assumes it is a user wildcard (as it is not only a glob wildcard) and will try to open a file named :file:`*.txt` which most likely will not exits and will fail. Usage ----- .. code-block:: sh bgqmap template "" -m -c -f ---- Usage: bgqmap template [OPTIONS] CMD Create a file template for execution with bgqmap. Conda environment and Easy build modules, if not provided are taken from the corresponding environment variables The commands accepts '{{...}}' and '*', '?', [x-y] (from glob module) as wildcards. If you want to use the value resulting of the expansion of that wildcard, name it '{{?name:...}}' and use it anywhere '{{?=name}}' and as many times as you want. First, items between '{{...}}' are expanded: If there only one element, it is assumed to be a file path, and the wildcard is replaced by any line in that file which is not empty or commented. If there are more ',' separated elements, it is assumed to be a list, and the wildcard replaced by each of the list members. If the inner value corresponds to one of the glob module wildcards, its expansion is postponed. In a second phase, '*' wildcards are substituted as in glob.glob. Wildcards with are not in a named group are expanded first, and the latter ones are expanded in a final iterative process. Options: -f, --file PATH File to write the template to -c, --cores INTEGER Number of cores to use -m, --memory TEXT Max memory -t, --wall_time TEXT Wall time --conda-env TEXT Conda environment. Default: current environment --module PATH Easy build modules. Default: current loaded modules -h, --help Show this message and exit. Examples -------- **Easybuild modules** and **conda environments** are recognized: .. code-block:: console :emphasize-lines: 1 $ bgqmap template "sleep 5" # module load anaconda3/4.4.0 # source activate test_bgqmap sleep 5 **Job parameters** can also be added: .. code-block:: console :emphasize-lines: 1 $ bgqmap template "sleep 5" -c 1 -m 1G ## cores=1, memory=1G sleep 5 Using **user wildcards** with lists: .. code-block:: console :emphasize-lines: 1 $ bgqmap template "sleep {{5,10}}" sleep 5 sleep 10 Using **user wildcards** with files: .. code-block:: console :emphasize-lines: 1 $ bgqmap template "sleep {{examples/input/sleep_times.txt}}" sleep 5 sleep 10 Using **glob wildcards**: .. code-block:: console :emphasize-lines: 1 $ bgqmap template "mypgrog --input examples/input/*.txt" mypgrog --input examples/input/sleep_times.txt mypgrog --input examples/input/empty.txt Using **named wildcards**: .. code-block:: console :emphasize-lines: 1 $ bgqmap template "myprog --input examples/input/{{?f_name:*}}.txt --variable {{?v_name:a,b}} --output {{?=f_name}}_{{?=v_name}}" myprog --input examples/input/sleep_times.txt --variable a --output sleep_times_a myprog --input examples/input/empty.txt --variable a --output empty_a myprog --input examples/input/sleep_times.txt --variable b --output sleep_times_b myprog --input examples/input/empty.txt --variable b --output empty_b .. bgqmap template -c 6 -m 12 "oncodriveml -i mutations/*.txt -e cds.txt" .. bgqmap template -c 4 "oncodriveml -i mutations/{{?cancer_type:*}}.txt -e cds.txt -o {{?=cancer_type}}"