Installing, configuring, using¶
Using CliMAF at CNRM, on Ciclad or Climserv: fast track¶
If working on IPSL’s Ciclad, Climserv, at CNRM or on MF’s Beaufix HPC machine, you do not need to install CliMAF; just do as indicated below (as e.g. in section Running CliMAF interactively), replacing
<some_installation_dir>
by :/cnrm/est/COMMON/climaf/climaf_x.y.z
at CNRM~senesi/climaf_installs/climaf_x.y.
on Beaufix
On the IPSL - ESPRI Mesocenter (Ciclad and Climserv):
- do not forget to submit an interactive session if you want to run your code interactively:
- qsub -IVX -q std -l mem=9g -l vmem=9g (don’t need the X if you
don’t want interactive display, and add
-l nodes=1:ppn=10
for parallel processing on 10 procs)
- qsub -IVX -q std -l mem=9g -l vmem=9g (don’t need the X if you
don’t want interactive display, and add
do:
module load climaf
- to set your environment:
- point to a CliMAF install ($CLIMAF environment variable)
- set your CliMAF cache ($CLIMAF_CACHE environment variable)
- use a miniconda install and activate a suitable environment
- load cdo, ncl, nco, and netcdf4 modules
See ‘Configuring CliMAF’ below to customize your environment (use another CliMAF install, another cache…)
use CliMAF in an interactive IPython session (launch IPython and import CliMAF at once):
climaf
if you want to use Jupyter notebooks, use:
climaf-notebook
and follow the instructions
get the CliMAF demo notebooks:
cp -r /ciclad-home/jservon/TP_CliMAF ~ cd TP_CliMAF climaf-notebook
lastly, use:
from climaf.api import *
at the beginning of your python script or notebook to import CliMAF
Installing CliMAF (if you are not at CNRM, on Ciclad or Climserv)¶
Installing CliMAF, if necessary, is quick, through only a few commands, using CliMAF GitHub repository ; this will also copy some data allowing for testing the installation and for running a few examples
first check the listed Requirements and acknowledgements ;
execute:
cd some_installation_dir git clone https://github.com/rigoudyg/climaf climaf cd climaf export PYTHONPATH=$PYTHONPATH:$(pwd) cd testing ./test_install.sh
and check the output of last command. Contact ‘climaf at meteo dot fr’ in case of problem at that stage
for getting the development version, you may rather execute:
git clone -b dev https://github.com/rigoudyg/climaf climaf
Configuring CliMAF¶
- CliMAF do interpret some environment variables :
- CLIMAF : path to CliMAF install
- CLIMAF_CACHE : a directory used for storing intermediate results, and those final results which are not explicitly copied elsewhere; defaults to ~/tmp/climaf_cache.
- CLIMAF_REMOTE_CACHE : a directory used for storing local copies of remote data; defaults to $CLIMAF_CACHE/remote_data.
- CLIMAF_LOG_DIR : for setting the directory holdong climaf logs (defaults to current working directory
- CLIMAF_LOG_LEVEL and CLIMAF_LOGFILE_LEVEL : for setting the verbosity level on stderr (resp. on file climaf.log); defaults to ‘error’ (resp. ‘info’). See
clog()
(resp.clog_file()
) for details
- CLIMAF_CHECK_DEPENDENCIES : if set to ‘no’, this will avoid that CLiMAF checks external tools version at startup; this speeds up the init stage but should be used only when you are sure these dependencies are satisfied on your system and environment.
- CLIMAF_FIX_NEMO_TIME : if set to anything but ‘no’, this will automatic fix CNRM’s Nemo old data time_axis issues. This adresses the wrong time coordinate variable t_ave_01month and t_ave_00086400. This will add processing cost
- CLIMAF_FIX_ALADIN_COORD : if set to anything but ‘no’, this will automatic fix Aladin outputs attribute ‘coordinates’ issues. This adresses the wrong variable attribute ‘coordinates’ with ‘lat lon’ to ‘latitude longitude’.
Configuration file : you may put in file
~/.climaf
any python code using CliMAF functions; this will be executed at the end of climaf import; the code must use fully qualified names for Python functions (as in e.g.climaf.operators.cscript
): it des not benefit from the intractive shortcuts defined in climaf.api (as described below in Running CliMAF interactively)Environment :
If running on Beaufix, you must setup your environment by:
$ module load python/2.7.5 nco ncview ncl
If running on Ciclad, you must setup your environment by:
$ module load climaf
On some systems, if CDO fails at allocating memory, you may have to put e.g. in your ~/.bash_profile:
$ ulimit -s unlimited
in csh:
$ unlimit stacksize
Running CliMAF interactively¶
For running CliMAF as easily as possible under the Python prompt, without having to know details about CliMAF functions location, and just mimicking one of the Examples, please first make sur you have write permission in the current directory (used for some log files); then, you can either :
use binary
climaf
for launching Python while importing CliMAF :set your PATH e.g. in your
~/.profile
file:$ export PATH=$PATH:<some_installation_dir>/bin
and then launch CliMAF
$ climaf
>>> #(this is the Python prompt)
you may of course also directly type
$ <some_installation_dir>/bin/climaf
or import
climaf.api.*
in your python environment :set your PYTHONPATH , e.g. in your
~/.profile
file:$ export PYTHONPATH=$PYTHONPATH:<some_installation_dir>/
type
$ python
>>> from climaf.api import * >>> ...
You may also use CDAT instead of Python. It is working at least for CDAT version using Python from 2.6.5.
Please see also : Examples
Using CliMAF as a back end in your scripts¶
Binary climaf
described above (and located in <some_installation_dir>/bin
) can
be used with a string argument which is a series of valid CliMAF
commands. It will then run silently in the background (up to the point
where an error occurs) and may be used e.g. to get the filename for a
result handled by CliMAF in its cache.
As an example, if your CliMAF startup file (see Configuring CliMAF) does import the necessary
modules for defining function season.clim
, you may write:
$ climaf "print cfile(season.clim('CNRM-CM','PRE6.2T127Cr2E','pr','JJAS','1980-1999'))"
or even:
$ file=$(climaf "print cfile(season.clim('CNRM-CM','PRE6.2T127Cr2E','pr','JJAS','1980-1999'))")
This can be handy for letting CliMAF handle your climatology files in its cache
Using CliMAF as a library¶
If you wish to have the same facilities (shortcuts) than in interactive sessions, then insert
>>> from climaf.api import *
in each module making use of CliMAF functions.
But you may prefer to make only explicit imports, and then use:
>>> import climaf
In that case:
you must use fully qualified python names for climaf functions, such as
climaf.classes.ds()
; you may have a look at module climaf.api to know in which module is each useful CliMAF functionplease note that all CliMAF operators declared using e.g.
cscript()
must be prefixed with “climaf.operators” as e.g. in>>> avg=climaf.operators.time_average(ds)
the same applies for macros, mutatis mutandis
>>> avg=climaf.macros.my_macro(ds)