HPC jobs using 1 cores crashes
When submitting jobs using only 1 core to the cluster we get this error
2019-10-23 13:45:16.472186: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2099750000 Hz
terminate called after throwing an instance of 'std::system_error'
what(): Resource temporarily unavailable
/opt/sge/default/spool/pac-one107/job_scripts/69886: line 20: 24311 Aborted (core dumped) OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMEXPR_NUM_THREADS=1 SINGULARITYENV_PYTHONPATH=/aloy/home/mbertoni/code/chemical_checker/package SINGULARITYENV_CC_CONFIG=/aloy/home/mbertoni/cc_config.json singularity exec /aloy/home/mbertoni/images/cc_py27.simg python sign3_predict_from_smiles_hpc.py sign3_predict_from_smiles_hpc.pkl
Apparently tensorflow requires an amount of memory that is proportional to the number of cores. The job would work requesting more memory (e.g. 18GB), so increasing the memory is a viable workaround.