chemical_checker issueshttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues2023-03-22T16:14:55Zhttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/155Slow image creation in CI2023-03-22T16:14:55ZMartino BertoniSlow image creation in CIBuilding the image on Coelho takes long time and sometimes this exceed the 2h timeout, giving errors in CI.
We should optimize image building.Building the image on Coelho takes long time and sometimes this exceed the 2h timeout, giving errors in CI.
We should optimize image building.https://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/152cc.name should not be read from path2022-02-24T14:55:57ZMartino Bertonicc.name should not be read from pathCurrently cc.name is basically the version of the cc taken from the path.
When we initialize with the default '/aloy/web_checker/current/' the version name used is current, which is not desirable.
cc.name is not used a lot, but is releva...Currently cc.name is basically the version of the cc taken from the path.
When we initialize with the default '/aloy/web_checker/current/' the version name used is current, which is not desirable.
cc.name is not used a lot, but is relevant for the diagnostic paths.Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/151Checking how to improve the Config file retrieval in the signature fitttings2022-03-01T17:24:00ZMartina LocatelliChecking how to improve the Config file retrieval in the signature fitttingsAnalysis of Config usage in the signatures generation pipelineAnalysis of Config usage in the signatures generation pipelineMartina LocatelliMartina Locatellihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/150Modifying the cc_update pipeline to get the latest Chembl version2022-03-01T17:24:16ZMartina LocatelliModifying the cc_update pipeline to get the latest Chembl versionNow default to 'chembl' DB in aloy-dbsrv.Now default to 'chembl' DB in aloy-dbsrv.Martina LocatelliMartina Locatellihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/149Update statistics page of CC web2022-03-01T17:23:36ZMartina LocatelliUpdate statistics page of CC webInclude Statistics page update in the cc_web.py pipeline to visualize updated version data.Include Statistics page update in the cc_web.py pipeline to visualize updated version data.https://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/148Change export of CC2022-03-11T10:11:36ZMartina LocatelliChange export of CCcc.export() method must create a zipped folder including the minimum files needed to run fully a cc_protocol with a user's dataset:
1. full: all sign0 (.h5 files + fit.ready file in models folder)
2. reference: sign1 models...cc.export() method must create a zipped folder including the minimum files needed to run fully a cc_protocol with a user's dataset:
1. full: all sign0 (.h5 files + fit.ready file in models folder)
2. reference: sign1 models folder (.pkl files only)
--> sign1 are going to be generated based on sign0
at the initialization of the ChemicalChecker instance
3. reference: sign2 models (savedmodel folder only)
--> sign2 are going to be generated based on sign1 and neig1
(also generated once sign1 is ready)Martina LocatelliMartina Locatellihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/146diagnostic plots at signature validation2022-02-15T16:05:52ZMartino Bertonidiagnostic plots at signature validationWe want the data for diagnostic plots to be computed when signature validation is performed...
``chemicalchecker/core/signature_base.py`` is the place where to add code, in function ``validate`` *that is run after each signature fit, so ...We want the data for diagnostic plots to be computed when signature validation is performed...
``chemicalchecker/core/signature_base.py`` is the place where to add code, in function ``validate`` *that is run after each signature fit, so it should be a general solution.nsolernsolerhttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/145Check with @mbertoni why an error occurs when performing diagnostic projectio...2022-02-15T16:06:02ZnsolerCheck with @mbertoni why an error occurs when performing diagnostic projection with Matplotlib 3.3.0 vs 3.2.1Patched for the moment by installing version 3.2.1 in the singularity config filePatched for the moment by installing version 3.2.1 in the singularity config filensolernsolerhttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/144sign0 for D1 takes very long2022-02-15T16:06:57ZMartino Bertonisign0 for D1 takes very longAfter discussing with @mduran and @nsoler we know it is expected to last 1 week approx and submit many jobs to the cluster.
The point is that LINCS data is not being updated anymore, so D1 is not changing.
We should implement an **md5s...After discussing with @mduran and @nsoler we know it is expected to last 1 week approx and submit many jobs to the cluster.
The point is that LINCS data is not being updated anymore, so D1 is not changing.
We should implement an **md5sum check** on downloaded files, so that if nothing has been updated we can keep the previous version of the spaceMartino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/143Sign0 predict save file in sign0.data_path2022-02-15T16:08:24ZOriol GuitartSign0 predict save file in sign0.data_pathTHe predict method save the output in sign0.data_path if destination is None. This is very dangerous since we could overwrite the original signature file by mistake, that it actually happens to me onceTHe predict method save the output in sign0.data_path if destination is None. This is very dangerous since we could overwrite the original signature file by mistake, that it actually happens to me onceMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgMiquel Duran-Frigolamiquel.duran@irbbarcelona.orghttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/142Sign0 predict key_type needs to be set2022-02-15T16:12:16ZOriol GuitartSign0 predict key_type needs to be setIn the new sign0 for the predict method if the parameter key_type is not set the method fails. The documentation says:
key_type(str): Type of key. May be inchikey or smiles. If None specified, no filtering is applied (default=None).
But ...In the new sign0 for the predict method if the parameter key_type is not set the method fails. The documentation says:
key_type(str): Type of key. May be inchikey or smiles. If None specified, no filtering is applied (default=None).
But it fails in the sorting.
```
2020-04-01 11:25:22,984 chemicalchecker.core.sign0.sign0 [DEBUG ] Setting input type
2020-04-01 11:25:22,984 chemicalchecker.core.sign0.sign0 [DEBUG ] Sorting
Traceback (most recent call last):
File "/aloy/scratch/oguitart/tmp/jobs_sign0_pred_74z5ryxm/sign0_pred_script.py", line 19, in <module>
sign_full.predict(**pars)
File "/aloy/home/oguitart/projects/source/chemical_checker/package/chemicalchecker/core/sign0.py", line 354, in predict
agg_method=self.agg_method)
File "/aloy/home/oguitart/projects/source/chemical_checker/package/chemicalchecker/core/sign0.py", line 174, in get_data
X, keys, keys_raw, features = self.sort(X, keys, keys_raw, features)
File "/aloy/home/oguitart/projects/source/chemical_checker/package/chemicalchecker/core/sign0.py", line 243, in sort
keys_raw = keys_raw[key_idxs]
TypeError: 'NoneType' object is not subscriptable
```Miquel Duran-Frigolamiquel.duran@irbbarcelona.orgMiquel Duran-Frigolamiquel.duran@irbbarcelona.orghttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/141Error with util performance2020-03-17T06:25:41ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgError with util performanceHi Oriol. When, inside `/aloy/home/mduran/myscripts/chemical_checker/package/scripts/preprocess/D2.001`, I run:
python run.py -o /aloy/home/mduran/myscripts/mosaic/manuscript/revisions/nbt/data/D2.001_sign0.h5
I get the following error...Hi Oriol. When, inside `/aloy/home/mduran/myscripts/chemical_checker/package/scripts/preprocess/D2.001`, I run:
python run.py -o /aloy/home/mduran/myscripts/mosaic/manuscript/revisions/nbt/data/D2.001_sign0.h5
I get the following error.
```
Traceback (most recent call last):
File "run.py", line 14, in <module>
from chemicalchecker.util.performance import gaussian_scale_impute
File "/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/util/performance/__init__.py", line 1, in <module>
from .linkprediction import LinkPrediction
File "/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/util/performance/linkprediction.py", line 6, in <module>
from .performance import PerformanceBinary
File "/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/util/performance/performance.py", line 6, in <module>
from sklearn.metrics import davies_bouldin_score, calinski_harabaz_score
ImportError: cannot import name 'davies_bouldin_score' from 'sklearn.metrics' (/opt/miniconda3/envs/py37/lib/python3.7/site-packages/sklearn/metrics/__init__.py)
```Oriol GuitartOriol Guitart2020-01-08https://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/140Define a CC matplotlib style sheet?2020-02-12T08:20:33ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgDefine a CC matplotlib style sheet?Hi @mbertoni I think it would be nice to have a CC stylesheet for matplotlib. What do you think?Hi @mbertoni I think it would be nice to have a CC stylesheet for matplotlib. What do you think?Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/139func_hpc remove folders when finished2022-02-15T16:16:11ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgfunc_hpc remove folders when finishedCan we add an argument to remove folders when func_hpc finishes?Can we add an argument to remove folders when func_hpc finishes?Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/138HPC jobs using 1 cores crashes2022-02-15T16:17:28ZMartino BertoniHPC jobs using 1 cores crashesWhen submitting jobs using only 1 core to the cluster we get this error
```bash
2019-10-23 13:45:16.472186: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2099750000 Hz
terminate called after throwing an instan...When submitting jobs using only 1 core to the cluster we get this error
```bash
2019-10-23 13:45:16.472186: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2099750000 Hz
terminate called after throwing an instance of 'std::system_error'
what(): Resource temporarily unavailable
/opt/sge/default/spool/pac-one107/job_scripts/69886: line 20: 24311 Aborted (core dumped) OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMEXPR_NUM_THREADS=1 SINGULARITYENV_PYTHONPATH=/aloy/home/mbertoni/code/chemical_checker/package SINGULARITYENV_CC_CONFIG=/aloy/home/mbertoni/cc_config.json singularity exec /aloy/home/mbertoni/images/cc_py27.simg python sign3_predict_from_smiles_hpc.py sign3_predict_from_smiles_hpc.pkl
```
Apparently tensorflow requires an amount of memory that is proportional to the number of cores.
The job would work requesting more memory (e.g. 18GB), so increasing the memory is a viable workaround.Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/137Turn off logging?2019-10-23T12:04:33ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgTurn off logging?Hi @mbertoni
Is there a way to turn logging of when working, for example, with a jupyter notebook?
Thanks!Hi @mbertoni
Is there a way to turn logging of when working, for example, with a jupyter notebook?
Thanks!Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/136Generic neig class has sign1 argument2022-02-15T16:20:20ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgGeneric neig class has sign1 argumentHi @oguitart
I have realised that the generic neig class has a sign1 argument. Shouldn't it be just sign?
I have not modified the script because I don't want to screw up.Hi @oguitart
I have realised that the generic neig class has a sign1 argument. Shouldn't it be just sign?
I have not modified the script because I don't want to screw up.Oriol GuitartOriol Guitarthttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/135Wiki page link broken2020-02-12T08:22:15ZpbadiaWiki page link brokenThe wiki page link on the main readme.md seems to be broken, it should be this one:
http://gitlabsbnb.irbbarcelona.org/project-specific-repositories/chemical_checker/wikis/homeThe wiki page link on the main readme.md seems to be broken, it should be this one:
http://gitlabsbnb.irbbarcelona.org/project-specific-repositories/chemical_checker/wikis/homehttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/134Chemistrix notebook needs update?2022-02-15T16:22:13ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgChemistrix notebook needs update?Hi @oguitart
In the chemistrix notebook, I find Molprop, which I don't think is part of the CC anymore... (correct me if I'm wrong)
Should we update it?Hi @oguitart
In the chemistrix notebook, I find Molprop, which I don't think is part of the CC anymore... (correct me if I'm wrong)
Should we update it?Oriol GuitartOriol Guitarthttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/133scale sign22020-02-12T08:21:47ZMartino Bertoniscale sign2We use Node2Vec to generate Sign2 whose components range roughly between -1.5 to 1.5.
To favor successive ML steps it might be convenient to transform the data so the it ranges between 0,1.
This can be applied as post-processing step dur...We use Node2Vec to generate Sign2 whose components range roughly between -1.5 to 1.5.
To favor successive ML steps it might be convenient to transform the data so the it ranges between 0,1.
This can be applied as post-processing step during Sign2 generation.Martino BertoniMartino Bertoni