chemical_checker issueshttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues2023-03-22T16:14:55Zhttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/155Slow image creation in CI2023-03-22T16:14:55ZMartino BertoniSlow image creation in CIBuilding the image on Coelho takes long time and sometimes this exceed the 2h timeout, giving errors in CI.
We should optimize image building.Building the image on Coelho takes long time and sometimes this exceed the 2h timeout, giving errors in CI.
We should optimize image building.https://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/154Verify Chemical Checker API2022-05-19T15:39:05ZMartina LocatelliVerify Chemical Checker APIStress test CC API:
https://chemicalchecker.com/api/db/getSignature/<CC SPACE e.g. A1>/<INCHIKEYS SEPARATED BY COMMA>
Possible memory leak which causes the web server to get down.Stress test CC API:
https://chemicalchecker.com/api/db/getSignature/<CC SPACE e.g. A1>/<INCHIKEYS SEPARATED BY COMMA>
Possible memory leak which causes the web server to get down.Martina LocatelliMartina Locatellihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/153sign4 train-test split2022-02-24T10:37:43ZMartino Bertonisign4 train-test splitAt the moment we are not saving which molecule goes to train and test, making a fair comparison to other approach difficult.
We need a splitter class which is also able to perform both random and scaffold splits.At the moment we are not saving which molecule goes to train and test, making a fair comparison to other approach difficult.
We need a splitter class which is also able to perform both random and scaffold splits.Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/152cc.name should not be read from path2022-02-24T14:55:57ZMartino Bertonicc.name should not be read from pathCurrently cc.name is basically the version of the cc taken from the path.
When we initialize with the default '/aloy/web_checker/current/' the version name used is current, which is not desirable.
cc.name is not used a lot, but is releva...Currently cc.name is basically the version of the cc taken from the path.
When we initialize with the default '/aloy/web_checker/current/' the version name used is current, which is not desirable.
cc.name is not used a lot, but is relevant for the diagnostic paths.Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/151Checking how to improve the Config file retrieval in the signature fitttings2022-03-01T17:24:00ZMartina LocatelliChecking how to improve the Config file retrieval in the signature fitttingsAnalysis of Config usage in the signatures generation pipelineAnalysis of Config usage in the signatures generation pipelineMartina LocatelliMartina Locatellihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/150Modifying the cc_update pipeline to get the latest Chembl version2022-03-01T17:24:16ZMartina LocatelliModifying the cc_update pipeline to get the latest Chembl versionNow default to 'chembl' DB in aloy-dbsrv.Now default to 'chembl' DB in aloy-dbsrv.Martina LocatelliMartina Locatellihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/149Update statistics page of CC web2022-03-01T17:23:36ZMartina LocatelliUpdate statistics page of CC webInclude Statistics page update in the cc_web.py pipeline to visualize updated version data.Include Statistics page update in the cc_web.py pipeline to visualize updated version data.https://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/148Change export of CC2022-03-11T10:11:36ZMartina LocatelliChange export of CCcc.export() method must create a zipped folder including the minimum files needed to run fully a cc_protocol with a user's dataset:
1. full: all sign0 (.h5 files + fit.ready file in models folder)
2. reference: sign1 models...cc.export() method must create a zipped folder including the minimum files needed to run fully a cc_protocol with a user's dataset:
1. full: all sign0 (.h5 files + fit.ready file in models folder)
2. reference: sign1 models folder (.pkl files only)
--> sign1 are going to be generated based on sign0
at the initialization of the ChemicalChecker instance
3. reference: sign2 models (savedmodel folder only)
--> sign2 are going to be generated based on sign1 and neig1
(also generated once sign1 is ready)Martina LocatelliMartina Locatellihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/147notebooks2022-02-15T17:45:35ZMartino BertoninotebooksThe notebook folder contains old, not supported examples.
We should replace them with some explanatory examples on the line of the protocol paper.
* [ ] Add a new non-competing dataset (i.e. the data is not directly related to any exist...The notebook folder contains old, not supported examples.
We should replace them with some explanatory examples on the line of the protocol paper.
* [ ] Add a new non-competing dataset (i.e. the data is not directly related to any existing CC space, e.g. M1.001)
* [ ] Add a new competing dataset (i.e. the raw data input is the same as other dataset difference is in the processing, e.g. D1.002)
* [ ] Property prediction: supervised learning inspired by MoleculeNet TDIcommons or other
* [ ] 2D projections of small compound collections, Neighbor identification, enrichment analysis?
* [ ] Signature reversion of Alzheimer's-related traits?
* [ ] Signature mimicking of bio-drugs?Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/146diagnostic plots at signature validation2022-02-15T16:05:52ZMartino Bertonidiagnostic plots at signature validationWe want the data for diagnostic plots to be computed when signature validation is performed...
``chemicalchecker/core/signature_base.py`` is the place where to add code, in function ``validate`` *that is run after each signature fit, so ...We want the data for diagnostic plots to be computed when signature validation is performed...
``chemicalchecker/core/signature_base.py`` is the place where to add code, in function ``validate`` *that is run after each signature fit, so it should be a general solution.nsolernsolerhttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/145Check with @mbertoni why an error occurs when performing diagnostic projectio...2022-02-15T16:06:02ZnsolerCheck with @mbertoni why an error occurs when performing diagnostic projection with Matplotlib 3.3.0 vs 3.2.1Patched for the moment by installing version 3.2.1 in the singularity config filePatched for the moment by installing version 3.2.1 in the singularity config filensolernsolerhttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/144sign0 for D1 takes very long2022-02-15T16:06:57ZMartino Bertonisign0 for D1 takes very longAfter discussing with @mduran and @nsoler we know it is expected to last 1 week approx and submit many jobs to the cluster.
The point is that LINCS data is not being updated anymore, so D1 is not changing.
We should implement an **md5s...After discussing with @mduran and @nsoler we know it is expected to last 1 week approx and submit many jobs to the cluster.
The point is that LINCS data is not being updated anymore, so D1 is not changing.
We should implement an **md5sum check** on downloaded files, so that if nothing has been updated we can keep the previous version of the spaceMartino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/143Sign0 predict save file in sign0.data_path2022-02-15T16:08:24ZOriol GuitartSign0 predict save file in sign0.data_pathTHe predict method save the output in sign0.data_path if destination is None. This is very dangerous since we could overwrite the original signature file by mistake, that it actually happens to me onceTHe predict method save the output in sign0.data_path if destination is None. This is very dangerous since we could overwrite the original signature file by mistake, that it actually happens to me onceMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgMiquel Duran-Frigolamiquel.duran@irbbarcelona.orghttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/142Sign0 predict key_type needs to be set2022-02-15T16:12:16ZOriol GuitartSign0 predict key_type needs to be setIn the new sign0 for the predict method if the parameter key_type is not set the method fails. The documentation says:
key_type(str): Type of key. May be inchikey or smiles. If None specified, no filtering is applied (default=None).
But ...In the new sign0 for the predict method if the parameter key_type is not set the method fails. The documentation says:
key_type(str): Type of key. May be inchikey or smiles. If None specified, no filtering is applied (default=None).
But it fails in the sorting.
```
2020-04-01 11:25:22,984 chemicalchecker.core.sign0.sign0 [DEBUG ] Setting input type
2020-04-01 11:25:22,984 chemicalchecker.core.sign0.sign0 [DEBUG ] Sorting
Traceback (most recent call last):
File "/aloy/scratch/oguitart/tmp/jobs_sign0_pred_74z5ryxm/sign0_pred_script.py", line 19, in <module>
sign_full.predict(**pars)
File "/aloy/home/oguitart/projects/source/chemical_checker/package/chemicalchecker/core/sign0.py", line 354, in predict
agg_method=self.agg_method)
File "/aloy/home/oguitart/projects/source/chemical_checker/package/chemicalchecker/core/sign0.py", line 174, in get_data
X, keys, keys_raw, features = self.sort(X, keys, keys_raw, features)
File "/aloy/home/oguitart/projects/source/chemical_checker/package/chemicalchecker/core/sign0.py", line 243, in sort
keys_raw = keys_raw[key_idxs]
TypeError: 'NoneType' object is not subscriptable
```Miquel Duran-Frigolamiquel.duran@irbbarcelona.orgMiquel Duran-Frigolamiquel.duran@irbbarcelona.orghttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/141Error with util performance2020-03-17T06:25:41ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgError with util performanceHi Oriol. When, inside `/aloy/home/mduran/myscripts/chemical_checker/package/scripts/preprocess/D2.001`, I run:
python run.py -o /aloy/home/mduran/myscripts/mosaic/manuscript/revisions/nbt/data/D2.001_sign0.h5
I get the following error...Hi Oriol. When, inside `/aloy/home/mduran/myscripts/chemical_checker/package/scripts/preprocess/D2.001`, I run:
python run.py -o /aloy/home/mduran/myscripts/mosaic/manuscript/revisions/nbt/data/D2.001_sign0.h5
I get the following error.
```
Traceback (most recent call last):
File "run.py", line 14, in <module>
from chemicalchecker.util.performance import gaussian_scale_impute
File "/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/util/performance/__init__.py", line 1, in <module>
from .linkprediction import LinkPrediction
File "/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/util/performance/linkprediction.py", line 6, in <module>
from .performance import PerformanceBinary
File "/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/util/performance/performance.py", line 6, in <module>
from sklearn.metrics import davies_bouldin_score, calinski_harabaz_score
ImportError: cannot import name 'davies_bouldin_score' from 'sklearn.metrics' (/opt/miniconda3/envs/py37/lib/python3.7/site-packages/sklearn/metrics/__init__.py)
```Oriol GuitartOriol Guitart2020-01-08https://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/140Define a CC matplotlib style sheet?2020-02-12T08:20:33ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgDefine a CC matplotlib style sheet?Hi @mbertoni I think it would be nice to have a CC stylesheet for matplotlib. What do you think?Hi @mbertoni I think it would be nice to have a CC stylesheet for matplotlib. What do you think?Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/139func_hpc remove folders when finished2022-02-15T16:16:11ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgfunc_hpc remove folders when finishedCan we add an argument to remove folders when func_hpc finishes?Can we add an argument to remove folders when func_hpc finishes?Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/138HPC jobs using 1 cores crashes2022-02-15T16:17:28ZMartino BertoniHPC jobs using 1 cores crashesWhen submitting jobs using only 1 core to the cluster we get this error
```bash
2019-10-23 13:45:16.472186: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2099750000 Hz
terminate called after throwing an instan...When submitting jobs using only 1 core to the cluster we get this error
```bash
2019-10-23 13:45:16.472186: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2099750000 Hz
terminate called after throwing an instance of 'std::system_error'
what(): Resource temporarily unavailable
/opt/sge/default/spool/pac-one107/job_scripts/69886: line 20: 24311 Aborted (core dumped) OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMEXPR_NUM_THREADS=1 SINGULARITYENV_PYTHONPATH=/aloy/home/mbertoni/code/chemical_checker/package SINGULARITYENV_CC_CONFIG=/aloy/home/mbertoni/cc_config.json singularity exec /aloy/home/mbertoni/images/cc_py27.simg python sign3_predict_from_smiles_hpc.py sign3_predict_from_smiles_hpc.pkl
```
Apparently tensorflow requires an amount of memory that is proportional to the number of cores.
The job would work requesting more memory (e.g. 18GB), so increasing the memory is a viable workaround.Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/137Turn off logging?2019-10-23T12:04:33ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgTurn off logging?Hi @mbertoni
Is there a way to turn logging of when working, for example, with a jupyter notebook?
Thanks!Hi @mbertoni
Is there a way to turn logging of when working, for example, with a jupyter notebook?
Thanks!Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/136Generic neig class has sign1 argument2022-02-15T16:20:20ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgGeneric neig class has sign1 argumentHi @oguitart
I have realised that the generic neig class has a sign1 argument. Shouldn't it be just sign?
I have not modified the script because I don't want to screw up.Hi @oguitart
I have realised that the generic neig class has a sign1 argument. Shouldn't it be just sign?
I have not modified the script because I don't want to screw up.Oriol GuitartOriol Guitart