chemical_checker issueshttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues2019-10-23T12:04:33Zhttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/137Turn off logging?2019-10-23T12:04:33ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgTurn off logging?Hi @mbertoni
Is there a way to turn logging of when working, for example, with a jupyter notebook?
Thanks!Hi @mbertoni
Is there a way to turn logging of when working, for example, with a jupyter notebook?
Thanks!Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/132predict_from_inchi method in signature3?2022-02-15T16:30:56ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgpredict_from_inchi method in signature3?Hi @mbertoni
The same way we have a predict_from_smiles method, couldn't we have a predict_from_inchi method as well? This would be very convenient.Hi @mbertoni
The same way we have a predict_from_smiles method, couldn't we have a predict_from_inchi method as well? This would be very convenient.Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/131Pickles in python 32019-09-20T14:34:08ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgPickles in python 3Hi @mbertoni and @oguitart ,
When writing a pickle file in python 3, it is necessary to open the file in "wb" mode (instead of "w"). For example, when I run:
```python
from chemicalchecker.core import ChemicalChecker
cc = ChemicalChec...Hi @mbertoni and @oguitart ,
When writing a pickle file in python 3, it is necessary to open the file in "wb" mode (instead of "w"). For example, when I run:
```python
from chemicalchecker.core import ChemicalChecker
cc = ChemicalChecker()
s3 = cc.get_signature('sign3', 'reference', 'B4.001')
p3 = cc.get_signature('proj3', 'reference', 'B4.001', proj_type='PCA')
p3.fit(s3)
p3.plot()
```
I get the following error:
```
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-3-50c49b599e0b> in <module>
5 s3 = cc.get_signature('sign3', 'reference', 'B4.001')
6 p3 = cc.get_signature('proj3', 'reference', 'B4.001', proj_type='PCA')
----> 7 p3.fit(s3)
8 p3.plot()
/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/core/proj.py in fit(self, signature, validations, *args, **kwargs)
39 def fit(self, signature, validations=True, *args, **kwargs):
40 """Take an input learn a 2D representation."""
---> 41 self.projector.fit(signature, validations, *args, **kwargs)
42
43 def predict(self, signature, destination, *args, **kwargs):
/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/core/projector/pca.py in fit(self, signature, validations, chunk_size)
74 self.__log.info("Projecting took %s" % t_delta)
75 # save model
---> 76 pickle.dump(self.algo, open(self.algo_path, 'w'))
77 # save h5
78 sdtype = DataSignature.string_dtype()
TypeError: write() argument must be str, not bytes
```Martino BertoniMartino Bertoni2019-09-21https://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/103Pidgin v3 wrapper2019-05-06T07:26:25ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgPidgin v3 wrapperAs agreed, let's do a wrapper of pidgin that we can use easily in the prop_calculator.py script.
I would like to run it with the orthologue mapping. Details can be found: https://pidginv3.readthedocs.io/en/latest/install.html#id2
I sug...As agreed, let's do a wrapper of pidgin that we can use easily in the prop_calculator.py script.
I would like to run it with the orthologue mapping. Details can be found: https://pidginv3.readthedocs.io/en/latest/install.html#id2
I suggest that we share the work here:
* @mbertoni does the wrapper (util)
* @mduran writes explores the necessary parameters and writes the corresponding method in the PropCalculator class.
Run snippets are available here:
https://pidginv3.readthedocs.io/en/latest/usage/started.html#generating-binary-predictions
These can be used to test the wrapper.Martino BertoniMartino Bertoni2019-05-03https://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/93HPC capabilities for fit() and predict()2019-05-10T16:09:56ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgHPC capabilities for fit() and predict()@mbertoni @oguitart @mduran
An important matter, in practical terms:
It would be very nice that fit() and predict() methods of the signature classes can be run in the cluster just by giving passing the argument hpc = True/False.
For ...@mbertoni @oguitart @mduran
An important matter, in practical terms:
It would be very nice that fit() and predict() methods of the signature classes can be run in the cluster just by giving passing the argument hpc = True/False.
For instance:
s1.fit(s0, hpc = False) would run locally, whereas s1.fit(s0, hpc = True, *params) would run in the cluster.
Same with predict...Oriol GuitartOriol Guitart2019-04-30https://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/92Database organization2022-02-15T17:32:00ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgDatabase organizationAfter a long discussion with Oriol, we've decided to divide the current mosaic database into 3 databases:
* cc_package (fixed number of tables, evergrowing rows)
* cc_calcdata (evergrowing tables and evergrowing rows)
* cc_web[version]
...After a long discussion with Oriol, we've decided to divide the current mosaic database into 3 databases:
* cc_package (fixed number of tables, evergrowing rows)
* cc_calcdata (evergrowing tables and evergrowing rows)
* cc_web[version]
I attach a snapshot of the cc_package schema [cc_packagedb.pdf](/uploads/44c100e112d5f41f45aae2d5444de836/cc_packagedb.pdf)
@mbertoni @mduran @oguitartOriol GuitartOriol Guitarthttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/86Issues map_dataset_datasource table2019-05-03T14:21:00ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgIssues map_dataset_datasource tableI am inspecting for the first time the map_dataset_datasource table.
Some suggestions and questions:
* I think it is better to call it dataset_has_datasource (more sql-like)
* The primary key of this table is (dataset_code, datasource_...I am inspecting for the first time the map_dataset_datasource table.
Some suggestions and questions:
* I think it is better to call it dataset_has_datasource (more sql-like)
* The primary key of this table is (dataset_code, datasource_name), so the incremental id is not really necessary
* I would need a small tutorial on how to add information to this table
* Why is smpdb_structures a datasource of A1.001?Oriol GuitartOriol Guitarthttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/84Current preprocess scripts may have errors2019-05-10T14:08:15ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgCurrent preprocess scripts may have errorsThis part of the code in datasets like A1.001 will not work anymore...
```python
from chemicalchecker.util import PropCalculator
from chemicalchecker.util import Converter
```
It should be:
```python
from chemicalchecker.util.parser i...This part of the code in datasets like A1.001 will not work anymore...
```python
from chemicalchecker.util import PropCalculator
from chemicalchecker.util import Converter
```
It should be:
```python
from chemicalchecker.util.parser import PropCalculator
from chemicalchecker.util.parser import Converter
```Oriol GuitartOriol Guitarthttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/82Memory error while trying to predict targets (molpropery) on large molrepo2019-05-03T12:28:29ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgMemory error while trying to predict targets (molpropery) on large molrepoWhen I run:
```
import shutil
import sys, os
from chemicalchecker.database import Molprop, Structure
from chemicalchecker.database import Molrepo
from chemicalchecker.util import HPC
from chemicalchecker.util import Config
molprops = [...When I run:
```
import shutil
import sys, os
from chemicalchecker.database import Molprop, Structure
from chemicalchecker.database import Molrepo
from chemicalchecker.util import HPC
from chemicalchecker.util import Config
molprops = ['chembl_target_predictions']
molrepo_ik_inchi = Molrepo.get_fields_by_molrepo_name("chembl",["inchikey","inchi"])
for mol in molprops:
job_path = "/aloy/scratch/mduran/jobs_molprop_chembl_" + mol
if os.path.isdir(job_path):
shutil.rmtree(job_path)
os.mkdir(job_path)
molprop = Molprop(mol)
# This method sends the job and waits for the job to finish
cluster = molprop.molprop_hpc(job_path,molrepo_ik_inchi)
```
I get the following memory error:
```
---------------------------------------------------------------------------
MemoryError Traceback (most recent call last)
<ipython-input-2-6ed1f126b830> in <module>()
20
21 # This method sends the job and waits for the job to finish
---> 22 cluster = molprop.molprop_hpc(job_path,molrepo_ik_inchi)
/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/database/molprop.pyc in molprop_hpc(job_path, inchikey_inchi, **kwargs)
228 # submit jobs
229 cluster = HPC(Config())
--> 230 cluster.submitMultiJob(command, **params)
231 return cluster
232
/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/util/hpc/hpc.pyc in submitMultiJob(self, command, **kwargs)
58
59 if self.job_id is None:
---> 60 self.job_id = self.hpc.submitMultiJob(command, **kwargs)
61 else:
62 raise Exception("HPC instance already in use")
/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/util/hpc/sge.pyc in submitMultiJob(self, command, **kwargs)
165
166 input_dict = dict()
--> 167 for cid, chunk in enumerate(self._chunks(elements, num_jobs), 1):
168 input_dict[str(cid)] = chunk
169 input_path = os.path.join(self.jobdir, str(uuid.uuid4()))
/aloy/home/mduran/myscripts/chemical_checker/package/chemicalchecker/util/hpc/sge.pyc in _chunks(self, l, n)
81 """Yield successive n-sized chunks from l."""
82 if isinstance(l, list):
---> 83 for i in np.array_split(l, n):
84 yield i
85 elif isinstance(l, dict):
/opt/miniconda2/envs/py27/lib/python2.7/site-packages/numpy/lib/shape_base.pyc in array_split(ary, indices_or_sections, axis)
760
761 sub_arys = []
--> 762 sary = _nx.swapaxes(ary, axis, 0)
763 for i in range(Nsections):
764 st = div_points[i]
/opt/miniconda2/envs/py27/lib/python2.7/site-packages/numpy/core/fromnumeric.pyc in swapaxes(a, axis1, axis2)
583
584 """
--> 585 return _wrapfunc(a, 'swapaxes', axis1, axis2)
586
587
/opt/miniconda2/envs/py27/lib/python2.7/site-packages/numpy/core/fromnumeric.pyc in _wrapfunc(obj, method, *args, **kwds)
64 # a downstream library like 'pandas'.
65 except (AttributeError, TypeError):
---> 66 return _wrapit(obj, method, *args, **kwds)
67
68
/opt/miniconda2/envs/py27/lib/python2.7/site-packages/numpy/core/fromnumeric.pyc in _wrapit(obj, method, *args, **kwds)
44 except AttributeError:
45 wrap = None
---> 46 result = getattr(asarray(obj), method)(*args, **kwds)
47 if wrap:
48 if not isinstance(result, mu.ndarray):
/opt/miniconda2/envs/py27/lib/python2.7/site-packages/numpy/core/numeric.pyc in asarray(a, dtype, order)
536
537 """
--> 538 return array(a, dtype, copy=False, order=order)
539
540
MemoryError:
```Oriol GuitartOriol Guitarthttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/80Do not calculate properties unless it is necessary at predict time when pre-p...2019-07-01T11:42:01ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgDo not calculate properties unless it is necessary at predict time when pre-processingIn the preprocess scripts (e.g. A1.001), it shouldn't be necessary to calculate properties if they are already present in the corresponding raw table.In the preprocess scripts (e.g. A1.001), it shouldn't be necessary to calculate properties if they are already present in the corresponding raw table.Oriol GuitartOriol Guitarthttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/78First attempt to expand the CC, dataset B4.0022019-04-25T05:24:55ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgFirst attempt to expand the CC, dataset B4.002OK, I think I am ready to write the preprocessing of B4.002.
This dataset will be built upon a molproperty.
I attach a notebook briefly explaining how this can be calculated.
Let's discuss how we should proceed and, if you agree, let'...OK, I think I am ready to write the preprocessing of B4.002.
This dataset will be built upon a molproperty.
I attach a notebook briefly explaining how this can be calculated.
Let's discuss how we should proceed and, if you agree, let's do this one together this time. So we are sure that I do everything correctly.[target_prediction.ipynb](/uploads/cf2654d53853619078032d14c94e4e00/target_prediction.ipynb)Oriol GuitartOriol Guitart2019-04-22https://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/70Two further proofs of concept of the pre-processings2019-04-17T10:02:08ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgTwo further proofs of concept of the pre-processingsI have found two new data that we may want to consider as a first test to start expanding the CC with new datasets.
* PIDGIN v3: https://pidginv3.readthedocs.io/
* L1000FWD: http://amp.pharm.mssm.edu/L1000FWD/
Let's talk about it. It s...I have found two new data that we may want to consider as a first test to start expanding the CC with new datasets.
* PIDGIN v3: https://pidginv3.readthedocs.io/
* L1000FWD: http://amp.pharm.mssm.edu/L1000FWD/
Let's talk about it. It shall be a good test scenario for the pre-processing.Oriol GuitartOriol Guitarthttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/68setup_singularity.sh does not work2019-04-10T10:54:44ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgsetup_singularity.sh does not workFollowing the instructions of the "Complete installation" didn't work for me:
```
mduran@cervantes:~$ sh setup_chemicalchecker.sh
false
true
SUCCESS: Chemical Checker local directory available. (/home/mduran/local_checker)
Identity ad...Following the instructions of the "Complete installation" didn't work for me:
```
mduran@cervantes:~$ sh setup_chemicalchecker.sh
false
true
SUCCESS: Chemical Checker local directory available. (/home/mduran/local_checker)
Identity added: /dev/fd/63 (interactome3d@irbbarcelona.org)
SUCCESS: def file cloned.
Removing old singularity image...
rm: cannot remove '/home/mduran/local_checker/cc.simg': No such file or directory
Creating singularity sandbox image...
ERROR: /home/mduran/local_checker/sandbox is not an image file
Cleaning up...
ERROR: creating sandbox image.
```Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/67Quick start does not work on my computer2019-04-10T10:11:25ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgQuick start does not work on my computerUnder the newest version of Ubuntu, installing pip with apt-get, I run:
`sudo pip install --index http://10.7.108.15:3141/root/dev/ --trusted-host 10.7.108.15 chemicalchecker`
And get the following error:
```
Matplotlib 3.0+ does ...Under the newest version of Ubuntu, installing pip with apt-get, I run:
`sudo pip install --index http://10.7.108.15:3141/root/dev/ --trusted-host 10.7.108.15 chemicalchecker`
And get the following error:
```
Matplotlib 3.0+ does not support Python 2.x, 3.0, 3.1, 3.2, 3.3, or 3.4.
Beginning with Matplotlib 3.0, Python 3.5 and above is required.
This may be due to an out of date pip.
Make sure you have pip >= 9.0.1.
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-UKl9yO/matplotlib/
```
I can solve this, I guess, but this is an error that other people is likely to get. This is why I post it here.Martino BertoniMartino Bertonihttps://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/57Rename some folders and write better documentation2019-02-26T17:39:08ZMiquel Duran-Frigolamiquel.duran@irbbarcelona.orgRename some folders and write better documentationAs discussed,
* Reshape/rename folder structure so that it is more explicit.
* Write better documentation, especially for installation.As discussed,
* Reshape/rename folder structure so that it is more explicit.
* Write better documentation, especially for installation.Martino BertoniMartino Bertoni2019-02-26https://gitlabsbnb.irbbarcelona.org/packages/chemical_checker/-/issues/55setup script not working on laptops2019-02-26T17:38:20ZMartino Bertonisetup script not working on laptopsAs personal laptops have no `/aloy` access the script fail while trying to copy the remote image.
Using `scp` insteadd of `rsync` might be an optionAs personal laptops have no `/aloy` access the script fail while trying to copy the remote image.
Using `scp` insteadd of `rsync` might be an optionMartino BertoniMartino Bertoni