run_vitamin fails

[MarlinSchaefer]

python run_vitamin.py --gen_train True --train True fails to execute. I used the current master-branch to run this command and used the test_sets provided by you. The only thing I changed in params is tot_dataset_size to 5000, so that it generates a bit quicker. I get the output:

17:10 bilby INFO    : Running bilby version: 0.5.5:
module 'basemap' is not installed
Skyplotting functionality is automatically disabled.

... Making training set

Generated: ./training_sets_3det_9par_256Hz/tset_tot-100000_split-1000/data_1000-5000.h5py ...
Generated: ./training_sets_3det_9par_256Hz/tset_tot-100000_split-1000/data_2000-5000.h5py ...
Generated: ./training_sets_3det_9par_256Hz/tset_tot-100000_split-1000/data_3000-5000.h5py ...
Generated: ./training_sets_3det_9par_256Hz/tset_tot-100000_split-1000/data_4000-5000.h5py ...
Generated: ./training_sets_3det_9par_256Hz/tset_tot-100000_split-1000/data_5000-5000.h5py ...

... mass_1 will be inferred
... mass_2 will be inferred
... luminosity_distance will be inferred
... geocent_time will be inferred
... theta_jn will be inferred
... ra will be inferred
... dec will be inferred


... Loading test sample -> ./test_sets/all_4_samplers/test_dynesty1/all_4_samplers_0.h5py

... Loading test sample -> ./test_sets/all_4_samplers/test_dynesty1/all_4_samplers_1.h5py
Traceback (most recent call last):
  File "run_vitamin.py", line 1227, in <module>
    train(params,bounds,fixed_vals)
  File "run_vitamin.py", line 706, in train
    XS_all = np.vstack((XS_all,np.expand_dims(XS[:params['n_samples'],:], axis=0)))
  File "<__array_function__ internals>", line 6, in vstack
  File "/home/marlin/environments/vitaminb/lib/python3.6/site-packages/numpy/core/shape_base.py", line 283, in vstack
    return _nx.concatenate(arrs, 0)
  File "<__array_function__ internals>", line 6, in concatenate
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 262 and the array at index 1 has size 1713

Not sure what this bug is caused by.

[MarlinSchaefer]

Deleting the files all_4_samplers_<x>.h5py with > 0 results in the code trying to open them anyways. It then seems to enter an infinite loop where it prints out
Unable to open file (unable to open file: name = './test_sets/all_4_samplers/test_dynesty1/all_4_samplers_<x>.h5py', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)
, iterates and repeats trying to open a file.

[hagabbar]

Investigating now. Might be due to some recent changes I've made to the code. I'll try and reproduce it.

[hagabbar]

@MarlinSchaefer So just tested the initial bug you were having and I can't seem to reproduce it. I made sure to also reduce the tot_dataset_size variable to 5000 and it still runs all the way through. Are you sure there's nothing else you changed from the params file?

Btw, I'm using the most recent up-to-date version of master.

[hagabbar]

Deleting the files all_4_samplers_<x>.h5py with > 0 results in the code trying to open them anyways. It then seems to enter an infinite loop where it prints out
Unable to open file (unable to open file: name = './test_sets/all_4_samplers/test_dynesty1/all_4_samplers_<x>.h5py', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)
, iterates and repeats trying to open a file.

Ah yes, so this is an issue that needs fixing. I think you may be requesting more test samples than are available in the r variable of the params.json file. Try setting r = 1 and see if that works. In any case, the code should throw up an exception if the user asks to use more testing samples than are actually available, so I'll code that up.

[hagabbar]

Recent pull request hopefully addresses all the issues you brought up.

#65

Try using the latest version of the code from master and let me know if that fixes things for you.

[MarlinSchaefer]

Tried the latest version. It still crashes for me, when I use r=2. However, with r=1 it starts to train.

[hagabbar]

@MarlinSchaefer Is it still the same error as before when using r=2? Or is it a new error?

[MarlinSchaefer]

@hagabbar Yes, it is still the same error with r=2.

[hagabbar]

@MarlinSchaefer Okay, it turns out that there was a small bug in the check that looks to see whether the user is asking to use more test samples than is readily available to them. I've now alleviated this and it should now through an exception message and exit the program if the user does this.

[hagabbar]

That is, assuming that the bug is in reference to your second comment where the code enters an infinite while loop.