You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for the code. I tried to execute it got the following error. can you please tell me why this problem occurred.
WARNING:tensorflow:From :10: Supervisor.init (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Starting standard services.
INFO:tensorflow:Starting queue runners.
parameter_count = 223104
progress epoch 1 step 50 image/sec 0.6 remaining 5695m
loss [[[[nan]
[nan]
[nan]
.....
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.InvalidArgumentError'>, Nan in summary histogram for: decode_3/batchnorm/offset/values
[[node decode_3/batchnorm/offset/values (defined at :12) ]]
The text was updated successfully, but these errors were encountered:
Hi @YaswinTeja Yaswin/ @RaymondByc, I used this code to train the model from scratch using the data given since in the checkpoint file @RaymondByc you have only mentioned the model12345 something which is not present in the repo, so I passed the checkpoint as None and trained using MODE train. As shown above, by Yaswin I also got the loss as nan. Can you please help. @YaswinTeja if you figured it out please share your edits.
Thanks for the code. I tried to execute it got the following error. can you please tell me why this problem occurred.
WARNING:tensorflow:From :10: Supervisor.init (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Starting standard services.
INFO:tensorflow:Starting queue runners.
parameter_count = 223104
progress epoch 1 step 50 image/sec 0.6 remaining 5695m
loss [[[[nan]
[nan]
[nan]
.....
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.InvalidArgumentError'>, Nan in summary histogram for: decode_3/batchnorm/offset/values
[[node decode_3/batchnorm/offset/values (defined at :12) ]]
The text was updated successfully, but these errors were encountered: