No simulators have connected for training

New Contributor

 

Screenshot 2022-06-30 191515.pnginkling_file.pngHi all, We have created a simulator in Anylogic and using that now we are try to create a brain in Bonsai. But training process is going on and on some time later the process stopped automatically.

Please find the logs and screenshot attached with this post for the reference.  

Logs:-----

6/30 3:28 PM: The training session is starting. Training engine version 3.6 is being used. (+1 more, latest occurrence: 6/30 3:29 PM)
6/30 3:29 PM: Still working to start training.
6/30 3:28 PM: Still working to start training.
6/30 3:27 PM: Still working to start training.
6/30 3:27 PM: The training session is starting. Training engine version 3.6 is being used.
6/30 3:26 PM: Still working to start training.
6/30 3:26 PM: The training session is starting. Training engine version 3.6 is being used.
6/30 3:26 PM: Still working to start training.
6/30 3:26 PM: The training session is starting. Training engine version 3.6 is being used.
6/30 3:26 PM: No simulators have connected for training for 3600 seconds. This training session will stop but can be resumed at any time.
6/30 2:24 PM: Algorithm selection started. Gathering samples for algorithm selection.
6/30 2:24 PM: The training session is starting. Training engine version 3.6 is being used.

5 Replies

Hello Anish,

 

I see for several of your brain versions your simulator was providing a field called "moisture_content_main" when your inkling specified "Moisture_content_main". Those errors should have shown up in the error log in the web UI. Did you fix that issue?

 

I don't see those errors for the final run. The simulators you registered appear to have been unavailable when the system attempted to use them, and after waiting for more simulators it shut itself down. Did your simulators crash, or de-register for some reason? Are you running managed or unmanaged simulators?

 

If your simulators are happily registered and idling, I'll investigate further to see why the platform was unable to use them.

- Moisture_content_main issue is already resolved from our side.
- Yes, we also believe that the registered simulator was unavailable to use by the system and eventually timed out. But not sure about this behavior.
- Our simulator runs in Anylogic without crashing and we did not see it getting de-registered as well.
- Can you please explain us what are managed or unmanaged simulators?
Managed simulators are run in containers launched by the Bonsai platform, using an ACR image you provide or one we offer as a demo. Unmanaged simulators are run by you wherever you please, and you manage launching, registering, and tearing them down when you're done.

I will attempt to look further into why your simulators weren't usable by the platform. They were clearly able to register, but all attempts to actually communicate with them failed. I may have to ask some of my colleagues for help on this one, but I'll let you know what we find.
We've investigated further and found your simulators are timing out while the platform waits for them to send a new state. This may indicate a problem with your simulator if it's taking longer than you anticipated to compute states, or a misconfiguration of your simulator timeout. You may need to increase the simulator timeout if your simulation is very slow to step.
Hi Anish,
I work on the Bonsai team and cover AnyLogic simulations. I am sending you a direct message so we can take this offline and then post the final resolution here. Thanks a ton!