I have a question about the implementation of the step() function. I am not 100% sure whether I am correct, but when I run experiments on the real platform and log how long the execution of the step() function takes, I get a strange behavior. E.g. if a frameskip of 10 is specified, the step() function takes 10ms. This is problematic as this basically means that our own code will never run at the specified frequency (at least if interpreting that setting frameskip defines the frequency).
I think the reason for this behavior is that in the step() function one is not only appending the action into the queue, but right after this action, directly the observation is retrieved, which is a blocking action and waits until the appended action is executed.
I am wondering whether you also experience this behavior and whether this is intended.
I think to circumvent this issue, one idea would be to only once retrieve the observation after having appended all of the actions. But then retrieve the observation with respect to the current time index (e.g. t = self.platform.get_current_timeindex()). This way, the step function would run as quickly as possible,…
Thanks and best,