skrl-v1.4.0

[1.4.0] - Unreleased

Added

• Utilities to operate on Gymnasium spaces (Box, Discrete, MultiDiscrete, Tuple and Dict)
• parse_device static method in ML framework configuration (used in library components to set up the device)
• Model instantiator support for different shared model structures in PyTorch
• Support for automatic mixed precision training in PyTorch
• init_state_dict method to initialize model's lazy modules in PyTorch
• Model instantiators fixed_log_std parameter to define immutable log standard deviations
• Define the stochastic_evaluation trainer config to allow the use of the actions returned by the agent's model
  as-is instead of deterministic actions (mean-actions in Gaussian-based models) during evaluation.
  Make the return of deterministic actions the default behavior.

Changed

• Call agent's pre_interaction method during evaluation
• Use spaces utilities to process states, observations and actions for all the library components
• Update model instantiators definitions to process supported fundamental and composite Gymnasium spaces
• Make flattened tensor storage in memory the default option (revert changed introduced in version 1.3.0)
• Drop support for PyTorch versions prior to 1.10 (the previous supported version was 1.9)
• Update KL Adaptive learning rate scheduler implementation to match Optax's behavior in JAX
• Update AMP agent to use the environment's terminated and truncated data, and the KL Adaptive learning rate scheduler
• Update runner implementations to support definition of arbitrary agents and their models
• Speed up PyTorch implementation:
  • Disable argument checking when instantiating distributions
  • Replace PyTorch's BatchSampler by Python slice when sampling data from memory

Changed (breaking changes: style)

• Format code using Black code formatter (it's ugly, yes, but it does its job)

Fixed

• Move the batch sampling inside gradient step loop for DQN, DDQN, DDPG (RNN), TD3 (RNN), SAC and SAC (RNN)
• Model state dictionary initialization for composite Gymnasium spaces in JAX
• Add missing reduction parameter to Gaussian model instantiator
• Optax's learning rate schedulers integration in JAX implementation
• Isaac Lab wrapper's multi-agent state retrieval with gymnasium 1.0
• Treat truncation signal when computing 'done' (environment reset)

Removed

• Remove OpenAI Gym (gym) from dependencies and source code. skrl continues to support gym environments,
  it is just not installed as part of the library. If it is needed, it needs to be installed manually.
  Any gym-based environment wrapper must use the convert_gym_space space utility to operate