How to use deep neural network correctly

[procodingjob]

first of all thank you for this good project and your great effort

how i can use deep neural network correctlt ?
also how we can set layers for projection and prediction ?

i try this : [512, 256, 128, 64, 32] but is like getting same number of neuron on each layer !!

policy=dict(
    
    model=dict(
        observation_shape=observation_shape,
        action_space_size=action_space_size,
        model_type='mlp',
        lstm_hidden_size=1024,
        latent_state_dim=1024,

        fc_reward_layers=[512, 256, 128, 64, 32],  
        fc_value_layers=[512, 256, 128, 64, 32],   
        fc_policy_layers=[512, 256, 128, 64, 32],  

        self_supervised_learning_loss=True,  # NOTE: default is False.
        discrete_action_encoding_type='one_hot',
        norm_type='BN',
    ),
    


  **see the output please** 
  
  
                  MuZeroModelMLP(                                                                                                                                                                                                            
                      (representation_network): RepresentationNetworkMLP(                                                                                                                                                                      
                        (fc_representation): Sequential(                                                                                                                                                                                       
                          (0): Linear(in_features=120, out_features=1024, bias=True)                                                                                                                                                           
                          (1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (2): GELU(approximate='tanh')                                                                                                                                                                                        
                          (3): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                        )                                                                                                                                                                                                                      
                        (sim_norm): SimNorm(dim=8)                                                                                                                                                                                             
                      )                                                                                                                                                                                                                        
                      (dynamics_network): DynamicsNetwork(                                                                                                                                                                                     
                        (fc_dynamics_1): Sequential(                                                                                                                                                                                           
                          (0): Linear(in_features=1027, out_features=1024, bias=True)                                                                                                                                                          
                          (1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                          (4): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (5): ReLU(inplace=True)                                                                                                                                                                                              
                        )                                                                                                                                                                                                                      
                        (fc_dynamics_2): Sequential(                                                                                                                                                                                           
                          (0): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                          (1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                          (4): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (5): ReLU(inplace=True)                                                                                                                                                                                              
                        )                                                                                                                                                                                                                      
                        (fc_reward_head): Sequential(                                                                                                                                                                                          
                          (0): Linear(in_features=1024, out_features=512, bias=True)                                                                                                                                                           
                          (1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=512, out_features=601, bias=True)                                                                                                                                                            
                        )                                                                                                                                                                                                                      
                      )                                                                                                                                                                                                                        
                      (prediction_network): PredictionNetworkMLP(                                                                                                                                                                              
                        (fc_prediction_common): Sequential(                                                                                                                                                                                    
                          (0): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                          (1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                          (4): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (5): ReLU(inplace=True)                                                                                                                                                                                              
                        )                                                                                                                                                                                                                      
                        (fc_value_head): Sequential(                                                                                                                                                                                           
                          (0): Linear(in_features=1024, out_features=512, bias=True)                                                                                                                                                           
                          (1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (4): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (5): ReLU(inplace=True)                                                                                                                                                                                              
                          (6): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (7): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (8): ReLU(inplace=True)                                                                                                                                                                                              
                          (9): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (10): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (11): ReLU(inplace=True)                                                                                                                                                                                             
                          (12): Linear(in_features=512, out_features=601, bias=True)                                                                                                                                                           
                        )                                                                                                                                                                                                                      
                        (fc_policy_head): Sequential(                                                                                                                                                                                          
                          (0): Linear(in_features=1024, out_features=512, bias=True)                                                                                                                                                           
                          (1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (4): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (5): ReLU(inplace=True)                                                                                                                                                                                              
                          (6): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (7): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (8): ReLU(inplace=True)                                                                                                                                                                                              
                          (9): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (10): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (11): ReLU(inplace=True)                                                                                                                                                                                             
                          (12): Linear(in_features=512, out_features=3, bias=True)                                                                                                                                                             
                        )                                                                                                                                                                                                                      
                      )                                                                                                                                                                                                                        
                      (projection): Sequential(                                                                                                                                                                                                
                        (0): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                            
                        (1): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)                                                                                                                                 
                        (2): ReLU(inplace=True)                                                                                                                                                                                                
                        (3): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                            
                        (4): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)                                                                                                                                 
                        (5): ReLU(inplace=True)                                                                                                                                                                                                
                        (6): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                            
                        (7): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)                                                                                                                                 
                      )                                                                                                                                                                                                                        
                      (prediction_head): Sequential(                                                                                                                                                                                           
                        (0): Linear(in_features=1024, out_features=512, bias=True)                                                                                                                                                             
                        (1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)                                                                                                                                  
                        (2): ReLU(inplace=True)                                                                                                                                                                                                
                        (3): Linear(in_features=512, out_features=1024, bias=True)                                                                                                                                                             
                      )                                                                                                                                                                                                                        
                    )  

thank you

[puyuan1996]

Hello,

Apologies for the late reply.

The previous implementation did not fully account for the functionality of specifying the hidden layer dimensions through fc_reward/value/policy_layers. This has been adjusted in the PR #314. You can test zoo/classic_control/cartpole/config/cartpole_unizero_config.py to see if it meets your expectations.

If you encounter any issues during the implementation, feel free to contact us at any time. Wishing you all the best!