[Major] major refactoring of collectors

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3233

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}23$. Worsened: $\large\color{#d91a1a}13$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	82.0186μs	80.7115μs	12.3898 KOps/s	12.3612 KOps/s	$\color{#35bf28}+0.23\%$
test_tensor_to_bytestream_speed[torch.save]	0.1376ms	0.1362ms	7.3405 KOps/s	7.1821 KOps/s	$\color{#35bf28}+2.21\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1192s	0.1188s	8.4164 Ops/s	7.9782 Ops/s	$\textbf{\color{#35bf28}+5.49\%}$
test_tensor_to_bytestream_speed[numpy]	3.1274μs	3.1126μs	321.2784 KOps/s	329.6488 KOps/s	$\color{#d91a1a}-2.54\%$
test_tensor_to_bytestream_speed[safetensors]	37.6290μs	37.4226μs	26.7218 KOps/s	27.0330 KOps/s	$\color{#d91a1a}-1.15\%$
test_simple	0.5505s	0.5407s	1.8493 Ops/s	1.8451 Ops/s	$\color{#35bf28}+0.23\%$
test_transformed	1.1994s	1.1135s	0.8981 Ops/s	0.9141 Ops/s	$\color{#d91a1a}-1.75\%$
test_serial	1.7004s	1.6901s	0.5917 Ops/s	0.6041 Ops/s	$\color{#d91a1a}-2.06\%$
test_parallel	1.2183s	1.1193s	0.8934 Ops/s	0.9397 Ops/s	$\color{#d91a1a}-4.93\%$
test_step_mdp_speed[True-True-True-True-True]	0.2906ms	43.3215μs	23.0832 KOps/s	22.5366 KOps/s	$\color{#35bf28}+2.43\%$
test_step_mdp_speed[True-True-True-True-False]	76.3820μs	24.4973μs	40.8208 KOps/s	40.5905 KOps/s	$\color{#35bf28}+0.57\%$
test_step_mdp_speed[True-True-True-False-True]	59.1810μs	24.3026μs	41.1478 KOps/s	40.2635 KOps/s	$\color{#35bf28}+2.20\%$
test_step_mdp_speed[True-True-True-False-False]	43.4410μs	13.4176μs	74.5288 KOps/s	74.9673 KOps/s	$\color{#d91a1a}-0.59\%$
test_step_mdp_speed[True-True-False-True-True]	92.7310μs	46.3454μs	21.5771 KOps/s	21.1201 KOps/s	$\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-True-False-True-False]	66.6810μs	27.0892μs	36.9150 KOps/s	36.7924 KOps/s	$\color{#35bf28}+0.33\%$
test_step_mdp_speed[True-True-False-False-True]	63.6310μs	27.3509μs	36.5618 KOps/s	36.6669 KOps/s	$\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-True-False-False-False]	47.2510μs	16.2763μs	61.4390 KOps/s	62.9256 KOps/s	$\color{#d91a1a}-2.36\%$
test_step_mdp_speed[True-False-True-True-True]	78.1010μs	49.5464μs	20.1831 KOps/s	20.2994 KOps/s	$\color{#d91a1a}-0.57\%$
test_step_mdp_speed[True-False-True-True-False]	55.8510μs	30.2455μs	33.0628 KOps/s	32.9697 KOps/s	$\color{#35bf28}+0.28\%$
test_step_mdp_speed[True-False-True-False-True]	58.4210μs	27.4362μs	36.4482 KOps/s	37.1781 KOps/s	$\color{#d91a1a}-1.96\%$
test_step_mdp_speed[True-False-True-False-False]	51.7810μs	16.2938μs	61.3729 KOps/s	63.2434 KOps/s	$\color{#d91a1a}-2.96\%$
test_step_mdp_speed[True-False-False-True-True]	88.7810μs	51.8321μs	19.2931 KOps/s	19.6085 KOps/s	$\color{#d91a1a}-1.61\%$
test_step_mdp_speed[True-False-False-True-False]	66.4810μs	32.4431μs	30.8232 KOps/s	30.8643 KOps/s	$\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-False-False-False-True]	65.5310μs	29.7081μs	33.6609 KOps/s	33.8371 KOps/s	$\color{#d91a1a}-0.52\%$
test_step_mdp_speed[True-False-False-False-False]	49.6510μs	18.8498μs	53.0510 KOps/s	54.7085 KOps/s	$\color{#d91a1a}-3.03\%$
test_step_mdp_speed[False-True-True-True-True]	85.7720μs	49.6940μs	20.1231 KOps/s	20.1206 KOps/s	$\color{#35bf28}+0.01\%$
test_step_mdp_speed[False-True-True-True-False]	71.5210μs	30.0667μs	33.2594 KOps/s	33.2848 KOps/s	$\color{#d91a1a}-0.08\%$
test_step_mdp_speed[False-True-True-False-True]	2.4853ms	31.7164μs	31.5294 KOps/s	31.9923 KOps/s	$\color{#d91a1a}-1.45\%$
test_step_mdp_speed[False-True-True-False-False]	48.4210μs	17.9006μs	55.8640 KOps/s	56.9535 KOps/s	$\color{#d91a1a}-1.91\%$
test_step_mdp_speed[False-True-False-True-True]	85.8610μs	52.3964μs	19.0853 KOps/s	19.2429 KOps/s	$\color{#d91a1a}-0.82\%$
test_step_mdp_speed[False-True-False-True-False]	64.9610μs	32.8927μs	30.4019 KOps/s	30.5611 KOps/s	$\color{#d91a1a}-0.52\%$
test_step_mdp_speed[False-True-False-False-True]	69.4710μs	33.9055μs	29.4938 KOps/s	29.9163 KOps/s	$\color{#d91a1a}-1.41\%$
test_step_mdp_speed[False-True-False-False-False]	51.3310μs	20.7024μs	48.3035 KOps/s	49.6043 KOps/s	$\color{#d91a1a}-2.62\%$
test_step_mdp_speed[False-False-True-True-True]	89.0420μs	54.8653μs	18.2265 KOps/s	18.3764 KOps/s	$\color{#d91a1a}-0.82\%$
test_step_mdp_speed[False-False-True-True-False]	60.0810μs	34.6673μs	28.8456 KOps/s	28.6615 KOps/s	$\color{#35bf28}+0.64\%$
test_step_mdp_speed[False-False-True-False-True]	63.7010μs	32.9999μs	30.3031 KOps/s	29.9540 KOps/s	$\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-False-True-False-False]	50.7810μs	20.4265μs	48.9560 KOps/s	49.9487 KOps/s	$\color{#d91a1a}-1.99\%$
test_step_mdp_speed[False-False-False-True-True]	88.2910μs	56.2513μs	17.7774 KOps/s	17.9211 KOps/s	$\color{#d91a1a}-0.80\%$
test_step_mdp_speed[False-False-False-True-False]	67.2310μs	37.4540μs	26.6994 KOps/s	26.6034 KOps/s	$\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-False-False-True]	64.8710μs	35.0602μs	28.5224 KOps/s	27.6745 KOps/s	$\color{#35bf28}+3.06\%$
test_step_mdp_speed[False-False-False-False-False]	49.4010μs	23.0062μs	43.4665 KOps/s	43.7322 KOps/s	$\color{#d91a1a}-0.61\%$
test_values[generalized_advantage_estimate-True-True]	11.0229ms	10.8261ms	92.3692 Ops/s	87.8694 Ops/s	$\textbf{\color{#35bf28}+5.12\%}$
test_values[vec_generalized_advantage_estimate-True-True]	20.5355ms	17.4733ms	57.2302 Ops/s	89.6853 Ops/s	$\textbf{\color{#d91a1a}-36.19\%}$
test_values[td0_return_estimate-False-False]	0.2257ms	0.1313ms	7.6141 KOps/s	7.7724 KOps/s	$\color{#d91a1a}-2.04\%$
test_values[td1_return_estimate-False-False]	29.5916ms	29.0828ms	34.3846 Ops/s	32.7156 Ops/s	$\textbf{\color{#35bf28}+5.10\%}$
test_values[vec_td1_return_estimate-False-False]	17.9858ms	17.4379ms	57.3464 Ops/s	88.8272 Ops/s	$\textbf{\color{#d91a1a}-35.44\%}$
test_values[td_lambda_return_estimate-True-False]	43.9686ms	43.3533ms	23.0663 Ops/s	22.1800 Ops/s	$\color{#35bf28}+4.00\%$
test_values[vec_td_lambda_return_estimate-True-False]	18.3773ms	17.4478ms	57.3140 Ops/s	89.8100 Ops/s	$\textbf{\color{#d91a1a}-36.18\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	9.5429ms	9.4605ms	105.7027 Ops/s	99.9554 Ops/s	$\textbf{\color{#35bf28}+5.75\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.6902ms	1.4963ms	668.3339 Ops/s	642.7391 Ops/s	$\color{#35bf28}+3.98\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4584ms	0.4195ms	2.3837 KOps/s	2.3125 KOps/s	$\color{#35bf28}+3.08\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	34.5822ms	33.8790ms	29.5168 Ops/s	39.4064 Ops/s	$\textbf{\color{#d91a1a}-25.10\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.8662ms	1.7556ms	569.5935 Ops/s	571.0357 Ops/s	$\color{#d91a1a}-0.25\%$
test_dqn_speed[False-None]	6.5179ms	1.4032ms	712.6771 Ops/s	711.7161 Ops/s	$\color{#35bf28}+0.14\%$
test_dqn_speed[False-backward]	1.9294ms	1.8827ms	531.1477 Ops/s	517.7750 Ops/s	$\color{#35bf28}+2.58\%$
test_dqn_speed[True-None]	0.7009ms	0.5492ms	1.8209 KOps/s	1.7182 KOps/s	$\textbf{\color{#35bf28}+5.98\%}$
test_dqn_speed[True-backward]	1.0781ms	1.0383ms	963.1276 Ops/s	802.2451 Ops/s	$\textbf{\color{#35bf28}+20.05\%}$
test_dqn_speed[reduce-overhead-None]	0.6659ms	0.5377ms	1.8598 KOps/s	1.7393 KOps/s	$\textbf{\color{#35bf28}+6.93\%}$
test_dqn_speed[reduce-overhead-backward]	1.0559ms	1.0165ms	983.7562 Ops/s	812.3381 Ops/s	$\textbf{\color{#35bf28}+21.10\%}$
test_ddpg_speed[False-None]	3.1716ms	2.8176ms	354.9082 Ops/s	344.7411 Ops/s	$\color{#35bf28}+2.95\%$
test_ddpg_speed[False-backward]	4.1264ms	4.0246ms	248.4748 Ops/s	245.1251 Ops/s	$\color{#35bf28}+1.37\%$
test_ddpg_speed[True-None]	1.6207ms	1.4054ms	711.5348 Ops/s	693.1728 Ops/s	$\color{#35bf28}+2.65\%$
test_ddpg_speed[True-backward]	2.4586ms	2.4075ms	415.3650 Ops/s	365.0846 Ops/s	$\textbf{\color{#35bf28}+13.77\%}$
test_ddpg_speed[reduce-overhead-None]	1.6239ms	1.3994ms	714.6038 Ops/s	689.1005 Ops/s	$\color{#35bf28}+3.70\%$
test_ddpg_speed[reduce-overhead-backward]	2.8297ms	2.4115ms	414.6733 Ops/s	368.3983 Ops/s	$\textbf{\color{#35bf28}+12.56\%}$
test_sac_speed[False-None]	8.2656ms	7.7603ms	128.8606 Ops/s	126.7231 Ops/s	$\color{#35bf28}+1.69\%$
test_sac_speed[False-backward]	11.5583ms	11.0702ms	90.3328 Ops/s	89.6236 Ops/s	$\color{#35bf28}+0.79\%$
test_sac_speed[True-None]	2.2536ms	2.0772ms	481.4167 Ops/s	473.0603 Ops/s	$\color{#35bf28}+1.77\%$
test_sac_speed[True-backward]	4.0962ms	3.9867ms	250.8326 Ops/s	249.2385 Ops/s	$\color{#35bf28}+0.64\%$
test_sac_speed[reduce-overhead-None]	2.3258ms	2.0695ms	483.2148 Ops/s	481.2163 Ops/s	$\color{#35bf28}+0.42\%$
test_sac_speed[reduce-overhead-backward]	4.0936ms	3.9938ms	250.3897 Ops/s	252.5141 Ops/s	$\color{#d91a1a}-0.84\%$
test_redq_speed[False-None]	10.6472ms	10.0890ms	99.1178 Ops/s	99.2238 Ops/s	$\color{#d91a1a}-0.11\%$
test_redq_speed[False-backward]	18.2256ms	17.5412ms	57.0087 Ops/s	59.2753 Ops/s	$\color{#d91a1a}-3.82\%$
test_redq_speed[True-None]	4.5473ms	4.1846ms	238.9714 Ops/s	228.4996 Ops/s	$\color{#35bf28}+4.58\%$
test_redq_speed[True-backward]	10.0537ms	9.7473ms	102.5921 Ops/s	98.7091 Ops/s	$\color{#35bf28}+3.93\%$
test_redq_speed[reduce-overhead-None]	4.3546ms	4.2147ms	237.2662 Ops/s	232.9536 Ops/s	$\color{#35bf28}+1.85\%$
test_redq_speed[reduce-overhead-backward]	9.9790ms	9.6808ms	103.2971 Ops/s	105.2501 Ops/s	$\color{#d91a1a}-1.86\%$
test_redq_deprec_speed[False-None]	11.1985ms	10.7827ms	92.7410 Ops/s	94.5950 Ops/s	$\color{#d91a1a}-1.96\%$
test_redq_deprec_speed[False-backward]	16.0609ms	15.6330ms	63.9673 Ops/s	65.5181 Ops/s	$\color{#d91a1a}-2.37\%$
test_redq_deprec_speed[True-None]	3.8073ms	3.5802ms	279.3107 Ops/s	282.2178 Ops/s	$\color{#d91a1a}-1.03\%$
test_redq_deprec_speed[True-backward]	7.7435ms	7.4732ms	133.8122 Ops/s	137.9919 Ops/s	$\color{#d91a1a}-3.03\%$
test_redq_deprec_speed[reduce-overhead-None]	3.8115ms	3.5192ms	284.1570 Ops/s	283.2338 Ops/s	$\color{#35bf28}+0.33\%$
test_redq_deprec_speed[reduce-overhead-backward]	7.6342ms	7.4378ms	134.4477 Ops/s	122.0781 Ops/s	$\textbf{\color{#35bf28}+10.13\%}$
test_td3_speed[False-None]	8.0518ms	7.8323ms	127.6764 Ops/s	126.8158 Ops/s	$\color{#35bf28}+0.68\%$
test_td3_speed[False-backward]	11.1714ms	10.6815ms	93.6195 Ops/s	92.6695 Ops/s	$\color{#35bf28}+1.03\%$
test_td3_speed[True-None]	1.8323ms	1.7856ms	560.0428 Ops/s	542.1973 Ops/s	$\color{#35bf28}+3.29\%$
test_td3_speed[True-backward]	3.7646ms	3.6332ms	275.2373 Ops/s	264.5244 Ops/s	$\color{#35bf28}+4.05\%$
test_td3_speed[reduce-overhead-None]	1.8128ms	1.7577ms	568.9399 Ops/s	550.0088 Ops/s	$\color{#35bf28}+3.44\%$
test_td3_speed[reduce-overhead-backward]	3.7587ms	3.6499ms	273.9796 Ops/s	242.8588 Ops/s	$\textbf{\color{#35bf28}+12.81\%}$
test_cql_speed[False-None]	29.8546ms	25.7002ms	38.9103 Ops/s	38.9676 Ops/s	$\color{#d91a1a}-0.15\%$
test_cql_speed[False-backward]	38.4572ms	35.4850ms	28.1809 Ops/s	28.4854 Ops/s	$\color{#d91a1a}-1.07\%$
test_cql_speed[True-None]	12.6292ms	12.2247ms	81.8014 Ops/s	81.6096 Ops/s	$\color{#35bf28}+0.24\%$
test_cql_speed[True-backward]	18.5478ms	18.2726ms	54.7266 Ops/s	53.3770 Ops/s	$\color{#35bf28}+2.53\%$
test_cql_speed[reduce-overhead-None]	15.1910ms	12.3464ms	80.9951 Ops/s	78.7321 Ops/s	$\color{#35bf28}+2.87\%$
test_cql_speed[reduce-overhead-backward]	18.5108ms	18.2520ms	54.7886 Ops/s	55.1664 Ops/s	$\color{#d91a1a}-0.68\%$
test_a2c_speed[False-None]	5.5391ms	5.3419ms	187.1998 Ops/s	184.1724 Ops/s	$\color{#35bf28}+1.64\%$
test_a2c_speed[False-backward]	12.8319ms	11.6798ms	85.6180 Ops/s	84.4054 Ops/s	$\color{#35bf28}+1.44\%$
test_a2c_speed[True-None]	3.9214ms	3.6702ms	272.4655 Ops/s	262.5047 Ops/s	$\color{#35bf28}+3.79\%$
test_a2c_speed[True-backward]	8.7552ms	8.5153ms	117.4362 Ops/s	114.5186 Ops/s	$\color{#35bf28}+2.55\%$
test_a2c_speed[reduce-overhead-None]	3.8675ms	3.7041ms	269.9734 Ops/s	271.7250 Ops/s	$\color{#d91a1a}-0.64\%$
test_a2c_speed[reduce-overhead-backward]	9.0100ms	8.8149ms	113.4438 Ops/s	105.0661 Ops/s	$\textbf{\color{#35bf28}+7.97\%}$
test_ppo_speed[False-None]	6.0027ms	5.6989ms	175.4728 Ops/s	171.6669 Ops/s	$\color{#35bf28}+2.22\%$
test_ppo_speed[False-backward]	12.8356ms	12.3298ms	81.1045 Ops/s	80.4575 Ops/s	$\color{#35bf28}+0.80\%$
test_ppo_speed[True-None]	3.8362ms	3.6524ms	273.7936 Ops/s	273.6616 Ops/s	$\color{#35bf28}+0.05\%$
test_ppo_speed[True-backward]	8.6155ms	8.3791ms	119.3445 Ops/s	116.3628 Ops/s	$\color{#35bf28}+2.56\%$
test_ppo_speed[reduce-overhead-None]	3.7626ms	3.6413ms	274.6301 Ops/s	273.4562 Ops/s	$\color{#35bf28}+0.43\%$
test_ppo_speed[reduce-overhead-backward]	8.9249ms	8.7696ms	114.0297 Ops/s	111.7978 Ops/s	$\color{#35bf28}+2.00\%$
test_reinforce_speed[False-None]	4.7396ms	4.5171ms	221.3807 Ops/s	219.5456 Ops/s	$\color{#35bf28}+0.84\%$
test_reinforce_speed[False-backward]	7.6592ms	7.3133ms	136.7371 Ops/s	135.9656 Ops/s	$\color{#35bf28}+0.57\%$
test_reinforce_speed[True-None]	3.0560ms	2.8330ms	352.9828 Ops/s	347.8646 Ops/s	$\color{#35bf28}+1.47\%$
test_reinforce_speed[True-backward]	7.8351ms	7.6314ms	131.0369 Ops/s	122.1174 Ops/s	$\textbf{\color{#35bf28}+7.30\%}$
test_reinforce_speed[reduce-overhead-None]	3.0048ms	2.8219ms	354.3684 Ops/s	338.5274 Ops/s	$\color{#35bf28}+4.68\%$
test_reinforce_speed[reduce-overhead-backward]	8.1229ms	7.8683ms	127.0918 Ops/s	127.1141 Ops/s	$\color{#d91a1a}-0.02\%$
test_iql_speed[False-None]	20.2898ms	19.3991ms	51.5488 Ops/s	49.9744 Ops/s	$\color{#35bf28}+3.15\%$
test_iql_speed[False-backward]	30.8195ms	29.8976ms	33.4475 Ops/s	33.5360 Ops/s	$\color{#d91a1a}-0.26\%$
test_iql_speed[True-None]	9.0713ms	8.4399ms	118.4855 Ops/s	117.8566 Ops/s	$\color{#35bf28}+0.53\%$
test_iql_speed[True-backward]	17.0879ms	16.6850ms	59.9342 Ops/s	58.2092 Ops/s	$\color{#35bf28}+2.96\%$
test_iql_speed[reduce-overhead-None]	8.6649ms	8.4085ms	118.9266 Ops/s	117.0903 Ops/s	$\color{#35bf28}+1.57\%$
test_iql_speed[reduce-overhead-backward]	17.3528ms	17.0884ms	58.5192 Ops/s	58.1695 Ops/s	$\color{#35bf28}+0.60\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.2761ms	5.9197ms	168.9261 Ops/s	167.6854 Ops/s	$\color{#35bf28}+0.74\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6371ms	0.3256ms	3.0710 KOps/s	3.5958 KOps/s	$\textbf{\color{#d91a1a}-14.59\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5255ms	0.3413ms	2.9300 KOps/s	3.8819 KOps/s	$\textbf{\color{#d91a1a}-24.52\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.8948ms	5.6608ms	176.6525 Ops/s	175.7010 Ops/s	$\color{#35bf28}+0.54\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.5291ms	0.3087ms	3.2398 KOps/s	3.6763 KOps/s	$\textbf{\color{#d91a1a}-11.87\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6439ms	0.3269ms	3.0592 KOps/s	3.9185 KOps/s	$\textbf{\color{#d91a1a}-21.93\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.5189ms	1.3041ms	766.7877 Ops/s	778.7664 Ops/s	$\color{#d91a1a}-1.54\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.5085ms	1.2909ms	774.6783 Ops/s	831.7100 Ops/s	$\textbf{\color{#d91a1a}-6.86\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	9.6938ms	5.9757ms	167.3445 Ops/s	170.1546 Ops/s	$\color{#d91a1a}-1.65\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8075ms	0.4553ms	2.1963 KOps/s	2.1331 KOps/s	$\color{#35bf28}+2.96\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8191ms	0.4431ms	2.2568 KOps/s	2.0526 KOps/s	$\textbf{\color{#35bf28}+9.95\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.8657ms	5.7002ms	175.4312 Ops/s	173.6241 Ops/s	$\color{#35bf28}+1.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.7863ms	0.3088ms	3.2385 KOps/s	3.0823 KOps/s	$\textbf{\color{#35bf28}+5.07\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6735ms	0.3034ms	3.2957 KOps/s	3.1512 KOps/s	$\color{#35bf28}+4.59\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.9654ms	5.6827ms	175.9736 Ops/s	175.5205 Ops/s	$\color{#35bf28}+0.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6612ms	0.2845ms	3.5144 KOps/s	2.9465 KOps/s	$\textbf{\color{#35bf28}+19.27\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.4637ms	0.2969ms	3.3677 KOps/s	3.2333 KOps/s	$\color{#35bf28}+4.16\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.9960ms	5.8221ms	171.7602 Ops/s	169.8753 Ops/s	$\color{#35bf28}+1.11\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.7545ms	0.4653ms	2.1490 KOps/s	1.9774 KOps/s	$\textbf{\color{#35bf28}+8.68\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6489ms	0.4335ms	2.3070 KOps/s	2.0779 KOps/s	$\textbf{\color{#35bf28}+11.02\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	6.5977ms	5.1394ms	194.5755 Ops/s	54.1392 Ops/s	$\textbf{\color{#35bf28}+259.40\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	9.7118ms	2.4122ms	414.5604 Ops/s	488.5993 Ops/s	$\textbf{\color{#d91a1a}-15.15\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.0567ms	1.0897ms	917.6743 Ops/s	871.8951 Ops/s	$\textbf{\color{#35bf28}+5.25\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.5139s	15.4087ms	64.8982 Ops/s	188.9791 Ops/s	$\textbf{\color{#d91a1a}-65.66\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	3.0031ms	1.3794ms	724.9488 Ops/s	488.1609 Ops/s	$\textbf{\color{#35bf28}+48.51\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	11.4565ms	1.2135ms	824.0381 Ops/s	909.5039 Ops/s	$\textbf{\color{#d91a1a}-9.40\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	7.1483ms	5.3883ms	185.5879 Ops/s	181.2510 Ops/s	$\color{#35bf28}+2.39\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	8.8254ms	2.1556ms	463.8985 Ops/s	461.6036 Ops/s	$\color{#35bf28}+0.50\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.4428ms	1.3361ms	748.4217 Ops/s	987.3227 Ops/s	$\textbf{\color{#d91a1a}-24.20\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	42.8118ms	41.3539ms	24.1815 Ops/s	23.5079 Ops/s	$\color{#35bf28}+2.87\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	18.8377ms	17.3932ms	57.4936 Ops/s	56.4889 Ops/s	$\color{#35bf28}+1.78\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	43.7650ms	42.4738ms	23.5439 Ops/s	23.3246 Ops/s	$\color{#35bf28}+0.94\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.2558ms	17.7138ms	56.4533 Ops/s	55.7533 Ops/s	$\color{#35bf28}+1.26\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	52.4495ms	45.5380ms	21.9597 Ops/s	18.6859 Ops/s	$\textbf{\color{#35bf28}+17.52\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.2713ms	19.1483ms	52.2240 Ops/s	51.2227 Ops/s	$\color{#35bf28}+1.95\%$