Merge remote-tracking branch 'origin/sy_dev' into sy_dev

CONTRIBUTING.md

@@ -4,9 +4,27 @@ To contribute, fork the repository and send a pull request.
 
 When submitting code, please make every effort to follow existing conventions and style in order to keep the code as readable as possible.
 
-Where appropriate, please provide unit tests or integration tests. Unit tests should be pytest based tests and be added to <project>/tests.
+Where appropriate, please provide unit tests or integration tests. Unit tests should be pytest based tests and be added to `<project>/tests`.
 
-Please make sure all tests pass before submitting a pull request. It is also good if you squash your commits and add the tags #major or #minor to the pull request title if need be, otherwise your pull request will be considered a patch bump.
+Please make sure all tests pass before submitting a pull request. It is also good if you squash your commits and add the tags `#major` or `#minor` to the pull request title if need be, otherwise your pull request will be considered a patch bump.
+
+## Contributing to MARLlib
+
+We welcome contributions to MARLlib! If you're interested in contributing, here are some areas where we would appreciate your help:
+
+### New Tasks
+
+MARLlib aims to support a wide range of multi-agent reinforcement learning tasks. If you have a new task in mind that you think would be valuable to include, please consider contributing its implementation. You can add the new task code under the `marllib/envs` directory.
+
+### New Algorithms
+
+We are always looking to expand the collection of algorithms available in MARLlib. If you have developed a new multi-agent RL algorithm or would like to implement an existing one that is missing, we encourage you to contribute. You can add the new algorithm code under the `/marllib/marl/algos` directory.
+
+### Bug fixes and enhancements
+
+Contributions to fix bugs or enhance the existing functionality of MARLlib are highly appreciated. If you come across any issues or have ideas for improvements, please feel free to contribute your changes. You can submit bug fixes and enhancements as pull requests to the repository.
+
+We appreciate all contributions and will review them promptly. Thank you for your interest in improving MARLlib!
 
 
 ## Testing the code locally

README.md

@@ -1,11 +1,20 @@
 <div align="center">
 
-<img src=docs/source/images/allenv.gif width=95% />
-
+<img src=docs/source/images/logo1.png width=75% />
 </div>
 
+
+
 <h1 align="center"> MARLlib: A Multi-agent Reinforcement Learning Library </h1>
 
+<div align="center">
+
+<img src=docs/source/images/allenv.gif width=99% />
+
+</div>
+
+&emsp;
+
 [![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)]()
 ![test](https://github.com/Replicable-MARL/MARLlib/workflows/test/badge.svg)
 ![coverage](https://github.com/Replicable-MARL/MARLlib/blob/sy_dev/coverage.svg)
@@ -23,6 +32,7 @@
 | **March 2023** We are excited to announce that a major update has just been released. For detailed version information, please refer to the [version info](https://github.com/Replicable-MARL/MARLlib/releases/tag/1.0.2).|
 | **May 2023** Exciting news! MARLlib now supports three more tasks: [MATE](https://marllib.readthedocs.io/en/latest/handbook/env.html#mate), [GoBigger](https://marllib.readthedocs.io/en/latest/handbook/env.html#gobigger), [Overcooked-AI](https://marllib.readthedocs.io/en/latest/handbook/env.html#overcooked-ai). Give them a try!|
 | **May 2023 (update)** Supports two more tasks: [MAPDN](https://marllib.readthedocs.io/en/latest/handbook/env.html#power-distribution-networks) and [AirCombat](https://marllib.readthedocs.io/en/latest/handbook/env.html#air-combat). Give them a try!|
+| **June 2023** OpenAI: [Hide and Seek](https://marllib.readthedocs.io/en/latest/handbook/env.html#hideandseek) incorporated into MARLlib.|
 
 
 **Multi-agent Reinforcement Learning Library ([MARLlib](https://arxiv.org/abs/2210.13708))** is ***a MARL library*** that utilizes [**Ray**](https://github.com/ray-project/ray) and one of its toolkits [**RLlib**](https://github.com/ray-project/ray/tree/master/rllib). It offers a comprehensive platform for developing, training, and testing MARL algorithms across various tasks and environments. 
@@ -33,22 +43,20 @@ Here's an example of how MARLlib can be used:
 from marllib import marl
 
 # prepare env
-env = marl.make_env(environment_name="mpe", map_name="simple_spread")
+env = marl.make_env(environment_name="mpe", map_name="simple_spread", force_coop=True)
 
 # initialize algorithm with appointed hyper-parameters
 mappo = marl.algos.mappo(hyperparam_source='mpe')
 
 # build agent model based on env + algorithms + user preference
-model = marl.build_model(env, mappo, {"core_arch": "gru", "encode_layer": "128-256"})
+model = marl.build_model(env, mappo, {"core_arch": "mlp", "encode_layer": "128-256"})
 
 # start training
 mappo.fit(env, model, stop={'timesteps_total': 1000000}, share_policy='group')
-
-# ready to control
-mappo.render(env, model, share_policy='group', restore_path='path_to_checkpoint')
 ```
 
 
+
 ## Why MARLlib?
 
 Here we provide a table for the comparison of MARLlib and existing work.
@@ -201,6 +209,7 @@ Most of the popular environments in MARL research are supported by MARLlib:
 | **[Overcooked-AI](https://marllib.readthedocs.io/en/latest/handbook/env.html#overcooked-ai)** | cooperative | Full | Discrete | 1D |
 | **[PDN](https://marllib.readthedocs.io/en/latest/handbook/env.html#power-distribution-networks)** | cooperative | Partial | Continuous | 1D |
 | **[AirCombat](https://marllib.readthedocs.io/en/latest/handbook/env.html#air-combat)** | cooperative + mixed | Partial | MultiDiscrete | 1D |
+| **[HideAndSeek](https://marllib.readthedocs.io/en/latest/handbook/env.html#hideandseek)** | competitive + mixed | Partial | MultiDiscrete | 1D |
 
 Each environment has a readme file, standing as the instruction for this task, including env settings, installation, and
 important notes.
@@ -286,11 +295,11 @@ Choose `mlp`, `gru`, or `lstm` as you like to build the complete model.
 from marllib import marl
 
 # prepare env
-env = marl.make_env(environment_name="mpe", map_name="simple_spread")
+env = marl.make_env(environment_name="smac", map_name="5m_vs_6m")
 # initialize algorithm with appointed hyper-parameters
-mappo = marl.algos.mappo(hyperparam_source="mpe")
+mappo = marl.algos.mappo(hyperparam_source="smac")
 # build agent model based on env + algorithms + user preference
-model = marl.build_model(env, mappo, {"core_arch": "mlp", "encode_layer": "128-256"})
+model = marl.build_model(env, mappo, {"core_arch": "gru", "encode_layer": "128-256"})
 # start training
 mappo.fit(
   env, model, 
@@ -302,8 +311,8 @@ mappo.fit(
 mappo.render(
   env, model, 
   local_mode=True, 
-  restore_path={'params_path': "checkpoint_000010/params.json",
-                'model_path': "checkpoint_000010/checkpoint-10"}
+  restore_path={'params_path': "checkpoint/params.json",
+                'model_path': "checkpoint/checkpoint-10"}
 )
 ```
 </details>
@@ -345,7 +354,6 @@ MARLlib provides some practical examples for you to refer to.
 
 Try MPE + MAPPO examples on Google Colaboratory!
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Replicable-MARL/MARLlib/blob/sy_dev/marllib.ipynb)
-
 More tutorial documentations are available [here](https://marllib.readthedocs.io/).
 
 ## Awesome List
@@ -379,7 +387,7 @@ If you use MARLlib in your research, please cite the [MARLlib paper](https://arx
 
 ```tex
 @article{hu2022marllib,
-      title={MARLlib: Extending RLlib for Multi-agent Reinforcement Learning},
+      title={MARLlib: A Scalable Multi-agent Reinforcement Learning Library},
       author={Hu, Siyi and Zhong, Yifan and Gao, Minquan and Wang, Weixun and Dong, Hao and Li, Zhihui and Liang, Xiaodan and Chang, Xiaojun and Yang, Yaodong},
       journal={arXiv preprint arXiv:2210.13708},
       year={2022}

ROADMAP.md

@@ -8,7 +8,7 @@ This list describes the planned features including breaking changes.
 
 ## Roadmap to v2.0.0
 - [x] test suits + coverage
-- [ ] support Windows
+- [ ] support Windows OS / MacOS
 - [ ] manually inference, refer to issue: https://github.com/Replicable-MARL/MARLlib/issues/86#issuecomment-1468188682
 - [ ] new environments
   - [x] MATE: https://github.com/UnrealTracking/mate
@@ -17,5 +17,7 @@ This list describes the planned features including breaking changes.
   - [x] Overcooked: https://github.com/HumanCompatibleAI/overcooked_ai
   - [x] CloseAirCombat: https://github.com/liuqh16/CloseAirCombat
   - [ ] HideAndSeek: https://github.com/jiayu-ch15/Variational-Automatic-Curriculum-Learning#3-train-hide-and-seek
+  - [ ] Melting Pot: https://github.com/deepmind/meltingpot
+
 - [ ] support Transformers
 - [x] support MultiDiscrete action space