Add homepage introduct to RL framework

microsoft · you-n-g · Nov 10, 2022 · Oct 19, 2022 · Oct 19, 2022 · Oct 20, 2022
commit c7b68b1d284199a7693dbcebca8a16eee0ee49f8
diff --git a/README.md b/README.md
@@ -11,6 +11,7 @@
 Recent released features
 | Feature | Status |
 | --                      | ------    |
+| RL Learning Framework | :chart_with_upwards_trend: Released on Oct 20, 2022. [#1322](https://github.com/microsoft/qlib/pull/1322), [#1316](https://github.com/microsoft/qlib/pull/1316),[#1299](https://github.com/microsoft/qlib/pull/1299),[#1263](https://github.com/microsoft/qlib/pull/1263), [#1244](https://github.com/microsoft/qlib/pull/1244), [#1169](https://github.com/microsoft/qlib/pull/1169), [#1125](https://github.com/microsoft/qlib/pull/1125), [#1076](https://github.com/microsoft/qlib/pull/1076)|
 | HIST and IGMTF models | :chart_with_upwards_trend: [Released](https://github.com/microsoft/qlib/pull/1040) on Apr 10, 2022 |
 | Qlib [notebook tutorial](https://github.com/microsoft/qlib/tree/main/examples/tutorial) | 📖 [Released](https://github.com/microsoft/qlib/pull/1037) on Apr 7, 2022 | 
 | Ibovespa index data | :rice: [Released](https://github.com/microsoft/qlib/pull/990) on Apr 6, 2022 |
@@ -114,7 +115,8 @@ At the module level, Qlib is a platform that consists of the above components. T
 | ------                 | -----                                                                                                                                                                                                                                                                                                                                                                   |
 | `Infrastructure` layer | `Infrastructure` layer provides underlying support for Quant research. `DataServer` provides a high-performance infrastructure for users to manage and retrieve raw data. `Trainer` provides a flexible interface to control the training process of models, which enable algorithms to control the training process.                                                       |
 | `Workflow` layer       | `Workflow` layer covers the whole workflow of quantitative investment. `Information Extractor` extracts data for models. `Forecast Model` focuses on producing all kinds of forecast signals (e.g. _alpha_, risk) for other modules. With these signals `Decision Generator` will generate the target trading decisions(i.e. portfolio, orders)  to be executed by `Execution Env` (i.e. the trading market).  There may be multiple levels of `Trading Agent` and `Execution Env` (e.g. an _order executor trading agent and intraday order execution environment_ could behave like an interday trading environment and nested in  _daily portfolio management trading agent and interday trading environment_  ) |
-| `Interface` layer      | `Interface` layer tries to present a user-friendly interface for the underlying system. `Analyser` module will provide users detailed analysis reports of forecasting signals, portfolios and execution results                                                                                                                                                                 |
+| `Learning Framework` layer| The `Forecast Model` and `Trading Agent` are learnable. They are learned based on the `Learning Framework` layer and then applied to multiple scenarios in `Workflow` layer. The supported learning paradigms can be categorized into reinforcement learning and supervised learning.  The learning framework leverages the `Workflow` layer as well(e.g. sharing `Information Extractor`, creating environments based on `Execution Env`).   |
+| `Interface` layer | `Interface` layer tries to present a user-friendly interface for the underlying system. `Analyser` module will provide users detailed analysis reports of forecasting signals, portfolios and execution results |

 * The modules with hand-drawn style are under development and will be released in the future.
 * The modules with dashed borders are highly user-customizable and extendible.
@@ -404,6 +406,19 @@ Dataset plays a very important role in Quant. Here is a list of the datasets bui
 [Here](https://qlib.readthedocs.io/en/latest/advanced/alpha.html) is a tutorial to build dataset with `Qlib`.
 Your PR to build new Quant dataset is highly welcomed.

+
+# Learning Framework
+Qlib is high customizable and a lot of components is learnable.
+The learnable components are instances of `Forecast Model` and `Trading Agent`. They are learned based on the `Learning Framework` layer and then applied to multiple scenarios in `Workflow` layer.
+The learning framework leverages the `Workflow` layer as well(e.g. sharing `Information Extractor`, creating environments based on `Execution Env`).
+
+Based on learning paradigms, they can be categorized into reinforcement learning and supervised learning.
+
+For supervised learning, the detailed docs can be found [here](https://qlib.readthedocs.io/en/latest/component/model.html).
+
+For reinforcement learning, the detailed docs can be found [here](https://qlib.readthedocs.io/en/latest/component/rl.html). Qlib's RL learning framework leverages `Execution Env` in `Workflow` layer to create environments.  It's worth noting that `NestedExecutor` is supported as well. This empowers users to optimize different level of strategies/models/agents together (e.g. optimizing an order execution strategy for a specific portfolios management strategy).
+
+
 # More About Qlib
 If you want to have a quick glance at the most frequently used components of qlib, you can try notebooks [here](examples/tutorial/).


diff --git a/docs/_static/img/framework.svg b/docs/_static/img/framework.svg
diff --git a/docs/introduction/introduction.rst b/docs/introduction/introduction.rst
@@ -23,30 +23,37 @@ At the module level, Qlib is a platform that consists of above components. The c



-========================  ==============================================================================
-Name                      Description
-========================  ==============================================================================
-`Infrastructure` layer    `Infrastructure` layer provides underlying support for Quant research.
-                          `DataServer` provides high-performance infrastructure for users to manage
-                          and retrieve raw data. `Trainer` provides flexible interface to control
-                          the training process of models which enable algorithms controlling the
-                          training process.
-
-`Workflow` layer          `Workflow` layer covers the whole workflow of quantitative investment.
-                          `Information Extractor` extracts data for models. `Forecast Model` focuses
-                          on producing all kinds of forecast signals (e.g. *alpha*, risk) for other
-                          modules.  With these signals `Decision Generator` will generate the target
-                          trading decisions(i.e. portfolio, orders)  to be executed by `Execution Env`
-                          (i.e. the trading market).  There may be multiple levels of `Trading Agent`
-                          and `Execution Env` (e.g. an *order executor trading agent and intraday
-                          order execution environment* could behave like an interday trading
-                          environment and nested in  *daily portfolio management trading agent and
-                          interday trading environment*  )
-
-`Interface` layer         `Interface` layer tries to present a user-friendly interface for the underlying
-                          system. `Analyser` module will provide users detailed analysis reports of
-                          forecasting signals, portfolios and execution results
-========================  ==============================================================================
+===========================  ==============================================================================
+Name                         Description
+===========================  ==============================================================================
+`Infrastructure` layer       `Infrastructure` layer provides underlying support for Quant research.
+                             `DataServer` provides high-performance infrastructure for users to manage
+                             and retrieve raw data. `Trainer` provides flexible interface to control
+                             the training process of models which enable algorithms controlling the
+                             training process.
+
+`Workflow` layer             `Workflow` layer covers the whole workflow of quantitative investment.
+                             `Information Extractor` extracts data for models. `Forecast Model` focuses
+                             on producing all kinds of forecast signals (e.g. *alpha*, risk) for other
+                             modules.  With these signals `Decision Generator` will generate the target
+                             trading decisions(i.e. portfolio, orders)  to be executed by `Execution Env`
+                             (i.e. the trading market).  There may be multiple levels of `Trading Agent`
+                             and `Execution Env` (e.g. an *order executor trading agent and intraday
+                             order execution environment* could behave like an interday trading
+                             environment and nested in  *daily portfolio management trading agent and
+                             interday trading environment*  )
+
+`Learning Framework` layer   The `Forecast Model` and `Trading Agent` are learnable. They are learned
+                             based on the `Learning Framework` layer and then applied to multiple scenarios
+                             in `Workflow` layer. The supported learning paradigms can be categorized into
+                             reinforcement learning and supervised learning.  The learning framework
+                             leverages the `Workflow` layer as well(e.g. sharing `Information Extractor`,
+                             creating environments based on `Execution Env`).
+
+`Interface` layer            `Interface` layer tries to present a user-friendly interface for the underlying
+                             system. `Analyser` module will provide users detailed analysis reports of
+                             forecasting signals, portfolios and execution results
+===========================  ==============================================================================

 - The modules with hand-drawn style are under development and will be released in the future.
 - The modules with dashed borders are highly user-customizable and extendible.