-
Notifications
You must be signed in to change notification settings - Fork 151
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: chainSpec based controller config; Types from apps-config (#351)
* feat: chainSpec based controller config; Types from apps-config Clean up * Update mock api to include derive getBlock - specs work * Update to reflect TS 4.1 * Clean up comments * Save * feat: Token query param for non-native token balance-info * Revert package.json * Initial reply david review * Update src/main.ts Co-authored-by: David <[email protected]> * Remove kulupu controller * Add chain builder integration guide * Patch CHAIN_INTEGRATION.md * Apply suggestions from code review Co-authored-by: joe petrowski <[email protected]> * Update CHAIN_INTEGRATION.md * Bump deps * Use whole options object for controller creation * Apply suggestions from code review Co-authored-by: David <[email protected]> * Respond to maciej feedback * Update chain integration guide * Remove excessive nullish colescing operators * Fix merge issue * Update CHAIN_INTEGRATION.md Co-authored-by: David <[email protected]> Co-authored-by: David <[email protected]> Co-authored-by: joe petrowski <[email protected]>
- Loading branch information
1 parent
c2a4937
commit 5936a1c
Showing
32 changed files
with
870 additions
and
362 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,77 @@ | ||
# Substrate Api Sidecar chain integration guide | ||
|
||
This guide aims to help chain builders integrate their Substrate FRAME based chain with Substrate API Sidecar. | ||
|
||
## Table of contents | ||
|
||
- [Polkadot-js API type definition support](#polkadot-js-API-type-definition-support) | ||
- [Controller configuration](controller-configuration) | ||
|
||
## Polkadot-js API type definition support | ||
|
||
In order decode the SCALE encoded data from a substrate based node, polkadot-js needs to have a registry of type definitions. Sidecar pulls in chain type definitions from the [@polkadot/apps-config package hosted on NPM](https://www.npmjs.com/package/@polkadot/apps-config). | ||
|
||
If the chain's type definitions do not already exist in [@polkadot/apps-config](https://github.com/polkadot-js/apps/tree/master/packages/apps-config) they will need to be added via PR to polkadot-js/apps by following their [instructions for API config](https://github.com/polkadot-js/apps/tree/master/packages/apps-config#api). | ||
|
||
Before taking any other steps to integrate a chain with Sidecar, a chain's up-to-date type definitions must be included in a published version of @polkadot/apps-config. | ||
|
||
## Controller configuration | ||
|
||
Sidecar offers the ability to configure which controllers to mount. Sidecar uses a chain's spec name to determine which controller config to use, and if no config is linked to a spec name, then the [default config](/src/chains-config/defaultControllers.ts) is used. | ||
|
||
A chain builder can follow the steps below and submit a chain's controller config via PR, where it will be reviewed and merged once deemed ready by the maintainers. | ||
|
||
#### 1) Create a controller config | ||
|
||
Create a controller config for your chain. The shape of the controller config is specified [here](/src/chains-config/ControllerConfig.ts). The `controller` property has keys from the [controller export](/src/controllers/index.ts), which is an exhaustive collection of the available controller classes. In order to see the path(s) associated with a controller one must look in the controller source code. | ||
|
||
The easiest way to start creating a controller config would be to copy [defaultControllers.ts](/src/chains-config/ControllerConfig.ts) and name the file and export `{specName}Controllers`. Then change the boolean values to indicate wether or not to mount a controller and its paths. Ensure to export the controller config from `chains-config` by adding `export * from './{specName}Controllers.ts'` in [/src/chains-config/index.ts](/src/chains-config/index.ts). | ||
|
||
To determine what controllers to include, one must consider the runtime logic, specifically what pallets the chain uses. It is important to keep in mind the assumptions the service's logic makes and what exact pallets the service queries. | ||
|
||
An example is in order. If we want to use [`PalletsStakingProgressController`](/src/controllers/pallets/PalletsStakingProgressController.ts), first check [`PalletsStakingProgressService.ts`](/src/services/pallets/PalletsStakingProgressService.ts); here we see it queries `staking`, `sessions`, `babe` pallets and makes certain assumptions about how the pallets are used together in the runtime. If we determine that both have all storage items queried, and the assumptions made about the staking system are correct, we can then declare in `{specName}Controllers` (the controller config object) that we want to mount the controller by giving `PalletsStakingProgressController` property `true` as the value (`{ PalletsStakingProgressController: true }`). Finally, we can test it out by starting up Sidecar against our chain and accessing the `pallets/staking/progress` endpoint, verifying that the information returned is correct. | ||
|
||
In some circumstances, a chain may need a new path, modify a path or alter business logic for a path. Path changes that help a chain support wallets will be given priority. Breaking changes to paths are usually rejected. | ||
|
||
##### Basic balance transfer support | ||
|
||
In order to support traditional balance transfers the chain's Sidecar endpoints should support account balance lookup, transaction submission, transaction material retrieval, and block queries. | ||
|
||
To support these features the following endpoints are necessary: | ||
|
||
| Path | Controller | Description | | ||
|:----------------------------------------:|:-----------------------------:|:--------------------------------------------------------------------------:| | ||
| GET `/transaction/material` | TransactionMaterialController | Get all the network information needed to construct a transaction offline. | | ||
| POST `/transaction` | TransactionSubmitController | Submit a transaction to the node's transaction pool. | | ||
| GET `/blocks/head` & `/blocks/{number}` | BlocksController | Get a block. | | ||
| GET `accounts/{accountId}/balance-info` | AccountsBalanceInfoController | Get balance information for an account. | | ||
|
||
#### 2) Update `specToControllerMap` | ||
|
||
In order for Sidecar to use your controller config, the `specToControllerMap` in [/src/chains-config/index.ts](/src/chains-config/index.ts) must be updated with the chain's `specName` and controller config by adding them as a property to `specToControllerMap`: | ||
|
||
```javascript | ||
const specToControllerMap = { | ||
kulupu: kulupuControllers, | ||
mandala: mandalaControllers, | ||
{specName}: {specName}Controllers, | ||
}; | ||
``` | ||
|
||
#### 3) Test | ||
|
||
Run it against a node running your chain in archive mode: | ||
|
||
- Ensure all the expected paths work, including the root path, preferably with tests | ||
- Exercise each query param of every path | ||
- Make sure transaction submission works | ||
- Try out historic queries across runtimes where types might change | ||
|
||
#### 4) Submit your PR | ||
|
||
Make sure it passes lint with `yarn lint --fix` and tests with `yarn test`. Then submit a PR for review. | ||
|
||
#### 5) Maintenance | ||
|
||
- Keep types up-to-date in `@polkadot/apps-config` | ||
- If the business logic or storage of a chain's pallet queried by a Sidecar endpoint is changed, ensure that the corresponding service logic is updated in Sidecar as well |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
import { ControllerConfig } from '../types/chains-config'; | ||
|
||
/** | ||
* Controllers that Sidecar will always default to. This will always be | ||
* the optimal controller selection for Polkadot and Kusama. | ||
*/ | ||
export const defaultControllers: ControllerConfig = { | ||
controllers: { | ||
Blocks: true, | ||
AccountsStakingPayouts: true, | ||
AccountsBalanceInfo: true, | ||
AccountsStakingInfo: true, | ||
AccountsVestingInfo: true, | ||
NodeNetwork: true, | ||
NodeVersion: true, | ||
NodeTransactionPool: true, | ||
RuntimeCode: true, | ||
RuntimeSpec: true, | ||
RuntimeMetadata: true, | ||
TransactionDryRun: true, | ||
TransactionMaterial: true, | ||
TransactionFeeEstimate: true, | ||
TransactionSubmit: true, | ||
PalletsStakingProgress: true, | ||
PalletsStorage: true, | ||
}, | ||
options: { | ||
finalizes: true, | ||
}, | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
import { ApiPromise } from '@polkadot/api'; | ||
import AbstractController from 'src/controllers/AbstractController'; | ||
import { AbstractService } from 'src/services/AbstractService'; | ||
|
||
import { controllers } from '../controllers'; | ||
import { ControllerConfig } from '../types/chains-config'; | ||
import { defaultControllers } from './defaultControllers'; | ||
import { kulupuControllers } from './kulupuControllers'; | ||
import { mandalaControllers } from './mandalaControllers'; | ||
|
||
const specToControllerMap = { | ||
kulupu: kulupuControllers, | ||
mandala: mandalaControllers, | ||
}; | ||
|
||
/** | ||
* Return an array of instantiated controller instances based off of a `specName`. | ||
* | ||
* @param api ApiPromise to inject into controllers | ||
* @param implName | ||
*/ | ||
export function getControllersForSpec( | ||
api: ApiPromise, | ||
specName: string | ||
): AbstractController<AbstractService>[] { | ||
if (specToControllerMap[specName]) { | ||
return getControllersFromConfig(api, specToControllerMap[specName]); | ||
} | ||
|
||
// If we don't have the specName in the specToControllerMap we use the default | ||
// contoller config | ||
return getControllersFromConfig(api, defaultControllers); | ||
} | ||
|
||
/** | ||
* Return an array of instantiated controller instances based off of a | ||
* `ControllerConfig`. | ||
* | ||
* @param api ApiPromise to inject into controllers | ||
* @param config controller mount configuration object | ||
*/ | ||
function getControllersFromConfig(api: ApiPromise, config: ControllerConfig) { | ||
// If we don't typecast here, tsc thinks its just [string, any][] | ||
const controllersToInclude = Object.entries(config.controllers) as [ | ||
keyof typeof controllers, | ||
boolean | ||
][]; | ||
|
||
return controllersToInclude.reduce((acc, [controllerName, shouldMount]) => { | ||
if (shouldMount) { | ||
acc.push(new controllers[controllerName](api, config.options)); | ||
} | ||
|
||
return acc; | ||
}, [] as AbstractController<AbstractService>[]); | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
import { ControllerConfig } from '../types/chains-config'; | ||
|
||
export const kulupuControllers: ControllerConfig = { | ||
controllers: { | ||
Blocks: true, | ||
AccountsStakingPayouts: false, | ||
AccountsBalanceInfo: true, | ||
AccountsStakingInfo: false, | ||
AccountsVestingInfo: false, | ||
NodeNetwork: true, | ||
NodeVersion: true, | ||
NodeTransactionPool: true, | ||
RuntimeCode: true, | ||
RuntimeSpec: true, | ||
RuntimeMetadata: true, | ||
TransactionDryRun: true, | ||
TransactionMaterial: true, | ||
TransactionFeeEstimate: true, | ||
TransactionSubmit: true, | ||
PalletsStakingProgress: false, | ||
PalletsStorage: true, | ||
}, | ||
options: { | ||
finalizes: false, | ||
}, | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
import { ControllerConfig } from '../types/chains-config'; | ||
|
||
/** | ||
* Controllers for mandala, acala's test network. | ||
*/ | ||
export const mandalaControllers: ControllerConfig = { | ||
controllers: { | ||
Blocks: true, | ||
AccountsStakingPayouts: true, | ||
AccountsBalanceInfo: true, | ||
AccountsStakingInfo: true, | ||
AccountsVestingInfo: true, | ||
NodeNetwork: true, | ||
NodeVersion: true, | ||
NodeTransactionPool: true, | ||
RuntimeCode: true, | ||
RuntimeSpec: true, | ||
RuntimeMetadata: true, | ||
TransactionDryRun: true, | ||
TransactionMaterial: true, | ||
TransactionFeeEstimate: true, | ||
TransactionSubmit: true, | ||
PalletsStakingProgress: true, | ||
PalletsStorage: true, | ||
}, | ||
options: { | ||
finalizes: true, | ||
}, | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.