Skip to content

Commit

Permalink
Http Get support for NFT feature (#22)
Browse files Browse the repository at this point in the history
* impl PayloadData which has JsonRpcPayload and GetUrlPayload; provide general generate_payload_from_req func; doc comms; add url crate

* remove Unnecessary trailing semicolon

* use get_proxy_route_by_inbound func in http_handler, add test_parse_get_url_payload

* rename GetUrl to HttpGet and GetUrlPayload to HttpGetPayload, some more doc comms

* provide PayloadData type in proxy and validation_middleware funcs

* impl proxy_http_get

* docker-compose.yml for local tests and app json config

* impl modify_request_uri func

* use modify_request_uri func in proxy_http_get

* remove `url` dependency, as for modify_request_uri functionality Uri type is enough

* return Url type in HttpGetPayload. we need it for Deserialize, Serialize, and for general impl path in req uri have to represent only inbound_route

* update note in modify_request_uri

* update log name in macro_rules! log_format

* provide error message in logs

* remove header::CONTENT_LENGTH from http get request

* simplify upsert_address_rate_in_pipe func, impl validation_middleware_http_get, have general validation_middleware func which handle all PayloadData types

* rename proxy_eth to proxy_json_rpc

* add get_req: bool param into parse_payload function

* provide optional rate_limiter field in ProxyRoute, return NOT_ACCEPTABLE error if rate exceed for HTTP GET request

* review: remove conf_docker_test.json and use .conf_test instead

* review: move quicknode and moralis logic into separate crates, pin redis:7.2.4-alpine3.19 version.
fix: Execute the pipeline once after setting all commands and make upsert_address_rate_in_pipe not async, as it just mutates Pipeline.

* remove #[allow(dead_code)] in ProofOfFundingError

* review: update doc comments and fields

* readme: provide Docker Compose commands and update the configuration file description

* remove empty lines between some modules in main.rs

* add #[allow(dead_code)] annotation

* impl parse_header_payload func

* impl remove_unnecessary_headers func to avoid some code duplication

* review: rename remove_unnecessary_headers to remove_hop_by_hop_headers

* review: update docker section in readme

* review: update Middleware description in readme

* update GET req handling for Moralis

* use req.uri().path() to find ProxyRote in websocket. Note: in the current stage only quicknode feature supports both websocket and http.

* refactor quicknode: move SignedMessage to Header

* review: remove note about inbound_route in test

* review: update docs, rename parse functions, modify_request_uri non async

* state unit types explicitly to adapt to new never type fallback

* impl `get_proxy_route_extracting_uri_inbound` func and `test_get_proxy_route_by_uri_inbound` test, update modify_request_uri logic

* review: avoid multiple iterations in get proxy with Uri functionality, move delete inbound from Uri to modify_request_uri

---------

Co-authored-by: Onur Özkan <[email protected]>
  • Loading branch information
laruh and onur-ozkan committed Jul 8, 2024
1 parent 73738f0 commit 547b493
Show file tree
Hide file tree
Showing 21 changed files with 1,386 additions and 563 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Generated by Cargo
# will have compiled files and executables
/target/

.idea
# These are backup files generated by rustfmt
**/*.rs.bk

Expand Down
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ tokio-tungstenite = { version = "0.20.0", features = ["native-tls"] }
bitcrypto = { git = "https://github.com/KomodoPlatform/atomicDEX-API", branch = "dev" }
ethkey = { git = "https://github.com/artemii235/parity-ethereum.git" }
serialization = { git = "https://github.com/KomodoPlatform/atomicDEX-API", branch = "dev" }
url = { version = "2.2.2", features = ["serde"] }

[target.x86_64-unknown-linux-gnu.dependencies]
jemallocator = "0.5.0"
Expand Down
107 changes: 75 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,29 +12,31 @@ Create the configuration file for app runtime.

```json
{
"port": 6150,
"pubkey_path": "/path_to_publick_key.pem",
"privkey_path": "/path_to_private_key.pem",
"redis_connection_string": "redis://localhost",
"token_expiration_time": 300,
"proxy_routes": [
{
"inbound_route": "/dev",
"outbound_route": "http://localhost:8000",
"authorized": false,
"allowed_methods": [
"eth_blockNumber",
"eth_gasPrice"
]
}
],
"rate_limiter": {
"rp_1_min": 30,
"rp_5_min": 100,
"rp_15_min": 200,
"rp_30_min": 350,
"rp_60_min": 575
}
"port": 6150,
"pubkey_path": "/path_to_publick_key.pem",
"privkey_path": "/path_to_private_key.pem",
"redis_connection_string": "redis://localhost",
"token_expiration_time": 300,
"proxy_routes": [
{
"inbound_route": "/dev",
"outbound_route": "http://localhost:8000",
"proxy_type": "quicknode",
"authorized": false,
"allowed_rpc_methods": [
"eth_blockNumber",
"eth_gasPrice"
],
"rate_limiter": null
}
],
"rate_limiter": {
"rp_1_min": 30,
"rp_5_min": 100,
"rp_15_min": 200,
"rp_30_min": 350,
"rp_60_min": 575
}
}
```

Expand All @@ -54,17 +56,34 @@ Expose configuration file's path as an environment variable in `AUTH_APP_CONFIG_

3) If the incoming request comes from the same network, step 4 will be by-passed.

4) Request will be handled in the middleware with:
- Status Checker: Checks if the wallet address status is blocked, allowed, or trusted and does the following:
- Blocked: Return `403 Forbidden` immediately
- Allowed: process continues with the rate limiter
- Trusted: bypass rate limiter and proof of funding
- Rate Limiter: First, verify the signed message, and if not valid, return 401 Unauthorized immediately. If valid, then calculate the request count with time interval specified in the application configuration. If the wallet address sent too many request than the expected amount, process continues with the proof of funding. If not, by-passes the proof of funding.
- Proof of Funding: Return `406 Not Acceptable` if wallet has 0 balance. Otherwise, we assume that request is valid and process continues as usual.
4) Request Handling in the Middleware:

5) Find target route by requested endpoint
**For Quicknode:**
- **Status Checker**:
- **Blocked**: Return `403 Forbidden` immediately.
- **Allowed**: Process continues with the rate limiter.
- **Trusted**: Bypass rate limiter and proof of funding.

6) Check if requested rpc call is allowed in application configuration
- **Rate Limiter**:
- First, verify the signed message. If not valid, return `401 Unauthorized` immediately.
- If valid, calculate the request count with the time interval specified in the application configuration. If the wallet address has sent too many requests than the expected amount, process continues with the proof of funding. If not, bypass the proof of funding.

- **Proof of Funding**:
- Return `406 Not Acceptable` if the wallet has a 0 balance. Otherwise, assume the request is valid and process it as usual.

**For Moralis:**
- **Status Checker**:
- **Blocked**: Return `403 Forbidden` immediately.
- **Allowed**: Process continues with the rate limiter.
- **Trusted**: Bypass the rate limiter.

- **Rate Limiter**:
- First, verify the signed message. If not valid, return `401 Unauthorized` immediately.
- If valid, calculate the request count with the time interval specified in the application configuration. If the wallet address has sent too many requests, return an error `406 Not Acceptable` indicating that the wallet address must wait for some time before making more requests.

5) Find target route by requested endpoint.

6) Check if requested rpc call is allowed in application configuration.

7) Generate JWT token with RSA algorithm using pub-priv keys specified in the application configuration, and insert the token to the request header.

Expand Down Expand Up @@ -97,3 +116,27 @@ curl -v --url "'$mm2_address'" -s --data '{
"id": 0
}'
```

### How to run KomodoDefi-Proxy Service with Docker Compose

If you want to test features locally, you can run Docker containers using Docker Compose commands.

1. **Update Configuration**:
In the `.config_test` file, update the `proxy_routes` field by adding `ProxyRoutes` with the necessary parameters.

2. **Run Containers in Detached Mode**:
To start the containers, run the following command. This will build the images if they are not already built or if changes are detected in the Dockerfile or the build context.
```sh
docker compose up -d
```

3. **Follow the Logs**:
Open a new terminal window or tab and execute this command to follow the logs of all services defined in the Docker Compose file. The `-f` (or `--follow`) option ensures that new log entries are continuously displayed as they are produced, while the `-t` (or `--timestamps`) option adds timestamps to each log entry.
```sh
docker compose logs -f -t
```

4. **Stop the Containers**:
```sh
docker compose down
```
98 changes: 72 additions & 26 deletions assets/.config_test
Original file line number Diff line number Diff line change
@@ -1,28 +1,74 @@
{
"port": 5000,
"redis_connection_string": "dummy-value",
"pubkey_path": "dummy-value",
"privkey_path": "dummy-value",
"token_expiration_time": 300,
"proxy_routes": [
{
"inbound_route": "/test",
"outbound_route": "https://komodoplatform.com",
"authorized": false,
"allowed_methods": []
},
{
"inbound_route": "/test-2",
"outbound_route": "https://atomicdex.io",
"authorized": false,
"allowed_methods": []
}
],
"rate_limiter": {
"rp_1_min": 555,
"rp_5_min": 555,
"rp_15_min": 555,
"rp_30_min": 555,
"rp_60_min": 555
}
"port": 6150,
"redis_connection_string": "redis://redis:6379",
"pubkey_path": "/usr/src/komodo-defi-proxy/assets/.pubkey_test",
"privkey_path": "/usr/src/komodo-defi-proxy/assets/.privkey_test",
"token_expiration_time": 300,
"proxy_routes": [
{
"inbound_route": "/test",
"outbound_route": "https://komodoplatform.com",
"proxy_type": "quicknode",
"authorized": false,
"allowed_rpc_methods": [],
"rate_limiter": null
},
{
"inbound_route": "/test-2",
"outbound_route": "https://atomicdex.io",
"proxy_type": "quicknode",
"authorized": false,
"allowed_rpc_methods": [],
"rate_limiter": null
},
{
"inbound_route": "/nft-test",
"outbound_route": "https://nft.proxy",
"proxy_type": "moralis",
"authorized": false,
"allowed_rpc_methods": [],
"rate_limiter": {
"rp_1_min": 60,
"rp_5_min": 200,
"rp_15_min": 700,
"rp_30_min": 1000,
"rp_60_min": 2000
}
},
{
"inbound_route": "/nft-test/special",
"outbound_route": "https://nft.special",
"proxy_type": "moralis",
"authorized": false,
"allowed_rpc_methods": [],
"rate_limiter": {
"rp_1_min": 60,
"rp_5_min": 200,
"rp_15_min": 700,
"rp_30_min": 1000,
"rp_60_min": 2000
}
},
{
"inbound_route": "/",
"outbound_route": "https://adex.io",
"proxy_type": "moralis",
"authorized": false,
"allowed_rpc_methods": [],
"rate_limiter": {
"rp_1_min": 60,
"rp_5_min": 200,
"rp_15_min": 700,
"rp_30_min": 1000,
"rp_60_min": 2000
}
}
],
"rate_limiter": {
"rp_1_min": 555,
"rp_5_min": 555,
"rp_15_min": 555,
"rp_30_min": 555,
"rp_60_min": 555
}
}
20 changes: 20 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
version: '3.8'
services:
redis:
image: redis:7.2.4-alpine3.19
restart: always
ports:
- "6379:6379"

proxy:
build:
context: ./
dockerfile: Containerfile
ports:
- "6150:6150"
depends_on:
- redis
environment:
AUTH_APP_CONFIG_PATH: /usr/src/komodo-defi-proxy/assets/.config_test
volumes:
- ./assets:/usr/src/komodo-defi-proxy/assets
Loading

0 comments on commit 547b493

Please sign in to comment.