Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Http Get support for NFT feature #22

Merged
merged 41 commits into from
Jul 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
2434bd3
impl PayloadData which has JsonRpcPayload and GetUrlPayload; provide …
laruh Apr 30, 2024
91aa895
remove Unnecessary trailing semicolon
laruh Apr 30, 2024
f4fab5a
use get_proxy_route_by_inbound func in http_handler, add test_parse_g…
laruh May 1, 2024
7abd17c
rename GetUrl to HttpGet and GetUrlPayload to HttpGetPayload, some mo…
laruh May 1, 2024
ac1c9e2
provide PayloadData type in proxy and validation_middleware funcs
laruh May 2, 2024
93d5563
impl proxy_http_get
laruh May 2, 2024
04d8728
docker-compose.yml for local tests and app json config
laruh May 3, 2024
3136f4b
impl modify_request_uri func
laruh May 3, 2024
84a1f62
use modify_request_uri func in proxy_http_get
laruh May 5, 2024
d980f64
remove `url` dependency, as for modify_request_uri functionality Uri …
laruh May 5, 2024
69fe4b8
return Url type in HttpGetPayload. we need it for Deserialize, Serial…
laruh May 6, 2024
e022f40
update note in modify_request_uri
laruh May 6, 2024
412739e
update log name in macro_rules! log_format
laruh May 6, 2024
bca2338
provide error message in logs
laruh May 7, 2024
80d92fb
remove header::CONTENT_LENGTH from http get request
laruh May 7, 2024
0294f19
simplify upsert_address_rate_in_pipe func, impl validation_middleware…
laruh May 8, 2024
cacce87
rename proxy_eth to proxy_json_rpc
laruh May 8, 2024
8a201da
add get_req: bool param into parse_payload function
laruh May 12, 2024
8bd9d87
provide optional rate_limiter field in ProxyRoute, return NOT_ACCEPTA…
laruh May 13, 2024
3456a42
review: remove conf_docker_test.json and use .conf_test instead
laruh May 14, 2024
1c60030
review: move quicknode and moralis logic into separate crates, pin re…
laruh May 14, 2024
d9922ee
remove #[allow(dead_code)] in ProofOfFundingError
laruh May 15, 2024
12f1944
review: update doc comments and fields
laruh May 16, 2024
95f909d
readme: provide Docker Compose commands and update the configuration …
laruh May 16, 2024
cd04389
remove empty lines between some modules in main.rs
laruh May 16, 2024
7e87c8f
add #[allow(dead_code)] annotation
laruh May 16, 2024
e8f296d
impl parse_header_payload func
laruh May 16, 2024
b28a244
impl remove_unnecessary_headers func to avoid some code duplication
laruh May 17, 2024
800de2c
review: rename remove_unnecessary_headers to remove_hop_by_hop_headers
laruh May 20, 2024
0508f33
review: update docker section in readme
laruh May 20, 2024
c639542
review: update Middleware description in readme
laruh May 20, 2024
e607cbb
update GET req handling for Moralis
laruh May 23, 2024
e98cf08
use req.uri().path() to find ProxyRote in websocket. Note: in the cur…
laruh May 24, 2024
d86b97f
refactor quicknode: move SignedMessage to Header
laruh Jun 5, 2024
226db21
review: remove note about inbound_route in test
laruh Jun 22, 2024
db6e9cd
Merge remote-tracking branch 'refs/remotes/origin/nft-support' into n…
laruh Jun 22, 2024
c66f316
Merge pull request #23 from KomodoPlatform/nft-support-refactor-quick…
onur-ozkan Jun 26, 2024
e81cfb9
review: update docs, rename parse functions, modify_request_uri non a…
laruh Jun 27, 2024
765f584
state unit types explicitly to adapt to new never type fallback
laruh Jun 27, 2024
6422433
impl `get_proxy_route_extracting_uri_inbound` func and `test_get_prox…
laruh Jun 28, 2024
e3d81fa
review: avoid multiple iterations in get proxy with Uri functionality…
laruh Jul 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Generated by Cargo
# will have compiled files and executables
/target/

.idea
# These are backup files generated by rustfmt
**/*.rs.bk

Expand Down
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ tokio-tungstenite = { version = "0.20.0", features = ["native-tls"] }
bitcrypto = { git = "https://github.com/KomodoPlatform/atomicDEX-API", branch = "dev" }
ethkey = { git = "https://github.com/artemii235/parity-ethereum.git" }
serialization = { git = "https://github.com/KomodoPlatform/atomicDEX-API", branch = "dev" }
url = { version = "2.2.2", features = ["serde"] }

[target.x86_64-unknown-linux-gnu.dependencies]
jemallocator = "0.5.0"
Expand Down
107 changes: 75 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,29 +12,31 @@ Create the configuration file for app runtime.

```json
{
"port": 6150,
"pubkey_path": "/path_to_publick_key.pem",
"privkey_path": "/path_to_private_key.pem",
"redis_connection_string": "redis://localhost",
"token_expiration_time": 300,
"proxy_routes": [
{
"inbound_route": "/dev",
"outbound_route": "http://localhost:8000",
"authorized": false,
"allowed_methods": [
"eth_blockNumber",
"eth_gasPrice"
]
}
],
"rate_limiter": {
"rp_1_min": 30,
"rp_5_min": 100,
"rp_15_min": 200,
"rp_30_min": 350,
"rp_60_min": 575
}
"port": 6150,
"pubkey_path": "/path_to_publick_key.pem",
"privkey_path": "/path_to_private_key.pem",
"redis_connection_string": "redis://localhost",
"token_expiration_time": 300,
"proxy_routes": [
{
"inbound_route": "/dev",
"outbound_route": "http://localhost:8000",
"proxy_type": "quicknode",
"authorized": false,
"allowed_rpc_methods": [
"eth_blockNumber",
"eth_gasPrice"
],
"rate_limiter": null
}
],
"rate_limiter": {
"rp_1_min": 30,
"rp_5_min": 100,
"rp_15_min": 200,
"rp_30_min": 350,
"rp_60_min": 575
}
}
```

Expand All @@ -54,17 +56,34 @@ Expose configuration file's path as an environment variable in `AUTH_APP_CONFIG_

3) If the incoming request comes from the same network, step 4 will be by-passed.

4) Request will be handled in the middleware with:
- Status Checker: Checks if the wallet address status is blocked, allowed, or trusted and does the following:
- Blocked: Return `403 Forbidden` immediately
- Allowed: process continues with the rate limiter
- Trusted: bypass rate limiter and proof of funding
- Rate Limiter: First, verify the signed message, and if not valid, return 401 Unauthorized immediately. If valid, then calculate the request count with time interval specified in the application configuration. If the wallet address sent too many request than the expected amount, process continues with the proof of funding. If not, by-passes the proof of funding.
- Proof of Funding: Return `406 Not Acceptable` if wallet has 0 balance. Otherwise, we assume that request is valid and process continues as usual.
4) Request Handling in the Middleware:

5) Find target route by requested endpoint
**For Quicknode:**
- **Status Checker**:
- **Blocked**: Return `403 Forbidden` immediately.
- **Allowed**: Process continues with the rate limiter.
- **Trusted**: Bypass rate limiter and proof of funding.

6) Check if requested rpc call is allowed in application configuration
- **Rate Limiter**:
- First, verify the signed message. If not valid, return `401 Unauthorized` immediately.
- If valid, calculate the request count with the time interval specified in the application configuration. If the wallet address has sent too many requests than the expected amount, process continues with the proof of funding. If not, bypass the proof of funding.

- **Proof of Funding**:
- Return `406 Not Acceptable` if the wallet has a 0 balance. Otherwise, assume the request is valid and process it as usual.

**For Moralis:**
- **Status Checker**:
- **Blocked**: Return `403 Forbidden` immediately.
- **Allowed**: Process continues with the rate limiter.
- **Trusted**: Bypass the rate limiter.

- **Rate Limiter**:
- First, verify the signed message. If not valid, return `401 Unauthorized` immediately.
- If valid, calculate the request count with the time interval specified in the application configuration. If the wallet address has sent too many requests, return an error `406 Not Acceptable` indicating that the wallet address must wait for some time before making more requests.

5) Find target route by requested endpoint.

6) Check if requested rpc call is allowed in application configuration.

7) Generate JWT token with RSA algorithm using pub-priv keys specified in the application configuration, and insert the token to the request header.

Expand Down Expand Up @@ -97,3 +116,27 @@ curl -v --url "'$mm2_address'" -s --data '{
"id": 0
}'
```

### How to run KomodoDefi-Proxy Service with Docker Compose

If you want to test features locally, you can run Docker containers using Docker Compose commands.

1. **Update Configuration**:
In the `.config_test` file, update the `proxy_routes` field by adding `ProxyRoutes` with the necessary parameters.

2. **Run Containers in Detached Mode**:
To start the containers, run the following command. This will build the images if they are not already built or if changes are detected in the Dockerfile or the build context.
```sh
docker compose up -d
```

3. **Follow the Logs**:
Open a new terminal window or tab and execute this command to follow the logs of all services defined in the Docker Compose file. The `-f` (or `--follow`) option ensures that new log entries are continuously displayed as they are produced, while the `-t` (or `--timestamps`) option adds timestamps to each log entry.
```sh
docker compose logs -f -t
```

4. **Stop the Containers**:
```sh
docker compose down
```
98 changes: 72 additions & 26 deletions assets/.config_test
Original file line number Diff line number Diff line change
@@ -1,28 +1,74 @@
{
"port": 5000,
"redis_connection_string": "dummy-value",
"pubkey_path": "dummy-value",
"privkey_path": "dummy-value",
"token_expiration_time": 300,
"proxy_routes": [
{
"inbound_route": "/test",
"outbound_route": "https://komodoplatform.com",
"authorized": false,
"allowed_methods": []
},
{
"inbound_route": "/test-2",
"outbound_route": "https://atomicdex.io",
"authorized": false,
"allowed_methods": []
}
],
"rate_limiter": {
"rp_1_min": 555,
"rp_5_min": 555,
"rp_15_min": 555,
"rp_30_min": 555,
"rp_60_min": 555
}
"port": 6150,
"redis_connection_string": "redis://redis:6379",
"pubkey_path": "/usr/src/komodo-defi-proxy/assets/.pubkey_test",
"privkey_path": "/usr/src/komodo-defi-proxy/assets/.privkey_test",
"token_expiration_time": 300,
"proxy_routes": [
{
"inbound_route": "/test",
"outbound_route": "https://komodoplatform.com",
"proxy_type": "quicknode",
"authorized": false,
"allowed_rpc_methods": [],
"rate_limiter": null
},
{
"inbound_route": "/test-2",
"outbound_route": "https://atomicdex.io",
"proxy_type": "quicknode",
"authorized": false,
"allowed_rpc_methods": [],
"rate_limiter": null
},
{
"inbound_route": "/nft-test",
"outbound_route": "https://nft.proxy",
"proxy_type": "moralis",
"authorized": false,
"allowed_rpc_methods": [],
"rate_limiter": {
"rp_1_min": 60,
"rp_5_min": 200,
"rp_15_min": 700,
"rp_30_min": 1000,
"rp_60_min": 2000
}
},
{
"inbound_route": "/nft-test/special",
"outbound_route": "https://nft.special",
"proxy_type": "moralis",
"authorized": false,
"allowed_rpc_methods": [],
"rate_limiter": {
"rp_1_min": 60,
"rp_5_min": 200,
"rp_15_min": 700,
"rp_30_min": 1000,
"rp_60_min": 2000
}
},
{
"inbound_route": "/",
"outbound_route": "https://adex.io",
"proxy_type": "moralis",
"authorized": false,
"allowed_rpc_methods": [],
"rate_limiter": {
"rp_1_min": 60,
"rp_5_min": 200,
"rp_15_min": 700,
"rp_30_min": 1000,
"rp_60_min": 2000
}
}
],
"rate_limiter": {
"rp_1_min": 555,
"rp_5_min": 555,
"rp_15_min": 555,
"rp_30_min": 555,
"rp_60_min": 555
}
}
20 changes: 20 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
version: '3.8'
services:
redis:
image: redis:7.2.4-alpine3.19
restart: always
ports:
- "6379:6379"

proxy:
build:
context: ./
dockerfile: Containerfile
ports:
- "6150:6150"
depends_on:
- redis
environment:
AUTH_APP_CONFIG_PATH: /usr/src/komodo-defi-proxy/assets/.config_test
volumes:
- ./assets:/usr/src/komodo-defi-proxy/assets
Loading
Loading