diff --git a/docs/blob-archiver-rs-docs/index.md b/docs/blob-archiver-rs-docs/index.md index 36d3b5c..cbc2d40 100644 --- a/docs/blob-archiver-rs-docs/index.md +++ b/docs/blob-archiver-rs-docs/index.md @@ -1,6 +1,24 @@ # blob-archiver-rs This is a Rust implementation of -the [Beacon Chain blob archiver](https://github.com/base-org/blob-archiver) +the [Beacon Chain blob archiver](https://github.com/base/blob-archiver) + +The Blob Archiver is a service to archive and allow querying of all historical blobs from the beacon chain. It consists +of two components: + +* **Archiver** - Tracks the beacon chain and writes blobs to a storage backend +* **API** - Implements the blob sidecars [API](https://ethereum.github.io/beacon-APIs/#/Beacon/getBlobSidecars), which + allows clients to retrieve blobs from the storage backend + +### Storage +There are currently two supported storage options: + +* On-disk storage - Blobs are written to disk in a directory +* S3 storage - Blobs are written to an S3 bucket (or compatible service) + +You can control which storage backend is used by setting the `STORAGE_TYPE` to +either `file` or `s3`. + +The `s3` backend will also work with (for example) Google Cloud Storage buckets (instructions [here](https://medium.com/google-cloud/using-google-cloud-storage-with-minio-object-storage-c994fe4aab6b)). ### Development ```sh @@ -21,4 +39,154 @@ to your beacon client and storage backend of choice. Then you can run the projec ```sh docker compose up -``` \ No newline at end of file +``` + +#### Get blobs +After successfully starting the service, you can use the following command to obtain the blob: + +- get blob by block id from api service: +```shell +# also there is support other type of block id, like: finalized,justified. +curl -X 'GET' 'http://localhost:8000/eth/v1/beacon/blob_sidecars/head' -H 'accept: application/json' +``` + +- get blob by slot number from api service: +```shell +curl -X 'GET' 'http://localhost:8000/eth/v1/beacon/blob_sidecars/7111008' -H 'accept: application/json' +``` + +#### Storage Dashboard +MinIO has started the dashboard, allowing you to view the status of blob storage. +By default, you can access it directly at: +```http +http://localhost:9999 +``` + +## Options + +### `verbose` + + +```shell +--verbose= +``` + +```shell +--verbose=2 +``` + +### `log_dir` + +```shell +--log_dir= +``` +```shell +--log_dir=/var/log/blob-archiver +``` + +### `log_rotation` + +```shell +--log_rotation= +``` +```shell +--log_rotation=DAILY +``` +### `beacon_endpoint` + +```shell +--beacon_endpoint= +``` +```shell +--beacon_endpoint=http://localhost:5052 +``` + +### `beacon_client_timeout` + +```shell +--beacon_client_timeout= +``` +```shell +--beacon_client_timeout=10 +``` + +### `poll_interval` + +```shell +--poll_interval= +``` +```shell +--poll_interval=6 +``` + +### `listen_addr` + +```shell +--listen_addr= +``` +```shell +--listen_addr=0.0.0.0:8000 +``` + +### `origin_block` + +```shell +--origin_block= +``` +```shell +--origin_block="0x0" +``` + +### `storage_type` + +```shell +--storage_type= +``` +```shell +--storage_type="s3" +``` + +### `s3_endpoint` + +```shell +--s3_endpoint= +``` +```shell +--s3_endpoint="http://localhost:9000" +``` + +### `s3_bucket` +```shell +--s3_bucket= +``` +```shell +--s3_bucket="blobs" +``` + +### `s3_path` + +```shell +--s3_path= +``` +```shell +--s3_path=/blobs +``` + +### `s3_compress` + +```shell +--s3_compress= +``` +```shell +--s3_compress=false +``` + +### `fs_dir` + +```shell +--fs_dir= +``` +```shell +--fs_dir=/blobs +``` + diff --git a/docs/hildr-docs/run-a-node/use_docker.md b/docs/hildr-docs/run-a-node/use_docker.md index f331f7e..3cd452a 100644 --- a/docs/hildr-docs/run-a-node/use_docker.md +++ b/docs/hildr-docs/run-a-node/use_docker.md @@ -25,7 +25,7 @@ cp /path/to/jwt.txt . ```shell docker run -it -p 11545:11545 \ -v :/jwt/jwt.hex \ - ghcr.io/optimism-java/op-besu:latest -- \ + ghcr.io/optimism-java/hildr:latest \ --network optimism-sepolia \ # can be either: "optimism","base","base-sepolia" --jwt-file /jwt/jwt.hex \ --l1-rpc-url http://localhost:9545 \ @@ -44,7 +44,7 @@ cp /path/to/jwt.txt . docker run -it -p 11545:11545 \ -v :/jwt/jwt.hex \ -v :/network-configs/rollup.json - ghcr.io/optimism-java/op-besu:latest -- \ + ghcr.io/optimism-java/hildr:latest \ --devnet \ --network=/network-configs/rollup.json \ --jwt-file /jwt/jwt.hex \ diff --git a/docs/op-besu-docs/run-a-node/use_binaries.md b/docs/op-besu-docs/run-a-node/use_binaries.md index 17f3297..045eb19 100644 --- a/docs/op-besu-docs/run-a-node/use_binaries.md +++ b/docs/op-besu-docs/run-a-node/use_binaries.md @@ -58,6 +58,7 @@ mkdir datadir ```shell ./bin/besu \ --network=OP_SEPOLIA \ + --sync-mode=FULL \ --p2p-enabled=false \ --discovery-enabled=false \ --data-path=datadir \ @@ -79,6 +80,7 @@ Also you could use the [optimism devnet tool](https://github.com/ethpandaops/opt ```shell ./bin/besu \ --genesis-file= \ + --network-id= --p2p-enabled=false \ --discovery-enabled=false \ --data-path=datadir \ diff --git a/docs/op-besu-docs/run-a-node/use_docker.md b/docs/op-besu-docs/run-a-node/use_docker.md index 9378580..4a6a63d 100644 --- a/docs/op-besu-docs/run-a-node/use_docker.md +++ b/docs/op-besu-docs/run-a-node/use_docker.md @@ -26,7 +26,8 @@ openssl rand -hex 32 > jwt.txt ## Prepare the data directory ```bash -mkdir datadir +mkdir datadir +chmod 777 datadir ``` ## Start a Op-besu node @@ -35,10 +36,11 @@ mkdir datadir ```shell -docker run -it -p 8545:8545 -p 8551:8551 -v ./jwt.txt:/jwt/jwtsecret \ +docker run -it --name op-besu -p 8545:8545 -p 8551:8551 -v ./jwt.txt:/jwt/jwtsecret \ -v ./datadir:/data/ \ ghcr.io/optimism-java/op-besu:latest \ --network=OP_SEPOLIA \ +--sync-mode=FULL \ --p2p-enabled=false \ --discovery-enabled=false \ --data-path="/data/" \ @@ -56,10 +58,12 @@ ghcr.io/optimism-java/op-besu:latest \ ```shell -docker run -it -p 8545:8545 -p 8551:8551 -v ./jwt.txt:/jwt/jwtsecret \ +docker run -it --name op-besu-devnet -p 8545:8545 -p 8551:8551 -v ./jwt.txt:/jwt/jwtsecret \ -v ./datadir:/data/ \ ghcr.io/optimism-java/op-besu:latest \ --genesis-file= \ +--network-id= \ +--sync-mode=FULL \ --p2p-enabled=false \ --discovery-enabled=false \ --data-path="/data/" \ @@ -83,4 +87,51 @@ The following settings are a security risk in production environments: - Setting `--host-allowlist` to `"*"` allows JSON-RPC API access from any host. - Setting `--rpc-http-cors-origins` to `"all"` or `"*"` allows cross-origin resource sharing (CORS) access from any domain. -::: \ No newline at end of file +::: + +## Start op-node to sync L2 state + +After starting the Op-Besu node, you can start the op-node to sync state. + +Pull docker image of hildr: + +```shell +docker pull us-docker.pkg.dev/oplabs-tools-artifacts/images/op-node:v1.12.2 +``` + +Get IP of the op-besu container, and op-node container will use it to connect to op-besu via the docker bridge: + +```shell +docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' op-besu +``` + +Start op-node node: + +```shell +docker run -d -it --name op-node -p 11545:11545 \ +-v ./jwt.txt:/jwt/jwt.txt \ +--entrypoint op-node \ +us-docker.pkg.dev/oplabs-tools-artifacts/images/op-node:v1.12.2 \ +--network op-sepolia \ +--l1.rpckind=basic \ +--l1= \ +--l2= \ +--rpc.addr=0.0.0.0 \ +--rpc.port=11545 \ +--l2.jwt-secret=/jwt/jwt.txt \ +--l1.trustrpc \ +--l1.beacon= \ +--syncmode=consensus-layer +``` + +The synchronization needs to handle empty messages at the beginning, and the actual block synchronization will take place about 15 minutes later. + +For example, use curl get block data from op-besu: + +```bash +curl --request POST 'https://localhost:8545' \ +--header 'Content-Type: application/json' \ +--data-raw '{"id":2, "jsonrpc":"2.0", "method": "eth_getBlockByNumber", "params":["0xe", true]}' +``` + +You can confirm whether the block and transaction information is correct through the [Sepolia network's blockchain explorer](https://sepolia-optimism.etherscan.io/). \ No newline at end of file