Benchmark deserialization #198

Stapox35 · Apr 29, 2025

Add 4 new benchmarks:

concurrent deserialization

This benchmark uses executeConcurrent endpoint to insert n rows containing uuid, int, timeuuid, inet, date, time into the database. Afterwards it uses executeConcurrent endpoint to select all (n) of the inserted rows from the database n times.

JS:

node concurrent_deser.js <driver> <Number of queries>

Rust:

CNT=<Number of queries> cargo run --bin concurrent_deser_benchmark -r

deserialization

This benchmark executes n client.execute queries, that insert a single row containing uuid, int, timeuuid, inet, date, time waiting for the result of the previous query before executing the next one. Afterwards it executes n client.execute queries, that select all (n) of the inserted rows, waiting for the result of the previous query before executing the next one.

JS:

node deser.js <driver> <Number of queries>

Rust:

CNT=<Number of queries> cargo run --bin deser_benchmark -r

concurrent serialization

This benchmark uses executeConcurrent endpoint to insert n*n rows containing uuid, int, timeuuid, inet, date, time into the database.

JS:

node concurrent_ser.js <driver> <Number of queries>

Rust:

CNT=<Number of queries> cargo run --bin concurrent_ser_benchmark -r

serialization

This benchmark executes n*n client.execute queries, that insert a single row containing uuid, int, timeuuid, inet, date, time waiting for the result of the previous query before executing the next one.

JS:

node ser.js <driver> <Number of queries>

Rust:

CNT=<Number of queries> cargo run --bin ser_benchmark -r

wprzytula · May 6, 2025

@Stapox35 If you already have charts drawn using code from this PR, please upload them here in a comment.

benchmark/README.md

benchmark/logic/concurrent_deser.js

benchmark/logic/ser.js

benchmark/logic_rust/concurrent_deser.rs

wprzytula · May 8, 2025

benchmark/logic/ser.js

+        function createKeyspace(next) {
+            // Keep replication one to reduce time spent in the database
+            const query =
+                "CREATE KEYSPACE IF NOT EXISTS benchmarks WITH replication = {'class': 'NetworkTopologyStrategy', 'replication_factor': '1' }";


🌱 We should compare results with tablets turned on and off.

benchmark/runner.py

uuid, int, timeuuid, inet, date, time

adespawn · May 14, 2025

benchmark/logic/utils.js

+    const tuid = cassandra.types.TimeUuid.fromString("8e14e760-7fa8-11eb-bc66-000000000001");
+    const ip = new cassandra.types.InetAddress(utils.allocBufferFromArray([198, 168, 1, 1]));


Can we add some randomness also to those values?

adespawn · May 14, 2025

benchmark/logic/deser.js

+        },
+        function createTable(next) {
+            const query =
+                "CREATE TABLE benchmarks.basic (id uuid, val int, tuuid timeuuid, ip inet, date date, time time, PRIMARY KEY(id))";


make this string a constant in utils

adespawn · May 14, 2025

benchmark/logic/deser.js

+        },
+        async function insert(next) {
+            const query =
+                "INSERT INTO benchmarks.basic (id, val, tuuid, ip, date, time) VALUES (?, ?, ?, ?, ?, ?)";


This can also be a constant

wprzytula · May 14, 2025

benchmark/logic_rust/concurrent_deser.rs

+use std::sync::Arc;
+use uuid::Uuid;
+
+const CONCURRENCY: usize = 2000;


❓ 🤔 Here I can see CONCURRENCY hard-coded. I don't see an equivalent constant in JS benchark code. Where is it?

wprzytula · May 14, 2025

benchmark/logic_rust/concurrent_deser.rs

+    let mut handles = vec![];
+    let session = Arc::new(session);
+
+    for i in 0..CONCURRENCY {
+        let session_clone = Arc::clone(&session);
+        let insert_query_clone = insert_query.clone();
+        handles.push(tokio::spawn(async move {
+            insert_data(session_clone, i, n, &insert_query_clone)
+                .await
+                .unwrap();
+        }));
+    }
+
+    let results = join_all(handles).await;
+
+    for result in results {
+        result.unwrap();
+    }
+
+    let select_query = session.prepare("SELECT * FROM benchmarks.basic").await?;
+
+    let mut handles = vec![];
+
+    for i in 0..CONCURRENCY {
+        let session_clone = Arc::clone(&session);
+        let select_query_clone = select_query.clone();
+        handles.push(tokio::spawn(async move {
+            select_data(session_clone, i, n, &select_query_clone)
+                .await
+                .unwrap();
+        }));
+    }
+
+    let results = join_all(handles).await;
+
+    for result in results {
+        result.unwrap();
+    }


🔧 The logic is duplicated. Please extract the common part to a higher-order function that will accept a closure/Future performing the action. Hint: I'd make the function accept impl Future<Item = Vec<Result<QueryResult, ExecutionError>>>.

wprzytula · May 14, 2025

benchmark/runner.py

 steps = {}

-step = 4
+step = 6


❓ What are these??

hot benchmarks commit is a temporary commit

Copilot

Pull Request Overview

This PR adds four new benchmarks targeting serialization and deserialization for both concurrent and sequential operations in JavaScript and Rust. Key changes include adjustments to the benchmark runner parameters and figure layout, additions of new Rust benchmark binaries, and corresponding updates to JavaScript benchmark scripts and documentation.

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
benchmark/runner.py	Adjusted benchmark parameters (repeat, step) and figure layout.
benchmark/logic_rust/ser.rs	Added new Rust benchmark for serialization.
benchmark/logic_rust/deser.rs	Added new Rust benchmark for deserialization.
benchmark/logic_rust/concurrent_ser.rs	Added new Rust benchmark for concurrent serialization.
benchmark/logic_rust/concurrent_deser.rs	Added new Rust benchmark for concurrent deserialization.
benchmark/logic/utils.js	Extended utilities to support new (de)serialization benchmarks.
benchmark/logic/ser.js	Added new JS benchmark for serialization.
benchmark/logic/deser.js	Added new JS benchmark for deserialization.
benchmark/logic/concurrent_ser.js	Added new JS benchmark for concurrent serialization.
benchmark/logic/concurrent_deser.js	Added new JS benchmark for concurrent deserialization.
benchmark/README.md	Updated documentation to include new benchmarks.
Cargo.toml	Updated dependencies to support new benchmarks.

Copilot · May 16, 2025

benchmark/runner.py

@@ -52,29 +52,38 @@ def run_process(command):

 # --------- parameters ------------



Reducing the repeat count from 3 to 1 may affect the consistency of benchmark results; please add a comment clarifying the rationale behind this change.

Suggested change

# Setting repeat to 1 for now to speed up testing or debugging.

# Increase this value (e.g., to 3) for more consistent benchmark results.

Copilot · May 16, 2025

benchmark/runner.py


 steps = {}

-step = 4
+step = 5


[nitpick] Consider adding a comment explaining the reasoning for increasing the step value to 5, to improve future maintainability.

Copilot · May 16, 2025

benchmark/runner.py

@@ -145,12 +154,12 @@ def run_process(command):

 libs.append("rust-driver")

-cols = 3
+cols = 4


[nitpick] A change in the subplot layout (increasing the number of columns and adjusting the figure size) should be accompanied by a brief comment to explain the intended visualization improvements.

Stapox35 force-pushed the benchmark-serialize branch from f0ba260 to 6a93dfb Compare April 29, 2025 10:41

Stapox35 marked this pull request as draft April 29, 2025 10:44

Stapox35 added the benchmark label Apr 29, 2025

Stapox35 self-assigned this Apr 29, 2025

ZuzaOsa requested a review from Copilot April 29, 2025 11:36

This comment was marked as abuse.

Sign in to view

Stapox35 force-pushed the benchmark-serialize branch from 540f93e to 03f999f Compare May 6, 2025 10:21

Stapox35 marked this pull request as ready for review May 6, 2025 10:31

Stapox35 requested review from ZuzaOsa, adespawn and wprzytula May 6, 2025 10:32

ZuzaOsa requested a review from Copilot May 6, 2025 10:39

This comment was marked as abuse.

Sign in to view

scylladb-zpp-2024-javascript-driver deleted a comment from Copilot AI May 6, 2025

ZuzaOsa approved these changes May 6, 2025

View reviewed changes

Stapox35 force-pushed the benchmark-serialize branch from ee940a2 to 6b118d8 Compare May 6, 2025 11:08

Stapox35 changed the title ~~Benchmark serialize~~ Benchmark deserialization May 6, 2025

Stapox35 force-pushed the benchmark-serialize branch from 0df8858 to 39998cc Compare May 6, 2025 12:54

adespawn requested changes May 7, 2025

View reviewed changes

benchmark/README.md Outdated Show resolved Hide resolved

benchmark/logic/concurrent_deser.js Outdated Show resolved Hide resolved

benchmark/logic/ser.js Outdated Show resolved Hide resolved

wprzytula requested changes May 8, 2025

View reviewed changes

Stapox35 force-pushed the benchmark-serialize branch 2 times, most recently from 8efc172 to 3e309f1 Compare May 12, 2025 21:42

Stapox35 added 5 commits May 12, 2025 23:44

Add new benchmark - deserialization types

7fe6220

uuid, int, timeuuid, inet, date, time

Add concurrent version of deserialization benchmark

d20f95d

Add new benchmark - serialization types

63f8bf0

uuid, int, timeuuid, inet, date, time

Add concurrent version of serialization benchmark

f869c47

Add description for new benchmarks to README.md

268b8d0

Stapox35 force-pushed the benchmark-serialize branch from 3e309f1 to 268b8d0 Compare May 12, 2025 21:44

Correction of the order of benchmarks on the chart

a0e3e27

Stapox35 requested review from adespawn and wprzytula May 13, 2025 10:37

adespawn reviewed May 14, 2025

View reviewed changes

wprzytula reviewed May 14, 2025

View reviewed changes

hot benchmarks

b9c72a6

adespawn force-pushed the benchmark-serialize branch from 044b350 to b9c72a6 Compare May 14, 2025 19:43

ZuzaOsa requested a review from Copilot May 16, 2025 07:58

Copilot AI reviewed May 16, 2025

View reviewed changes

adespawn added this to the Benchmark day milestone May 17, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Benchmark deserialization #198

Benchmark deserialization #198

Stapox35 commented Apr 29, 2025 •

edited

Loading

This comment was marked as abuse.

This comment was marked as abuse.

wprzytula commented May 6, 2025

wprzytula May 8, 2025

adespawn May 14, 2025

adespawn May 14, 2025

adespawn May 14, 2025

wprzytula May 14, 2025

wprzytula May 14, 2025

wprzytula May 14, 2025

adespawn May 14, 2025

Copilot AI left a comment

Copilot AI May 16, 2025

Copilot AI May 16, 2025

Copilot AI May 16, 2025

		const tuid = cassandra.types.TimeUuid.fromString("8e14e760-7fa8-11eb-bc66-000000000001");
		const ip = new cassandra.types.InetAddress(utils.allocBufferFromArray([198, 168, 1, 1]));

		@@ -52,29 +52,38 @@ def run_process(command):

		# --------- parameters ------------



	# Setting repeat to 1 for now to speed up testing or debugging.
	# Increase this value (e.g., to 3) for more consistent benchmark results.

Search code, repositories, users, issues, pull requests...

Benchmark deserialization #198

Are you sure you want to change the base?

Benchmark deserialization #198

Conversation

Stapox35 commented Apr 29, 2025 • edited Loading

This comment was marked as abuse.

This comment was marked as abuse.

wprzytula commented May 6, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Copilot AI May 16, 2025

Choose a reason for hiding this comment

Copilot AI May 16, 2025

Choose a reason for hiding this comment

Copilot AI May 16, 2025

Choose a reason for hiding this comment

Stapox35 commented Apr 29, 2025 •

edited

Loading