fix: harden warehouse SQL identifier quoting#7187
fix: harden warehouse SQL identifier quoting#7187rudder-devbox[bot] wants to merge 5 commits intomasterrudderlabs/rudder-server:masterfrom harness/INT-6791-fix-harden-warehouse-sql-identi-362a1da7rudderlabs/rudder-server:harness/INT-6791-fix-harden-warehouse-sql-identi-362a1da7Copy head branch name to clipboard
Conversation
|
👋 I'm rudder-devbox. I opened this PR from a Linear ticket. Mention |
|
@rudder-devbox With the recent changes, can we update the postgres integration as well. |
|
Updated the Postgres integration as well. The Postgres SQL-building paths now use the shared warehouse identifier helpers instead of direct Validation:
Commit: e47314f |
|
I fixed the CI failure and pushed the change. Root cause from the Snowflake integration log: Fix: quoted the Snowflake users MERGE target/source table identifiers with Validation passed:
Commit: ca04eb1 |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #7187 +/- ##
==========================================
- Coverage 80.13% 80.07% -0.07%
==========================================
Files 595 596 +1
Lines 66151 66442 +291
==========================================
+ Hits 53011 53201 +190
- Misses 10021 10096 +75
- Partials 3119 3145 +26 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@achettyiitr shouldn't there be any updates in each warehouse test cases ? |
🔒 Scanned for secrets using gitleaks 8.29.1
Nothing changes per integrations; currently, in some places, we were not using quoted identifiers, so we added those. With or without quote identifiers, SQL would work. Everything is covered in integration tests. |
|
Remaining injection vector not covered by this pass — return fmt.Sprintf("LOCATION '%s/%s/%s'", externalLocation, d.Namespace, tableName)
Low severity (namespace is config-controlled and it is a storage-path literal rather than DDL), but it is a genuine breakout path. (Posted as a top-level comment because line 571 is outside this PR's diff, so GitHub could not anchor it inline.) |
🔒 Scanned for secrets using gitleaks 8.29.1
d2314fb to
c99e78f
Compare
Integration Tests are covering only the happy path and proving current changes is not causing a regression. But we need negative test cases with malicious SQL's for proving that SQL has been hardened. In the previous Postgres PR, unit tests did that - so good to add for all warehouses too ? |
|
Sandbox became unreachable (its file API was unavailable too long). |
Description
Summary
warehouse/utils: dialect quoters (double-quote, backtick, bracket), qualified-name and comma-separated joiners, a composite-key splitter (QuoteCommaSeparatedIdentifiers), and a string-literal escaper (SQLStringLiteral). Every quoter escapes by doubling its delimiter —"→"",`→``,]→]],'→''.%qinterpolation of schema, table, column, and staging-table names.TestLoadTable, and BigQuery view/delete SQL.row_id, column_name, table_name) into individually quoted columns instead of quoting the whole string as one identifier.schemaExistswith bind parameters.Linear Ticket
INT-6791
Security