…STSEC-2026-0001` (#19666) (#19686)

## Which issue does this PR close?

- part of #18566

## Rationale for this change

Let's resolve the audit workflow on branch-52 by removing our dependency
on the `rust_decimal` library, per @xudong963 's request here
#19666 (review)

## What changes are included in this PR?

- Backport  #19666 to branch-52

## Are these changes tested?

By CI

## Are there any user-facing changes?
No, this is an internal dev tool change

… constructors (#19681)

Backport of #19668 to branch-52.

This PR cherry-picks commit 680ddcc from main.

Includes:
- Split of BatchPartitioner::try_new into hash and round-robin
constructors
- Documentation improvements
- No behavior changes


part of #18566

Co-authored-by: Your Name <youremail@example.com>

…sCache table scoped (#19704)

## Which issue does this PR close?

- part of #18566

## Rationale for this change

Backport the fix for this regression into 52 release branch:
-  #19573 

## What changes are included in this PR?

Backport these two commits to `branch-52` (cherry-pick was clean)
- 1037f0a / #19388
- e6049de / #19616

<details><summary>Commands</summary>
<p>

```shell
andrewlamb@Andrews-MacBook-Pro-3:~/Software/datafusion$ git cherry-pick 1037f0a
[branch-52 1fc70ac] feat: add list_files_cache table function for `datafusion-cli` (#19388)
 Author: jizezhang <jizez@uw.edu>
 Date: Tue Jan 6 05:23:39 2026 -0800
 5 files changed, 446 insertions(+), 31 deletions(-)
andrewlamb@Andrews-MacBook-Pro-3:~/Software/datafusion$ git cherry-pick  e6049de
Auto-merging datafusion/core/src/execution/context/mod.rs
[branch-52 aa3d413] Make default ListingFilesCache table scoped (#19616)
 Author: jizezhang <jizez@uw.edu>
 Date: Thu Jan 8 06:34:10 2026 -0800
 10 files changed, 474 insertions(+), 184 deletions(-)
```

</p>
</details> 

## Are these changes tested?

By CI and new tests

## Are there any user-facing changes?

A new datafusion-cli function and dropping a external table now clears
the listing cache

---------

Co-authored-by: jizezhang <jizez@uw.edu>

…19659) (#19705)

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- part of #18566

## Rationale for this change

I propose back porting the fix for
#19641 to 52 release

## What changes are included in this PR?

- Backport #19659

## Are these changes tested?
eYes

## Are there any user-facing changes?

bug fix

Co-authored-by: Kumar Ujjawal <ujjawalpathak6@gmail.com>

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Part of #18566

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- part of #19784.

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

This is a follow-up of #19573

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

Backport 
- #19804

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

Existing tests

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
no
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

…itedBatchCoalescer (#19785) (#19836)

## Which issue does this PR close?

- part of #19784 
- related to #19785
- backport fix of #19781

## Rationale for this change

Backport a regression found by @bert-beyondloops into the 52 release
line

## What changes are included in this PR?
Backport fix for #19781,
#19785 to branch-52

## Are these changes tested?

Yes
## Are there any user-facing changes?

bug fix

Co-authored-by: Bert Vermeiren <103956021+bert-beyondloops@users.noreply.github.com>
Co-authored-by: Bert Vermeiren <bert.vermeiren@datadobi.com>

Brings #19853 into `branch-52`

Co-authored-by: Pierre Lacave <pierre.lacave@datadoghq.com>

…19877)

- part of #19784

- Brings #19716 from @adriangb
into `branch-52`

Co-authored-by: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com>

## Which issue does this PR close?

- part of #19784
## Rationale for this change

Prepare for release

## What changes are included in this PR?

Update release version to 52.1.0 and add changelog

## Are these changes tested?

By CI
## Are there any user-facing changes?

New version

…20019)

DataFusion's current `AggregateMode` enum has four variants covering
three of the four cells in the input/output matrix:

| | Input: raw data          | Input: partial state |
| - | - | - |
| Output: final values | `Single` / `SinglePartitioned` | `Final` /
`FinalPartitioned` |
| Output: partial state | `Partial`                   | ??? |

This PR adds `AggregateMode::PartialReduce` to fill in the missing cell:
it takes partially-reduced values as input, and reduces them further,
but without finalizing.

This is useful because it's the key component needed to implement
distributed tree-reduction (as seen in e.g. the Scuba or Honeycomb
papers): a set of worker nodes each perform multithreaded `Partial`
aggregations, feed those into a `PartialReduce` to reduce all of this
node's values into a single row, and then a head node collects the
outputs from all nodes' `PartialReduce` to feed into a `Final`
reduction.

PR can be reviewed commit by commit: first commit is pure
refactor/simplification; most places we were matching on `AggregateMode`
we were actually just trying to either check which row of the above
table we were in, or else which column. So now we have `is_first_stage`
(tells you which column) and `is_last_stage` (tells you which row) and
we use them everywhere.

Second commit adds `PartialReduce`, and is pretty small because
`is_first_stage`/`is_last_stage` do most of the heavy lifting. It also
adds a test demonstrating a minimal Partial -> PartialReduce -> Final
tree-reduction.

)

remap RecordBatches and Schemas

- This causes problems with delta tables, where there are fields marked non-null in the delta schema, but null in the parquet file schema