Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

[libc++] Refactor and add benchmark coverage for [alg.sort] #128236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 14, 2025

Conversation

ldionne
Copy link
Member

@ldionne ldionne commented Feb 21, 2025

This patch adds missing benchmark coverage for partial_sort, partial_sort_copy, is_sorted and is_sorted_until.

It also refactors the existing benchmarks for sort and stable_sort to follow the consistent style of the new algorithm benchmarks. However, these benchmarks were notoriously slow to run since they tested multiple data patterns on multiple data types. To try to alleviate this, I reduced the benchmarks to only run on integral types and on a single non-integral type, which should faithfully show how the algorithm behaves for anything non-integral. However, this is technically a reduction in coverage.

@ldionne ldionne requested a review from a team as a code owner February 21, 2025 21:29
@llvmbot llvmbot added the libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. label Feb 21, 2025
@llvmbot
Copy link
Member

llvmbot commented Feb 21, 2025

@llvm/pr-subscribers-libcxx

Author: Louis Dionne (ldionne)

Changes

This patch adds missing benchmark coverage for partial_sort, partial_sort_copy, is_sorted and is_sorted_until.

It also refactors the existing benchmarks for sort and stable_sort to follow the consistent style of the new algorithm benchmarks. However, these benchmarks were notoriously slow to run since they tested multiple data patterns on multiple data types. To try to alleviate this, I reduced the benchmarks to only run on integral types and on a single non-integral type, which should faithfully show how the algorithm behaves for anything non-integral. However, this is technically a reduction in coverage.


Patch is 40.08 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/128236.diff

12 Files Affected:

  • (removed) libcxx/test/benchmarks/algorithms/pstl.stable_sort.bench.cpp (-42)
  • (removed) libcxx/test/benchmarks/algorithms/ranges_sort.bench.cpp (-40)
  • (removed) libcxx/test/benchmarks/algorithms/ranges_stable_sort.bench.cpp (-40)
  • (removed) libcxx/test/benchmarks/algorithms/sort.bench.cpp (-38)
  • (added) libcxx/test/benchmarks/algorithms/sorting/common.h (+141)
  • (added) libcxx/test/benchmarks/algorithms/sorting/is_sorted.bench.cpp (+81)
  • (added) libcxx/test/benchmarks/algorithms/sorting/is_sorted_until.bench.cpp (+81)
  • (added) libcxx/test/benchmarks/algorithms/sorting/partial_sort.bench.cpp (+95)
  • (added) libcxx/test/benchmarks/algorithms/sorting/partial_sort_copy.bench.cpp (+90)
  • (added) libcxx/test/benchmarks/algorithms/sorting/sort.bench.cpp (+92)
  • (added) libcxx/test/benchmarks/algorithms/sorting/stable_sort.bench.cpp (+93)
  • (removed) libcxx/test/benchmarks/algorithms/stable_sort.bench.cpp (-40)
diff --git a/libcxx/test/benchmarks/algorithms/pstl.stable_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/pstl.stable_sort.bench.cpp
deleted file mode 100644
index a385185ec7fe5..0000000000000
--- a/libcxx/test/benchmarks/algorithms/pstl.stable_sort.bench.cpp
+++ /dev/null
@@ -1,42 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// UNSUPPORTED: c++03, c++11, c++14, c++17
-// UNSUPPORTED: libcpp-has-no-incomplete-pstl
-
-#include <algorithm>
-#include <execution>
-
-#include "common.h"
-
-namespace {
-template <class ValueType, class Order>
-struct StableSort {
-  size_t Quantity;
-
-  void run(benchmark::State& state) const {
-    runOpOnCopies<ValueType>(state, Quantity, Order(), BatchSize::CountBatch, [](auto& Copy) {
-      std::stable_sort(std::execution::par, Copy.begin(), Copy.end());
-    });
-  }
-
-  bool skip() const { return Order() == ::Order::Heap; }
-
-  std::string name() const {
-    return "BM_pstl_stable_sort" + ValueType::name() + Order::name() + "/" + std::to_string(Quantity);
-  }
-};
-} // namespace
-
-int main(int argc, char** argv) {
-  benchmark::Initialize(&argc, argv);
-  if (benchmark::ReportUnrecognizedArguments(argc, argv))
-    return 1;
-  makeCartesianProductBenchmark<StableSort, AllValueTypes, AllOrders>(Quantities);
-  benchmark::RunSpecifiedBenchmarks();
-}
diff --git a/libcxx/test/benchmarks/algorithms/ranges_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_sort.bench.cpp
deleted file mode 100644
index d145a159a21fd..0000000000000
--- a/libcxx/test/benchmarks/algorithms/ranges_sort.bench.cpp
+++ /dev/null
@@ -1,40 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// UNSUPPORTED: c++03, c++11, c++14, c++17
-
-#include <algorithm>
-
-#include "common.h"
-
-namespace {
-template <class ValueType, class Order>
-struct Sort {
-  size_t Quantity;
-
-  void run(benchmark::State& state) const {
-    runOpOnCopies<ValueType>(state, Quantity, Order(), BatchSize::CountElements, [](auto& Copy) {
-      std::ranges::sort(Copy);
-    });
-  }
-
-  bool skip() const { return Order() == ::Order::Heap; }
-
-  std::string name() const {
-    return "BM_RangesSort" + ValueType::name() + Order::name() + "_" + std::to_string(Quantity);
-  }
-};
-} // namespace
-
-int main(int argc, char** argv) {
-  benchmark::Initialize(&argc, argv);
-  if (benchmark::ReportUnrecognizedArguments(argc, argv))
-    return 1;
-  makeCartesianProductBenchmark<Sort, AllValueTypes, AllOrders>(Quantities);
-  benchmark::RunSpecifiedBenchmarks();
-}
diff --git a/libcxx/test/benchmarks/algorithms/ranges_stable_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_stable_sort.bench.cpp
deleted file mode 100644
index acc2f3f755fb8..0000000000000
--- a/libcxx/test/benchmarks/algorithms/ranges_stable_sort.bench.cpp
+++ /dev/null
@@ -1,40 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// UNSUPPORTED: c++03, c++11, c++14, c++17
-
-#include <algorithm>
-
-#include "common.h"
-
-namespace {
-template <class ValueType, class Order>
-struct StableSort {
-  size_t Quantity;
-
-  void run(benchmark::State& state) const {
-    runOpOnCopies<ValueType>(state, Quantity, Order(), BatchSize::CountElements, [](auto& Copy) {
-      std::ranges::stable_sort(Copy);
-    });
-  }
-
-  bool skip() const { return Order() == ::Order::Heap; }
-
-  std::string name() const {
-    return "BM_RangesStableSort" + ValueType::name() + Order::name() + "_" + std::to_string(Quantity);
-  }
-};
-} // namespace
-
-int main(int argc, char** argv) {
-  benchmark::Initialize(&argc, argv);
-  if (benchmark::ReportUnrecognizedArguments(argc, argv))
-    return 1;
-  makeCartesianProductBenchmark<StableSort, AllValueTypes, AllOrders>(Quantities);
-  benchmark::RunSpecifiedBenchmarks();
-}
diff --git a/libcxx/test/benchmarks/algorithms/sort.bench.cpp b/libcxx/test/benchmarks/algorithms/sort.bench.cpp
deleted file mode 100644
index 7f3ce6ff7a07e..0000000000000
--- a/libcxx/test/benchmarks/algorithms/sort.bench.cpp
+++ /dev/null
@@ -1,38 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// UNSUPPORTED: c++03, c++11, c++14, c++17
-
-#include <algorithm>
-
-#include "common.h"
-
-namespace {
-template <class ValueType, class Order>
-struct Sort {
-  size_t Quantity;
-
-  void run(benchmark::State& state) const {
-    runOpOnCopies<ValueType>(state, Quantity, Order(), BatchSize::CountElements, [](auto& Copy) {
-      std::sort(Copy.begin(), Copy.end());
-    });
-  }
-
-  bool skip() const { return Order() == ::Order::Heap; }
-
-  std::string name() const { return "BM_Sort" + ValueType::name() + Order::name() + "_" + std::to_string(Quantity); };
-};
-} // namespace
-
-int main(int argc, char** argv) {
-  benchmark::Initialize(&argc, argv);
-  if (benchmark::ReportUnrecognizedArguments(argc, argv))
-    return 1;
-  makeCartesianProductBenchmark<Sort, AllValueTypes, AllOrders>(Quantities);
-  benchmark::RunSpecifiedBenchmarks();
-}
diff --git a/libcxx/test/benchmarks/algorithms/sorting/common.h b/libcxx/test/benchmarks/algorithms/sorting/common.h
new file mode 100644
index 0000000000000..8195e9a2dc8d0
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/sorting/common.h
@@ -0,0 +1,141 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LIBCXX_TEST_BENCHMARKS_ALGORITHMS_SORTING_COMMON_H
+#define LIBCXX_TEST_BENCHMARKS_ALGORITHMS_SORTING_COMMON_H
+
+#include <algorithm>
+#include <cstddef>
+#include <numeric>
+#include <random>
+#include <type_traits>
+#include <vector>
+
+namespace support {
+
+// This function creates a vector with N int-like values.
+//
+// These values are arranged in such a way that they would invoke O(N^2)
+// behavior on any quick sort implementation that satisifies certain conditions.
+// Details are available in the following paper:
+//
+//  "A Killer Adversary for Quicksort", M. D. McIlroy, Software-Practice &
+//  Experience Volume 29 Issue 4 April 10, 1999 pp 341-344.
+//  https://dl.acm.org/doi/10.5555/311868.311871.
+template <class T>
+std::vector<T> quicksort_adversarial_data(std::size_t n) {
+  static_assert(std::is_integral_v<T>);
+  assert(n > 0);
+
+  // If an element is equal to gas, it indicates that the value of the element
+  // is still to be decided and may change over the course of time.
+  T gas = n - 1;
+
+  std::vector<T> v;
+  v.resize(n);
+  for (unsigned int i = 0; i < n; ++i) {
+    v[i] = gas;
+  }
+  // Candidate for the pivot position.
+  int candidate = 0;
+  int nsolid    = 0;
+  // Populate all positions in the generated input to gas.
+  std::vector<int> ascending_values(v.size());
+
+  // Fill up with ascending values from 0 to v.size()-1.  These will act as
+  // indices into v.
+  std::iota(ascending_values.begin(), ascending_values.end(), 0);
+  std::sort(ascending_values.begin(), ascending_values.end(), [&](int x, int y) {
+    if (v[x] == gas && v[y] == gas) {
+      // We are comparing two inputs whose value is still to be decided.
+      if (x == candidate) {
+        v[x] = nsolid++;
+      } else {
+        v[y] = nsolid++;
+      }
+    }
+    if (v[x] == gas) {
+      candidate = x;
+    } else if (v[y] == gas) {
+      candidate = y;
+    }
+    return v[x] < v[y];
+  });
+  return v;
+}
+
+// ascending sorted values
+template <class T>
+std::vector<T> ascending_sorted_data(std::size_t n) {
+  std::vector<T> v(n);
+  std::iota(v.begin(), v.end(), 0);
+  return v;
+}
+
+// descending sorted values
+template <class T>
+std::vector<T> descending_sorted_data(std::size_t n) {
+  std::vector<T> v(n);
+  std::iota(v.begin(), v.end(), 0);
+  std::reverse(v.begin(), v.end());
+  return v;
+}
+
+// pipe-organ pattern
+template <class T>
+std::vector<T> pipe_organ_data(std::size_t n) {
+  std::vector<T> v(n);
+  std::iota(v.begin(), v.end(), 0);
+  auto half = v.begin() + v.size() / 2;
+  std::reverse(half, v.end());
+  return v;
+}
+
+// heap pattern
+template <class T>
+std::vector<T> heap_data(std::size_t n) {
+  std::vector<T> v(n);
+  std::iota(v.begin(), v.end(), 0);
+  std::make_heap(v.begin(), v.end());
+  return v;
+}
+
+// shuffled randomly
+template <class T>
+std::vector<T> shuffled_data(std::size_t n) {
+  std::vector<T> v(n);
+  std::iota(v.begin(), v.end(), 0);
+  std::mt19937 rng;
+  std::shuffle(v.begin(), v.end(), rng);
+  return v;
+}
+
+// single element in the whole sequence
+template <class T>
+std::vector<T> single_element_data(std::size_t n) {
+  std::vector<T> v(n);
+  return v;
+}
+
+struct NonIntegral {
+  NonIntegral() : value_(0) {}
+  NonIntegral(int i) : value_(i) {}
+  friend auto operator<(NonIntegral const& a, NonIntegral const& b) { return a.value_ < b.value_; }
+  friend auto operator>(NonIntegral const& a, NonIntegral const& b) { return a.value_ > b.value_; }
+  friend auto operator<=(NonIntegral const& a, NonIntegral const& b) { return a.value_ <= b.value_; }
+  friend auto operator>=(NonIntegral const& a, NonIntegral const& b) { return a.value_ >= b.value_; }
+  friend auto operator==(NonIntegral const& a, NonIntegral const& b) { return a.value_ == b.value_; }
+  friend auto operator!=(NonIntegral const& a, NonIntegral const& b) { return a.value_ != b.value_; }
+
+private:
+  int value_;
+};
+
+} // namespace support
+
+#endif // LIBCXX_TEST_BENCHMARKS_ALGORITHMS_SORTING_COMMON_H
diff --git a/libcxx/test/benchmarks/algorithms/sorting/is_sorted.bench.cpp b/libcxx/test/benchmarks/algorithms/sorting/is_sorted.bench.cpp
new file mode 100644
index 0000000000000..cb07d87cbd2c8
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/sorting/is_sorted.bench.cpp
@@ -0,0 +1,81 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+
+int main(int argc, char** argv) {
+  auto std_is_sorted      = [](auto first, auto last) { return std::is_sorted(first, last); };
+  auto std_is_sorted_pred = [](auto first, auto last) {
+    return std::is_sorted(first, last, [](auto x, auto y) {
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      return x < y;
+    });
+  };
+  auto ranges_is_sorted_pred = [](auto first, auto last) {
+    return std::ranges::is_sorted(first, last, [](auto x, auto y) {
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      return x < y;
+    });
+  };
+
+  // Benchmark {std,ranges}::is_sorted on a sorted sequence (the worst case).
+  {
+    auto bm = []<class Container>(std::string name, auto is_sorted) {
+      benchmark::RegisterBenchmark(
+          name,
+          [is_sorted](auto& st) {
+            std::size_t const size = st.range(0);
+            using ValueType        = typename Container::value_type;
+            std::vector<ValueType> data;
+            std::generate_n(std::back_inserter(data), size, [] { return Generate<ValueType>::random(); });
+            std::sort(data.begin(), data.end());
+
+            Container c(data.begin(), data.end());
+
+            for ([[maybe_unused]] auto _ : st) {
+              benchmark::DoNotOptimize(c);
+              auto result = is_sorted(c.begin(), c.end());
+              benchmark::DoNotOptimize(result);
+            }
+          })
+          ->Arg(8)
+          ->Arg(1024)
+          ->Arg(8192);
+    };
+    bm.operator()<std::vector<int>>("std::is_sorted(vector<int>)", std_is_sorted);
+    bm.operator()<std::deque<int>>("std::is_sorted(deque<int>)", std_is_sorted);
+    bm.operator()<std::list<int>>("std::is_sorted(list<int>)", std_is_sorted);
+    bm.operator()<std::vector<int>>("rng::is_sorted(vector<int>)", std::ranges::is_sorted);
+    bm.operator()<std::deque<int>>("rng::is_sorted(deque<int>)", std::ranges::is_sorted);
+    bm.operator()<std::list<int>>("rng::is_sorted(list<int>)", std::ranges::is_sorted);
+
+    bm.operator()<std::vector<int>>("std::is_sorted(vector<int>, pred)", std_is_sorted_pred);
+    bm.operator()<std::deque<int>>("std::is_sorted(deque<int>, pred)", std_is_sorted_pred);
+    bm.operator()<std::list<int>>("std::is_sorted(list<int>, pred)", std_is_sorted_pred);
+    bm.operator()<std::vector<int>>("rng::is_sorted(vector<int>, pred)", ranges_is_sorted_pred);
+    bm.operator()<std::deque<int>>("rng::is_sorted(deque<int>, pred)", ranges_is_sorted_pred);
+    bm.operator()<std::list<int>>("rng::is_sorted(list<int>, pred)", ranges_is_sorted_pred);
+  }
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/sorting/is_sorted_until.bench.cpp b/libcxx/test/benchmarks/algorithms/sorting/is_sorted_until.bench.cpp
new file mode 100644
index 0000000000000..1b96c36c66a1a
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/sorting/is_sorted_until.bench.cpp
@@ -0,0 +1,81 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <cstddef>
+#include <deque>
+#include <list>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "../../GenerateInput.h"
+
+int main(int argc, char** argv) {
+  auto std_is_sorted_until      = [](auto first, auto last) { return std::is_sorted_until(first, last); };
+  auto std_is_sorted_until_pred = [](auto first, auto last) {
+    return std::is_sorted_until(first, last, [](auto x, auto y) {
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      return x < y;
+    });
+  };
+  auto ranges_is_sorted_until_pred = [](auto first, auto last) {
+    return std::ranges::is_sorted_until(first, last, [](auto x, auto y) {
+      benchmark::DoNotOptimize(x);
+      benchmark::DoNotOptimize(y);
+      return x < y;
+    });
+  };
+
+  // Benchmark {std,ranges}::is_sorted_until on a sorted sequence (the worst case).
+  {
+    auto bm = []<class Container>(std::string name, auto is_sorted_until) {
+      benchmark::RegisterBenchmark(
+          name,
+          [is_sorted_until](auto& st) {
+            std::size_t const size = st.range(0);
+            using ValueType        = typename Container::value_type;
+            std::vector<ValueType> data;
+            std::generate_n(std::back_inserter(data), size, [] { return Generate<ValueType>::random(); });
+            std::sort(data.begin(), data.end());
+
+            Container c(data.begin(), data.end());
+
+            for ([[maybe_unused]] auto _ : st) {
+              benchmark::DoNotOptimize(c);
+              auto result = is_sorted_until(c.begin(), c.end());
+              benchmark::DoNotOptimize(result);
+            }
+          })
+          ->Arg(8)
+          ->Arg(1024)
+          ->Arg(8192);
+    };
+    bm.operator()<std::vector<int>>("std::is_sorted_until(vector<int>)", std_is_sorted_until);
+    bm.operator()<std::deque<int>>("std::is_sorted_until(deque<int>)", std_is_sorted_until);
+    bm.operator()<std::list<int>>("std::is_sorted_until(list<int>)", std_is_sorted_until);
+    bm.operator()<std::vector<int>>("rng::is_sorted_until(vector<int>)", std::ranges::is_sorted_until);
+    bm.operator()<std::deque<int>>("rng::is_sorted_until(deque<int>)", std::ranges::is_sorted_until);
+    bm.operator()<std::list<int>>("rng::is_sorted_until(list<int>)", std::ranges::is_sorted_until);
+
+    bm.operator()<std::vector<int>>("std::is_sorted_until(vector<int>, pred)", std_is_sorted_until_pred);
+    bm.operator()<std::deque<int>>("std::is_sorted_until(deque<int>, pred)", std_is_sorted_until_pred);
+    bm.operator()<std::list<int>>("std::is_sorted_until(list<int>, pred)", std_is_sorted_until_pred);
+    bm.operator()<std::vector<int>>("rng::is_sorted_until(vector<int>, pred)", ranges_is_sorted_until_pred);
+    bm.operator()<std::deque<int>>("rng::is_sorted_until(deque<int>, pred)", ranges_is_sorted_until_pred);
+    bm.operator()<std::list<int>>("rng::is_sorted_until(list<int>, pred)", ranges_is_sorted_until_pred);
+  }
+
+  benchmark::Initialize(&argc, argv);
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+  return 0;
+}
diff --git a/libcxx/test/benchmarks/algorithms/sorting/partial_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/sorting/partial_sort.bench.cpp
new file mode 100644
index 0000000000000..aa25ad25043ac
--- /dev/null
+++ b/libcxx/test/benchmarks/algorithms/sorting/partial_sort.bench.cpp
@@ -0,0 +1,95 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
+#include <algorithm>
+#include <array>
+#include <cstddef>
+#include <deque>
+#include <string>
+#include <vector>
+
+#include "benchmark/benchmark.h"
+#include "common.h"
+
+int main(int argc, char** argv) {
+  auto std_partial_sort = [](auto first, auto mid, auto last) { return std::partial_sort(first, mid, last); };
+
+  // Benchmark {std,ranges}::partial_sort on various types of data. We always partially sort only
+  // half of the full range.
+  //
+  // We perform this benchmark in a batch because we need to restore the
+  // state of the container after the operation.
+  //
+  // Also note that we intentionally don't benchmark the predicated version of the algorithm
+  // because that makes the benchmark run too slowly.
+  {
+    auto bm = []<class Container>(std::string name, auto partial_sort, auto generate_data) {
+      benchmark::RegisterBenchmark(
+          name,
+          [partial_sort, generate_data](auto& st) {
+            std::size_t const size          = st.range(0);
+            constexpr std::size_t BatchSize = 32;
+            using ValueType                 = typename Container::value_type;
+            std::vector<ValueType> data     = generate_data(size);
+            std::array<Container, BatchSize> c;
+            std::fill_n(c.begin(), BatchSize, Container(data.begin(), data.end()));
+
+ ...
[truncated]

@ldionne ldionne force-pushed the review/benchmarks-add-sorting branch from 86fbaa8 to d71cb07 Compare March 20, 2025 19:00
ldionne added 3 commits May 7, 2025 12:08
This patch adds missing benchmark coverage for partial_sort,
partial_sort_copy, is_sorted and is_sorted_until.

It also refactors the existing benchmarks for sort and stable_sort
to follow the consistent style of the new algorithm benchmarks.
However, these benchmarks were notoriously slow to run since they
tested multiple data patterns on multiple data types. To try to
alleviate this, I reduced the benchmarks to only run on integral
types and on a single non-integral type, which should faithfully
show how the algorithm behaves for anything non-integral. However,
this is technically a reduction in coverage.
@ldionne ldionne force-pushed the review/benchmarks-add-sorting branch from d71cb07 to 7d797b8 Compare May 7, 2025 16:09
@ldionne ldionne merged commit 8c43588 into llvm:main May 14, 2025
84 checks passed
@ldionne ldionne deleted the review/benchmarks-add-sorting branch May 14, 2025 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.