Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 21de1f7

Browse filesBrowse files
spavloffspavloff
committed
Minimal support of floating-point operand bundles
This is a lite version of #109798, where code changes are minimized to facilitate discussion about the implementation. The motivations and ideas behind the new floating-point operation support are described in that PR and in the discussion https://discourse.llvm.org/t/rfc-change-of-strict-fp-operation-representation-in-ir/85021. There are concerns that the proposed changes are too invasive and a new approach is required to make the transition smoother. This implementation is essentially a subset of PR109798, where everything beyond the minimum is removed. It tries to build eventually the same implementation as that PR but in different steps. The patch does not attempt to modify the existing implementation based on the constrained intrinsics. Instead it introduces a new one using operand bundles. This new implementation initially has very limited functionality, which latter will be extended and finally can replace the existing one. This PR introduces the notion of floating-point operation, this is an intrinsic, that is listed in the file "FloatingPointOps.def". These have two additional properties: 1. In the strict environment (a function with strictfp attribute) calls to these operations acquire side effect, now it is read/write access to inaccessible memory, just as constrained intrinsics do. 2. Calls to these operations may have floating-point operand bundles. There are two kinds of such bundles, tagged with "fp.control" and "fp.except", which are used to carry additional information about control modes and exception handling. Initially the set of control modes consists of rounding mode only. The set of operations enlisted in "FloatingPointOps.def" and in "ConstrainedOps.def" are completely independent, an intrinsic may be in one list or in both. The set of floating-point operations is expected to grow and finally all FP intrinsics will be available in the new implementation. In this patch set of intrinsics in "FloatingPointOps.def" is minimum necessary for tests.
1 parent 009971a commit 21de1f7
Copy full SHA for 21de1f7
Expand file treeCollapse file tree

18 files changed

+716
-21
lines changed

‎llvm/docs/LangRef.rst

Copy file name to clipboardExpand all lines: llvm/docs/LangRef.rst
+48-3Lines changed: 48 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3071,6 +3071,51 @@ A "convergencectrl" operand bundle is only valid on a ``convergent`` operation.
30713071
When present, the operand bundle must contain exactly one value of token type.
30723072
See the :doc:`ConvergentOperations` document for details.
30733073

3074+
.. _ob_fp:
3075+
3076+
Floating-point Operand Bundles
3077+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3078+
3079+
These operand bundles are used for calls that involve floating-point
3080+
operations and interact with :ref:`floating-point environment <floatenv>` or
3081+
depend on floating-point options, such as rounding mode, denormal modes, etc.
3082+
There are two kinds of such operand bundles, which represent the value of
3083+
floating-point control modes and the treatment of status bits respectively.
3084+
3085+
An operand bundle tagged with "fp.control" contains information about the
3086+
control modes used for the operation execution. Operands specified in this
3087+
bundle represent particular options. Currently, only rounding mode is supported.
3088+
It is represented by a metadata string value, which specifies the rounding mode
3089+
to be used for the operation evaluation. Possible values are:
3090+
3091+
::
3092+
3093+
"rtz" - toward zero
3094+
"rte" - to nearest, ties to even
3095+
"rtp" - toward positive infinity
3096+
"rtn" - toward negative infinity
3097+
"rmm" - to nearest, ties away from zero
3098+
"dyn" - rounding mode is taken from control register
3099+
3100+
If "fp.control" is absent, the default rounding rounding mode is taken from the
3101+
control register (dynamic rounding). In the particular case of
3102+
:ref:`default floating-point environment <floatenv>`, it must be rounding to
3103+
nearest, ties to even.
3104+
3105+
An operand bundle tagged with "fp.except" may be associated with operations
3106+
that can read or write floating-point exception flags. It contains a single
3107+
metadata string value, which can have one of the following values:
3108+
3109+
::
3110+
3111+
"ignore"
3112+
"strict"
3113+
"maytrap"
3114+
3115+
It has the same meaning as the corresponding argument in
3116+
:ref:`constrained intrinsics <constrainedfp>`.
3117+
3118+
30743119
.. _moduleasm:
30753120

30763121
Module-Level Inline Assembly
@@ -3776,9 +3821,9 @@ round-to-nearest rounding mode, and subnormals are assumed to be preserved.
37763821
Running LLVM code in an environment where these assumptions are not met
37773822
typically leads to undefined behavior. The ``strictfp`` and ``denormal-fp-math``
37783823
attributes as well as :ref:`Constrained Floating-Point Intrinsics
3779-
<constrainedfp>` can be used to weaken LLVM's assumptions and ensure defined
3780-
behavior in non-default floating-point environments; see their respective
3781-
documentation for details.
3824+
<constrainedfp>` or :ref:`floating-point operand bundles<ob_fp>` can be used to
3825+
weaken LLVM's assumptions and ensure defined behavior in non-default
3826+
floating-point environments; see their respective documentation for details.
37823827

37833828
.. _floatnan:
37843829

‎llvm/docs/ReleaseNotes.md

Copy file name to clipboardExpand all lines: llvm/docs/ReleaseNotes.md
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@ Changes to the LLVM IR
6464

6565
* Updated semantics of `llvm.type.checked.load.relative` to match that of
6666
`llvm.load.relative`.
67+
* Floating-point operand bundles have been added.
6768

6869
Changes to LLVM infrastructure
6970
------------------------------

‎llvm/include/llvm/IR/FPEnv.h

Copy file name to clipboardExpand all lines: llvm/include/llvm/IR/FPEnv.h
+18Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,18 +48,36 @@ enum ExceptionBehavior : uint8_t {
4848
/// metadata.
4949
std::optional<RoundingMode> convertStrToRoundingMode(StringRef);
5050

51+
/// Returns a valid RoundingMode enumerator given a string that is used as
52+
/// rounding mode specifier in operand bundles.
53+
std::optional<RoundingMode> convertBundleToRoundingMode(StringRef);
54+
5155
/// For any RoundingMode enumerator, returns a string valid as input in
5256
/// constrained intrinsic rounding mode metadata.
5357
std::optional<StringRef> convertRoundingModeToStr(RoundingMode);
5458

59+
/// For any RoundingMode enumerator, returns a string to be used in operand
60+
/// bundles.
61+
std::optional<StringRef> convertRoundingModeToBundle(RoundingMode);
62+
5563
/// Returns a valid ExceptionBehavior enumerator when given a string
5664
/// valid as input in constrained intrinsic exception behavior metadata.
5765
std::optional<fp::ExceptionBehavior> convertStrToExceptionBehavior(StringRef);
5866

67+
/// Returns a valid ExceptionBehavior enumerator given a string from the operand
68+
/// bundle argument.
69+
std::optional<fp::ExceptionBehavior>
70+
convertBundleToExceptionBehavior(StringRef);
71+
5972
/// For any ExceptionBehavior enumerator, returns a string valid as
6073
/// input in constrained intrinsic exception behavior metadata.
6174
std::optional<StringRef> convertExceptionBehaviorToStr(fp::ExceptionBehavior);
6275

76+
/// Return string representing the given exception behavior for use in operand
77+
/// bundles
78+
std::optional<StringRef>
79+
convertExceptionBehaviorToBundle(fp::ExceptionBehavior);
80+
6381
/// Returns true if the exception handling behavior and rounding mode
6482
/// match what is used in the default floating point environment.
6583
inline bool isDefaultFPEnvironment(fp::ExceptionBehavior EB, RoundingMode RM) {

‎llvm/include/llvm/IR/FloatingPointOps.def

Copy file name to clipboard
+24Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
//===- llvm/IR/FloatingPointOps.def - FP intrinsics -------------*- C++ -*-===//
2+
//
3+
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
4+
// See https://llvm.org/LICENSE.txt for license information.
5+
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
6+
//
7+
//===----------------------------------------------------------------------===//
8+
//
9+
// Defines set of intrinsics, which are classified as floating-point operations.
10+
//
11+
//===----------------------------------------------------------------------===//
12+
13+
#ifndef FUNCTION
14+
#define FUNCTION(N,D)
15+
#endif
16+
17+
// Arguments of the entries are:
18+
// - intrinsic function name,
19+
// - DAG node corresponding to the intrinsic.
20+
21+
FUNCTION(nearbyint, FNEARBYINT)
22+
FUNCTION(trunc, FTRUNC)
23+
24+
#undef FUNCTION

‎llvm/include/llvm/IR/IRBuilder.h

Copy file name to clipboardExpand all lines: llvm/include/llvm/IR/IRBuilder.h
+31-14Lines changed: 31 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1004,6 +1004,16 @@ class IRBuilderBase {
10041004
ArrayRef<Value *> Args, FMFSource FMFSource = {},
10051005
const Twine &Name = "");
10061006

1007+
/// Create a call to intrinsic \p ID with \p Args, mangled using \p Types and
1008+
/// with operand bundles.
1009+
/// If \p FMFSource is provided, copy fast-math-flags from that instruction to
1010+
/// the intrinsic.
1011+
CallInst *CreateIntrinsic(Intrinsic::ID ID, ArrayRef<Type *> Types,
1012+
ArrayRef<Value *> Args,
1013+
ArrayRef<OperandBundleDef> OpBundles,
1014+
Instruction *FMFSource = nullptr,
1015+
const Twine &Name = "");
1016+
10071017
/// Create a call to non-overloaded intrinsic \p ID with \p Args. If
10081018
/// \p FMFSource is provided, copy fast-math-flags from that instruction to
10091019
/// the intrinsic.
@@ -2492,24 +2502,13 @@ class IRBuilderBase {
24922502
CallInst *CreateCall(FunctionType *FTy, Value *Callee,
24932503
ArrayRef<Value *> Args = {}, const Twine &Name = "",
24942504
MDNode *FPMathTag = nullptr) {
2495-
CallInst *CI = CallInst::Create(FTy, Callee, Args, DefaultOperandBundles);
2496-
if (IsFPConstrained)
2497-
setConstrainedFPCallAttr(CI);
2498-
if (isa<FPMathOperator>(CI))
2499-
setFPAttrs(CI, FPMathTag, FMF);
2500-
return Insert(CI, Name);
2505+
return CreateCall(FTy, Callee, Args, DefaultOperandBundles, Name,
2506+
FPMathTag);
25012507
}
25022508

25032509
CallInst *CreateCall(FunctionType *FTy, Value *Callee, ArrayRef<Value *> Args,
25042510
ArrayRef<OperandBundleDef> OpBundles,
2505-
const Twine &Name = "", MDNode *FPMathTag = nullptr) {
2506-
CallInst *CI = CallInst::Create(FTy, Callee, Args, OpBundles);
2507-
if (IsFPConstrained)
2508-
setConstrainedFPCallAttr(CI);
2509-
if (isa<FPMathOperator>(CI))
2510-
setFPAttrs(CI, FPMathTag, FMF);
2511-
return Insert(CI, Name);
2512-
}
2511+
const Twine &Name = "", MDNode *FPMathTag = nullptr);
25132512

25142513
CallInst *CreateCall(FunctionCallee Callee, ArrayRef<Value *> Args = {},
25152514
const Twine &Name = "", MDNode *FPMathTag = nullptr) {
@@ -2731,6 +2730,24 @@ class IRBuilderBase {
27312730
/// Create an assume intrinsic call that represents an dereferencable
27322731
/// assumption on the provided pointer.
27332732
CallInst *CreateDereferenceableAssumption(Value *PtrValue, Value *SizeValue);
2733+
2734+
/// Create an operand bundle in the provided bundle set to represent given FP
2735+
/// rounding mode.
2736+
///
2737+
/// If the rounding mode is not defined, adds the default rounding mode,
2738+
/// stored in this builder object.
2739+
void
2740+
createFPRoundingBundle(SmallVectorImpl<OperandBundleDef> &Bundles,
2741+
std::optional<RoundingMode> Rounding = std::nullopt);
2742+
2743+
/// Create an operand bundle in the provided bundle set to represent FP
2744+
/// exception behavior.
2745+
///
2746+
/// If the exception behavior is not defined, adds the default behavior,
2747+
/// stored in this builder object.
2748+
void createFPExceptionBundle(
2749+
SmallVectorImpl<OperandBundleDef> &Bundles,
2750+
std::optional<fp::ExceptionBehavior> Except = std::nullopt);
27342751
};
27352752

27362753
/// This provides a uniform API for creating instructions and inserting

‎llvm/include/llvm/IR/InstrTypes.h

Copy file name to clipboardExpand all lines: llvm/include/llvm/IR/InstrTypes.h
+16Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
#include "llvm/IR/CallingConv.h"
2626
#include "llvm/IR/DerivedTypes.h"
2727
#include "llvm/IR/FMF.h"
28+
#include "llvm/IR/FPEnv.h"
2829
#include "llvm/IR/Function.h"
2930
#include "llvm/IR/Instruction.h"
3031
#include "llvm/IR/LLVMContext.h"
@@ -1091,6 +1092,13 @@ template <typename InputTy> class OperandBundleDefT {
10911092
using OperandBundleDef = OperandBundleDefT<Value *>;
10921093
using ConstOperandBundleDef = OperandBundleDefT<const Value *>;
10931094

1095+
void addFPRoundingBundle(LLVMContext &Ctx,
1096+
SmallVectorImpl<OperandBundleDef> &Bundles,
1097+
RoundingMode Rounding);
1098+
void addFPExceptionBundle(LLVMContext &Ctx,
1099+
SmallVectorImpl<OperandBundleDef> &Bundles,
1100+
fp::ExceptionBehavior Except);
1101+
10941102
//===----------------------------------------------------------------------===//
10951103
// CallBase Class
10961104
//===----------------------------------------------------------------------===//
@@ -1150,6 +1158,8 @@ class CallBase : public Instruction {
11501158
/// number of extra operands.
11511159
unsigned getNumSubclassExtraOperandsDynamic() const;
11521160

1161+
MemoryEffects getFloatingPointMemoryEffects() const;
1162+
11531163
public:
11541164
using Instruction::getContext;
11551165

@@ -2155,6 +2165,12 @@ class CallBase : public Instruction {
21552165
return false;
21562166
}
21572167

2168+
/// Return rounding mode specified by operand bundles.
2169+
RoundingMode getRoundingMode() const;
2170+
2171+
/// Return exception behavior specified by operand bundles.
2172+
std::optional<fp::ExceptionBehavior> getExceptionBehavior() const;
2173+
21582174
/// Used to keep track of an operand bundle. See the main comment on
21592175
/// OperandBundleUser above.
21602176
struct BundleOpInfo {

‎llvm/include/llvm/IR/IntrinsicInst.h

Copy file name to clipboardExpand all lines: llvm/include/llvm/IR/IntrinsicInst.h
+8Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,14 @@ class IntrinsicInst : public CallInst {
128128
/// course of IR transformations
129129
static bool mayLowerToFunctionCall(Intrinsic::ID IID);
130130

131+
/// Check if \p ID represents a function that may access FP environment and
132+
/// may have FP operand bundles.
133+
///
134+
/// Access to FP environment means that in the strict FP environment the
135+
/// function has read/write memory effect, which is used to maintain proper
136+
/// instructions ordering.
137+
static bool isFloatingPointOperation(Intrinsic::ID IID);
138+
131139
/// Methods for support type inquiry through isa, cast, and dyn_cast:
132140
static bool classof(const CallInst *I) {
133141
if (const Function *CF = I->getCalledFunction())

‎llvm/include/llvm/IR/LLVMContext.h

Copy file name to clipboardExpand all lines: llvm/include/llvm/IR/LLVMContext.h
+2Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,8 @@ class LLVMContext {
9696
OB_ptrauth = 7, // "ptrauth"
9797
OB_kcfi = 8, // "kcfi"
9898
OB_convergencectrl = 9, // "convergencectrl"
99+
OB_fp_control = 10, // "fp.control"
100+
OB_fp_except = 11, // "fp.except"
99101
};
100102

101103
/// getMDKindID - Return a unique non-zero ID for the specified metadata kind.

‎llvm/include/llvm/Support/ModRef.h

Copy file name to clipboardExpand all lines: llvm/include/llvm/Support/ModRef.h
+5Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,11 @@ template <typename LocationEnum> class MemoryEffectsBase {
234234
return getWithoutLoc(Location::InaccessibleMem).doesNotAccessMemory();
235235
}
236236

237+
/// Whether this function accesses inaccessible memory.
238+
bool doesAccessInaccessibleMem() const {
239+
return isModOrRefSet(getModRef(Location::InaccessibleMem));
240+
}
241+
237242
/// Whether this function only (at most) accesses errno memory.
238243
bool onlyAccessesErrnoMem() const {
239244
return getWithoutLoc(Location::ErrnoMem).doesNotAccessMemory();

‎llvm/lib/IR/FPEnv.cpp

Copy file name to clipboardExpand all lines: llvm/lib/IR/FPEnv.cpp
+64Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,17 @@ std::optional<RoundingMode> convertStrToRoundingMode(StringRef RoundingArg) {
3434
.Default(std::nullopt);
3535
}
3636

37+
std::optional<RoundingMode> convertBundleToRoundingMode(StringRef RoundingArg) {
38+
return StringSwitch<std::optional<RoundingMode>>(RoundingArg)
39+
.Case("dyn", RoundingMode::Dynamic)
40+
.Case("rte", RoundingMode::NearestTiesToEven)
41+
.Case("rmm", RoundingMode::NearestTiesToAway)
42+
.Case("rtn", RoundingMode::TowardNegative)
43+
.Case("rtp", RoundingMode::TowardPositive)
44+
.Case("rtz", RoundingMode::TowardZero)
45+
.Default(std::nullopt);
46+
}
47+
3748
std::optional<StringRef> convertRoundingModeToStr(RoundingMode UseRounding) {
3849
std::optional<StringRef> RoundingStr;
3950
switch (UseRounding) {
@@ -61,6 +72,33 @@ std::optional<StringRef> convertRoundingModeToStr(RoundingMode UseRounding) {
6172
return RoundingStr;
6273
}
6374

75+
std::optional<StringRef> convertRoundingModeToBundle(RoundingMode UseRounding) {
76+
std::optional<StringRef> RoundingStr;
77+
switch (UseRounding) {
78+
case RoundingMode::Dynamic:
79+
RoundingStr = "dyn";
80+
break;
81+
case RoundingMode::NearestTiesToEven:
82+
RoundingStr = "rte";
83+
break;
84+
case RoundingMode::NearestTiesToAway:
85+
RoundingStr = "rmm";
86+
break;
87+
case RoundingMode::TowardNegative:
88+
RoundingStr = "rtn";
89+
break;
90+
case RoundingMode::TowardPositive:
91+
RoundingStr = "rtp";
92+
break;
93+
case RoundingMode::TowardZero:
94+
RoundingStr = "rtz";
95+
break;
96+
default:
97+
break;
98+
}
99+
return RoundingStr;
100+
}
101+
64102
std::optional<fp::ExceptionBehavior>
65103
convertStrToExceptionBehavior(StringRef ExceptionArg) {
66104
return StringSwitch<std::optional<fp::ExceptionBehavior>>(ExceptionArg)
@@ -70,6 +108,15 @@ convertStrToExceptionBehavior(StringRef ExceptionArg) {
70108
.Default(std::nullopt);
71109
}
72110

111+
std::optional<fp::ExceptionBehavior>
112+
convertBundleToExceptionBehavior(StringRef ExceptionArg) {
113+
return StringSwitch<std::optional<fp::ExceptionBehavior>>(ExceptionArg)
114+
.Case("ignore", fp::ebIgnore)
115+
.Case("maytrap", fp::ebMayTrap)
116+
.Case("strict", fp::ebStrict)
117+
.Default(std::nullopt);
118+
}
119+
73120
std::optional<StringRef>
74121
convertExceptionBehaviorToStr(fp::ExceptionBehavior UseExcept) {
75122
std::optional<StringRef> ExceptStr;
@@ -87,6 +134,23 @@ convertExceptionBehaviorToStr(fp::ExceptionBehavior UseExcept) {
87134
return ExceptStr;
88135
}
89136

137+
std::optional<StringRef>
138+
convertExceptionBehaviorToBundle(fp::ExceptionBehavior UseExcept) {
139+
std::optional<StringRef> ExceptStr;
140+
switch (UseExcept) {
141+
case fp::ebStrict:
142+
ExceptStr = "strict";
143+
break;
144+
case fp::ebIgnore:
145+
ExceptStr = "ignore";
146+
break;
147+
case fp::ebMayTrap:
148+
ExceptStr = "maytrap";
149+
break;
150+
}
151+
return ExceptStr;
152+
}
153+
90154
Intrinsic::ID getConstrainedIntrinsicID(const Instruction &Instr) {
91155
Intrinsic::ID IID = Intrinsic::not_intrinsic;
92156
switch (Instr.getOpcode()) {

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.