Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit b8bff07

Browse filesBrowse files
hlinnakahlinnaka
committed
Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a single array and hash to hold all kinds of resources. This makes it possible to introduce new resource "kinds" without having to modify the ResourceOwnerData struct. In particular, this makes it possible for extensions to register custom resource kinds. The old approach was to have a small array of resources of each kind, and if it fills up, switch to a hash table. The new approach also uses an array and a hash, but now the array and the hash are used at the same time. The array is used to hold the recently added resources, and when it fills up, they are moved to the hash. This keeps the access to recent entries fast, even when there are a lot of long-held resources. All the resource-specific ResourceOwnerEnlarge*(), ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have been replaced with three generic functions that take resource kind as argument. For convenience, we still define resource-specific wrapper macros around the generic functions with the old names, but they are now defined in the source files that use those resource kinds. The release callback no longer needs to call ResourceOwnerForget on the resource being released. ResourceOwnerRelease unregisters the resource from the owner before calling the callback. That needed some changes in bufmgr.c and some other files, where releasing the resources previously always called ResourceOwnerForget. Each resource kind specifies a release priority, and ResourceOwnerReleaseAll releases the resources in priority order. To make that possible, we have to restrict what you can do between phases. After calling ResourceOwnerRelease(), you are no longer allowed to remember any more resources in it or to forget any previously remembered resources by calling ResourceOwnerForget. There was one case where that was done previously. At subtransaction commit, AtEOSubXact_Inval() would handle the invalidation messages and call RelationFlushRelation(), which temporarily increased the reference count on the relation being flushed. We now switch to the parent subtransaction's resource owner before calling AtEOSubXact_Inval(), so that there is a valid ResourceOwner to temporarily hold that relcache reference. Other end-of-xact routines make similar calls to AtEOXact_Inval() between release phases, but I didn't see any regression test failures from those, so I'm not sure if they could reach a codepath that needs remembering extra resources. There were two exceptions to how the resource leak WARNINGs on commit were printed previously: llvmjit silently released the context without printing the warning, and a leaked buffer io triggered a PANIC. Now everything prints a WARNING, including those cases. Add tests in src/test/modules/test_resowner. Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu Reviewed-by: Peter Eisentraut, Andres Freund Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
1 parent b70c214 commit b8bff07
Copy full SHA for b8bff07
Expand file treeCollapse file tree

36 files changed

+2278
-1144
lines changed

‎src/backend/access/common/tupdesc.c

Copy file name to clipboardExpand all lines: src/backend/access/common/tupdesc.c
+49-2Lines changed: 49 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,34 @@
2727
#include "common/hashfn.h"
2828
#include "utils/builtins.h"
2929
#include "utils/datum.h"
30-
#include "utils/resowner_private.h"
30+
#include "utils/resowner.h"
3131
#include "utils/syscache.h"
3232

33+
/* ResourceOwner callbacks to hold tupledesc references */
34+
static void ResOwnerReleaseTupleDesc(Datum res);
35+
static char *ResOwnerPrintTupleDesc(Datum res);
36+
37+
static const ResourceOwnerDesc tupdesc_resowner_desc =
38+
{
39+
.name = "tupdesc reference",
40+
.release_phase = RESOURCE_RELEASE_AFTER_LOCKS,
41+
.release_priority = RELEASE_PRIO_TUPDESC_REFS,
42+
.ReleaseResource = ResOwnerReleaseTupleDesc,
43+
.DebugPrint = ResOwnerPrintTupleDesc
44+
};
45+
46+
/* Convenience wrappers over ResourceOwnerRemember/Forget */
47+
static inline void
48+
ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc)
49+
{
50+
ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_desc);
51+
}
52+
53+
static inline void
54+
ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc)
55+
{
56+
ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_desc);
57+
}
3358

3459
/*
3560
* CreateTemplateTupleDesc
@@ -364,7 +389,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc)
364389
{
365390
Assert(tupdesc->tdrefcount >= 0);
366391

367-
ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner);
392+
ResourceOwnerEnlarge(CurrentResourceOwner);
368393
tupdesc->tdrefcount++;
369394
ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc);
370395
}
@@ -847,3 +872,25 @@ TupleDescGetDefault(TupleDesc tupdesc, AttrNumber attnum)
847872

848873
return result;
849874
}
875+
876+
/* ResourceOwner callbacks */
877+
878+
static void
879+
ResOwnerReleaseTupleDesc(Datum res)
880+
{
881+
TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res);
882+
883+
/* Like DecrTupleDescRefCount, but don't call ResourceOwnerForget() */
884+
Assert(tupdesc->tdrefcount > 0);
885+
if (--tupdesc->tdrefcount == 0)
886+
FreeTupleDesc(tupdesc);
887+
}
888+
889+
static char *
890+
ResOwnerPrintTupleDesc(Datum res)
891+
{
892+
TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res);
893+
894+
return psprintf("TupleDesc %p (%u,%d)",
895+
tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod);
896+
}

‎src/backend/access/transam/xact.c

Copy file name to clipboardExpand all lines: src/backend/access/transam/xact.c
+14Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5172,9 +5172,23 @@ AbortSubTransaction(void)
51725172
ResourceOwnerRelease(s->curTransactionOwner,
51735173
RESOURCE_RELEASE_BEFORE_LOCKS,
51745174
false, false);
5175+
51755176
AtEOSubXact_RelationCache(false, s->subTransactionId,
51765177
s->parent->subTransactionId);
5178+
5179+
5180+
/*
5181+
* AtEOSubXact_Inval sometimes needs to temporarily bump the refcount
5182+
* on the relcache entries that it processes. We cannot use the
5183+
* subtransaction's resource owner anymore, because we've already
5184+
* started releasing it. But we can use the parent resource owner.
5185+
*/
5186+
CurrentResourceOwner = s->parent->curTransactionOwner;
5187+
51775188
AtEOSubXact_Inval(false);
5189+
5190+
CurrentResourceOwner = s->curTransactionOwner;
5191+
51785192
ResourceOwnerRelease(s->curTransactionOwner,
51795193
RESOURCE_RELEASE_LOCKS,
51805194
false, false);

‎src/backend/jit/jit.c

Copy file name to clipboardExpand all lines: src/backend/jit/jit.c
-2Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,6 @@
2626
#include "jit/jit.h"
2727
#include "miscadmin.h"
2828
#include "utils/fmgrprotos.h"
29-
#include "utils/resowner_private.h"
3029

3130
/* GUCs */
3231
bool jit_enabled = true;
@@ -140,7 +139,6 @@ jit_release_context(JitContext *context)
140139
if (provider_successfully_loaded)
141140
provider.release_context(context);
142141

143-
ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context));
144142
pfree(context);
145143
}
146144

‎src/backend/jit/llvm/llvmjit.c

Copy file name to clipboardExpand all lines: src/backend/jit/llvm/llvmjit.c
+42-3Lines changed: 42 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@
4545
#include "portability/instr_time.h"
4646
#include "storage/ipc.h"
4747
#include "utils/memutils.h"
48-
#include "utils/resowner_private.h"
48+
#include "utils/resowner.h"
4949

5050
#define LLVMJIT_LLVM_CONTEXT_REUSE_MAX 100
5151

@@ -131,6 +131,30 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm);
131131
static char *llvm_error_message(LLVMErrorRef error);
132132
#endif /* LLVM_VERSION_MAJOR > 11 */
133133

134+
/* ResourceOwner callbacks to hold JitContexts */
135+
static void ResOwnerReleaseJitContext(Datum res);
136+
137+
static const ResourceOwnerDesc jit_resowner_desc =
138+
{
139+
.name = "LLVM JIT context",
140+
.release_phase = RESOURCE_RELEASE_BEFORE_LOCKS,
141+
.release_priority = RELEASE_PRIO_JIT_CONTEXTS,
142+
.ReleaseResource = ResOwnerReleaseJitContext,
143+
.DebugPrint = NULL /* the default message is fine */
144+
};
145+
146+
/* Convenience wrappers over ResourceOwnerRemember/Forget */
147+
static inline void
148+
ResourceOwnerRememberJIT(ResourceOwner owner, LLVMJitContext *handle)
149+
{
150+
ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_desc);
151+
}
152+
static inline void
153+
ResourceOwnerForgetJIT(ResourceOwner owner, LLVMJitContext *handle)
154+
{
155+
ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_desc);
156+
}
157+
134158
PG_MODULE_MAGIC;
135159

136160

@@ -220,15 +244,15 @@ llvm_create_context(int jitFlags)
220244

221245
llvm_recreate_llvm_context();
222246

223-
ResourceOwnerEnlargeJIT(CurrentResourceOwner);
247+
ResourceOwnerEnlarge(CurrentResourceOwner);
224248

225249
context = MemoryContextAllocZero(TopMemoryContext,
226250
sizeof(LLVMJitContext));
227251
context->base.flags = jitFlags;
228252

229253
/* ensure cleanup */
230254
context->base.resowner = CurrentResourceOwner;
231-
ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context));
255+
ResourceOwnerRememberJIT(CurrentResourceOwner, context);
232256

233257
llvm_jit_context_in_use_count++;
234258

@@ -300,6 +324,9 @@ llvm_release_context(JitContext *context)
300324
llvm_jit_context->handles = NIL;
301325

302326
llvm_leave_fatal_on_oom();
327+
328+
if (context->resowner)
329+
ResourceOwnerForgetJIT(context->resowner, llvm_jit_context);
303330
}
304331

305332
/*
@@ -1394,3 +1421,15 @@ llvm_error_message(LLVMErrorRef error)
13941421
}
13951422

13961423
#endif /* LLVM_VERSION_MAJOR > 11 */
1424+
1425+
/*
1426+
* ResourceOwner callbacks
1427+
*/
1428+
static void
1429+
ResOwnerReleaseJitContext(Datum res)
1430+
{
1431+
JitContext *context = (JitContext *) DatumGetPointer(res);
1432+
1433+
context->resowner = NULL;
1434+
jit_release_context(context);
1435+
}

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.