Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
Greenplum
Gpdb
提交
f2e3f621
G
Gpdb
项目概览
Greenplum
/
Gpdb
通知
7
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
G
Gpdb
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
f2e3f621
编写于
7月 15, 2000
作者:
T
Tom Lane
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Update implementation notes for new memory management logic.
上级
e40492ec
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
52 addition
and
41 deletion
+52
-41
src/backend/utils/mmgr/README
src/backend/utils/mmgr/README
+52
-41
未找到文件。
src/backend/utils/mmgr/README
浏览文件 @
f2e3f621
Proposal for memory allocation fixes, take 2 21-Jun
-2000
--------------------------------------
------
Notes about memory allocation redesign 14-Jul
-2000
--------------------------------------
We know that Postgres has serious problems with memory leakage during
large queries that process a lot of pass-by-reference data. There is
no provision for recycling memory until end of query. This needs to be
Up through version 7.0, Postgres has serious problems with memory leakage
during large queries that process a lot of pass-by-reference data. There
is
no provision for recycling memory until end of query. This needs to be
fixed, even more so with the advent of TOAST which will allow very large
chunks of data to be passed around in the system. So, here is a proposal.
...
...
@@ -193,30 +193,53 @@ pathnodes; this will allow it to release the bulk of its temporary space
usage (which can be a lot, for large joins) at completion of planning.
The completed plan tree will be in TransactionCommandContext.
The executor will have contexts with lifetime similar to plan nodes
(I'm not sure at the moment whether there's need for one such context
per plan level, or whether a single context is sufficient). These
contexts will hold plan-node-local execution state and related items.
There will also be a context on each plan level that is reset at the start
of each tuple processing cycle. This per-tuple context will be the normal
CurrentMemoryContext during evaluation of expressions and so forth. By
resetting it, we reclaim transient memory that was used during processing
of the prior tuple. That should be enough to solve the problem of running
out of memory on large queries. We must have a per-tuple context in each
plan node, and we must reset it at the start of a tuple cycle rather than
the end, so that each plan node can use results of expression evaluation
as part of the tuple it returns to its parent node.
By resetting the per-tuple context, we will be able to free memory after
each tuple is processed, rather than only after the whole plan is
processed. This should solve our memory leakage problems pretty well;
yet we do not need to add very much new bookkeeping logic to do it.
In particular, we do *not* need to try to keep track of individual values
palloc'd during expression evaluation.
Note we assume that resetting a context is a cheap operation. This is
true already, and we can make it even more true with a little bit of
tuning in aset.c.
The top-level executor routines, as well as most of the "plan node"
execution code, will normally run in TransactionCommandContext. Much
of the memory allocated in these routines is intended to live until end
of query, so this is appropriate for those purposes. We already have
a mechanism --- "tuple table slots" --- for avoiding leakage of tuples,
which is the major kind of short-lived data handled by these routines.
This still leaves a certain amount of explicit pfree'ing needed by plan
node code, but that code largely exists already and is probably not worth
trying to remove. I looked at the possibility of running in a shorter-
lived context (such as a context that gets reset per-tuple), but this
seems fairly impractical. The biggest problem with it is that code in
the index access routines, as well as some other complex algorithms like
tuplesort.c, assumes that palloc'd storage will live across tuples.
For example, rtree uses a palloc'd state stack to keep track of an index
scan.
The main improvement needed in the executor is that expression evaluation
--- both for qual testing and for computation of targetlist entries ---
needs to not leak memory. To do this, each ExprContext (expression-eval
context) created in the executor will now have a private memory context
associated with it, and we'll arrange to switch into that context when
evaluating expressions in that ExprContext. The plan node that owns the
ExprContext is responsible for resetting the private context to empty
when it no longer needs the results of expression evaluations. Typically
the reset is done at the start of each tuple-fetch cycle in the plan node.
Note that this design gives each plan node its own expression-eval memory
context. This appears necessary to handle nested joins properly, since
an outer plan node might need to retain expression results it has computed
while obtaining the next tuple from an inner node --- but the inner node
might execute many tuple cycles and many expressions before returning a
tuple. The inner node must be able to reset its own expression context
more often than once per outer tuple cycle. Fortunately, memory contexts
are cheap enough that giving one to each plan node doesn't seem like a
problem.
A problem with running index accesses and sorts in TransactionMemoryContext
is that these operations invoke datatype-specific comparison functions,
and if the comparators leak any memory then that memory won't be recovered
till end of query. The comparator functions all return bool or int32,
so there's no problem with their result data, but there could be a problem
with leakage of internal temporary data. In particular, comparator
functions that operate on TOAST-able data types will need to be careful
not to leak detoasted versions of their inputs. This is annoying, but
it appears a lot easier to make the comparators conform than to fix the
index and sort routines, so that's what I propose to do for 7.1. Further
cleanup can be left for another day.
There will be some special cases, such as aggregate functions. nodeAgg.c
needs to remember the results of evaluation of aggregate transition
...
...
@@ -365,15 +388,3 @@ chunk of memory is allocated in (by checking the required standard chunk
header), so nodeAgg can determine whether or not it's safe to reset
its working context; it doesn't have to rely on the transition function
to do what it's expecting.
It might be that the executor per-run contexts described above should
be tied directly to executor "EState" nodes, that is, one context per
EState. I'm not real clear on the lifespan of EStates or the situations
where we have just one or more than one, so I'm not sure. Comments?
It would probably be possible to adapt the existing "portal" memory
management mechanism to do what we need. I am instead proposing setting
up a totally new mechanism, because the portal code strikes me as
extremely crufty and unwieldy. It may be that we can eventually remove
portals entirely, or perhaps reimplement them with this mechanism
underneath.
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录