Fix used_segspace counter leak
GPDB currently has 2 APIs of creating work files 1. Using workfile_mgr : 1. This uses a workfile set to contain all the related workfiles. 2. All workfiles are created under the same directory. As a result, when it is time to delete the workfiles, we can simply delete the directory. 3. There is no mechanism to control the exact name of the workfile created. All workfiles are referenced using an ID. `"%s/spillfile\_f%u", work\_set-\>path, file\_no);` 4. Deletion of workiles is automatically handled in case of an exception, as all the files are contained in one directory, thus no counter/files are leaked. 2. Using ExecWorkfile : 1. There is a Create/Open/Read/Write/Flush/Close API. 2. This is the lower-level API also used by workfile_mgr to manage the workfiles directly. 3. Each operator is resposible to correctly close the workfile it creates. When we write to a workfile, we reserve and add the number of bytes to used_segspace, and add that amount to `workfile->size`. Then, we call `workfile_update_in_progress_size`. If the workfile has an owning workset, we update the workset's `in_progress_size` counter. However, if it has no such workset, we completely ignore the count - and this is what ultimately leaks the `used_segspace` counter. This is a fix to account for bytes written in workfiles that are not part of any workfile set - that is those created using the lower level API. Hence when we abort or error out, we subtract this amount from the `used_segspace` counter. Authors: Shreedhar Hardikar <shardikar@pivotal.io> Xin Zhang <xzhang@pivotal.io> Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com> Nikos Armenatzoglou <nikos.armenatzoglou@gmail.com>
Showing
想要评论请 注册 或 登录