Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1229494 - pmprintf shouldn't use /var/tmp files
Summary: pmprintf shouldn't use /var/tmp files
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: pcp
Version: 29
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Frank Ch. Eigler
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-08 23:58 UTC by Frank Ch. Eigler
Modified: 2018-11-19 20:07 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-19 20:07:14 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Frank Ch. Eigler 2015-06-08 23:58:53 UTC
libpcp/src/util.c's use of /var/tmp/$FILEs for buffering each pmprintf call is inefficient (leads to excess syscall & physical disk traffic).  On linux and probably elsewhere, it could easily use fmemopen(3) or open_memstream(3) instead.

Comment 1 Nathan Scott 2015-06-09 00:36:00 UTC
There's no evidence of physical disk traffic presented here (an efficient filesystem should do none for this short-lived-file, to which libpcp issues buffered I/O only), nor is there any evidence of "excess" syscalls being provided here.

If you intend to "fix" this (non-problem) please provide both a tested patch for more detailed discussion (this will help you realise its not going to be as trivial a change as you're thinking based on your initial reading of the code), some actual evidence of an actual observed problem being solved by it, and re-open.

Comment 2 Frank Ch. Eigler 2015-06-09 01:03:50 UTC
(In reply to Nathan Scott from comment #1)
> There's no evidence of physical disk traffic presented here (an efficient
> filesystem should do none for this short-lived-file, to which libpcp issues
> buffered I/O only)

It does not take much thought to see why comment #0 true.  (Use of "buffered I/O" at the glibc level is irrelevant.)  The libpcp code is opening files on /var/tmp, which is normally a physical partition, not even a tmpfs.  It creates, writes-to, and closes the file.  Later it opens, reads, closes, and unlinks it.  A journaling filesystem can require physical & barriered I/O for several of these operations.


% cat foo.c
#include <pcp/pmapi.h>
int main ()
{ 
  for(;;) {
    pmprintf("hello");
    pmflush();
  }
}
% gcc foo.c -lpcp
% ./a.out 2>/dev/null

iostat shows at least journaling-related traffic; more if sync(1)'s are going in in the background.


> nor is there any evidence of "excess" syscalls being provided here.

Even one syscall is an excess over the apprx. zero needed for an in-memory buffering.  strace -t -t -T:

20:58:24.154035 open("/var/tmp/pcp-KzKLEM", O_RDWR|O_CREAT|O_EXCL, 0600) = 3 <0.
000023>
20:58:24.154067 umask(022)              = 0177 <0.000005>
20:58:24.154079 fcntl(3, F_GETFL)       = 0x8002 (flags O_RDWR|O_LARGEFILE) <0.0
00005>
20:58:24.154091 fcntl(3, F_SETFL, O_RDWR|O_APPEND|O_LARGEFILE) = 0 <0.000005>
20:58:24.154104 lseek(3, 0, SEEK_END)   = 0 <0.000005>
20:58:24.154128 fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0 <0.000005>
20:58:24.154142 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ab1c1db8000 <0.000006>
20:58:24.154158 write(3, "hello\n", 6)  = 6 <0.000009>
20:58:24.154175 lseek(3, 0, SEEK_SET)   = 0 <0.000005>
20:58:24.154187 read(3, "hello\n", 256) = 6 <0.000006>
20:58:24.154201 write(2, "hello\n", 6)  = 6 <0.000005>
20:58:24.154214 read(3, "", 256)        = 0 <0.000004>
20:58:24.154226 close(3)                = 0 <0.000006>
20:58:24.154239 munmap(0x2ab1c1db8000, 4096) = 0 <0.000007>
20:58:24.154254 unlink("/var/tmp/pcp-KzKLEM") = 0 <0.000013>
20:58:24.154277 umask(0177)             = 022 <0.000004>



> If you intend to "fix" this (non-problem) please provide both a tested patch

That is not a legitimate requirement for a bug report (or RFE).

Comment 3 Nathan Scott 2015-06-09 01:16:59 UTC
OK, its all yours.  Those numbers so far don't suggest a real problem to me.

FWLIW, delayed allocation often means no disk traffic is actually required for short-lived files.  Buffered I/O is extremely important in this regard - if not used, then allocation & writing to disk is indeed unavoidable.

You will find the -file option to pmconfirm(1) complicates your proposed libpcp changes here, just as a heads-up.

Comment 4 Jan Kurik 2015-07-15 14:02:47 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle.
Changing version to '23'.

(As we did not run this process for some time, it could affect also pre-Fedora 23 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora23

Comment 5 Fedora End Of Life 2016-11-24 11:52:24 UTC
This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 23 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 6 Fedora End Of Life 2017-02-28 09:45:01 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.

Comment 7 Frank Ch. Eigler 2018-02-18 16:30:05 UTC
kenj is kindly fixing this problem
https://github.com/kmcdonell/pcp/commit/cb92d3474c5fea8f6da6b47e43d7e238fcea55fe

Comment 8 Fedora End Of Life 2018-05-03 08:50:09 UTC
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 9 Jan Kurik 2018-08-14 10:23:29 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle.
Changing version to '29'.

Comment 10 Frank Ch. Eigler 2018-11-19 20:07:14 UTC
fixed by commit cb92d3474c5fea8f6da6b47e43d7e238fcea55fe
Author: Ken McDonell <kenj.au>
Date:   Sun Feb 18 15:28:43 2018 +1100


Note You need to log in before you can comment on or make changes to this bug.