[CALUG] More I/O buffering foolishness

[Dave Dodge]

On Wed, Mar 01, 2006 at 09:39:22PM -0500, Jason C. Miller wrote:
> 1. It is a Solaris 8 machine

> (which, after more reading, I understand a lot more about how programs 
> see what devices STDOUT is attached to and how they act accordingly).

Some programs check the input and output devices on their own, for
example to determine if they should present a prompt to the user.  But
even if they don't check, the C library itself can (and probably does)
make this check internally, and changes the default buffering for
stdout under the hood.

> 3. As mentioned, it doesn't matter what command (perl, awk, sed, grep, 
> etc) comes after that next pipe.  All the STDIN for those programs are 
> block-buffered.

Most likely what's happening is that as soon as you add the pipe, the
C library sees a pipe instead of a terminal device, and changes tar's
stdout to be block-buffered.  If you "truss" the tar process and watch
the write operations it's performing on descriptor 1, you'll probably
see them change from being one-per-line to one-per-block when you add
the pipe.

If this is indeed the problem, then you have to do something like:

  - Change tar's behavior somehow.  There's probably no command-line
    option to do this, so you're forced to either patch the code or
    perhaps use an LD_PRELOAD trick to slip a bit of your own code
    into tar nondestructively.  Here's an example of the LD_PRELOAD
    technique on Linux; Solaris may be similar:

      http://www.dalkescientific.com/writings/diary/archive/2005/04/18/wrapping_command_line_programs_IV.html

  - Slip a pseudoterminal device between tar and the next command.
    This is messy and extremely unportable.  I've done it before with
    "expect" and a script that poked around in the Linux "/proc"
    filesystem.  Yuck.

                                                  -Dave Dodge