[CALUG] More I/O buffering foolishness

[Jason C. Miller]

Well....if I do ...

---------------------------------
(cd /blah ; tar cvf - * 2>/dev/null) | (cd /blah2 ; tar xvf - 2>/tmp/log)
---------------------------------
 
... stdout from the tar extraction works just fine.  Even if I try to pipe 
that output into 'grep', it'll still buffer everthing in chunks and I'm 
really perplexed as to why that is.  The output comes out of tar just 
fine, but it seems that something happens to only allow it to be passed in 
chunks into the next command.  Really weird.  

And, for all those who are curious, I've done all the usual 
stdout/stderr redirection testing.  



On Wed, 1 Mar 2006, Allbritten, Mark wrote:

> Hi Jason, 
> Perl is taking standard in as one chunk.  I believe you need to use a while readline loop...  But if that's all your using perl for it would be easier to do something like:
> while read X ^Jdo^Jecho $X^Jdone 
> I don't have a perl while readline snippet handy..... 
> Best Regards, 
> Mark 
> 
> 
> -----Original Message-----
> From: lug-bounces at calug.com [mailto:lug-bounces at calug.com]On Behalf Of Jason C. Miller
> Sent: Wednesday, March 01, 2006 9:43 AM
> To: lug at calug.com
> Subject: [CALUG] More I/O buffering foolishness
> 
> I've asked a buffering question on here before and have now run into a
> similar-yet-different issue and was wondering if anyone had any insight.
> 
> Here's the short story...
> 
> 
> At the command line (bash) ...
> -----------------------------------
> (cd /blah ; tar cvf - * 2>/dev/null) | (cd /blah2 ; tar xvf - 2>/tmp/log) | \
> perl -e 'while (<STDIN>) { print }'
> -----------------------------------
> ... works fine.  All it does is print the stdout from the tar extraction. 
> 
> 
> However, once inserted into a script, the exact same string of commands
> will be block-buffered instead of line-buffered (ie. prints output in
> chunks).  Granted, this isn't unheard of.  This is particularly annoying
> because the stdin is being used as input for another program (Xdialog).  
> 
> 
> The oddness is that, inside the script ...
> -----------------------------------
> (cd /blah ; tar cvf - * 2>/dev/null) | (cd /blah2 ; tar xvf - 2>/tmp/log)
> -----------------------------------
> ... will line-buffer just fine.  It's when I try to pipe that into
> anything else AFTER that the buffering issues come up.  I've tried it with
> both perl and awk and they both do the same thing.  I tried forcing the
> flush in the perl, but that doesn't do anything and I really don't know if
> that's the issue or not. 
> 
> 
> Any ideas?
> 
>                                            -jason
> 
> _______________________________________________
> Columbia, Maryland Linux User's Group (CALUG) mailing list
> CALUG Website: http://www.calug.com
> Email postings to: lug at calug.com
> Change your list subscription options: http://calug.com/mailman/listinfo/lug
> 

-- 
***************************************************
My blog: http://millersplace.blogspot.com/ 
***************************************************