[CALUG] More I/O buffering foolishness

[Allbritten, Mark]

Hi Jason,

I have a couple of different idea's then since we are talking perl.  I've only encountered this a few times, but I do have some scripts that had to do system calls to process the data (proprietary app).  I assume you've checked and there are no perl modules you can add or compile to do the task, right?  Native perl modules are always cleaner.

That assumption being made have you tried anything like below (it's a snippet from a script I use on an Samsung Contact system)? 

In this one, I pull the info into a file handle and then process and push to an array:

open (FH, "/opt/openmail/bin/omlist -m|");
  while (<FH>){
    chomp ();
      ($MSGNumber,$MSGData)=split " ",$_,2;
    push (@MSGList, "$MSGNumber -- $MSGData\n");
  }
  close (FH);

Another method I like even less is (since you say it works from the command line) the following:

system ("/opt/openmail/omlist -m >> /tmp/file");
open (FH, "/tmp/file");
while (<FH>){
    chomp ();
      ($MSGNumber,$MSGData)=split " ",$_,2;
    push (@MSGList, "$MSGNumber -- $MSGData\n");
  }
  close (FH);
unlink ("/tmp/file");

Best Regards,
Mark




-----Original Message-----
From: lug-bounces at calug.com [mailto:lug-bounces at calug.com]On Behalf Of Jason C. Miller
Sent: Wednesday, March 01, 2006 9:39 PM
To: Chris Edillon
Cc: lug at calug.com
Subject: Re: [CALUG] More I/O buffering foolishness

Arg.  Alright....some more details.  :)

1. It is a Solaris 8 machine
2. My code is not as simple as a single perl print() statement.  That
was simply there to provide a simple example that doesn't work on my
system.  It works fine at the command prompt but not from a script
(which, after more reading, I understand a lot more about how programs
see what devices STDOUT is attached to and how they act accordingly).
3. As mentioned, it doesn't matter what command (perl, awk, sed, grep,
etc) comes after that next pipe.  All the STDIN for those programs are
block-buffered.

Anyhoo.  Yet another day in paradise.  :)


Chris Edillon wrote:

>On Wed, 2006-03-01 at 09:43 -0500, Jason C. Miller wrote:
>
> 
>
>>I've asked a buffering question on here before and have now run into a
>>similar-yet-different issue and was wondering if anyone had any insight.
>>
>>Here's the short story...
>>
>>
>>At the command line (bash) ...
>>-----------------------------------
>>(cd /blah ; tar cvf - * 2>/dev/null) | (cd /blah2 ; tar xvf - 2>/tmp/log) | \
>>perl -e 'while (<STDIN>) { print }'
>>-----------------------------------
>>... works fine.  All it does is print the stdout from the tar extraction. 
>>
>>   
>>
>  perhaps i'm missing something, but why pipe the output
>to perl?  it's already printing to stdout.
>
> 
>
>>The oddness is that, inside the script ...
>>-----------------------------------
>>(cd /blah ; tar cvf - * 2>/dev/null) | (cd /blah2 ; tar xvf - 2>/tmp/log)
>>-----------------------------------
>>... will line-buffer just fine.  It's when I try to pipe that into
>>anything else AFTER that the buffering issues come up.  I've tried it with
>>both perl and awk and they both do the same thing.  I tried forcing the
>>flush in the perl, but that doesn't do anything and I really don't know if
>>that's the issue or not. 
>>
>>   
>>
>  like mark, i can't recreate this either.  i wrote this little snippit
>and it doesn't seem to buffer the output (with or without the perl bit):
>
>#!/bin/sh
>tar -cvf - /usr/share/doc/* 2>/dev/null | (cd /tmp/test && tar -xvf -
>2>/tmp/testlog) | perl -pe ''
>
>if this is input for Xdialog, perhaps it is buffering before
>printing anything?
>
>chris
>
> 
>


--
***************************************************
My blog: http://millersplace.blogspot.com/
***************************************************

_______________________________________________
Columbia, Maryland Linux User's Group (CALUG) mailing list
CALUG Website: http://www.calug.com
Email postings to: lug at calug.com
Change your list subscription options: http://calug.com/mailman/listinfo/lug