I'm wondering how to capture data written to stdout (or stderr) in real
time using popen3. The problem occurs with the following sample
scripts:

sleep.rb
--------
puts "before sleep"
sleep 3
puts "after sleep"


capture.rb
---------
require 'open3'

Open3.popen3("ruby sleep.rb") do |stdin, stdout, stderr|

Thread.new do
loop do
out = stdout.gets
puts out if out
end
end

sleep 5
end


capture.rb outputs both "before sleep" and "after sleep" after 3 seconds
have passed. When using stderr, however, output is seen in real time.

Note, I'm avoiding STDOUT.flush in sleep.rb since I won't be able to
change the actual scripts I'll be passing to popen3.

Thanks for any .
Paul
--
Posted via http://www.ruby-forum.com/.

On May 9, 2008, at 6:08 PM, Paul Winward wrote:
>
> Note, I'm avoiding STDOUT.flush in sleep.rb since I won't be able to
> change the actual scripts I'll be passing to popen3.

popen3 is going to be using fork/exec under the hood and should
inherit the sync setting of STDOUT, so try setting

STDOUT.sync = true

before the popen and see if that s - untested, but i'm running out
the door....

cheers.

a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Thanks for the suggestion but I'm still having the problem.

Paul

ara.t.howard wrote:
> On May 9, 2008, at 6:08 PM, Paul Winward wrote:
>>
>> Note, I'm avoiding STDOUT.flush in sleep.rb since I won't be able to
>> change the actual scripts I'll be passing to popen3.
>
> popen3 is going to be using fork/exec under the hood and should
> inherit the sync setting of STDOUT, so try setting
>
> STDOUT.sync = true
>
> before the popen and see if that s - untested, but i'm running out
> the door....
>
> cheers.
>
> a @ http://codeforpeople.com/

--
Posted via http://www.ruby-forum.com/.

Another approach I had tried was to use IO.select, as shown in
http://whynotwiki.com/Ruby_/_Process_management ( attributed to Jamis
Buck). This too, produced the same delay in output, but is shown below:

Thank you for any

def read_until(pipe, stop_at, verbose = true)
lines = []
line = ""
while result = IO.select([pipe]) #, nil, nil, 10)
next if result.empty?

c = pipe.read(1)
break if c.nil?

line << c
break if line =~ stop_at

# Start a new line?
if line[-1] == ?\n
puts line if verbose
lines << line
line = ""
end
end
lines
end



Paul Winward wrote:
> Thanks for the suggestion but I'm still having the problem.
>
> Paul
>
> ara.t.howard wrote:
>> On May 9, 2008, at 6:08 PM, Paul Winward wrote:
>>>
>>> Note, I'm avoiding STDOUT.flush in sleep.rb since I won't be able to
>>> change the actual scripts I'll be passing to popen3.
>>
>> popen3 is going to be using fork/exec under the hood and should
>> inherit the sync setting of STDOUT, so try setting
>>
>> STDOUT.sync = true
>>
>> before the popen and see if that s - untested, but i'm running out
>> the door....
>>
>> cheers.
>>
>> a @ http://codeforpeople.com/

--
Posted via http://www.ruby-forum.com/.

I found a perl solution at http://www.perlmonks.org/?node_id=305812 that
makes use of pseudo tty to get line-buffered output. Unfortunately, I
can find very little documentation on how to use the ruby module PTY,
and it looks like it is no longer supported
(http://www.ruby-forum.com/topic/134771#new). Also, I wonder if a
pseudo tty can capture stderr as well
(http://unix.derkeiler.com/Newsgroups...3-07/0544.html)

The perl solution is system dependent, but it worked for me on Linux
2.6.22. I did see a debian package libpty-ruby but I don't see any
documentation and am not sure if this is still supported.

The notion of a pseudo tty is new to me and I'd appreciate any . Is
there a ruby equivalent for the perl solution below?

Thanks

#!/usr/bin/perl -w
use strict;
use IO::Pty;

my $pty = new IO::Pty;
my $slave = $pty->slave();

my $pid = fork();
die "Couldn't fork: $!" unless defined $pid;

if($pid){ # dup STDOUT to Pty and run external program:
$pty->close_slave();
open(STDOUT, ">&",$pty)||die $!;
system "perl blackbox.pl";
print "\cD"; # send ^d to end

} else { # this is your monitoring process
$pty->make_slave_controlling_terminal();
print "*$_" while <$slave>;
exit;
}
__END__




Paul Winward wrote:
> Another approach I had tried was to use IO.select, as shown in
> http://whynotwiki.com/Ruby_/_Process_management ( attributed to Jamis
> Buck). This too, produced the same delay in output, but is shown below:
>
> Thank you for any
>
> def read_until(pipe, stop_at, verbose = true)
> lines = []
> line = ""
> while result = IO.select([pipe]) #, nil, nil, 10)
> next if result.empty?
>
> c = pipe.read(1)
> break if c.nil?
>
> line << c
> break if line =~ stop_at
>
> # Start a new line?
> if line[-1] == ?\n
> puts line if verbose
> lines << line
> line = ""
> end
> end
> lines
> end
>
>
>
> Paul Winward wrote:
>> Thanks for the suggestion but I'm still having the problem.
>>
>> Paul
>>
>> ara.t.howard wrote:
>>> On May 9, 2008, at 6:08 PM, Paul Winward wrote:
>>>>
>>>> Note, I'm avoiding STDOUT.flush in sleep.rb since I won't be able to
>>>> change the actual scripts I'll be passing to popen3.
>>>
>>> popen3 is going to be using fork/exec under the hood and should
>>> inherit the sync setting of STDOUT, so try setting
>>>
>>> STDOUT.sync = true
>>>
>>> before the popen and see if that s - untested, but i'm running out
>>> the door....
>>>
>>> cheers.
>>>
>>> a @ http://codeforpeople.com/

--
Posted via http://www.ruby-forum.com/.

On May 10, 2008, at 1:28 PM, Paul Winward wrote:
> I found a perl solution at http://www.perlmonks.org/?node_id=305812
> that
> makes use of pseudo tty to get line-buffered output. Unfortunately, I
> can find very little documentation on how to use the ruby module PTY,
> and it looks like it is no longer supported
> (http://www.ruby-forum.com/topic/134771#new). Also, I wonder if a
> pseudo tty can capture stderr as well
> (http://unix.derkeiler.com/Newsgroups...3-07/0544.html
> )
>
> The perl solution is system dependent, but it worked for me on Linux
> 2.6.22. I did see a debian package libpty-ruby but I don't see any
> documentation and am not sure if this is still supported.
>
> The notion of a pseudo tty is new to me and I'd appreciate any
> . Is
> there a ruby equivalent for the perl solution below?

pty is supported and the dist has a bunch of demo code in it (ext/pty/
*) but you cannot capture stderr akaikt.

a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

On May 10, 2008, at 12:08 AM, Paul Winward wrote:
> Thanks for the suggestion but I'm still having the problem.

if the child program makes a call to setvbuf (as ruby does) there is
little you can do to alter this behaviour. otherwise the 'normal'
approach will work:

cfp:~ > cat a.rb
r, w = IO.pipe

child = fork{ STDOUT.reopen w; w.close; system 'echo 4 && sleep 1 &&
echo 2' }

w.close

while(( char = r.read(1) ))
printf "%f : %s\n", Time.now.to_f, char.inspect
end


cfp:~ > ruby a.rb
1210456103.163009 : "4"
1210456103.163153 : "\n"
1210456104.166967 : "2"
1210456104.167023 : "\n"


i might be wrong, but i really don't think there is anything you can
do with a child program that alters the buffering mode itself.

a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Thanks for the pointers. I wasn't aware of setvbuf. I'll look more into
pty.

ara.t.howard wrote:
> On May 10, 2008, at 12:08 AM, Paul Winward wrote:
>> Thanks for the suggestion but I'm still having the problem.
>
> if the child program makes a call to setvbuf (as ruby does) there is
> little you can do to alter this behaviour. otherwise the 'normal'
> approach will work:
>
--
Posted via http://www.ruby-forum.com/.