I'm having a problem "read"ing array variables via a pipe, but it works
fine using redirection of stdin.

$ bash --version
GNU bash, version 3.1.17(1)-release ...

"${function}" produces e.g.:

566 269 732 435

"${function}" | read -a var; echo ${#var[@]} ${var[*]}
0

however:

"${function}" > file; read -a var < file; echo ${#var[@]} ${var[*]}
4 566 269 732 435

Is it possible to get the array variables assigned using the pipe
structure please? I don't wish to generate a temporary file if at all
possible.

Greg Russell wrote:
> I'm having a problem "read"ing array variables via a pipe, but it works
> fine using redirection of stdin.
>
> $ bash --version
> GNU bash, version 3.1.17(1)-release ...
>
> "${function}" produces e.g.:
>
> 566 269 732 435
>
> "${function}" | read -a var; echo ${#var[@]} ${var[*]}
> 0
>
> however:
>
> "${function}" > file; read -a var < file; echo ${#var[@]} ${var[*]}
> 4 566 269 732 435

$ echo 566 269 732 435 | while read -a var; do echo ${#var[@]} ${var[*]}; done
4 566 269 732 435

> Is it possible to get the array variables assigned using the pipe
> structure please? I don't wish to generate a temporary file if at all
> possible.

--
Best regards | Monica Lewinsky's X-Boyfriend's
Cyrus | Wife for President

On Tue, 18 Mar 2008 07:04:36 +0100, Cyrus Kriticos wrote:

>> "${function}" | read -a var; echo ${#var[@]} ${var[*]}
>> 0
>>
>> however:
>>
>> "${function}" > file; read -a var < file; echo ${#var[@]} ${var[*]}
>> 4 566 269 732 435
>
> $ echo 566 269 732 435 | while read -a var; do echo ${#var[@]}
> ${var[*]}; done 4 566 269 732 435

Yes, that works, but I really don't wish to "do ...;done" anything at the
time, just assign the variable array. "read -a var" should, according to
the bash man page, assign the array from stdin. I don't understand the
need to echo the values during a single-pass while loop.

$ echo 566 269 732 435 | while read -a var; do cat /dev/null; done
$ echo ${#var[@]} ${var[*]}
0

2008-03-18, 05:25(+00), Greg Russell:
> I'm having a problem "read"ing array variables via a pipe, but it works
> fine using redirection of stdin.
>
> $ bash --version
> GNU bash, version 3.1.17(1)-release ...
>
> "${function}" produces e.g.:
>
> 566 269 732 435
>
> "${function}" | read -a var; echo ${#var[@]} ${var[*]}
> 0
[...]

A pipe, is an IPC mechanism, IPC standing for inter-process
communication. So, each element of a pipe line is run by a
different process. Typically, the read command on bash is run in
a subshell.

"$function" | awk '{print NF, $0}'

--
Stéphane

You can try:
"${function}" | { read -a var; echo ${#var[@]} ${var[*]};}
or:
read -a var < <("${function}"); echo ${#var[@]} ${var[*]}


> "${function}" | read -a var; echo ${#var[@]} ${var[*]}
> 0

On Tue, 18 Mar 2008 10:00:34 +0000, Stephane CHAZELAS wrote:

> A pipe, is an IPC mechanism, IPC standing for inter-process
> communication. So, each element of a pipe line is run by a different
> process. Typically, the read command on bash is run in a subshell.

read -a var < file
is a bash builtin. Are you really stating that a bash builtin "is run in
a subshell"? If so, I'd like a reference please so that I can consider
the idea more carefully, as I can't make any sense of it.

The bash man page states "The read builtin accepts a -a option to assign
a list of words read from the standard input to an array." It also states
in the Pipelines section example that "The standard output of command is
connected via a pipe to the standard input of command2." This is what I
expect to occur with

echo a b c d | read -a var

since the stdout of the builtin echo command is piped into the stdin of
the read builtin command. I can't find any documentation to suggest that
it should be otherwise, your example notwithstanding, although I really
have no idea what you're trying to illustrate with it other than that awk
is a separate PID from ${function}.

> "${function}" | awk '{print NF, $0}'

If your implication is that awk is a separate process to illustrate the
IPC use of the pipe, then please explain how "echo a b c d | while read
-a var; do echo ${#var[@]} ${var[*]};done" utilizes IPC when it consists
entirely of bash builtins and logical structures, as does "echo a b c d |
read -a var".

2008-03-18, 12:25(+00), Greg Russell:
> On Tue, 18 Mar 2008 10:00:34 +0000, Stephane CHAZELAS wrote:
>
>> A pipe, is an IPC mechanism, IPC standing for inter-process
>> communication. So, each element of a pipe line is run by a different
>> process. Typically, the read command on bash is run in a subshell.
>
> read -a var < file
> is a bash builtin. Are you really stating that a bash builtin "is run in
> a subshell"? If so, I'd like a reference please so that I can consider
> the idea more carefully, as I can't make any sense of it.

There's no pipe in
read < file

In the Bourne shell, redirecting also involved forking a new
process. Modern shells use a hack when running builtins to avoid
the need to fork. The shell saves the current stdin to another
fd using the dup2() system call, open the "file" for reading on
the stdin fd (0) and run the builtin.

> The bash man page states "The read builtin accepts a -a option to assign
> a list of words read from the standard input to an array." It also states
> in the Pipelines section example that "The standard output of command is
> connected via a pipe to the standard input of command2." This is what I
> expect to occur with
>
> echo a b c d | read -a var
>
> since the stdout of the builtin echo command is piped into the stdin of
> the read builtin command. I can't find any documentation to suggest that
> it should be otherwise,

You're confusing commands and processes. A single command (like
bash) can spawn several processes. When running external
commands, the shell spawns a new bash process and execs the
command in that process.

For running several commands concurrently connected by pipes,
bash *needs* to run them in different processes, how would you
do it otherwise.

So, builtin or not, bash needs to make sure "read" is run in a
different process as the command that pipes its output to it.

Some shells like zsh and some versions of ksh have an
optimisation in that the right-most command in a pipeline, if
builtin, is run by the current shell process. Most other shells
(ash, pdksh, bash) run every command in child processes of the
current shell.

This is covered by question E4 of the bash FAQ
(http://www.faqs.org/faqs/unix-faq/shell/bash/)

echo foo | read bar

runs too commands concurrently. They are both built in the
shell. zsh will run the rightmost one (read bar) in the current
shell process which explains why it the bar variable is still
readable after that pipeline has completed.

echo foo | read bar | true

will run "read bar" in a subshell in every shell.

echo foo | {
read bar
printf '%s\n' "$bar"
}

Will run "echo foo" and "read bar; printf '%s\n' "$bar"" in 2
differents processes (in zsh the second one will run in the
current shell).

> your example notwithstanding, although I really
> have no idea what you're trying to illustrate with it other than that awk
> is a separate PID from ${function}.

It wasn't an example, it was a suggestion to use awk instead
since awk is the command to split text into fields and a shell
is a command to run commands.

--
Stéphane

Stephane CHAZELAS wrote:
>
> Some shells like zsh and some versions of ksh have an

I think that _all_ ksh's (88 and 93) would behave the way you
describe below. Or did you count that PD thing also as "ksh"?
(Which I would find more confusing than ful, specifically
because of that fundamental difference.)

> optimisation in that the right-most command in a pipeline, if
> builtin, is run by the current shell process. Most other shells
> (ash, pdksh, bash) run every command in child processes of the
> current shell.

What about the Bourne shell's behaviour? (I've none available
to check.)

Janis

On Tue, 18 Mar 2008 05:00:17 -0700, mop2 wrote:

> You can try:
> "${function}" | { read -a var; echo ${#var[@]} ${var[*]};} or:
> read -a var < <("${function}"); echo ${#var[@]} ${var[*]}

Thank you. The first suggestion has problems, but the second will be
useful for my needs; I hadn't seen that redirection technique before.

On Tue, 18 Mar 2008 13:21:36 +0000, Stephane CHAZELAS wrote:

> This is covered by question E4 of the bash FAQ
> (http://www.faqs.org/faqs/unix-faq/shell/bash/)

Thank you. I hadn't appreciated the subtleties of builtins wrt pipe
structures.

2008-03-18, 14:44(+01), Janis Papanagnou:
> Stephane CHAZELAS wrote:
>>
>> Some shells like zsh and some versions of ksh have an
>
> I think that _all_ ksh's (88 and 93) would behave the way you
> describe below. Or did you count that PD thing also as "ksh"?
> (Which I would find more confusing than ful, specifically
> because of that fundamental difference.)

I call ksh every ksh implementation like the AT&T ones, the PD
one or the zsh one.

The AT&T implementations of ksh I generally call AT&T ksh, ksh88
or ksh93. Also note that there exist many variants of those on
various systems.

pdksh is the implementation of ksh I will most likely be using,
ksh88 being not free software and ksh93 being more like an
experimental/joke shell in my opinion. If you search the
archives of this group, you'll notice I'm not a big fan of ksh
holding it responsible for most of the flaws in modern shells
design.

>> optimisation in that the right-most command in a pipeline, if
>> builtin, is run by the current shell process. Most other shells
>> (ash, pdksh, bash) run every command in child processes of the
>> current shell.
>
> What about the Bourne shell's behaviour? (I've none available
> to check.)
[...]

Very easy. It was always forking, even for redirections.

--
Stéphane

Stephane CHAZELAS wrote:
>
> pdksh is the implementation of ksh I will most likely be using,

On commercial systems, too?

> ksh88 being not free software

But it's the one available on most (all?) commercial Unixes.

> and ksh93 being more like an
> experimental/joke shell in my opinion.

It has a stable base for 15 years. Beyond that its development
proceeds; only in this respect (i.e. the new features they add)
I might call it experimental. Usually, as long as new features
are in the evaluation/experimentation phase they are left quite
latent in the code or docs and will become published eventually.

> If you search the
> archives of this group, you'll notice I'm not a big fan of ksh
>

I need not search to know about that :-) And that's the only
explanation why you're using the term "joke" in this context.

> holding it responsible for most of the flaws in modern shells
> design.

ksh? or rather the bourne shell base?

Janis

On Tue, 18 Mar 2008 14:46:17 +0000, Greg Russell wrote:

> On Tue, 18 Mar 2008 05:00:17 -0700, mop2 wrote:
>
>> You can try:
>> "${function}" | { read -a var; echo ${#var[@]} ${var[*]};} or: read -a
>> var < <("${function}"); echo ${#var[@]} ${var[*]}
>
> Thank you. The first suggestion has problems, but the second will be
> useful for my needs; I hadn't seen that redirection technique before.

Have you tried:

var=($(function))

According to the bash man page, the outer enclosing parentheses allow a
list to be assigned to an array in a single operation. I've used this
syntax on occasion myself, and it does work.

--
Conrad J. Sabatier <conrads@cox.net>

"Procrastinate now; don't put it off." -- Ellen Degeneres

On Thu, 20 Mar 2008 02:34:28 +0000, Conrad J. Sabatier wrote:

> Have you tried:
> var=($(function))

Most excellent, thank you.

Stephane CHAZELAS wrote:

> read < file
>
> In the Bourne shell, redirecting also involved forking a new process.

Not in this case (simple commands), - but only when redirecting a
control structure like "while", "for" and "if", as well as "{ list;}"
--
<http://www.in-ulm.de/~mascheck/bourne/common.html>