On Sep 17, 4:30 am, Alex Shulgin <alex.shul...@gmail.com> wrote:
> On Sep 17, 12:19 am, Robert Klemme <shortcut...@googlemail.com> wrote:
>
>
>
> > If you do not need to treat every word before it is read from the input
> > you could do this:
>
> > $stdin.each do |line|
> > line.scan /\w+/ do |word|
> > puts word
> > end
> > end
>
> > If your definition of "word" is different (i.e. non whitespace
> > characters) you need a different regexp (for example /\S+/).
>
> > If you want to read to word boundaries only it becomes more difficult.
>
> This is more or less the same code as I use, maybe a bit more
> readable, tough. :-)
>
> So there is no way in Ruby to read words w/o reading the whole line of
> input and then splitting/scanning the line (which takes us two nested
> loops anyway)? Looks very odd to me...
>
> Alex

Awk is a very popular tool for text processing, but there is no
way to make it treat a sequence of whitespace characters as a
record-separator. So in awk, as in Ruby, text is almost always
read a line at a time.
Gawk added the ability to set the record-separator to a
regular expression:

gawk 'BEGIN{RS="[ \t\n]+"} 1'