Hello,
is it possible to append some data on a text output file appending
the data to each row, and without use too much computational I/O ??

Instead of the classical way:

VARX VARY VARZ...
year1 x1 y1 z1
year2 x2 y2 z2

... I would like to use the much more redeable:

year1 year2 ..
VARX x1 x2 ...
VARY y1 y2 ...
VARZ z1 z2 ...
....

where the 1,2 series are wrote at different times.. a sort of a
appendByRow() function.. and I would avoid of loading the whole file
in memory and then printing it line-by-line, as I have to do it on
approx 40,000 (few KB) files.. and I am worry that the second approach
would take years :-)))

is it possible to do it in a efficient way (e.g. seeking to the right
place, delete the newline and appending there) ??

sylvaticus wrote:
> Hello,
> is it possible to append some data on a text output file appending
> the data to each row, and without use too much computational I/O ??

There is no such thing as "each row" when files are concerned. Each
file is a stream of characters. Only your interpretation of some
specific character (like the "newline" \n for example) makes the file
*appear* to contain "rows" (we usually call them "lines").

>
> Instead of the classical way:
>
> VARX VARY VARZ...
> year1 x1 y1 z1
> year2 x2 y2 z2
>
> .. I would like to use the much more redeable:
>
> year1 year2 ..
> VARX x1 x2 ...
> VARY y1 y2 ...
> VARZ z1 z2 ...
> ...
>
> where the 1,2 series are wrote at different times.. a sort of a
> appendByRow() function.. and I would avoid of loading the whole file
> in memory and then printing it line-by-line, as I have to do it on
> approx 40,000 (few KB) files.. and I am worry that the second approach
> would take years :-)))
>
> is it possible to do it in a efficient way (e.g. seeking to the right
> place, delete the newline and appending there) ??

No. The usual approach is writing a new file where "each line" is
replicated and "expanded" to contain new information.

V
--
Please remove capital 'A's when replying by e-mail
I do not respond to top-posted replies, please don't ask

On 2007-10-16 16:46, sylvaticus wrote:
> Hello,
> is it possible to append some data on a text output file appending
> the data to each row, and without use too much computational I/O ??
>
> Instead of the classical way:
>
> VARX VARY VARZ...
> year1 x1 y1 z1
> year2 x2 y2 z2
>
> .. I would like to use the much more redeable:
>
> year1 year2 ..
> VARX x1 x2 ...
> VARY y1 y2 ...
> VARZ z1 z2 ...
> ...
>
> where the 1,2 series are wrote at different times.. a sort of a
> appendByRow() function.. and I would avoid of loading the whole file
> in memory and then printing it line-by-line, as I have to do it on
> approx 40,000 (few KB) files.. and I am worry that the second approach
> would take years :-)))

Files of that size is really nothing much for a modern computer, not
even if there are 40,000 of them. In fact, you could probably load them
all into memory without much trouble (though I think that it would be
faster to just open them one by one).

> is it possible to do it in a efficient way (e.g. seeking to the right
> place, delete the newline and appending there) ??

While such an approach would be possible it would require reading the
whole file into a buffer large enough to hold the final result, and
would involve a lot of copying making it very slow. By operating on one
line at at time you get much better memory usage and a much better design.

--
Erik Wikström

On Oct 16, 6:01 pm, Erik Wikström <Erik-wikst...@telia.com> wrote:
> On 2007-10-16 16:46, sylvaticus wrote:
>
>
>
> > Hello,
> > is it possible to append some data on a text output file appending
> > the data to each row, and without use too much computational I/O ??
>
> > Instead of the classical way:
>
> > VARX VARY VARZ...
> > year1 x1 y1 z1
> > year2 x2 y2 z2
>
> > .. I would like to use the much more redeable:
>
> > year1 year2 ..
> > VARX x1 x2 ...
> > VARY y1 y2 ...
> > VARZ z1 z2 ...
> > ...
>
> > where the 1,2 series are wrote at different times.. a sort of a
> > appendByRow() function.. and I would avoid of loading the whole file
> > in memory and then printing it line-by-line, as I have to do it on
> > approx 40,000 (few KB) files.. and I am worry that the second approach
> > would take years :-)))
>
> Files of that size is really nothing much for a modern computer, not
> even if there are 40,000 of them. In fact, you could probably load them
> all into memory without much trouble (though I think that it would be
> faster to just open them one by one).
>
> > is it possible to do it in a efficient way (e.g. seeking to the right
> > place, delete the newline and appending there) ??
>
> While such an approach would be possible it would require reading the
> whole file into a buffer large enough to hold the final result, and
> would involve a lot of copying making it very slow. By operating on one
> line at at time you get much better memory usage and a much better design.
>
> --
> Erik Wikström

Thanks both.. I will try it opening each one at a time, and if it will
be too slow I'll go for the "normal" approach...