On Oct 16, 6:01 pm, Erik Wikström <Erik-wikst...@telia.com> wrote:
> On 2007-10-16 16:46, sylvaticus wrote:
>
>
>
> > Hello,
> > is it possible to append some data on a text output file appending
> > the data to each row, and without use too much computational I/O ??
>
> > Instead of the classical way:
>
> > VARX VARY VARZ...
> > year1 x1 y1 z1
> > year2 x2 y2 z2
>
> > .. I would like to use the much more redeable:
>
> > year1 year2 ..
> > VARX x1 x2 ...
> > VARY y1 y2 ...
> > VARZ z1 z2 ...
> > ...
>
> > where the 1,2 series are wrote at different times.. a sort of a
> > appendByRow() function.. and I would avoid of loading the whole file
> > in memory and then printing it line-by-line, as I have to do it on
> > approx 40,000 (few KB) files.. and I am worry that the second approach
> > would take years :-)))
>
> Files of that size is really nothing much for a modern computer, not
> even if there are 40,000 of them. In fact, you could probably load them
> all into memory without much trouble (though I think that it would be
> faster to just open them one by one).
>
> > is it possible to do it in a efficient way (e.g. seeking to the right
> > place, delete the newline and appending there) ??
>
> While such an approach would be possible it would require reading the
> whole file into a buffer large enough to hold the final result, and
> would involve a lot of copying making it very slow. By operating on one
> line at at time you get much better memory usage and a much better design.
>
> --
> Erik Wikström

Thanks both.. I will try it opening each one at a time, and if it will
be too slow I'll go for the "normal" approach...