How to pass an UTF-8 string to a shell script and to save a file as UTF-8?

07/12/2006, 22h53

I have a C++ program that sends a string encoded in UTF-8 to a shell
script. This shell script performs a text replacement in a text file
encoded in UTF-8. The command is like this:

%./TextProc new_string_here

Problem is that when I execute the C++ program, the shell script saves
the file as ISO-8859-1 instead of keeping the original character
encoding (UTF-8). And therefore, Latin/Asian characters are turnt into
squares.

Do you know how I could save my UTF-8 string in the same character set
all the way?
Thanks,

07/12/2006, 23h33

On 7 Dec 2006 14:53:09 -0800, Charles A. Landemaine
<landemaine@gmail.com> wrote:
> I have a C++ program that sends a string encoded in UTF-8 to a shell
> script. This shell script performs a text replacement in a text file
> encoded in UTF-8. The command is like this:
>
> %./TextProc new_string_here
>
> Problem is that when I execute the C++ program, the shell script saves
> the file as ISO-8859-1 instead of keeping the original character
> encoding (UTF-8). And therefore, Latin/Asian characters are turnt into
> squares.
>
> Do you know how I could save my UTF-8 string in the same character set
> all the way?
> Thanks,
>
Make sure the shell script runs in a UTF-8 locale. Start it with
export LANG=en_US.utf8
or replace "en_US" with the appropriate language and country codes.

--
Ever wonder if taxation without representation might have been cheaper?

07/12/2006, 23h37

2006-12-7, 14:53(-08), Charles A. Landemaine:
> I have a C++ program that sends a string encoded in UTF-8 to a shell
> script. This shell script performs a text replacement in a text file
> encoded in UTF-8. The command is like this:
>
> %./TextProc new_string_here
>
> Problem is that when I execute the C++ program, the shell script saves
> the file as ISO-8859-1 instead of keeping the original character
> encoding (UTF-8). And therefore, Latin/Asian characters are turnt into
> squares.
>
> Do you know how I could save my UTF-8 string in the same character set
> all the way?
[...]

Not too sure what you mean, but no UTF-8 character other that
some of the ASCII ones are special to any shell. No shell will
do any conversion of charsets on its own.

You'll have to give more details about what commands are called
within your script and what you want to achieve.

--
Stéphane

08/12/2006, 12h17

Bill Marcum wrote:
> Make sure the shell script runs in a UTF-8 locale. Start it with
> export LANG=en_US.utf8
> or replace "en_US" with the appropriate language and country codes.

Thank you guys! The shells weren't localized into UTF-8. Exporting the
LANG variable fixed it

07/12/2006, 22h53	#1
Charles A. Landemaine Messages: n/a Hébergeur:	How to pass an UTF-8 string to a shell script and to save a file as UTF-8? I have a C++ program that sends a string encoded in UTF-8 to a shell script. This shell script performs a text replacement in a text file encoded in UTF-8. The command is like this: %./TextProc new_string_here Problem is that when I execute the C++ program, the shell script saves the file as ISO-8859-1 instead of keeping the original character encoding (UTF-8). And therefore, Latin/Asian characters are turnt into squares. Do you know how I could save my UTF-8 string in the same character set all the way? Thanks,

07/12/2006, 23h33	#2
Bill Marcum Messages: n/a Hébergeur:	Re: How to pass an UTF-8 string to a shell script and to save a file as UTF-8? On 7 Dec 2006 14:53:09 -0800, Charles A. Landemaine <landemaine@gmail.com> wrote: > I have a C++ program that sends a string encoded in UTF-8 to a shell > script. This shell script performs a text replacement in a text file > encoded in UTF-8. The command is like this: > > %./TextProc new_string_here > > Problem is that when I execute the C++ program, the shell script saves > the file as ISO-8859-1 instead of keeping the original character > encoding (UTF-8). And therefore, Latin/Asian characters are turnt into > squares. > > Do you know how I could save my UTF-8 string in the same character set > all the way? > Thanks, > Make sure the shell script runs in a UTF-8 locale. Start it with export LANG=en_US.utf8 or replace "en_US" with the appropriate language and country codes. -- Ever wonder if taxation without representation might have been cheaper?

07/12/2006, 23h37	#3
Stephane CHAZELAS Messages: n/a Hébergeur:	Re: How to pass an UTF-8 string to a shell script and to save a file as UTF-8? 2006-12-7, 14:53(-08), Charles A. Landemaine: > I have a C++ program that sends a string encoded in UTF-8 to a shell > script. This shell script performs a text replacement in a text file > encoded in UTF-8. The command is like this: > > %./TextProc new_string_here > > Problem is that when I execute the C++ program, the shell script saves > the file as ISO-8859-1 instead of keeping the original character > encoding (UTF-8). And therefore, Latin/Asian characters are turnt into > squares. > > Do you know how I could save my UTF-8 string in the same character set > all the way? [...] Not too sure what you mean, but no UTF-8 character other that some of the ASCII ones are special to any shell. No shell will do any conversion of charsets on its own. You'll have to give more details about what commands are called within your script and what you want to achieve. -- Stéphane

08/12/2006, 12h17	#4
Charles A. Landemaine Messages: n/a Hébergeur:	Re: How to pass an UTF-8 string to a shell script and to save a file as UTF-8? Bill Marcum wrote: > Make sure the shell script runs in a UTF-8 locale. Start it with > export LANG=en_US.utf8 > or replace "en_US" with the appropriate language and country codes. Thank you guys! The shells weren't localized into UTF-8. Exporting the LANG variable fixed it

Outils de la discussion
Afficher une version imprimable Envoyer un lien vers cette page par email