Hi,

Ive got to compare file sizes from two remote linux machines. Because
of which I made the following script. It is not complete cos Im having
the following problem:


1) On the localhost Im doing `ls -l` on the /path/to/files and cutting
the file size.
2) Feeding the output of `ls -l` into an array
3) Doing ssh remote_host "ls -l /path/to/files" and then feeding it
into an array

Im trying to compare the values of the two arrays, and if the the size
of each file is equal Ill put in the `rm`command to delete the files
from the local host..

Ive been able to feed the output into the array, but am not able to
figure out how to compare the value of the arrays....

Would like to have your advise on it....

#!/bin/sh

array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f5`)

len=${#array[*]}

i=0

while [ $i -lt $len ]; do
echo "${array[$i]}"
let i++
done

echo "ssh to 192.168.10.98 starts here"

array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \"
\"|cut -d \" \" -f5 "`

length=${#array1[*]}

j=0

while [ $j -lt $length ]; do
echo "${array1[$j]}"
let j++
done

Thanks
Danish

On 2006-08-19, Danish wrote:
> Hi,
>
> Ive got to compare file sizes from two remote linux machines. Because
> of which I made the following script. It is not complete cos Im having
> the following problem:
>
>
> 1) On the localhost Im doing `ls -l` on the /path/to/files and cutting
> the file size.
> 2) Feeding the output of `ls -l` into an array
> 3) Doing ssh remote_host "ls -l /path/to/files" and then feeding it
> into an array
>
> Im trying to compare the values of the two arrays, and if the the size
> of each file is equal Ill put in the `rm`command to delete the files
> from the local host..
>
> Ive been able to feed the output into the array, but am not able to
> figure out how to compare the value of the arrays....
>
> Would like to have your advise on it....
>
> #!/bin/sh
>
> array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f5`)
>
> len=${#array[*]}
>
> i=0
>
> while [ $i -lt $len ]; do
> echo "${array[$i]}"
> let i++

I recommend using the portable syntax:

i=$(( $i + 1 ))

> done

You don't need a loop for that:

printf "%s\n" "${array[@]}"

> echo "ssh to 192.168.10.98 starts here"
>
> array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \"
> \"|cut -d \" \" -f5 "`

n=0
while [ $n -lt ${#array[@]} ]
do
[ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"
done

> length=${#array1[*]}
>
> j=0
>
> while [ $j -lt $length ]; do
> echo "${array1[$j]}"
> let j++
> done


--
Chris F.A. Johnson, author <http://cfaj.freeshell.org>
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
===== My code in this post, if any, assumes the POSIX locale
===== and is released under the GNU General Public Licence

Chris F.A. Johnson wrote:
> On 2006-08-19, Danish wrote:
> > Hi,
> >
> > Ive got to compare file sizes from two remote linux machines. Because
> > of which I made the following script. It is not complete cos Im having
> > the following problem:
> >
> >
> > 1) On the localhost Im doing `ls -l` on the /path/to/files and cutting
> > the file size.
> > 2) Feeding the output of `ls -l` into an array
> > 3) Doing ssh remote_host "ls -l /path/to/files" and then feeding it
> > into an array
> >
> > Im trying to compare the values of the two arrays, and if the the size
> > of each file is equal Ill put in the `rm`command to delete the files
> > from the local host..
> >
> > Ive been able to feed the output into the array, but am not able to
> > figure out how to compare the value of the arrays....
> >
> > Would like to have your advise on it....
> >
> > #!/bin/sh
> >
> > array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f5`)
> >
> > len=${#array[*]}
> >
> > i=0
> >
> > while [ $i -lt $len ]; do
> > echo "${array[$i]}"
> > let i++
>
> I recommend using the portable syntax:
>
> i=$(( $i + 1 ))
>
> > done
>
> You don't need a loop for that:
>
> printf "%s\n" "${array[@]}"
>
> > echo "ssh to 192.168.10.98 starts here"
> >
> > array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \"
> > \"|cut -d \" \" -f5 "`
>
> n=0
> while [ $n -lt ${#array[@]} ]
> do
> [ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"
> done
>
> > length=${#array1[*]}
> >
> > j=0
> >
> > while [ $j -lt $length ]; do
> > echo "${array1[$j]}"
> > let j++
> > done
>
>
> --
> Chris F.A. Johnson, author <http://cfaj.freeshell.org>
> Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
> ===== My code in this post, if any, assumes the POSIX locale
> ===== and is released under the GNU General Public Licence

Thank you very much for the suggestions, but when I run the script..the
following error is reported....

integer expression expected
../ls2.sh: line 18: [: 0

The following is the script

#!/bin/sh

array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f5`)

len=${#array[*]}

i=$(($i + 1))

printf "%s\n" "${array[@]}"

echo "ssh to 192.168.10.98 starts here"

array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \" \"|cut
-d \" \" -f5 "`

n=0
while [ $n -lt ${#array1[@]} ]
do
[ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"
done


Thanks you
Danish

On 21 Aug 2006 02:35:54 -0700, Danish wrote:
[...]
> integer expression expected
> ./ls2.sh: line 18: [: 0
[...]
> n=0
> while [ $n -lt ${#array1[@]} ]
> do
> [ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"
> done
[...]

Chances are the file is in MSDOS text formant where lines are
terminated by the <CR><LF> sequence instead of <LF> as Unix
expects it.

So $n contains "0<CR>" which is not a valid integer expression.

Use sed l < your-script to identify where the CRs are.

Note that $n is not incremented in your loop.

You're using arrays in a sh script. Arrays are not a standard sh
feature, you should specify which specific sh implementation
that script is intended for as you rely on non-standard
extension.

So

#! /bin/bash -

or

#! /bin/ksh -

....

--
Stephane

Stephane Chazelas wrote:
> On 21 Aug 2006 02:35:54 -0700, Danish wrote:
> [...]
> > integer expression expected
> > ./ls2.sh: line 18: [: 0
> [...]
> > n=0
> > while [ $n -lt ${#array1[@]} ]
> > do
> > [ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"
> > done
> [...]
>
> Chances are the file is in MSDOS text formant where lines are
> terminated by the <CR><LF> sequence instead of <LF> as Unix
> expects it.
>
> So $n contains "0<CR>" which is not a valid integer expression.
>
> Use sed l < your-script to identify where the CRs are.
>
> Note that $n is not incremented in your loop.
>
> You're using arrays in a sh script. Arrays are not a standard sh
> feature, you should specify which specific sh implementation
> that script is intended for as you rely on non-standard
> extension.
>
> So
>
> #! /bin/bash -
>
> or
>
> #! /bin/ksh -
>
> ...
>
> --
> Stephane

I did sed my-script.sh

But it gave me the following error....

#sed ls2.sh
sed: -e expression #1, char 2: extra characters after command

Thanks
Danish

On 21 Aug 2006 04:44:55 -0700, Danish wrote:
[...]
>> Use sed l < your-script to identify where the CRs are.
[...]
> I did sed my-script.sh
>
> But it gave me the following error....
>
> #sed ls2.sh
> sed: -e expression #1, char 2: extra characters after command
[...]

I wrote:

sed l < ls2.sh

That the letter l (el) after sed.

Or

cat -vte < ls2.sh

or

od -c < ls2.sh

Or

vi ls2.sh

Then

:set list

within vi.

--
Stephane

Stephane Chazelas wrote:
> On 21 Aug 2006 04:44:55 -0700, Danish wrote:
> [...]
> >> Use sed l < your-script to identify where the CRs are.
> [...]
> > I did sed my-script.sh
> >
> > But it gave me the following error....
> >
> > #sed ls2.sh
> > sed: -e expression #1, char 2: extra characters after command
> [...]
>
> I wrote:
>
> sed l < ls2.sh
>
> That the letter l (el) after sed.
>
> Or
>
> cat -vte < ls2.sh
>
> or
>
> od -c < ls2.sh
>
> Or
>
> vi ls2.sh
>
> Then
>
> :set list
>
> within vi.
>
> --
> Stephane

Sorry for being a dud...Actually Im really troubled by this
script..Here is the output after dong

#sed l < ls2.sh
and
# cat -vte ls2.sh

#!/bin/sh$
$
array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f5`)$
$
len=${#array[*]}$
$
i=$(($i + 1))$
$
printf "%s\n" "${array[@]}" $
$
echo "ssh to 192.168.10.98 starts here"$
$
array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \" \"|cut
-d \" \" -f5 "`$
$
n=0$
while [ $n -lt ${#array1[@]} ]$
do$
[ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"$
done$
$
$

Thanks
Danish

On 21 Aug 2006 05:32:58 -0700, Danish wrote:
[...]
> array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \" \"|cut
> -d \" \" -f5 "`$

That's not how you define an array.

> $
> n=0$
> while [ $n -lt ${#array1[@]} ]$
> do$
> [ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"$

Then it will be that "[" invocation that fails. Maybe $array,
$array1 contain <CR>s.

Try ./ls2.sh 2>&1 | sed l

To see what "invalid integer expression" [ is complaining about.

--
Stephane

Stephane Chazelas wrote:
> On 21 Aug 2006 05:32:58 -0700, Danish wrote:
> [...]
> > array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \" \"|cut
> > -d \" \" -f5 "`$
>
> That's not how you define an array.
>
> > $
> > n=0$
> > while [ $n -lt ${#array1[@]} ]$
> > do$
> > [ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"$
>
> Then it will be that "[" invocation that fails. Maybe $array,
> $array1 contain <CR>s.
>
> Try ./ls2.sh 2>&1 | sed l
>
> To see what "invalid integer expression" [ is complaining about.
>
> --
> Stephane

Thanks for your efforts. I appreciate it....

After running
#./ls2.sh 2>&1 | sed l
# ./ls2.sh 2>&1 | sed l
0$
0
0$
0
0$
0
0$
0
102$
102
24$
24
0$
0
70$
70
ssh to 192.168.10.98 starts here$
ssh to 192.168.10.98 starts here
../ls2.sh: line 18: [: : integer expression expected
../ls2.sh: line 18: [: : integer expression expected$
../ls2.sh: line 18: [: : integer expression expected
../ls2.sh: line 18: [: : integer expression expected$
../ls2.sh: line 18: [: : integer expression expected
../ls2.sh: line 18: [: : integer expression expected$

The above error then goes into an infinite loop...

> That's not how you define an array.
If I was wrong can you please give me an example as to how to define an
array



Thanks
Danish

On 22 Aug 2006 02:56:40 -0700, Danish wrote:
[...]
>> > array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \" \"|cut
>> > -d \" \" -f5 "`$
>>
>> That's not how you define an array.
>>
>> > $
>> > n=0$
>> > while [ $n -lt ${#array1[@]} ]$
>> > do$
>> > [ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"$
[...]
> ./ls2.sh: line 18: [: : integer expression expected

That's not the error you had initially.

Now, "[" is telling you that either operand of "-ne" on line 18
is the empty string.

> ./ls2.sh: line 18: [: : integer expression expected$
> ./ls2.sh: line 18: [: : integer expression expected
> ./ls2.sh: line 18: [: : integer expression expected$
> ./ls2.sh: line 18: [: : integer expression expected
> ./ls2.sh: line 18: [: : integer expression expected$
>
> The above error then goes into an infinite loop...

You don't increment $n, so there's no way you could break out of
the loop.

>> That's not how you define an array.
> If I was wrong can you please give me an example as to how to define an
> array
[...]

You did it right for $array.

array=(list of values)

or

set -A array -- list of values # ksh before ksh93

Not

normal_variable=one_single_value


--
Stephane

Stephane Chazelas wrote:
> On 22 Aug 2006 02:56:40 -0700, Danish wrote:
> [...]
> >> > array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \" \"|cut
> >> > -d \" \" -f5 "`$
> >>
> >> That's not how you define an array.
> >>
> >> > $
> >> > n=0$
> >> > while [ $n -lt ${#array1[@]} ]$
> >> > do$
> >> > [ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"$
> [...]
> > ./ls2.sh: line 18: [: : integer expression expected
>
> That's not the error you had initially.
>
> Now, "[" is telling you that either operand of "-ne" on line 18
> is the empty string.
>
> > ./ls2.sh: line 18: [: : integer expression expected$
> > ./ls2.sh: line 18: [: : integer expression expected
> > ./ls2.sh: line 18: [: : integer expression expected$
> > ./ls2.sh: line 18: [: : integer expression expected
> > ./ls2.sh: line 18: [: : integer expression expected$
> >
> > The above error then goes into an infinite loop...
>
> You don't increment $n, so there's no way you could break out of
> the loop.
>
> >> That's not how you define an array.
> > If I was wrong can you please give me an example as to how to define an
> > array
> [...]
>
> You did it right for $array.
>
> array=(list of values)
>
> or
>
> set -A array -- list of values # ksh before ksh93
>
> Not
>
> normal_variable=one_single_value
>
>
> --
> Stephane

Ive got mixed up again...Apologise for that...Im posting the original
script by doing

# sed l < ls2.sh
#!/bin/sh$
#!/bin/sh
$

array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f5`)$
array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f5`)
$

len=${#array[*]}$
len=${#array[*]}
$

i=$(($i + 1))$
i=$(($i + 1))
$

printf "%s\\n" "${array[@]}" $
printf "%s\n" "${array[@]}"
$

echo "ssh to 192.168.10.98 starts here"$
echo "ssh to 192.168.10.98 starts here"
$

array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \\" \\"|c\
ut -d \\" \\" -f5 "`$
array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \" \"|cut
-d \" \" -f5 "`
$

n=0$
n=0
while [ $n -lt ${#array1[@]} ]$
while [ $n -lt ${#array1[@]} ]
do$
do
[ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"$
[ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"
done$
done
$

$

Then I did..

# ./ls2.sh 2>&1 | sed l
0$
0
0$
0
0$
0
0$
0
102$
102
24$
24
0$
0
70$
70
ssh to 192.168.10.98 starts here$
ssh to 192.168.10.98 starts here
../ls2.sh: line 18: [: 0$
../ls2.sh: line 18: [: 0
0$
0
0$
0
0$
0
102$
102
24$
24
0$
0
70: integer expression expected$
70: integer expression expected
../ls2.sh: line 18: [: 0$
../ls2.sh: line 18: [: 0

After which I incremented "n" in the block
do
[ "${array[$n]}" -ne "${array1[$n]}" ] && echo "Error"
let n++
done

after which I got the following error...

# ./ls2.sh
0
0
0
0
102
24
0
70
ssh to 192.168.10.98 starts here
../ls2.sh: line 18: [: 0
0
0
0
102
24
0
70: integer expression expected

Thanks
Danish

On 22 Aug 2006 04:38:05 -0700, Danish wrote:
[...]
> array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f5`)

OK.

> array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \\" \\"|c\
> ut -d \\" \\" -f5 "`$

not OK

The above is equivelant to:

array1[0]=`ssh....`

Or array1=("`ssh...`")

You want:

array1=(`ssh...`)
or
set -A -- `ssh...`

as you want each word from the output of ssh to be put in a
separate array element rather than the whole output of ssh to be
put in the first element of array1.

--
Stephane

Stephane Chazelas wrote:
> On 22 Aug 2006 04:38:05 -0700, Danish wrote:
> [...]
> > array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f5`)
>
> OK.
>
> > array1=`ssh 192.168.10.98 " vdir /home/danish/scp/* | tr -s \\" \\"|c\
> > ut -d \\" \\" -f5 "`$
>
> not OK
>
> The above is equivelant to:
>
> array1[0]=`ssh....`
>
> Or array1=("`ssh...`")
>
> You want:
>
> array1=(`ssh...`)
> or
> set -A -- `ssh...`
>
> as you want each word from the output of ssh to be put in a
> separate array element rather than the whole output of ssh to be
> put in the first element of array1.
>
> --
> Stephane

After your suggestions, I modified the code a bit..Im pasting the code
below.

#!/bin/sh

array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f9`)

len=${#array[*]}

i=$(($i + 1))

printf "%s\n" "${array[@]}"

echo "ssh to 192.168.10.98 starts here"

array1=(`ssh 192.168.10.98 " cd /home/danish/scp; vdir * | tr -s \"
\"|cut -d \" \" -f9 "`)

length=${#array1[*]}

j=$(($j + 1))

printf "%s\n" "${array1[@]}"

This portion is working perfectly well. Now I would like to equate the
names of both the files ( on localhost as well as on the remote server.
For eg..If after running the script I get as output, the names of files
"a", "b" on both the machines. After comparing the names of the files
the script should delete the files "a", "b" on the local machine, if
they are present on both the machines..

I was wondering if this comparing can be done using the array
elements...like

if [ $array[0] == $array1[0] ]
then
remove files on localmachine
fi

Thanks
Danish

On 23 Aug 2006 04:01:43 -0700, Danish wrote:
[...]
> After your suggestions, I modified the code a bit..Im pasting the code
> below.
>
> #!/bin/sh
>
> array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f9`)
>
> len=${#array[*]}
>
> i=$(($i + 1))
>
> printf "%s\n" "${array[@]}"
>
> echo "ssh to 192.168.10.98 starts here"
>
> array1=(`ssh 192.168.10.98 " cd /home/danish/scp; vdir * | tr -s \"
> \"|cut -d \" \" -f9 "`)
>
> length=${#array1[*]}
>
> j=$(($j + 1))
>
> printf "%s\n" "${array1[@]}"
>
> This portion is working perfectly well. Now I would like to equate the
> names of both the files ( on localhost as well as on the remote server.
> For eg..If after running the script I get as output, the names of files
> "a", "b" on both the machines. After comparing the names of the files
> the script should delete the files "a", "b" on the local machine, if
> they are present on both the machines..
[...]

Ok, you want to compare the content of two directories one local
and one remote, and you want to delete the local files, if a
file by the same name can be found on the remote machine?

Then:

#! /bin/sh -

cd /home/danish/scp || exit

{
ls # list local files
ssh -n 192.168.10.98 '
cd /home/danish/scp && ls' # list remote files
} | sort | uniq -d | sed 's/./\\&/g' | xargs rm -f --
# uniq -d reports the dupplicates, sed makes that output
# compatible with xargs expected input format and xargs gives
# as argument to rm the names of the files it read from its
# standard input

There's no need for arrays. That's not how you do shell
scripting.

--
Stephane

Stephane Chazelas wrote:
> On 23 Aug 2006 04:01:43 -0700, Danish wrote:
> [...]
> > After your suggestions, I modified the code a bit..Im pasting the code
> > below.
> >
> > #!/bin/sh
> >
> > array=(`ls -l /home/danish/scp | tr -s " " | cut -d " " -f9`)
> >
> > len=${#array[*]}
> >
> > i=$(($i + 1))
> >
> > printf "%s\n" "${array[@]}"
> >
> > echo "ssh to 192.168.10.98 starts here"
> >
> > array1=(`ssh 192.168.10.98 " cd /home/danish/scp; vdir * | tr -s \"
> > \"|cut -d \" \" -f9 "`)
> >
> > length=${#array1[*]}
> >
> > j=$(($j + 1))
> >
> > printf "%s\n" "${array1[@]}"
> >
> > This portion is working perfectly well. Now I would like to equate the
> > names of both the files ( on localhost as well as on the remote server.
> > For eg..If after running the script I get as output, the names of files
> > "a", "b" on both the machines. After comparing the names of the files
> > the script should delete the files "a", "b" on the local machine, if
> > they are present on both the machines..
> [...]
>
> Ok, you want to compare the content of two directories one local
> and one remote, and you want to delete the local files, if a
> file by the same name can be found on the remote machine?
>
> Then:
>
> #! /bin/sh -
>
> cd /home/danish/scp || exit
>
> {
> ls # list local files
> ssh -n 192.168.10.98 '
> cd /home/danish/scp && ls' # list remote files
> } | sort | uniq -d | sed 's/./\\&/g' | xargs rm -f --
> # uniq -d reports the dupplicates, sed makes that output
> # compatible with xargs expected input format and xargs gives
> # as argument to rm the names of the files it read from its
> # standard input
>
> There's no need for arrays. That's not how you do shell
> scripting.
>
> --
> Stephane

I copied you script and named it "remove_files.sh". But when I ran it
,the following error was reported.

../remove_files.sh
sed: -e expression #1, char 8: unterminated `s' command
#

Thanks
Danish

On 23 Aug 2006 05:47:22 -0700, Danish wrote:
[...]
>> #! /bin/sh -
>>
>> cd /home/danish/scp || exit
>>
>> {
>> ls # list local files
>> ssh -n 192.168.10.98 '
>> cd /home/danish/scp && ls' # list remote files
>> } | sort | uniq -d | sed 's/./\\&/g' | xargs rm -f --
[...]
> I copied you script and named it "remove_files.sh". But when I ran it
> ,the following error was reported.
>
> ./remove_files.sh
> sed: -e expression #1, char 8: unterminated `s' command
[...]

That script as it is may have some errors as I've not tested it.
However, the sed command should be OK.

I'd suggest you double check what you've copied into your file,
and maybe as well start to think by yourself, and read sed man
page for instance to understand why it doesn't work.

--
Stephane

Stephane Chazelas wrote:
> On 23 Aug 2006 05:47:22 -0700, Danish wrote:
> [...]
> >> #! /bin/sh -
> >>
> >> cd /home/danish/scp || exit
> >>
> >> {
> >> ls # list local files
> >> ssh -n 192.168.10.98 '
> >> cd /home/danish/scp && ls' # list remote files
> >> } | sort | uniq -d | sed 's/./\\&/g' | xargs rm -f --
> [...]
> > I copied you script and named it "remove_files.sh". But when I ran it
> > ,the following error was reported.
> >
> > ./remove_files.sh
> > sed: -e expression #1, char 8: unterminated `s' command
> [...]
>
> That script as it is may have some errors as I've not tested it.
> However, the sed command should be OK.
>
> I'd suggest you double check what you've copied into your file,
> and maybe as well start to think by yourself, and read sed man
> page for instance to understand why it doesn't work.
>
> --
> Stephane

> I'd suggest you double check what you've copied into your file,
> and maybe as well start to think by yourself, and read sed man
> page for instance to understand why it doesn't work.

Yes, you are correct...Im acting like a dud, but basically this script
is troubling me a lot..Ill get back to you as soon as I finish it

Thanks
Danish