Hi,
I have a huge file having ~500,000 records. I want to remove certain
records satisfying some criteria (that field 3 and 4 both have value
0). I wrote following script, but I am getting errors:

#!/bin/sh
lno=0
while read line
do
lno=`expr $lno + 1`
echo "line read" $lno
annot =`echo $line | cut -f 3 -d " "`
gen = `echo $line | cut -f 4 -d " "`

if [ $annot -eq 0 ]
then
if [ $gen -eq 0 ]
then
echo "delete it"
fi
else
echo $line >> balancedset
fi
done < ./newTrainingset12.dat
-----------------------------------------------#
Errors:
removeoutliers.sh: line 8: annot: command not found
removeoutliers.sh: line 9: gen: command not found
removeoutliers.sh: line 11: [: -eq: unary operator expected


-----
I tried doing the command substitution on the prompt, and it works
there.
could anyone tell me what's wrong here?
Thanks

On 3/23/2008 6:51 PM, musiclover wrote:
> Hi,
> I have a huge file having ~500,000 records. I want to remove certain
> records satisfying some criteria (that field 3 and 4 both have value
> 0).

awk '$3!=0 || $4!=0' file

Ed.

On 2008-03-23, musiclover <oramaster1@gmail.com> wrote:
>
>
> Hi,
> I have a huge file having ~500,000 records. I want to remove certain
> records satisfying some criteria (that field 3 and 4 both have value
> 0). I wrote following script, but I am getting errors:
>
> #!/bin/sh
> lno=0
> while read line
> do
> lno=`expr $lno + 1`
> echo "line read" $lno
> annot =`echo $line | cut -f 3 -d " "`
> gen = `echo $line | cut -f 4 -d " "`
>
When you assign a variable, there should be no space between the
variable name and the = sign.

Ed Morton <morton@lsupcaemnt.com> writes:

> On 3/23/2008 6:51 PM, musiclover wrote:
> > Hi,
> > I have a huge file having ~500,000 records. I want to remove certain
> > records satisfying some criteria (that field 3 and 4 both have value
> > 0).
>
> awk '$3!=0 || $4!=0' file
>
> Ed.

Ed's solution will be much faster. Your solution will be executing 2.5
million programs. Ed's only needs to execute one.