I'm not too good with awk, but I don't think this can be done
with sed.

I pass email/usenet messages through my message filter to 'clean'
them up. Currently I have the script below [1](with some personal
lines removed). As you can see, the message is piped through
sed, then t-prot (a tidying up script I found by Jochen Striepe),
then finally 'mail-to-filter' (a perl script by Gary Johnson).

I want to add an awk script that adds a blank line to messages just
above the signature delimiter if there isn't a blank line there
already. So if the end of the email ends with:

....
Thanks,
Fred
--
Fred's sig here
blahblah

I'd like my display filter to add blank line like this:

....
Thanks,
Fred

--
Fred's sig here
blahblah

If there was already a blank line there, then do nothing.
Any ideas or pushes in the right direction?

[1]
-----8<-----
#!/bin/bash

sed -e '

# fix incorrect sig delimiters
s/^--$/-- /
s/^_____.*/-- /
s/^\*\*\*\*\*.*/-- /

# some lines deleted

' \
\
| t-prot -aceklmt -S=5 --bigq=20,10 --diff -Mmutt --sigsmax
--spass -L${HOME}/.mutt/mlfooters -A${HOME}/.mutt/adfooters \
\
| mail-to-filter
-----8<-----

--
Troy Piggins
,-o
o )
`-o

Troy Piggins wrote:
> I'm not too good with awk, but I don't think this can be done
> with sed.
>

sed -e '
$!N
/^\n/{P;D;}
/\n.*signature/!{P;D;}
s/\n/&&/
'

* Rakesh Sharma <sharma__r@hotmail.com> :
>
> Troy Piggins wrote:
> > I'm not too good with awk, but I don't think this can be done
> > with sed.

Apparently not too good with sed either ;-\

> sed -e '

If this can be done with sed then great, but I'll need a little
walk-through on your solution. It's a little above my
current/basic sed knowledge. I've been reading
http://sed.sourceforge.net/sedfaq.html and other resources, but
can't find one that satisfactorily explains what you've done.

> $!N

This adds Next line to the pattern space, but not the last line
of file?

> /^\n/{P;D;}

This matches beginning immediately followed by newline, not sure
what the {P;D;} part is?

> /\n.*signature/!{P;D;}

The word signature here throws me - should it be the sig
delimiter '-- '? Again not sure about {P;D;}?

> s/\n/&&/

&& ?

> '

Thanks for you , just need a little explanation of the above.

--
Troy Piggins
,-o
o ) Ubuntu linux 6.06 http://ubuntu.com RLU#415538 http://counter.li.org
`-o uptime: 17:35:59 up 12 days,10:32,2 users,load average:0.00,0.00,0.00

the tool 'sed' normally operates on a line-by-line basis with it's
input. but in your case we need to carry with us two lines all
the time, since we need to base our decision after looking at the
signature line and also the line preceding it (whether it was a blank
or not).

$!N -> will make the pattern space comprise 2 lines separated by
a newline (\n), unless we reach the end of file.

now we need to decide whether the first part (i.e., the part of pattern
space preceding the \n is a blank or not). now, it is a blank line
then we discard any further processing on that part since we are
interested in a non-blank line followed by our signature line.

whence,
/^\n/{P;D;} -> will print the first part (viz., portion preceding the
\n)
and also delete the first part. after the D, we go back to top of
script
in effect read in the next line.

once we jump upto here, that means the first part is a nonblank and we
need further tests to verify whether the next line was a signature
line.

N.B. I didn't know how u look for a signature line. so i just put in
the
regular expression 'signature'.

so we check whether the second portion (viz., portion after the \n)
contains the signature. if it doesn't then just print out the first
portion
and delete that part and go onto read in the next line.

lastly if it matches our condition of a non-blank line followed by
the signature line, *** WHAT WE NEED ***, then we need to insert
a blank line in between these.

s/\n/&&/ -> will insert a blank line.

So we can recast our code as under:

sed -e '
## read in the next line also
$!N

## first part is blank, we dont need this so
## print out first portion n delete that portion
/^\n/{
P
D
}

## first part is blank, but we must test the 2nd part now
## 2nd part doesnot contain signature then print 1st part n delete
it
/\n.*signature/!{
P
D
}

## if we are able to reach here then that means first part was a
nonblank
## and 2nd part had a signature. SO WE MUST INSERT A BLANK HERE
s/\n/&&/
' yourfile

************************************************** ************************************************** ********

Troy Piggins wrote:
> * Rakesh Sharma <sharma__r@hotmail.com> :
>
>>Troy Piggins wrote:
>>
>>>I'm not too good with awk, but I don't think this can be done
>>>with sed.
>
>
> Apparently not too good with sed either ;-\
>
>
>>sed -e '
>
>
> If this can be done with sed then great,

Not really. Isn't this:

awk '
/^--$/ && !prev { print "" }
{ print; prev = $0 }
'

a little more easily understandable than the posted sed solution:

sed -e '
$!N
/^\n/{P;D;}
/\n.*signature/!{P;D;}
s/\n/&&/
'

sed is a fine tool for simple operations, but there's just no point
attempting anything more with it.

Regards,

Ed.

* Rakesh Sharma <sharma__r@hotmail.com> :
>
> the tool 'sed' normally operates on a line-by-line basis with it's
> input. but in your case we need to carry with us two lines all
> the time, since we need to base our decision after looking at the
> signature line and also the line preceding it (whether it was a blank
> or not).
>
> $!N -> will make the pattern space comprise 2 lines separated by
> a newline (\n), unless we reach the end of file.
[excellent description snipped]

Thankyou Rakesh for taking the time to type all that. Your
patience is appreciated. I think I understand most of what you
said. I'll read some more.

For the record, I added the following lines into my script and it
works a treat:

sed -e '
# snip other lines

# put a space before sig delimiters
$!N
/^\n/{P;D;}
/\n.*-- /!{P;D;}
s/\n/&&/
'

The standard email sig delimiter, as you know, is "-- ".

Cheers

--
Troy Piggins
,-o
o ) Ubuntu linux 6.06 http://ubuntu.com RLU#415538 http://counter.li.org
`-o uptime: 23:03:56 up 12 days,16:00,2 users,load average:0.10,0.12,0.05

* Ed Morton <morton@lsupcaemnt.com> :
> Troy Piggins wrote:
> >* Rakesh Sharma <sharma__r@hotmail.com> :
> >>Troy Piggins wrote:
> >>
> >>>I'm not too good with awk, but I don't think this can be done
> >>>with sed.
> >Apparently not too good with sed either ;-\
> >>sed -e '
> >If this can be done with sed then great,
>
> Not really. Isn't this:
>
> awk '
> /^--$/ && !prev { print "" }
> { print; prev = $0 }
> '
>
> a little more easily understandable than the posted sed
> solution:
>
> sed -e '
> $!N
> /^\n/{P;D;}
> /\n.*signature/!{P;D;}
> s/\n/&&/
> '

Agreed.

> sed is a fine tool for simple operations, but there's just no
> point attempting anything more with it.

That's what I thought, and that's why my OP assumed awk would be
the tool. Thankyou for your solution. I'll do some reading
tomorrow when I get a chance to disect it. It certainly reads
easy.

Cheers

--
Troy Piggins
,-o
o ) Ubuntu linux 6.06 http://ubuntu.com RLU#415538 http://counter.li.org
`-o uptime: 23:07:48 up 12 days,16:04,2 users,load average:0.00,0.05,0.03

Troy Piggins wrote:
> I'm not too good with awk, but I don't think this can be done
> with sed.
>
> I pass email/usenet messages through my message filter to 'clean'
> them up. Currently I have the script below [1](with some personal
> lines removed). As you can see, the message is piped through
> sed, then t-prot (a tidying up script I found by Jochen Striepe),
> then finally 'mail-to-filter' (a perl script by Gary Johnson).
>
> I want to add an awk script that adds a blank line to messages just
> above the signature delimiter if there isn't a blank line there
> already. So if the end of the email ends with:
>
> ...
> Thanks,
> Fred
> --
> Fred's sig here
> blahblah
>
> I'd like my display filter to add blank line like this:
>
> ...
> Thanks,
> Fred
>
> --
> Fred's sig here
> blahblah
>
> If there was already a blank line there, then do nothing.
> Any ideas or pushes in the right direction?
>
> [1]

There are some more ways:

sed '/^--/{g;s/^$/--/;t;s/.*/\n--/};h'
awk 'BEGIN{RS=ORS="\n--"}sub(/\n*$/,"\n")'
perl -0777pe 's/(?<=\S)(?=\n--)/\n/'
perl -00pe 's/(?=\n--)/\n/'

Xicheng

* Xicheng Jia <xicheng@gmail.com> :
> Troy Piggins wrote:
> > I'm not too good with awk, but I don't think this can be done
> > with sed.
> >
> > I pass email/usenet messages through my message filter to 'clean'
> > them up. Currently I have the script below [1](with some personal
> > lines removed). As you can see, the message is piped through
> > sed, then t-prot (a tidying up script I found by Jochen Striepe),
> > then finally 'mail-to-filter' (a perl script by Gary Johnson).
> >
> > I want to add an awk script that adds a blank line to messages just
> > above the signature delimiter if there isn't a blank line there
> > already. So if the end of the email ends with:
> >
> > ...
> > Thanks,
> > Fred
> > --
> > Fred's sig here
> > blahblah
> >
> > I'd like my display filter to add blank line like this:
> >
> > ...
> > Thanks,
> > Fred
> >
> > --
> > Fred's sig here
> > blahblah
> >
> > If there was already a blank line there, then do nothing.
> > Any ideas or pushes in the right direction?
> >
> > [1]
>
> There are some more ways:
>
> sed '/^--/{g;s/^$/--/;t;s/.*/\n--/};h'

I've adopted this one. Note that there should be a space after
the "--"

sed '/^-- /{g;s/^$/-- /;t;s/.*/\n-- /};h'

> awk 'BEGIN{RS=ORS="\n--"}sub(/\n*$/,"\n")'
> perl -0777pe 's/(?<=\S)(?=\n--)/\n/'
> perl -00pe 's/(?=\n--)/\n/'

Cheers

--
Troy Piggins
,-o
o ) Ubuntu linux 6.06 http://ubuntu.com RLU#415538 http://counter.li.org
`-o uptime: 10:30:06 up 13 days,3:26,2 users,load average:0.00,0.00,0.00