Dear list

I am currently working on a new project (actually it is very old in my
head >3 years) which I called Rubyfilter.
I know exactly what I want it to do, not bad . But before I ask the
question it might be useful to describe its context.

Rubyfilter will be more or less what the name says, a classic filter
program -as defined in the good ol'Unix way- taking some input and
producing some output.

The way how the output is produced by the input is almost completely
undefined and depends on what I call a mode. My intention is to
provide at least an html mode with the first version and a rich?
toolbox to implement new modes easily.

The mode is defined in the first line of the input and then everything
else is passed into the "handler" as defined for that mode.

It shall be *extremely* easy to write new modes and I have come up
with the following model.


(1) There is one main file to be required called
lib/rubyfilter/runner.rb it defines a class Rubyfilter and a class
method run_file, there will be a commandline frontend bin/rubyfilter
too.

(2) Writing a mode means putting a file with the name <mode>.rb into
lib/rubyfilter/modes and defining an instance method
RubyfilterMode#process_data. This method will be called with two IO
objects the first being the input (containing all input but the
modeline), the second being the output. RubyfilterMode is a module
that will be used to extend Rubyfilter which I intend to use as a
singleton.

(3) lib/rubyfilter/tools contains the toolbox and groups methods
depending on their use into different modules. It is up to the toolbox
user to include these modules into the RubyfilterMode module and thus
into the singleton Rubyfilter or create her own objects with the
behavior of these modules.

These design decisions come from thinking, but thinking cannot replace
doing and I do not have a big experience in code organization. But
this decision surely is crucial for the acceptance of the toolkit by
potential mode implementors and shall not be object to change. I would
therefore love to have some input on this from you folks.

Thanx in advance
Robert

--=20
http://ruby-smalltalk.blogspot.com/

---
Les m=EAmes questions qu'on se pose
On part vers o=F9 et vers qui
Et comme indice pas grand-chose
Des roses et des orties.
-
Francis Cabrel

2008/6/19 Robert Dober <robert.dober@gmail.com>:
> Dear list
>
> I am currently working on a new project (actually it is very old in my
> head >3 years) which I called Rubyfilter.

Hey, this sounds like fun!

> I know exactly what I want it to do, not bad . But before I ask the
> question it might be useful to describe its context.
>
> Rubyfilter will be more or less what the name says, a classic filter
> program -as defined in the good ol'Unix way- taking some input and
> producing some output.
>
> The way how the output is produced by the input is almost completely
> undefined and depends on what I call a mode. My intention is to
> provide at least an html mode with the first version and a rich?
> toolbox to implement new modes easily.
>
> The mode is defined in the first line of the input and then everything
> else is passed into the "handler" as defined for that mode.

I am not sure whether this is a good idea: you force selection of the
algorithm into the data stream. This will make some tasks ugly,
namely you would have to do something like this

{ echo "myMode; ls -lR; } | rubyfilter
{ echo "myMode; ls -lR; } | ruby -r rubyfilter


I'd prefer making the mode part of the filter command line, i.e.

ls -lR | rubyfilter myMode
ls -lR | ruby -r rubyfilter myMode

or even chaining with

ls -lR | rubyfilter myMode,myOtherMode
ls -lR | ruby -r rubyfilter myMode,myOtherMode

> It shall be *extremely* easy to write new modes and I have come up
> with the following model.
>
>
> (1) There is one main file to be required called
> lib/rubyfilter/runner.rb it defines a class Rubyfilter and a class
> method run_file, there will be a commandline frontend bin/rubyfilter
> too.

What about just requiring rubyfilter and let that automatically do the work?

> (2) Writing a mode means putting a file with the name <mode>.rb into
> lib/rubyfilter/modes and defining an instance method
> RubyfilterMode#process_data. This method will be called with two IO
> objects the first being the input (containing all input but the
> modeline), the second being the output. RubyfilterMode is a module
> that will be used to extend Rubyfilter which I intend to use as a
> singleton.

I would at least provide one other filter type that is line based
because this will be a typical scenario. You could do that by
supplying a base class RubyLineFilter which implements #process_data
as a loop and calls another method (say #process_line).

> (3) lib/rubyfilter/tools contains the toolbox and groups methods
> depending on their use into different modules. It is up to the toolbox
> user to include these modules into the RubyfilterMode module and thus
> into the singleton Rubyfilter or create her own objects with the
> behavior of these modules.

I am not sure I have completely understood what you need the singleton
for. Personally I would reverse the registration logic, i.e. not
everybody needs including their filter in a module but rather record
every time a module / class is inherited:

An alternative approach to registration: have a base module / base
class which listens to #inherited and records all instances that are
subclassing it. Make the filter provide a identification method
(could be the class name by default) and update a Hash with the name
to class mapping.

> These design decisions come from thinking, but thinking cannot replace
> doing and I do not have a big experience in code organization. But
> this decision surely is crucial for the acceptance of the toolkit by
> potential mode implementors and shall not be object to change. I would
> therefore love to have some input on this from you folks.

Hopefully I could provide some input despite my limited time.

Kind regards

robert


--
use.inject do |as, often| as.you_can - without end

On Thu, Jun 19, 2008 at 3:22 PM, Robert Klemme
<shortcutter@googlemail.com> wrote:

> I am not sure whether this is a good idea: you force selection of the
> algorithm into the data stream.
Excellent point, that was what I wanted to avoid in the first place.
<snip>
>
> I'd prefer making the mode part of the filter command line, i.e.
>
> ls -lR | rubyfilter myMode
> ls -lR | ruby -r rubyfilter myMode
>
> or even chaining with
>
> ls -lR | rubyfilter myMode,myOtherMode
> ls -lR | ruby -r rubyfilter myMode,myOtherMode
I will probably go for

rubyfilter -m mode
rubyfilter --mode mode

this is the unix way I guess

the rest of your remarks need more thinking, thx.
R

On Jun 19, 2008, at 7:54 AM, Robert Dober wrote:

> On Thu, Jun 19, 2008 at 3:22 PM, Robert Klemme
> <shortcutter@googlemail.com> wrote:
>
>> I am not sure whether this is a good idea: you force selection of the
>> algorithm into the data stream.
> Excellent point, that was what I wanted to avoid in the first place.
> <snip>
>>
>> I'd prefer making the mode part of the filter command line, i.e.
>>
>> ls -lR | rubyfilter myMode
>> ls -lR | ruby -r rubyfilter myMode
>>
>> or even chaining with
>>
>> ls -lR | rubyfilter myMode,myOtherMode
>> ls -lR | ruby -r rubyfilter myMode,myOtherMode
> I will probably go for
>
> rubyfilter -m mode
> rubyfilter --mode mode
>
> this is the unix way I guess
>
> the rest of your remarks need more thinking, thx.
> R
>

i rather like the concept of writing a custom filter entailing

#! /usr/bin/env ruby

require 'filter'
require 'uri'

filter do |line|

uris = URI.extract( line, 'http' ) + URI.extract( line, 'https' ) +
URI.extract( line, 'ftp' )

line = uris.join ','

end


requiring rubyfilter would do, roughly, this

class Filter
def initialize
# ...
end

def run
# ...
end
end

class Object
def filter *a, &b
filter = Filter.new(*a, &b)
filter.run if $0 == __FILE__
end
end


cheers.

a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Thank you both for your input, it showed me that in matter of fact my
concern about flexibility of the approach was a sign of misconception.

What I really want to provide is not a framework but a toolset, I just
shall not care about how it is called, it shall just deliver a maximum
of value for a certain job to do, which happens to be text processing
(just short of parsing, I guess).

My main problem however is to decide what I want to be done by
filtering text and what I want to be done by DSL-like template code
and how to make the two approaches work together.

I guess I am off to the design board again, some rapid prototyping
might too.

Cheers
Robert
--=20
http://ruby-smalltalk.blogspot.com/

---
Les m=EAmes questions qu'on se pose
On part vers o=F9 et vers qui
Et comme indice pas grand-chose
Des roses et des orties.
-
Francis Cabrel

On 20 Jun., 16:29, Robert Dober <robert.do...@gmail.com> wrote:
> Thank you both for your input, it showed me that in matter of fact my
> concern about flexibility of the approach was a sign of misconception.
>
> What I really want to provide is not a framework but a toolset, I just
> shall not care about how it is called, it shall just deliver a maximum
> of value for a certain job to do, which happens to be text processing
> (just short of parsing, I guess).
>
> My main problem however is to decide what I want to be done by
> filtering text and what I want to be done by DSL-like template code
> and how to make the two approaches work together.
>
> I guess I am off to the design board again, some rapid prototyping
> might too.

Are there any particularities about the kind of text processing that
you intend? I can imagine a lot of different things from line based
conversions and filtering, over regular expression manipulation to
parsing of context free (or even sensitive) grammars.

You might actually discover that the toolset you have in mind is
actually Ruby's standard library because your requirements /
processing options are too different.

Kind regards

robert

On Fri, Jun 20, 2008 at 5:13 PM, Robert Klemme
<shortcutter@googlemail.com> wrote:
> On 20 Jun., 16:29, Robert Dober <robert.do...@gmail.com> wrote:
>> Thank you both for your input, it showed me that in matter of fact my
>> concern about flexibility of the approach was a sign of misconception.
>>
>> What I really want to provide is not a framework but a toolset, I just
>> shall not care about how it is called, it shall just deliver a maximum
>> of value for a certain job to do, which happens to be text processing
>> (just short of parsing, I guess).
>>
>> My main problem however is to decide what I want to be done by
>> filtering text and what I want to be done by DSL-like template code
>> and how to make the two approaches work together.
>>
>> I guess I am off to the design board again, some rapid prototyping
>> might too.
>
> Are there any particularities about the kind of text processing that
> you intend? I can imagine a lot of different things from line based
> conversions and filtering, over regular expression manipulation to
> parsing of context free (or even sensitive) grammars.
>
> You might actually discover that the toolset you have in mind is
> actually Ruby's standard library because your requirements /
> processing options are too different.
>
> Kind regards
>
> robert
>
>

Robert you just gave me the "simple" answer to what I really want, and
you are absolutely right it is almost pure Ruby as it turns out to be
a DSL.

Here is a simple example

html do
body do
headline :level =3D> 1, :text =3D> "The Ultimate Solution To All
Problems AKA 42"
table :headers =3D> true, :text =3D> <<-EOT ### And this is the
point I missed until I tried to answer your mail
number meaning
1 one
42 yes
EOT
table :from_file =3D> "some_cvs_file", :format =3D> :cvs
etc.etc.


and now I have just to write lots of tests and do uninteresting work
like implementing them
But I guess it will not be worth a project anymore, so many have
written that kind of DSL already.
And IIRC something like above is already implemented somewhere?

Cheers
Robert

--=20
http://ruby-smalltalk.blogspot.com/

---
Les m=EAmes questions qu'on se pose
On part vers o=F9 et vers qui
Et comme indice pas grand-chose
Des roses et des orties.
-
Francis Cabrel

On 20.06.2008 19:43, Robert Dober wrote:
> On Fri, Jun 20, 2008 at 5:13 PM, Robert Klemme
> <shortcutter@googlemail.com> wrote:
>> On 20 Jun., 16:29, Robert Dober <robert.do...@gmail.com> wrote:

>> You might actually discover that the toolset you have in mind is
>> actually Ruby's standard library because your requirements /
>> processing options are too different.

> Robert you just gave me the "simple" answer to what I really want, and
> you are absolutely right it is almost pure Ruby as it turns out to be
> a DSL.

That's great to see that I could be of any .

> Here is a simple example
>
> html do
> body do
> headline :level => 1, :text => "The Ultimate Solution To All
> Problems AKA 42"
> table :headers => true, :text => <<-EOT ### And this is the
> point I missed until I tried to answer your mail
> number meaning
> 1 one
> 42 yes
> EOT
> table :from_file => "some_cvs_file", :format => :cvs
> etc.etc.

I assume this generates a HTML page, doesn't it?

> and now I have just to write lots of tests and do uninteresting work
> like implementing them
> But I guess it will not be worth a project anymore, so many have
> written that kind of DSL already.
> And IIRC something like above is already implemented somewhere?

I believe class CGI has something similar. And you can even tell it
which HTML version to create.

http://ruby-doc.org/stdlib/libdoc/cg...asses/CGI.html

Kind regards

robert

On Fri, Jun 20, 2008 at 1:43 PM, Robert Dober <robert.dober@gmail.com> wrote:
> On Fri, Jun 20, 2008 at 5:13 PM, Robert Klemme
> <shortcutter@googlemail.com> wrote:
>> On 20 Jun., 16:29, Robert Dober <robert.do...@gmail.com> wrote:
>>> Thank you both for your input, it showed me that in matter of fact my
>>> concern about flexibility of the approach was a sign of misconception.
>>>
>>> What I really want to provide is not a framework but a toolset, I just
>>> shall not care about how it is called, it shall just deliver a maximum
>>> of value for a certain job to do, which happens to be text processing
>>> (just short of parsing, I guess).
>>>
>>> My main problem however is to decide what I want to be done by
>>> filtering text and what I want to be done by DSL-like template code
>>> and how to make the two approaches work together.
>>>
>>> I guess I am off to the design board again, some rapid prototyping
>>> might too.
>>
>> Are there any particularities about the kind of text processing that
>> you intend? I can imagine a lot of different things from line based
>> conversions and filtering, over regular expression manipulation to
>> parsing of context free (or even sensitive) grammars.
>>
>> You might actually discover that the toolset you have in mind is
>> actually Ruby's standard library because your requirements /
>> processing options are too different.
>>
>> Kind regards
>>
>> robert
>>
>>
>
> Robert you just gave me the "simple" answer to what I really want, and
> you are absolutely right it is almost pure Ruby as it turns out to be
> a DSL.
>
> Here is a simple example
>
> html do
> body do
> headline :level => 1, :text => "The Ultimate Solution To All
> Problems AKA 42"
> table :headers => true, :text => <<-EOT ### And this is the
> point I missed until I tried to answer your mail
> number meaning
> 1 one
> 42 yes
> EOT
> table :from_file => "some_cvs_file", :format => :cvs
> etc.etc.
>
>
> and now I have just to write lots of tests and do uninteresting work
> like implementing them
> But I guess it will not be worth a project anymore, so many have
> written that kind of DSL already.
> And IIRC something like above is already implemented somewhere?

Maybe you're thinking about http://code.whytheluckystiff.net/markaby/

Michael Guterl

On Jun 20, 2008, at 11:43 AM, Robert Dober wrote:

> Robert you just gave me the "simple" answer to what I really want, and
> you are absolutely right it is almost pure Ruby as it turns out to be
> a DSL.
>
> Here is a simple example
>
> html do
> body do
> headline :level => 1, :text => "The Ultimate Solution To All
> Problems AKA 42"
> table :headers => true, :text => <<-EOT ### And this is the
> point I missed until I tried to answer your mail
> number meaning
> 1 one
> 42 yes
> EOT
> table :from_file => "some_cvs_file", :format => :cvs
> etc.etc.



it's been done for you

http://codeforpeople.com/lib/ruby/ta...z-4.2.0/README

;-)


a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama