PHWinfo - Afficher un message - Texture/image similarity with ruby? (computer vision)

23/06/2008, 13h09

Axel Etzold wrote on Thu, 19 Jun 2008 20:33:43 +0900

> well, what does similarity of two images/textures mean for you ?

Perceptual similarity as a human subject would experience it. Let me
expand (=ramble) on this:

At the moment I am focusing on following the problem: Given any single
photograph and a random set of photos (20-100), which of the random set
is most similar, perceptually, to the target photo.

I have made some simple tests that divide image into color-channel
components and a luminosity channel, downsample the channels into a
16x16 arrays, and calculates the difference between the target photo to
each of the random ones. Difference hashing its called?

Results are rather confusing. Most of the time perceived similarity (as
I experience it) does not exist, even if statistically the images might
be similar.

IOW, random bw-noise-image gets rather good similarity scores whilst
morphologically having zero resemblance to the target photo.

But anyone who has studied this subject even superficially already knows
this - hashing wont produce the same kind of similarity we experience
with our senses.

> Given a (hopefully large) set of images, you could divide it into a
> training set and a set of images to be classified into sets of mutually
> similar images. One way to perform both the training and the classification
> is using Support Vector Machines. I found these Ruby bindings to a library
> providing these by a Google search, haven't used them yet :
>
> http://sourceforge.net/projects/rubysvm/

This is one of the possible avenues. Gabor features sets used this kind
of approach I believe. Thank you for that link.

But, I don't see this as the most interesting approach. The particular
problem I wrestle with has rather small sets, and training would have to
be performed for every photo. Not really practical or efficient. May be
training a nn is the only way to really do this.

> Most probably, you'll need to read out image information at the pixel level at some point.
> Imagemagick is a very powerful library to do this, and Tim Hunter provides
> a wonderfully rich Ruby binding to it: RMagick.

I have been using RMagick, AND have been looking at Hornets Eye, which
might be useful at some point, but at the moment doesn't cater to this
specific case.

> Psychologists who have conducted image similarity studies compare many different
> measures and filtering methods,e.g., here:
>
> @misc{ karl-perception,
> author = "Dirk Neumann Karl",
> title = "Perception Based Image Retrieval",
> url = "citeseer.ist.psu.edu/635475.html" },

Too bad that service isnt available, as I have not read that article.

> At some point, most of these methods use a (discrete) Fourier transform of some of the
> image information and compare the results of the transforms of two images to assess their
> similarity.
> You could use Ruby-GSL, or Narray with fftw3 to perform that.

Narray is what I was planning to use once theres a working model for the
similarity comparison in place.

E. Borasky - Yes, I have the tools down, but clearly dont have a
suitable perceptual image comparison algo yet.

Thanks also to Ron Fox for pointing out its not going to be easy.

So, I guess I could use the rest of my lunch break to elaborate on the
Question.

What kind of computational algorithm would provide the perceptual
similarity score, rating or hash of some kind between two or more images
that would match the way humans perceive best?

I guess one would need two distinct classifications: similarity of
morphological appearance (features, shapes, ? in image) and similarity
of the colors (of the areas).

I assume there is no answer in existance to this question, well, that I
know of.

23/06/2008, 13h09	#5
Casimir Messages: n/a Hébergeur:	Re: Texture/image similarity with ruby? (computer vision) Axel Etzold wrote on Thu, 19 Jun 2008 20:33:43 +0900 > well, what does similarity of two images/textures mean for you ? Perceptual similarity as a human subject would experience it. Let me expand (=ramble) on this: At the moment I am focusing on following the problem: Given any single photograph and a random set of photos (20-100), which of the random set is most similar, perceptually, to the target photo. I have made some simple tests that divide image into color-channel components and a luminosity channel, downsample the channels into a 16x16 arrays, and calculates the difference between the target photo to each of the random ones. Difference hashing its called? Results are rather confusing. Most of the time perceived similarity (as I experience it) does not exist, even if statistically the images might be similar. IOW, random bw-noise-image gets rather good similarity scores whilst morphologically having zero resemblance to the target photo. But anyone who has studied this subject even superficially already knows this - hashing wont produce the same kind of similarity we experience with our senses. > Given a (hopefully large) set of images, you could divide it into a > training set and a set of images to be classified into sets of mutually > similar images. One way to perform both the training and the classification > is using Support Vector Machines. I found these Ruby bindings to a library > providing these by a Google search, haven't used them yet : > > http://sourceforge.net/projects/rubysvm/ This is one of the possible avenues. Gabor features sets used this kind of approach I believe. Thank you for that link. But, I don't see this as the most interesting approach. The particular problem I wrestle with has rather small sets, and training would have to be performed for every photo. Not really practical or efficient. May be training a nn is the only way to really do this. > Most probably, you'll need to read out image information at the pixel level at some point. > Imagemagick is a very powerful library to do this, and Tim Hunter provides > a wonderfully rich Ruby binding to it: RMagick. I have been using RMagick, AND have been looking at Hornets Eye, which might be useful at some point, but at the moment doesn't cater to this specific case. > Psychologists who have conducted image similarity studies compare many different > measures and filtering methods,e.g., here: > > @misc{ karl-perception, > author = "Dirk Neumann Karl", > title = "Perception Based Image Retrieval", > url = "citeseer.ist.psu.edu/635475.html" }, Too bad that service isnt available, as I have not read that article. > At some point, most of these methods use a (discrete) Fourier transform of some of the > image information and compare the results of the transforms of two images to assess their > similarity. > You could use Ruby-GSL, or Narray with fftw3 to perform that. Narray is what I was planning to use once theres a working model for the similarity comparison in place. E. Borasky - Yes, I have the tools down, but clearly dont have a suitable perceptual image comparison algo yet. Thanks also to Ron Fox for pointing out its not going to be easy. So, I guess I could use the rest of my lunch break to elaborate on the Question. What kind of computational algorithm would provide the perceptual similarity score, rating or hash of some kind between two or more images that would match the way humans perceive best? I guess one would need two distinct classifications: similarity of morphological appearance (features, shapes, ? in image) and similarity of the colors (of the areas). I assume there is no answer in existance to this question, well, that I know of.