Websites like amazon.com, newegg.com, bestbuy.com, etc, have pages,
replete with information, for each and every single product they
sell. My question is... where do they get this information from?

Websites like facebook.com and digg.com have API's that let you pull
information in an easy to parse XML format that, in theory, should
never change, even whilst the layout of the website does. Is that
what websites like bestbuy.com does? Does bestbuy.com pull it's
information in an easy-to-parse from it's vendor?

Or maybe bestbuy.com just uses a bunch of regular expressions and
parses it's vendors webpage that way? If so, they're liable to have
to recode their parsing tools every time their vendor gets a facelift.

The third possibility that occurs to me is that maybe they just have
people enter this information into a computer, all day?

Any ideas? If they have an easy to parse API format, is that
something that other vendors are likely to have, as well?

"yawnmoth" <terra1024@yahoo.com> wrote in message
news:1190080457.925765.136030@22g2000hsm.googlegro ups.com...
> Websites like amazon.com, newegg.com, bestbuy.com, etc, have pages,
> replete with information, for each and every single product they
> sell. My question is... where do they get this information from?
>
> Websites like facebook.com and digg.com have API's that let you pull
> information in an easy to parse XML format that, in theory, should
> never change, even whilst the layout of the website does. Is that
> what websites like bestbuy.com does? Does bestbuy.com pull it's
> information in an easy-to-parse from it's vendor?

All this information is provided by the manufacturer/producer/publisher/etc.

The vendor will store this information in their own database.

And/Either:
a) The vendor will check routinely for updates on product information
b) The manufacturer/producer/publisher/etc will contact the vendor when
the product information changes

On Tue, 18 Sep 2007 01:54:17 -0000, yawnmoth put finger to keyboard
and typed:

>Websites like amazon.com, newegg.com, bestbuy.com, etc, have pages,
>replete with information, for each and every single product they
>sell. My question is... where do they get this information from?

Their suppliers, usually.

>Websites like facebook.com and digg.com have API's that let you pull
>information in an easy to parse XML format that, in theory, should
>never change, even whilst the layout of the website does. Is that
>what websites like bestbuy.com does? Does bestbuy.com pull it's
>information in an easy-to-parse from it's vendor?

That depends on the vendor. Some make their data available via XML
over the web, others supply CSV files via FTP, others supply the data
in proprietory format on disk, and some just provide a printed
catalogue.

>Or maybe bestbuy.com just uses a bunch of regular expressions and
>parses it's vendors webpage that way? If so, they're liable to have
>to recode their parsing tools every time their vendor gets a facelift.

Not all vendors put their catalogue on the web, but where they do,
this is often the way it's done.

>The third possibility that occurs to me is that maybe they just have
>people enter this information into a computer, all day?

For some vendors, yes, this is the only way to get the data into your
database.

>Any ideas? If they have an easy to parse API format, is that
>something that other vendors are likely to have, as well?

There are no standards. Each vendor does it their own way.

Mark
--
http://www.BritishSurnames.co.uk - What does your surname say about you?
"Life is bigger, it's bigger than you"

On 18 Sep, 02:54, yawnmoth <terra1...@yahoo.com> wrote:
> Websites like amazon.com, newegg.com, bestbuy.com, etc, have pages,
> replete with information, for each and every single product they
> sell. My question is... where do they get this information from?

I suggest you use pliers, and the vendors' fingernails. 8-(

Given that you're trying to sell the vendors' products for them, it is
still _ridiculously_ difficult to get product information from them,
particularly in a machine-readable form. Most vendors regard this as
some sort of attempted copyright theft and can become downright
aggressive over refusing to supply it.

The few industries I've seen personally that aren't quite so clueless
are mobile phones and DIY power tools. It seems to be an attribute of
the market sector: maybe once one vendor gets a clue, they all have to
to keep up. Fashion is still terrible, and the cost of re-doing good
quality product photography is massive.

If anyone wanted a technical solution, I could witter for days about
the many lovely ways in which we could now do this quickly and easily,
with huge cost savings for all concerned, and better product
descriptions that ed to sell more products. Back in reality
though, it's still a matter of butting heads with the perversely
unful idiots to be found in most sales or marketing departments.