On Sat, 14 Oct 2006, Gary R. Schmidt wrote:
> > SVR4 only has POSIX style fcntl() locking. The BSD man pages call the
> > semantics of POSIX locking "completely stupid"; I don't disagree. I
> > have quoted the relevant paragraph from the BSD man page many times
> > already; I hope that I don't have to do so again. UW imapd is obliged
> > to do considerable more work on SVR4 systems than it does on BSD and
> > Linux systems which offer flock() locking as an alternative to POSIX
> > locking.
> There's been an flock() on top of fcntl in System V since Release 2.

I'm afraid that you completely misunderstood the issue. "Semantics" have
nothing to do with a calling interface. It has to do with what gets done
by the call.

IEEE Std 1003.1-1988 (a.k.a. POSIX.1) requires that all fcntl() locks
associated with a file for a given process are removed when *any* file
descriptor for that file is closed by that process. This, in turn,
requires a global knowledge of file activity between an application and
all of the API/libraries that the application uses.

Put another way, before any library routine opens a file, it must be aware
of what files the application and any other libraries have open and
locked; otherwise the library routine will remove the lock unexpectedly
when it closes the file.

This gets even more fun when an server is multi-threaded, and is servicing
clients that have no knowledge of each other.

Solaris' flock() on top of fcntl() does not address this problem.
Solaris's pseudo flock() call, being based upon fcntl(), has the problem.
The programmer (or the autoconf) must be aware that Solaris' flock() is
not real flock(), and thus can not assume that "since I am using flock(),
the locks are safe from this problem."

Linux and BSD have both fcntl() and real flock() locking, and these locks
operate independently of each other. Tru64 UNIX once did as well, but
later on they removed real flock() in favor of a Solaris style jacket.

Please read the BSD man page on fcntl() locking, which has a detailed
paragraph on this matter, before commenting on this issue further.

Not that flock() locking is perfect -- far from it! flock() does not have
true lock promotion/demotion. Instead, an attempt to promote/demote a
lock *frees* the current lock and then seeks to acquire a lock of the new
type.

> > Many (most?) SVR4 filesystems are poor performers compared to BSD and
> > Linux filesystems.
> Are they? Please quote some reliable _reproducible_ figures.

As our few remaining SVR4 systems are being retired, I don't have much
opportunity to do new testing. I know what I experienced: we had to make
/tmp be a RAMdisk, since even zero-length files on /tmp tied up the disk
too much; directory access slowed down dramatically as the number of files
in the directory increased (news spools!); etc.

This is the first time that I have heard someone assert that SVR4
filesystems perform comparably to BSD and Linux filesystems. Perhaps
newer SVR4 filesystems do so. Do you have some "reliable reproducible
figures" that show a Solaris filesystem performing comparably to Linux?

If so, that is interesting. But, since Solaris is increasingly irrelevant
while industry standardizes upon Linux, it doesn't change much.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Sat, 14 Oct 2006, Willi Burmeister wrote:
> >and with other versions of Solaris.
> That's simply not true. Compatibility has always been the major point for
> Solaris. We have 15 year old software running without any problems on the
> newest release of Solaris. And even old drivers will run on newer releases.

Just to pick one example:

ustat() is a call that changed semantics incompatibly between versions of
Solaris. SUN eventually called ustat() "deprecated" and said to use
statfs() or statvfs(), both of which were unstable for a few versions
before their semantics got nailed down. No portable application can
depend upon any of these functions; a binary that uses one of these
functions and works in an earlier version of Solaris can fail with a later
version of Solaris.

> >> >Solaris is SVR4 based, which is
> >> >not a recommended platform due to its deficiencies
> >> what deficiencies?
> >The deficiencies impact performance and reliability.

The lack of flock() locking is a big one.

The nightmare of rpc.statd/rpc.lockd is related. For years, there was a
game with SUN where they claimed that rpc.statd/rpc.lockd were "fixed",
and I would proceed to demonstrate (by locking up an entire cluster) that
it wasn't.

> P.S. http://www.softpanorama.org/Articles...vs_linux.shtml

ext2 used as the standard to judge Linux filesystem performance?!?

Yeah, right.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Sat, 14 Oct 2006, Mark Crispin wrote:
> IEEE Std 1003.1-1988 (a.k.a. POSIX.1) requires that all fcntl() locks
> associated with a file for a given process are removed when *any* file
> descriptor for that file is closed by that process.

To add to the fun, Cygwin implements locking by using Windows locks, which
are mandatory instead of advisory.

One of the attractions of the maildir format that is frequently (and
justifiably!) cited by its fan community is that maildir does not depend
upon locking at all and thus supposedly avoids the problems faced by
formats that depend upon various locking. Not everybody agrees that
maildir really works without locking (e.g., see the Wikipedia article
about maildir), but to the extent that it is true it is attractive.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Sat, 14 Oct 2006, Mark Crispin wrote:
>On Sat, 14 Oct 2006, Willi Burmeister wrote:
>>On Fri, 13 Oct 2006, Mark Crispin wrote:
>> >Suffice it to say that Solaris has a long
>> >history of incompatibility with other UNIX-like systems and with other
>> >versions of Solaris.
>> That's simply not true. Compatibility has always been the major point for
>> Solaris. We have 15 year old software running without any problems on the
>> newest release of Solaris. And even old drivers will run on newer releases.

>Just to pick one example:

>ustat() is a call that changed semantics incompatibly between versions of
>Solaris. SUN eventually called ustat() "deprecated" and said to use
>statfs() or statvfs(), both of which were unstable for a few versions
>before their semantics got nailed down. No portable application can
>depend upon any of these functions; a binary that uses one of these
>functions and works in an earlier version of Solaris can fail with a later
>version of Solaris.

I'm not an expert in such details, so I changed the subject and crossposted
this to comp.unix.solaris. Maybe someone there with more Solaris knowledge
is willing to comment your statement.

Compatiblity is a big point for us, so I'm always looking for statements
in this area.

Willi

On Fri, 13 Oct 2006 14:58:38 -0700, Mark Crispin <MRC@CAC.Washington.EDU> wrote:
> I can only guess what will happen when we switch to 128-bit processors.
> Will we have "long long long"? Or "long long long long"? Or "really
> long"?

I suspect we will have something like "too long" :-)!

Cheers, Liam

Willi Burmeister wrote:
> On Sat, 14 Oct 2006, Mark Crispin wrote:
> >On Sat, 14 Oct 2006, Willi Burmeister wrote:
> >>On Fri, 13 Oct 2006, Mark Crispin wrote:
> >> >Suffice it to say that Solaris has a long
> >> >history of incompatibility with other UNIX-like systems and with other
> >> >versions of Solaris.
> >> That's simply not true. Compatibility has always been the major point for
> >> Solaris. We have 15 year old software running without any problems on the
> >> newest release of Solaris. And even old drivers will run on newer releases.
>
> >Just to pick one example:
>
> >ustat() is a call that changed semantics incompatibly between versions of
> >Solaris. SUN eventually called ustat() "deprecated" and said to use
> >statfs() or statvfs(), both of which were unstable for a few versions
> >before their semantics got nailed down. No portable application can
> >depend upon any of these functions; a binary that uses one of these
> >functions and works in an earlier version of Solaris can fail with a later
> >version of Solaris.
>
> I'm not an expert in such details, so I changed the subject and crossposted
> this to comp.unix.solaris. Maybe someone there with more Solaris knowledge
> is willing to comment your statement.
>
> Compatiblity is a big point for us, so I'm always looking for statements
> in this area.
>
> Willi


In general, you'll find that Solaris engineering maintains
binary compatibility quite slavishly. Does this mean that
we're perfect? Of course not. Sometimes we have no
choice, as when filesystems and files exceeded the traditional
32bit file size limits, and filesystems exceeded 2 TB.

In this case, old applications not compiled large file
aware will have problems on large filesystems or with
large files on more modern versions of Solaris; those
apps need to be recompiled either 64 bit or using the
32bit large file options (see man -s 5 lfcompile).

The ustat(2) system call has been marked obsolete
for many years; it apparently doesn't have a large file
version for some reason. Since every operating system
I've come across notes that this call is obsolete and
statvfs should be used, I'd port the code to use that.
It should be simple enough.

- Bart

wib+usenet@cs.uni-kiel.de (Willi Burmeister) writes:

>On Sat, 14 Oct 2006, Mark Crispin wrote:

>>Just to pick one example:

>>ustat() is a call that changed semantics incompatibly between versions of
>>Solaris. SUN eventually called ustat() "deprecated" and said to use
>>statfs() or statvfs(), both of which were unstable for a few versions
>>before their semantics got nailed down. No portable application can
>>depend upon any of these functions; a binary that uses one of these
>>functions and works in an earlier version of Solaris can fail with a later
>>version of Solaris.

I would certainly want to see proof of this; (or, at least,some documentation
to the fact that they are different)

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

Willi Burmeister wrote:

> On Sat, 14 Oct 2006, Mark Crispin wrote:
>
>>On Sat, 14 Oct 2006, Willi Burmeister wrote:
>>
>>>On Fri, 13 Oct 2006, Mark Crispin wrote:
>>>
>>>>Suffice it to say that Solaris has a long
>>>>history of incompatibility with other UNIX-like systems and with other
>>>>versions of Solaris.
>>>
>>>That's simply not true. Compatibility has always been the major point for
>>>Solaris. We have 15 year old software running without any problems on the
>>>newest release of Solaris. And even old drivers will run on newer releases.
>
>
>>Just to pick one example:
>
>
>>ustat() is a call that changed semantics incompatibly between versions of
>>Solaris. SUN eventually called ustat() "deprecated" and said to use
>>statfs() or statvfs(), both of which were unstable for a few versions
>>before their semantics got nailed down. No portable application can
>>depend upon any of these functions; a binary that uses one of these
>>functions and works in an earlier version of Solaris can fail with a later
>>version of Solaris.
>
>
> I'm not an expert in such details, so I changed the subject and crossposted
> this to comp.unix.solaris. Maybe someone there with more Solaris knowledge
> is willing to comment your statement.
>
> Compatiblity is a big point for us, so I'm always looking for statements
> in this area.
>
From the man pages on my RHEL4 AS box:
NOTES
ustat is deprecated and has only been provided for
compatibility. All
new programs should use statfs(2) instead.

(Seems rather the same as on Solaris, no?)

Cheers,
Gary B-)

--
__________________________________________________ ____________________________
Armful of chairs: Something some people would not know
whether you were up them with or not
- Barry Humphries