PHWinfo banniere

Titres
PORTAIL ANNUAIRE ARTICLES COMPARATEUR HÉBERGEURS DEVIS FORUMS RÉDUCTEUR D'URL
Précédent   PHWinfo > Forums Hébergement > Forum Logiciels d'hébergement > comp.mail.imap > UW imap-2006b: 64 bit problem
S'inscrire FAQ Membres Recherche Messages du jour Marquer les forums comme lus
comp.mail.imap Discussion of IMAP-based mail systems.

UW imap-2006b: 64 bit problem

Réponse
 
LinkBack Outils de la discussion
Vieux 12/10/2006, 23h00   #1
Jens Elkner
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut UW imap-2006b: 64 bit problem

Hi,

just made a Solaris 10 sparc package for UW imap-2006b (using Studio 11
compiler), but using -xarch=v9 aka 64bit seems to break imapd over SSL
(v8 aka 32bit works): connection timeouts. Traces show, that in the
64bit version tcp_serverport() always returns -1 instead of 993 (as the
32bit version does). So is it possible, that the unix/ip_*|tcp_* source
is not 64bit safe? Or do you think, it might be a compiler problem ?

BTW: for reproduction e.g.:

cd c-client;make soc EXTRACFLAGS='-g -xarch=v9
-I/tmp/build/root4build/usr/include -I/usr/sfw/include -KPIC'\
EXTRALDFLAGS='-L/tmp/build/root4build/usr/lib/sparcv9
-L/usr/sfw/lib/sparcv9'\
EXTRADRIVERS='mbox'\
EXTRAAUTHENTICATORS=''\
PASSWDTYPE=std SSLTYPE=unix IP=4\

ln -s os_soln.h os_sol.h
make build EXTRACFLAGS='-g -xarch=v9
-I/tmp/build/root4build/usr/include -I/usr/sfw/include -KPIC'
EXTRALDFLAGS='-L/tmp/build/root4build/usr/lib/sparcv9
-L/usr/sfw/lib/sparcv9' EXTRADRIVERS='mbox' EXTRAAUTHENTICATORS=''
PASSWDTYPE=std SSLTYPE=unix IP=4 `cat SPECIALS` OS=sol \
SIGTYPE=psx CHECKPW=psx CRXTYPE=nfs \
SPOOLDIR=/var/spool MAILSPOOL=/var/mail \
ACTIVEFILE=/var/news/active \
RSHPATH=/usr/bin/rsh \
BASECFLAGS= \
BASELDFLAGS="-lsocket -lnsl -lgen" \
RANLIB=true CC=cc
....
  Réponse avec citation
Vieux 12/10/2006, 23h16   #2
Mark Crispin
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

In general, 64-bit builds are untested and not recommended in UW imapd.
There is generally not much benefit to a 64-bit build.

I do not have easy access to a Solaris system, and there's a long history
of Solaris being troublesome to support. Solaris is SVR4 based, which is
not a recommended platform due to its deficiencies (Linux and BSD are
recommended platforms).

However, there may be a simple explanation and fix for your problem.

By default, Solaris builds use IPv4 only code. If getsockname() returns
an IPv6 socket, that would cause the -1 port that you experienced.

Try adding IP=6 to your make command (you'll need to "make clean" first).
That'll select the IPv6 code which is also IPv4 savvy.

That may be the magic bullet that solves the problem.

Good luck!

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
  Réponse avec citation
Vieux 13/10/2006, 00h16   #3
Jens Elkner
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

Mark Crispin wrote:
> In general, 64-bit builds are untested and not recommended in UW imapd.



> There is generally not much benefit to a 64-bit build.


Yes, I know. But since I do not want to package each SW twice, and some SW
really benefits from 64bit, but unfortunately one can't mix 32 and 64bit
objects/libs, I decided to make all packages 64bit, only.

> Solaris is SVR4 based,
> which is not a recommended platform due to its deficiencies (Linux and
> BSD are recommended platforms).


Well, have not really a choice ;-)

> However, there may be a simple explanation and fix for your problem.
>
> By default, Solaris builds use IPv4 only code. If getsockname() returns
> an IPv6 socket, that would cause the -1 port that you experienced.
>
> Try adding IP=6 to your make command (you'll need to "make clean"
> first). That'll select the IPv6 code which is also IPv4 savvy.
>
> That may be the magic bullet that solves the problem.


No - same problem. Must be something different...
BTW: What is the easiest way to programatically enable the tcbdebug
stuff, so that I can do some "printf" debugging, when I find the time
for it ?

Regards,
jens.
  Réponse avec citation
Vieux 13/10/2006, 00h32   #4
Mark Crispin
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

On Fri, 13 Oct 2006, Jens Elkner wrote:
>> Try adding IP=6 to your make command (you'll need to "make clean" first).
>> That'll select the IPv6 code which is also IPv4 savvy.

> No - same problem. Must be something different...


Hmm. This suggests that it's not getting the correct value for sa_family
from the sockaddr. Maybe some includes need to be different if you do
64-bit builds on Solaris.

I don't see what you can't make an exception for imapd and build it
32-bit.

> BTW: What is the easiest way to programatically enable the tcbdebug stuff, so
> that I can do some "printf" debugging, when I find the time for it ?


The tcpdebug stuff only s in client code, not server code. You'll be
better off just doing logging code (via syslog(), not printf()) yourself.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
  Réponse avec citation
Vieux 13/10/2006, 01h51   #5
Jens Elkner
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

Mark Crispin wrote:
> On Fri, 13 Oct 2006, Jens Elkner wrote:
>
> Hmm. This suggests that it's not getting the correct value for
> sa_family from the sockaddr.


Yepp.

> I don't see what you can't make an exception for imapd and build it 32-bit.

As I said: XOR - e.g. PHP+libc-client.so (maintainance)

>> BTW: What is the easiest way to programatically enable the tcbdebug
>> stuff, so that I can do some "printf" debugging, when I find the time
>> for it ?

>
> The tcpdebug stuff only s in client code, not server code. You'll
> be better off just doing logging code (via syslog(), not printf())
> yourself.


Hmmm, ok - may be it isn't worth to debug/fix it (already spent to much
time on this). After 10 years of WU-IMAP may be it's time to change
something and to have a look at cyrus ...

Thanx a lot,
jens.
  Réponse avec citation
Vieux 13/10/2006, 08h47   #6
Willi Burmeister
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

In <Pine.WNT.4.65.0610121505590.2320@Tomobiki-Cho.CAC.Washington.EDU> Mark Crispin <MRC@CAC.Washington.EDU> writes:

>I do not have easy access to a Solaris system, and there's a long history
>of Solaris being troublesome to support.


why?

>Solaris is SVR4 based, which is
>not a recommended platform due to its deficiencies


what deficiencies?


Willi

  Réponse avec citation
Vieux 13/10/2006, 18h55   #7
Mark Crispin
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

On Fri, 13 Oct 2006, Willi Burmeister wrote:
> >I do not have easy access to a Solaris system, and there's a long history
> >of Solaris being troublesome to support.

> why?


I don't want to go into detail; this newsgroup is not the appropriate
place for Solaris-bashing. Suffice it to say that Solaris has a long
history of incompatibility with other UNIX-like systems and with other
versions of Solaris.

> >Solaris is SVR4 based, which is
> >not a recommended platform due to its deficiencies

> what deficiencies?


The deficiencies impact performance and reliability. I know of sites that
have fixed their IMAP performance problems simply by replacing SVR4 based
systems with (less expensive!) Linux systems.

SVR4 only has POSIX style fcntl() locking. The BSD man pages call the
semantics of POSIX locking "completely stupid"; I don't disagree. I have
quoted the relevant paragraph from the BSD man page many times already; I
hope that I don't have to do so again. UW imapd is obliged to do
considerable more work on SVR4 systems than it does on BSD and Linux
systems which offer flock() locking as an alternative to POSIX locking.

Many (most?) SVR4 filesystems are poor performers compared to BSD and
Linux filesystems.

I do not recommend Solaris, or any other SVR4 based system (AIX, HP-UX,
etc.) as a platform for an IMAP server. This is not just for UW imapd.
Linux and BSD are more suitable platforms for other IMAP server
implementations as well. Other IMAP server implementations may not be as
severely impacted by the choice of SVR4; but I can't imagine any IMAP
server implementation performing better on SVR4 than on Linux or BSD.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
  Réponse avec citation
Vieux 13/10/2006, 19h08   #8
Philip Guenther
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

On Oct 12, 5:16 pm, Jens Elkner <elkner+news...@cs.uni-magdeburg.de>
wrote:
> Mark Crispin wrote:
> > There is generally not much benefit to a 64-bit build.

>
> Yes, I know. But since I do not want to package each SW twice, and some SW
> really benefits from 64bit, but unfortunately one can't mix 32 and 64bit
> objects/libs, I decided to make all packages 64bit, only.


Nothing personal, but I find that bizarre. I mean, the only files for
which you need separate 32bit and 64bit copies are libraries. You
would rather replace your software or run software that hasn't been
fully tested for 64bit compilation than keep around two copies to some
libraries. (How many non-Sun libraries does UW-imap actually require?)
What about programs that run *faster* as 32bit binaries because the
cache-footprint of their data is smaller?

Nevermind, it's your call. You'll even be doing a service to the
community by reporting problems that developers may not have the
hardware to check themselves.


Philip Guenther

  Réponse avec citation
Vieux 13/10/2006, 22h10   #9
Jens Elkner
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

Philip Guenther wrote:
> On Oct 12, 5:16 pm, Jens Elkner <elkner+news...@cs.uni-magdeburg.de>
> wrote:


>>Yes, I know. But since I do not want to package each SW twice, and some SW
>>really benefits from 64bit, but unfortunately one can't mix 32 and 64bit
>>objects/libs, I decided to make all packages 64bit, only.

>
>
> Nothing personal, but I find that bizarre.


;-)

> I mean, the only files for
> which you need separate 32bit and 64bit copies are libraries. You
> would rather replace your software or run software that hasn't been
> fully tested for 64bit compilation than keep around two copies to some
> libraries.


Ehhm - no: I prefer to replace it, since such problems are an indicator
for me, that the software might have other flaws as well or is a bad
hack at all ;-). BTW: It actually seems to have an problem with mozilla
IMAPS connections (SSL error status: error:140D308A:SSL
routines:{TLS1|SSL3}_SETUP_KEY_BLOCK:cipher or hash unavailable ...).

> (How many non-Sun libraries does UW-imap actually require?)


Well, at least PHP... But anyway, this doesn't matter, since one never
knows, which other software comes up or will be written by students here ...

> What about programs that run *faster* as 32bit binaries because the
> cache-footprint of their data is smaller?


What is *faster*? Some nano/milliseconds? Well, as many software
developers say, RAM is not an issue since cheap, "gets faster", etc.
I apply the analogon to CPUs ;-)

> Nevermind, it's your call. You'll even be doing a service to the
> community by reporting problems that developers may not have the
> hardware to check themselves.


May be.

Regards,
jens.
  Réponse avec citation
Vieux 13/10/2006, 22h58   #10
Mark Crispin
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

On Fri, 13 Oct 2006, Jens Elkner wrote:
> Ehhm - no: I prefer to replace it, since such problems are an indicator for
> me, that the software might have other flaws as well or is a bad hack at all


The problems with 64-bit that I have seen have invariably been related to
types or sizes that didn't exist when the code was written.

In K&R days, there were three integral types: short (at least 16 bits),
int (at least 16 bits, maybe 32), and long (at least 32 bits). Four
integral types, if you call "char" one (at least 8 bits).

Some applications came to assume that "long == 32 bits". The people who
decided matters for 64-bits want to break those applications. So, they
created "long long" for 64-bit values instead of changing "long" to
64-bit.

The problem is, some other applications followed K&R and assumed that
"long" was the wide type (because it *was* prior to 64-bit). The
requirement for "long long" broke those applications.

What's amazing is not that 64-bit issues come up, but rather than there
are so few 64-bit issues.

I can only guess what will happen when we switch to 128-bit processors.
Will we have "long long long"? Or "long long long long"? Or "really
long"?

> BTW: It actually seems to have an problem with mozilla IMAPS connections
> (SSL error status: error:140D308A:SSL
> routines:{TLS1|SSL3}_SETUP_KEY_BLOCK:cipher or hash unavailable ...).


That sounds like Mozilla may be using the wrong client methods.

For imaps (port 993), clients should use the SSLv23 client method; for
IMAP TLS (port 143 + STARTTLS) clients should use the TLSv1 client method.

>> (How many non-Sun libraries does UW-imap actually require?)

> Well, at least PHP... But anyway, this doesn't matter, since one never knows,
> which other software comes up or will be written by students here ...


UW imapd does not require PHP. PHP apparently requires the UW IMAP
c-client library.

The 64-bit issue is not in UW imapd, but rather in the c-client
library which UW imapd also uses. It makes no sense to abandon UW imapd,
which doesn't have the problem itself, without abandoning the library
which causes it to have the problem.

Since you therefore choose to abandon c-client, that means that you must
also abandon PHP.

Or you can just do the sensible thing, and accept that some applications
still need to be 32-bit. It gains nothing to build them as 64-bit if they
aren't ready for such building; and especially not if they don't use
64-bit for anything.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
  Réponse avec citation
Vieux 13/10/2006, 23h09   #11
Mark Crispin
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

On Fri, 13 Oct 2006, Mark Crispin wrote:
> The problems with 64-bit that I have seen have invariably been related to
> types or sizes that didn't exist when the code was written.


Also, C library/syscall types have changed. For example, the third
argument to getsockname() is a socklen_t* in modern systems. It used to
be an int*, and socklen_t didn't exist.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
  Réponse avec citation
Vieux 13/10/2006, 23h40   #12
Sam
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

Mark Crispin writes:

> On Fri, 13 Oct 2006, Mark Crispin wrote:
>> The problems with 64-bit that I have seen have invariably been related to
>> types or sizes that didn't exist when the code was written.

>
> Also, C library/syscall types have changed. For example, the third
> argument to getsockname() is a socklen_t* in modern systems. It used to
> be an int*, and socklen_t didn't exist.


So what?

This problem has been solved a long time ago. Courier builds fine on system
that use socklen_t, and on systems that don't; on systems that define IPv6
structures, and on systems that do not; etc…; without having me manually
keep track who does what.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQBFMBXIx9p3GYHlUOIRAn8gAJ4tKL0qlJSwyAWCNjaO0o 8DdblTlACePuFJ
bOBoFlRwJbs20t419mdXKr4=
=PAyp
-----END PGP SIGNATURE-----

  Réponse avec citation
Vieux 14/10/2006, 00h35   #13
Mark Crispin
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

On Fri, 13 Oct 2006, Sam wrote:
> This problem has been solved a long time ago.


That misses the point. In order to solve a problem, it's necessary for
someone, somewhere, to know that the problem exists. Even if the solution
is punted to an external mechanism such as autoconf, it's still necessary
for that mechanism to be updated to know about new problems and deploy the
necessary solutions.

It is quite unlikely that any code written in 2006 will fully comply with
the world of 2016.

IMHO, the sooner that all C library functions replace *all* their usage of
C language types with xxxx_t typedefs, the better.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
  Réponse avec citation
Vieux 14/10/2006, 02h12   #14
Sam
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

Mark Crispin writes:

> On Fri, 13 Oct 2006, Sam wrote:
>> This problem has been solved a long time ago.

>
> That misses the point. In order to solve a problem, it's necessary for
> someone, somewhere, to know that the problem exists. Even if the solution
> is punted to an external mechanism such as autoconf, it's still necessary
> for that mechanism to be updated to know about new problems and deploy the
> necessary solutions.


And once you know about a new problem, you then go and solve _THE_ problem,
instead of starting a list of all platforms that are vulnerable to the
problem, and then having to maintain that list forever, until our Sun
explodes.

I don't remember how long ago I encountered the socklen_t issue. Four,
maybe five years ago. Maybe longer. But once I did, I wrote one autoconf
test for it, and that's the last time I had to waste any mental energy on
it. I have more important thing on my mind than having to constantly update
a list of all platforms that do or not define socklen_t, and set the
compiler flags accordingly. And when a new version of some Linux, or BSD,
or Solaris comes out, I don't need to waste an evening figuring out whether
it does or does not define socklen_t, and a myriad of other mundane things.
The macros will do it for me.

Another one is the missing declaration of gethostname() in unistd.h.

Why do I need to research which one of the hundreds (thousands?) of
operating systems where my code might get compiled declare or do not
declare gethostname? Why do I need to maintain such a list?

I'd rather test this with autoconf, and have the code compiled accordingly.
I don't give a flip whether this specific platform does or does not define
gethostname. I don't need to know that.

Let's see, what else…

Oh, this is a good one. tm_gmtoff? tm_isdst? Or, altzone? Ugh. Let's
see, Debian is about to make their first release in the new millenium? Do I
care which particular way they maintain daylight savings time? No, I really
don't. I don't want to know.

And speaking of that, the United States must now be the laughing stock of
the entire world. If not for the more obvious reasons, at least for the
fact that starting next year, the so-called alternate time will be in effect
longer then the so-called standard time.

Stupid Americans -- we can't even decide what time of day it is.



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQBFMDlxx9p3GYHlUOIRAp20AJ9qpOmhi7WrgECMWg43+r COCxLOOQCeKe++
PfYhEvKZ1DqRYGWYwqVHrxw=
=1VCE
-----END PGP SIGNATURE-----

  Réponse avec citation
Vieux 14/10/2006, 06h44   #15
Gary R. Schmidt
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

Mark Crispin wrote:

> On Fri, 13 Oct 2006, Willi Burmeister wrote:
>
>> >I do not have easy access to a Solaris system, and there's a long

>> history >of Solaris being troublesome to support.
>> why?

>
>
> I don't want to go into detail; this newsgroup is not the appropriate
> place for Solaris-bashing. Suffice it to say that Solaris has a long
> history of incompatibility with other UNIX-like systems and with other
> versions of Solaris.

How bizarre - I've never had problems with Solaris, admittedly I've only
developed on/for 2.5.1, 2.6, 7, 8, 9, and 10.

>> >Solaris is SVR4 based, which is >not a recommended platform due to

>> its deficiencies what deficiencies?

>
>
> The deficiencies impact performance and reliability. I know of sites
> that have fixed their IMAP performance problems simply by replacing SVR4
> based systems with (less expensive!) Linux systems.

Names, dates, etcetera, please. And if you are talking about someone
swapping a 10-year old proprietary UNIX system for a current x86-based
solution, just don't.

> SVR4 only has POSIX style fcntl() locking. The BSD man pages call the
> semantics of POSIX locking "completely stupid"; I don't disagree. I
> have quoted the relevant paragraph from the BSD man page many times
> already; I hope that I don't have to do so again. UW imapd is obliged
> to do considerable more work on SVR4 systems than it does on BSD and
> Linux systems which offer flock() locking as an alternative to POSIX
> locking.

There's been an flock() on top of fcntl in System V since Release 2.
But I will admit that I've never found there to be any strangeness with
advisory locking on the various UNIX and UNIX-like systems I've used
over the years, perhaps the multi-user databases I was involved in
coding simply didn't enter these problematic areas.

> Many (most?) SVR4 filesystems are poor performers compared to BSD and
> Linux filesystems.

Are they? Please quote some reliable _reproducible_ figures.

> I do not recommend Solaris, or any other SVR4 based system (AIX, HP-UX,
> etc.) as a platform for an IMAP server. This is not just for UW imapd.
> Linux and BSD are more suitable platforms for other IMAP server
> implementations as well. Other IMAP server implementations may not be
> as severely impacted by the choice of SVR4; but I can't imagine any IMAP
> server implementation performing better on SVR4 than on Linux or BSD.


You know, for someone who claims 30-odd years experience you seem
somewhat limited in your views. Perhaps it's just a case of the recent
convert being overzealous?

I've only been developing and administering computers for 25 or so
years, and most (~20 years) of that development has had two distinct
requirements, performance and portability, both between different
architectures and different Operating Systems, and when I've run across
code that doesn't work well on OS n or architecture z it's usually been
because the code has been hand-optimised to work only on one system, or
it is simply bad, and doesn't run well anywhere.

I admit I've not looked closely at the source of UW-imapd, or, for that
matter, any other IMAP implementation, but if it's so *bad* on System
V-like systems (by the way, neither AIX nor HP-UX are SVR4 based, both
are more SVR2 based) perhaps someone needs to review the basic design
and the assumptions that lead to it?

Cheers,
Gary B-)

--
__________________________________________________ ____________________________
Armful of chairs: Something some people would not know
whether you were up them with or not
- Barry Humphries
  Réponse avec citation
Vieux 14/10/2006, 10h42   #16
Willi Burmeister
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

In <alpine.OSX.0.7.0610131023290.12100@pangtzu.panda. com> Mark Crispin <mrc@CAC.Washington.EDU> writes:

>On Fri, 13 Oct 2006, Willi Burmeister wrote:
>> >I do not have easy access to a Solaris system, and there's a long history
>> >of Solaris being troublesome to support.

>> why?


>I don't want to go into detail; this newsgroup is not the appropriate
>place for Solaris-bashing.


then don't do that. Just tell us facts.

>Suffice it to say that Solaris has a long
>history of incompatibility with other UNIX-like systems


Porting software developed on Solaris to other systems is much easier
than the other way.

>and with other versions of Solaris.


That's simply not true. Compatibility has always been the major point for
Solaris. We have 15 year old software running without any problems on the
newest release of Solaris. And even old drivers will run on newer releases.

>> >Solaris is SVR4 based, which is
>> >not a recommended platform due to its deficiencies

>> what deficiencies?


>The deficiencies impact performance and reliability.


Do you have any facts?


Willi

P.S. http://www.softpanorama.org/Articles...vs_linux.shtml

  Réponse avec citation
Vieux 14/10/2006, 20h16   #17
Mark Crispin
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

On Sat, 14 Oct 2006, Gary R. Schmidt wrote:
> > SVR4 only has POSIX style fcntl() locking. The BSD man pages call the
> > semantics of POSIX locking "completely stupid"; I don't disagree. I
> > have quoted the relevant paragraph from the BSD man page many times
> > already; I hope that I don't have to do so again. UW imapd is obliged
> > to do considerable more work on SVR4 systems than it does on BSD and
> > Linux systems which offer flock() locking as an alternative to POSIX
> > locking.

> There's been an flock() on top of fcntl in System V since Release 2.


I'm afraid that you completely misunderstood the issue. "Semantics" have
nothing to do with a calling interface. It has to do with what gets done
by the call.

IEEE Std 1003.1-1988 (a.k.a. POSIX.1) requires that all fcntl() locks
associated with a file for a given process are removed when *any* file
descriptor for that file is closed by that process. This, in turn,
requires a global knowledge of file activity between an application and
all of the API/libraries that the application uses.

Put another way, before any library routine opens a file, it must be aware
of what files the application and any other libraries have open and
locked; otherwise the library routine will remove the lock unexpectedly
when it closes the file.

This gets even more fun when an server is multi-threaded, and is servicing
clients that have no knowledge of each other.

Solaris' flock() on top of fcntl() does not address this problem.
Solaris's pseudo flock() call, being based upon fcntl(), has the problem.
The programmer (or the autoconf) must be aware that Solaris' flock() is
not real flock(), and thus can not assume that "since I am using flock(),
the locks are safe from this problem."

Linux and BSD have both fcntl() and real flock() locking, and these locks
operate independently of each other. Tru64 UNIX once did as well, but
later on they removed real flock() in favor of a Solaris style jacket.

Please read the BSD man page on fcntl() locking, which has a detailed
paragraph on this matter, before commenting on this issue further.

Not that flock() locking is perfect -- far from it! flock() does not have
true lock promotion/demotion. Instead, an attempt to promote/demote a
lock *frees* the current lock and then seeks to acquire a lock of the new
type.

> > Many (most?) SVR4 filesystems are poor performers compared to BSD and
> > Linux filesystems.

> Are they? Please quote some reliable _reproducible_ figures.


As our few remaining SVR4 systems are being retired, I don't have much
opportunity to do new testing. I know what I experienced: we had to make
/tmp be a RAMdisk, since even zero-length files on /tmp tied up the disk
too much; directory access slowed down dramatically as the number of files
in the directory increased (news spools!); etc.

This is the first time that I have heard someone assert that SVR4
filesystems perform comparably to BSD and Linux filesystems. Perhaps
newer SVR4 filesystems do so. Do you have some "reliable reproducible
figures" that show a Solaris filesystem performing comparably to Linux?

If so, that is interesting. But, since Solaris is increasingly irrelevant
while industry standardizes upon Linux, it doesn't change much.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
  Réponse avec citation
Vieux 14/10/2006, 20h37   #18
Mark Crispin
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

On Sat, 14 Oct 2006, Willi Burmeister wrote:
> >and with other versions of Solaris.

> That's simply not true. Compatibility has always been the major point for
> Solaris. We have 15 year old software running without any problems on the
> newest release of Solaris. And even old drivers will run on newer releases.


Just to pick one example:

ustat() is a call that changed semantics incompatibly between versions of
Solaris. SUN eventually called ustat() "deprecated" and said to use
statfs() or statvfs(), both of which were unstable for a few versions
before their semantics got nailed down. No portable application can
depend upon any of these functions; a binary that uses one of these
functions and works in an earlier version of Solaris can fail with a later
version of Solaris.

> >> >Solaris is SVR4 based, which is
> >> >not a recommended platform due to its deficiencies
> >> what deficiencies?

> >The deficiencies impact performance and reliability.


The lack of flock() locking is a big one.

The nightmare of rpc.statd/rpc.lockd is related. For years, there was a
game with SUN where they claimed that rpc.statd/rpc.lockd were "fixed",
and I would proceed to demonstrate (by locking up an entire cluster) that
it wasn't.

> P.S. http://www.softpanorama.org/Articles...vs_linux.shtml


ext2 used as the standard to judge Linux filesystem performance?!?

Yeah, right.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
  Réponse avec citation
Vieux 14/10/2006, 22h28   #19
Mark Crispin
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

On Sat, 14 Oct 2006, Mark Crispin wrote:
> IEEE Std 1003.1-1988 (a.k.a. POSIX.1) requires that all fcntl() locks
> associated with a file for a given process are removed when *any* file
> descriptor for that file is closed by that process.


To add to the fun, Cygwin implements locking by using Windows locks, which
are mandatory instead of advisory.

One of the attractions of the maildir format that is frequently (and
justifiably!) cited by its fan community is that maildir does not depend
upon locking at all and thus supposedly avoids the problems faced by
formats that depend upon various locking. Not everybody agrees that
maildir really works without locking (e.g., see the Wikipedia article
about maildir), but to the extent that it is true it is attractive.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
  Réponse avec citation
Vieux 15/10/2006, 16h47   #20
Willi Burmeister
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut ustat not upward compatible? [was: UW imap-2006b: 64 bit problem]

On Sat, 14 Oct 2006, Mark Crispin wrote:
>On Sat, 14 Oct 2006, Willi Burmeister wrote:
>>On Fri, 13 Oct 2006, Mark Crispin wrote:
>> >Suffice it to say that Solaris has a long
>> >history of incompatibility with other UNIX-like systems and with other
>> >versions of Solaris.

>> That's simply not true. Compatibility has always been the major point for
>> Solaris. We have 15 year old software running without any problems on the
>> newest release of Solaris. And even old drivers will run on newer releases.


>Just to pick one example:


>ustat() is a call that changed semantics incompatibly between versions of
>Solaris. SUN eventually called ustat() "deprecated" and said to use
>statfs() or statvfs(), both of which were unstable for a few versions
>before their semantics got nailed down. No portable application can
>depend upon any of these functions; a binary that uses one of these
>functions and works in an earlier version of Solaris can fail with a later
>version of Solaris.


I'm not an expert in such details, so I changed the subject and crossposted
this to comp.unix.solaris. Maybe someone there with more Solaris knowledge
is willing to comment your statement.

Compatiblity is a big point for us, so I'm always looking for statements
in this area.

Willi

  Réponse avec citation
Vieux 15/10/2006, 17h22   #21
Liam Greenwood
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: UW imap-2006b: 64 bit problem

On Fri, 13 Oct 2006 14:58:38 -0700, Mark Crispin <MRC@CAC.Washington.EDU> wrote:
> I can only guess what will happen when we switch to 128-bit processors.
> Will we have "long long long"? Or "long long long long"? Or "really
> long"?


I suspect we will have something like "too long" :-)!

Cheers, Liam
  Réponse avec citation
Vieux 15/10/2006, 18h40   #22
barts@smaalders.net
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: ustat not upward compatible? [was: UW imap-2006b: 64 bit problem]

Willi Burmeister wrote:
> On Sat, 14 Oct 2006, Mark Crispin wrote:
> >On Sat, 14 Oct 2006, Willi Burmeister wrote:
> >>On Fri, 13 Oct 2006, Mark Crispin wrote:
> >> >Suffice it to say that Solaris has a long
> >> >history of incompatibility with other UNIX-like systems and with other
> >> >versions of Solaris.
> >> That's simply not true. Compatibility has always been the major point for
> >> Solaris. We have 15 year old software running without any problems on the
> >> newest release of Solaris. And even old drivers will run on newer releases.

>
> >Just to pick one example:

>
> >ustat() is a call that changed semantics incompatibly between versions of
> >Solaris. SUN eventually called ustat() "deprecated" and said to use
> >statfs() or statvfs(), both of which were unstable for a few versions
> >before their semantics got nailed down. No portable application can
> >depend upon any of these functions; a binary that uses one of these
> >functions and works in an earlier version of Solaris can fail with a later
> >version of Solaris.

>
> I'm not an expert in such details, so I changed the subject and crossposted
> this to comp.unix.solaris. Maybe someone there with more Solaris knowledge
> is willing to comment your statement.
>
> Compatiblity is a big point for us, so I'm always looking for statements
> in this area.
>
> Willi



In general, you'll find that Solaris engineering maintains
binary compatibility quite slavishly. Does this mean that
we're perfect? Of course not. Sometimes we have no
choice, as when filesystems and files exceeded the traditional
32bit file size limits, and filesystems exceeded 2 TB.

In this case, old applications not compiled large file
aware will have problems on large filesystems or with
large files on more modern versions of Solaris; those
apps need to be recompiled either 64 bit or using the
32bit large file options (see man -s 5 lfcompile).

The ustat(2) system call has been marked obsolete
for many years; it apparently doesn't have a large file
version for some reason. Since every operating system
I've come across notes that this call is obsolete and
statvfs should be used, I'd port the code to use that.
It should be simple enough.

- Bart

  Réponse avec citation
Vieux 16/10/2006, 09h25   #23
Casper H.S. Dik
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: ustat not upward compatible? [was: UW imap-2006b: 64 bit problem]

wib+usenet@cs.uni-kiel.de (Willi Burmeister) writes:

>On Sat, 14 Oct 2006, Mark Crispin wrote:


>>Just to pick one example:


>>ustat() is a call that changed semantics incompatibly between versions of
>>Solaris. SUN eventually called ustat() "deprecated" and said to use
>>statfs() or statvfs(), both of which were unstable for a few versions
>>before their semantics got nailed down. No portable application can
>>depend upon any of these functions; a binary that uses one of these
>>functions and works in an earlier version of Solaris can fail with a later
>>version of Solaris.


I would certainly want to see proof of this; (or, at least,some documentation
to the fact that they are different)

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
  Réponse avec citation
Vieux 16/10/2006, 11h03   #24
Gary R. Schmidt
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: ustat not upward compatible? [was: UW imap-2006b: 64 bit problem]

Willi Burmeister wrote:

> On Sat, 14 Oct 2006, Mark Crispin wrote:
>
>>On Sat, 14 Oct 2006, Willi Burmeister wrote:
>>
>>>On Fri, 13 Oct 2006, Mark Crispin wrote:
>>>
>>>>Suffice it to say that Solaris has a long
>>>>history of incompatibility with other UNIX-like systems and with other
>>>>versions of Solaris.
>>>
>>>That's simply not true. Compatibility has always been the major point for
>>>Solaris. We have 15 year old software running without any problems on the
>>>newest release of Solaris. And even old drivers will run on newer releases.

>
>
>>Just to pick one example:

>
>
>>ustat() is a call that changed semantics incompatibly between versions of
>>Solaris. SUN eventually called ustat() "deprecated" and said to use
>>statfs() or statvfs(), both of which were unstable for a few versions
>>before their semantics got nailed down. No portable application can
>>depend upon any of these functions; a binary that uses one of these
>>functions and works in an earlier version of Solaris can fail with a later
>>version of Solaris.

>
>
> I'm not an expert in such details, so I changed the subject and crossposted
> this to comp.unix.solaris. Maybe someone there with more Solaris knowledge
> is willing to comment your statement.
>
> Compatiblity is a big point for us, so I'm always looking for statements
> in this area.
>

From the man pages on my RHEL4 AS box:
NOTES
ustat is deprecated and has only been provided for
compatibility. All
new programs should use statfs(2) instead.

(Seems rather the same as on Solaris, no?)

Cheers,
Gary B-)

--
__________________________________________________ ____________________________
Armful of chairs: Something some people would not know
whether you were up them with or not
- Barry Humphries