Andrew Haley noted that there is an active group of developers trying to improve GCC, requesting, "give us a chance." Returning to the original compiler misoptimization that started the whole discussion, he noted that a fix was being committed to all open GCC branches, "we're back-porting the patch to all open branches. However, this patch only affects one particular case where gcc introduces a data race; we're sure there are others not fixed." Andrew also noted that they were actively continuing to audit the code to find and remove similar optimization bugs.
From: Bart Van Assche <bart.vanassche@...>
Subject: Re: Is gcc thread-unsafe?
Date: Oct 26, 10:40 am 2007
On 10/25/07, Linus Torvalds <> wrote:
> The gcc developers seem to have had a total disregard for what people
> want or need, and every time some code generation issue comes up, there's
> a lot of people on the list that do language-lawyering, rather than admit
> that there might be a problem.
Please make a proposal for how gcc should be modified instead of just
shooting on the gcc people -- the root cause here is the way the C/C++
memory model is defined. (Note: I'm not in any way involved in gcc
development.)
You can find my proposal to improve gcc here:
http://gcc.gnu.org/ml/gcc/2007-10/msg00465.html
Bart Van Assche.
-
From: Linus Torvalds <torvalds@...>
Subject: Re: Is gcc thread-unsafe?
Date: Oct 26, 11:09 am 2007
On Fri, 26 Oct 2007, Bart Van Assche wrote:
> On 10/25/07, Linus Torvalds <> wrote:
> >
> > The gcc developers seem to have had a total disregard for what people
> > want or need, and every time some code generation issue comes up, there's
> > a lot of people on the list that do language-lawyering, rather than admit
> > that there might be a problem.
>
> Please make a proposal for how gcc should be modified instead of just
> shooting on the gcc people -- the root cause here is the way the C/C++
> memory model is defined. (Note: I'm not in any way involved in gcc
> development.)
Note that I'm very ambivalent about gcc. I think it's a *great* compiler.
I have my doubts about some of the things it does, but hey, it is an old
project, it has accumulated cruft over time, and cleaning things up is
often almost impossible.
And bugs happen. I'm not complaining about that at all either, even if
sometimes a compiler bug is damn frustrating.
And the fact is, most of the gcc developers are great.
So my basic worry about gcc is in fact none of the individual technical
problems themselves - those can be fixed. No, the problem I've seen in gcc
is that _some_ of the developers seem to be total d*ckheads when it comes
to "language definition", and seem to think that it's more important to
read the language spec like a lawyer than it is to solve actual user
problems.
And that has come up before. It has nothing to do with this particular
"gcc doesn't create thread-safe code" issue. We had the exact same issue
with gcc creating totally unusable code due to having a fundamentally
braindead memory aliasing model. The language-lawyering people basically
could push their *idiotic* model onto gcc, just by quoting the standard,
and not caring about actual users at all.
And that's a problem. In the kernel, we've historically always cared a lot
about POSIX and SuS, but while conforming to standards have been primary
goals since pretty much day one (ie I asked around about POSIX before the
very first release, and it's how I met some suckers^Wupstanding citizens
to try those early kernels), it has *always* taken a back seat to things
like compatibility with existing apps.
The gcc lists seem to often get to the point where people quote the
standard, and that's that. In that environment, the paper standard (that
hass *nothing* to do with reality) trumps any other argument. "What we do
is _allowed_ by the standard" seems to be a good argument, even if it
breaks real code and there is no sane way to avoid doing it.
And I really don't think it's everybody. At all. But I think it's the case
that sometimes it's easier to quote the standard than write good code, and
so gcc lists have more people quoting the papers than trying to fix some
problem.
Linus
-
From: Andrew Haley <aph@...>
Subject: Re: Is gcc thread-unsafe?
Date: Oct 26, 11:34 am 2007
Linus Torvalds writes:
> So my basic worry about gcc is in fact none of the individual
> technical problems themselves - those can be fixed. No, the problem
> I've seen in gcc is that _some_ of the developers seem to be total
> d*ckheads when it comes to "language definition", and seem to think
> that it's more important to read the language spec like a lawyer
> than it is to solve actual user problems.
Well, yeah. I know what you mean. However, at this moment, some gcc
developers are trying really hard not to be total d*ckheads about this
issue, but get gcc fixed. Give us a chance.
Andrew.
--
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903
-
From: David Schwartz <davids@...>
Subject: RE: Is gcc thread-unsafe?
Date: Oct 26, 2:06 pm 2007
> Well, yeah. I know what you mean. However, at this moment, some gcc
> developers are trying really hard not to be total d*ckheads about this
> issue, but get gcc fixed. Give us a chance.
>
> Andrew.
Can we get some kind of consensus that 'optimizations' that add writes to
any object that the programmer might have taken the address of are invalid
on any platform that supports memory protection? That seems like obvious
common sense to me.
And it has the advantage that it can't be language-lawyered. There is no
document that states the rational requirements of a compiler that's going to
support a memory protection model. So they can be anything rational people
think they should be.
DS
-
From: Andrew Haley <aph@...>
Subject: RE: Is gcc thread-unsafe?
Date: Oct 30, 6:20 am 2007
David Schwartz writes:
>
> > Well, yeah. I know what you mean. However, at this moment, some
> > gcc developers are trying really hard not to be total d*ckheads
> > about this issue, but get gcc fixed. Give us a chance.
>
> Can we get some kind of consensus that 'optimizations' that add
> writes to any object that the programmer might have taken the
> address of are invalid on any platform that supports memory
> protection?
That's what the proposed standard language says, kinda-sorta. There's
an informal description at
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2338.html.
Anyway, we have fixed this bug and are committing it to all open gcc
branches. Credit to Ian Taylor for writing the patch.
Andrew.
--
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903
-
From: Bart Van Assche <bart.vanassche@...>
Subject: Re: Is gcc thread-unsafe?
Date: Nov 2, 11:29 am 2007
On 10/30/07, Andrew Haley <aph@redhat.com> wrote:
> David Schwartz writes:
> >
> > Can we get some kind of consensus that 'optimizations' that add
> > writes to any object that the programmer might have taken the
> > address of are invalid on any platform that supports memory
> > protection?
>
> That's what the proposed standard language says, kinda-sorta. There's
> an informal description at
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2338.html.
There is other important information in the cited text. A.o. it is
explained that register promotion of potentially shared variables can
introduce data races. Or: register promotion can introduce bugs in
multithreaded software when compiled with optimization enabled. Are
there any register promotion transformations implemented in gcc that
can introduce data races in multithreaded software ? This is very
important information both for kernel developers and for developers of
multithreaded userspace applications.
Another conclusion from the cited text is that in contrast with what
was stated before on the gcc mailing list, it is not required to
declare thread-shared variables volatile if that thread-shared data is
consistently protected by calls to locking functions.
Bart Van Assche.
-
From: Andrew Haley <aph@...>
Subject: Re: Is gcc thread-unsafe?
Date: Nov 2, 11:38 am 2007
Bart Van Assche writes:
> On 10/30/07, Andrew Haley <aph@redhat.com> wrote:
> > David Schwartz writes:
> > >
> > > Can we get some kind of consensus that 'optimizations' that add
> > > writes to any object that the programmer might have taken the
> > > address of are invalid on any platform that supports memory
> > > protection?
> >
> > That's what the proposed standard language says, kinda-sorta. There's
> > an informal description at
> > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2338.html.
>
> There is other important information in the cited text. A.o. it is
> explained that register promotion of potentially shared variables
> can introduce data races. Or: register promotion can introduce bugs
> in multithreaded software when compiled with optimization
> enabled. Are there any register promotion transformations
> implemented in gcc that can introduce data races in multithreaded
> software ?
I expect so. We're going to have to audit this whole enormous code
base to find them all and take them out.
Note that some of these optimizations have been around since gcc 3.4.
> This is very important information both for kernel developers and
> for developers of multithreaded userspace applications.
> Another conclusion from the cited text is that in contrast with
> what was stated before on the gcc mailing list, it is not required
> to declare thread-shared variables volatile if that thread-shared
> data is consistently protected by calls to locking functions.
Well, let's be clear: ISO 9899:1999 doesn't say so, but the proposed
standard language does.
Andrew.
--
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903
-
From: Bart Van Assche <bart.vanassche@...>
Subject: Re: Is gcc thread-unsafe?
Date: Nov 4, 11:13 am 2007
On 11/2/07, Andrew Haley <aph@redhat.com> wrote:
> Bart Van Assche writes:
> > On 10/30/07, Andrew Haley <aph@redhat.com> wrote:
> > > That's what the proposed standard language says, kinda-sorta. There's
> > > an informal description at
> > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2338.html.
> >
> > There is other important information in the cited text. A.o. it is
> > explained that register promotion of potentially shared variables
> > can introduce data races. Or: register promotion can introduce bugs
> > in multithreaded software when compiled with optimization
> > enabled. Are there any register promotion transformations
> > implemented in gcc that can introduce data races in multithreaded
> > software ?
>
> I expect so. We're going to have to audit this whole enormous code
> base to find them all and take them out.
>
> Note that some of these optimizations have been around since gcc 3.4.
Has it already been decided who will do this audit, and when this
audit will happen ? Has a target date been set when this audit should
be complete, or is the completion of this audit a requirement for the
release of a specific gcc version ?
And if there would exist register promotion transformations in gcc
that can introduce data races, which would be the optimization levels
that enable these transformations ?
Bart Van Assche.
-
From: Linus Torvalds <torvalds@...>
Subject: Re: Is gcc thread-unsafe?
Date: Nov 4, 1:45 pm 2007
On Sun, 4 Nov 2007, Bart Van Assche wrote:
>
> Has it already been decided who will do this audit, and when this
> audit will happen ? Has a target date been set when this audit should
> be complete, or is the completion of this audit a requirement for the
> release of a specific gcc version ?
I am told that the gcc people realized that was indeed a bug (people were
able to show problems even in non-threaded environments with mprotect()),
and have now fixed it in the current gcc sources. That still leaves the
old versions with potential problems, but I think it makes it much less
interesting to audit for these things.
Linus
-
From: Andrew Haley <aph@...>
Subject: Re: Is gcc thread-unsafe?
Date: Nov 4, 1:58 pm 2007
Linus Torvalds writes:
>
>
> On Sun, 4 Nov 2007, Bart Van Assche wrote:
> >
> > Has it already been decided who will do this audit, and when this
> > audit will happen ? Has a target date been set when this audit
> > should be complete, or is the completion of this audit a
> > requirement for the release of a specific gcc version ?
>
> I am told that the gcc people realized that was indeed a bug
> (people were able to show problems even in non-threaded
> environments with mprotect()), and have now fixed it in the current
> gcc sources. That still leaves the old versions with potential
> problems, but I think it makes it much less interesting to audit
> for these things.
We're back-porting the patch to all open branches. However, this
patch only affects one paticular case where gcc introduces a data
race; we're sure there are others not fixed.
Andrew.
--
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903
-
"and seem to think that it's
"and seem to think that it's more important to read the language spec like a lawyer than it is to solve actual user problems."
I don't get it. If he doesn't want to program in C then he should find another language.
Allow me to assist in clarifying.
The issue is not that he doesn't like C. The issue, from where I sit, is that the C standard should not be a straitjacket that requires obviously wrong behavior. The fact is, a language standard is like any other document: no matter how clearly specified, no matter how precise the language, inevitably there are going to be multiple ways to interpret at least some of it. In law, this is why we have lawyers and judges. In computing, however, we have a more objective means of determining the proper interpretation: which approach actually does the right thing? A standard is supposed to be a guide in making sure things work correctly on different platforms, with different implementations, it's not meant to be a justification for broken behavior.
In this case, the standard is being used to justify doing the wrong thing, because it's allowed by the standard. Well, yes, it may well be; the standard also allows, as in does not forbid, the compiler to trigger a supernova, or make weasels come out of your nose, if you dereference a null pointer ("undefined" means it could do anything, after all!), but that doesn't mean we would call such behavior correct by any means.
no
Actually this is a case where the rules of C are not well defined (and
the standards body recognized that and are slowly working to improve it)
for threaded programs.
What the gcc guys could do is be pragmatic and say that "ok, we'll implement
something that is sane and allows people to get on with their work rather
than waiting for the standards to come out". And the gcc guys are doing just
that, so they indeed recognize it is a problem too.
I don't get it. If he
Part of the problem is rather that C itself (or at least its GCC implementation) has been changing, moving away from its roots as a "portable assembler": a systems programming language that does just what the programmer tells it to do without having too many ideas of its own about how things should be done. A low-level-code programmer needs a "do what I say" language, not an "I know best" language.
going away from portable assembler
User-Agent: Lynx + jed (x86_64-pc-linux-glibc-debian)
> Part of the problem is rather that C itself (or at least its GCC
> implementation) has been changing, moving away from its roots as a
> "portable assembler": a systems programming language that does just
> what the programmer tells it to do without having too many ideas of
> its own about how things should be done. A low-level-code programmer
> needs a "do what I say" language, not an "I know best" language.
I agree absolutely. While C itself have much things to do behind the
scene, optimizing G(C)C added much more. Many system programmers don't
do *how* they doing things, relying on that compiler. Like, ahh i'll
add -Os and everything will be OK.
Linus asking them to read produced assembly, to stop spaghetti-like
programming and to think, what they thing they do[0][1].
[0] alpine.LFD.0.999.0707171957300.27353@woody.linux-foundation.org
[1] alpine.LFD.0.999.0707181050572.27353@woody.linux-foundation.org
(this web engine is screwed up, guys)
Some of those ``famous'' coders are even doing long-known for CS
professors crap, i.e. wrong usage of "gotos"[1]. In system programming
gotos are good, but only if used appropriately.
[2] 20071007205954.4053C13E06@wotan.suse.de
Funny, that today i've posted thoughts about stretching C a$$, not
knowing text processing tools.
klibc.mail, klibc2.mail on my ftp/upload
IMHO all that is going nowhere, post below about Makefile is a
confirmation. But i want to create new build system, based on shell
and text-processing assembler -- `sed`. More easy, more comprehensive,
more efficient.
--
-o--=O`C
#oo'L O
<___=E M signature will be damaged>
"i'm still reading how the
"i'm still reading how the gcc 'Makefile' works, to see how they can produce 'config.h' automagically"
.
.
.
(1 years later...)
"120 more lines... be patient, i'm still working on it"