The funny thing is, he manages to actually lead the project and get things done despite his personality flaws.
Programmers who can't grasp the basics of memory manipulation and management, on the other hand, frequently totally fuck up whatever project they're working on - and people like me get paid 1/2 as much come in last minute and fix things.
I totally understand the mindset of people like Theo.
However, I disagree with him in this particular case (a discussion about adding some documentation about the behavior of strlcpy and strlcat): there's no reason not to just add a note to the manpages about overlapping memory regions, even if the behavior should be obvious from the algorithm. You can't stop there from being people who don't understand the fundamentals of the tools they're working with, so anything you can do to help them learn those fundamentals is a good thing for everyone involved.
Wow, that's actually a new restriction to me, but I suppose it makes sense. A legal (but surprising) implementation *could* copy strings in the reverse direction.
I know I've written code in the past that does stuff like this (effectively):
/* Remove a prefix from a string */
len = strlen(prefix);
if (!strncmp(prefix, string, len))
strcpy(string, string + len + 1);
This should work for most obvious implementations of strcpy—pretty much any implementation that copies in the forward direction—although technically it can be considered an "overlapping string" if strlen(prefix) < (strlen(string) - strlen(prefix)).
I guess I question the rationale for leaving such a sentence out of a man page. It is one of the more obscure restrictions, and where it comes up can be subtle and non-obvious. Who are these man pages for, anyway? If you're so good that you can remember all the obscure restrictions, then why do you need to be reminded what the function prototype is? You can grep for that.
FWIW, the Linux man page does mention this restriction:
The strcpy() function copies the string pointed to by src, including
the terminating null byte (’\0’), to the buffer pointed to by dest. The strings may not overlap, and the destination string dest must be
large enough to receive the copy.
Of course, the man page also includes this rather, erm, professional bit of prose:
If the destination string of a strcpy() is not large enough (that is,
if the programmer was stupid or lazy, and failed to check the size
before copying) then anything might happen. Overflowing fixed length
strings is a favorite cracker technique.
While it's true that many buffer overflows come from such programmer oversights, "stupid" and "lazy" aren't the only reasons. Incompetence, forgetfulness and just plain human error are additional reasons. Not exactly a model of how man pages ought to be written either, IMHO. It seems like it ought to be possible to warn programmers without insulting learners.
(I should add, one can be incompetent without being stupid. A smart person can be ignorant, especially about the peculiarities of one language when they're not fluent in it, vs. another that they might be fluent in. I would say that that person is not a competent programmer in the language they're not fluent in. I, for instance, consider myself a competent programmer generally, but I'm not competent in, say, LISP.)
-- Program Intellivision and play Space Patrol!
The ruder a comment the more likely it is that the reader will actually pay attention. If you are trying to learn and you see a comment like "that is, if the programmer was stupid or lazy", I would sure pay attention and think it would be important to follow the advice.
I don't want people thinking I am stupid. Lazy can be a virtue, but not in this case.
This kind of in-your-face comment provides a motivation that is better than a very polite and dry "The strings may not overlap".
I am in favor of the man page pointing out the nuances of the function, but the more interesting the man page, the more beneficial. Heck, we are all looking at it right now because of the "erm, professional bit of prose".
Rude (or at least unabashed and unreserved) comments have more place in forums, mailing lists and private messages than they do in documentation. I personally would rather hold man pages to similar editorial and stylistic standards as any other vendor-produced documentation.
You can still get results without insulting the reader. Strong language != insulting language. I personally would lean toward something like this:
BUGS
If the destination buffer for strcpy is not large enough, strcpy will write beyond the end of the destination buffer, leading to unpredictable program behavior. This is a very common source of security holes. Programmers MUST ensure that there is sufficient room for the string in the destination before copying to avoid undesirable program behavior including security holes. Alternate functions such as strlcpy may be more appropriate in many circumstances.
Then you might as well go ahead and explain in strlen(3) and friends that you must pass a string, not just any char array. IMHO this has no place in a man page.
If you don't know what a string is, then you don't know C. If you don't know that you cannot modify string literals, then you don't know C. If you don't know the difference between memcpy and memmove, then you don't know C. You learn C by reading books, writing code, and reading the standard to get the finer details often left out in the books (and to write sweet portable code with well defined behaviour ;-) ).
Some things are more obscure than others. For example, the overlapping string aspect of strcpy, I admit, hadn't occurred to me. I always think in terms of it doing a forward copy. I don't recall K&R talking about it when discussing strings, actually. And they know C.
I actually went and dug up my K&R C book (the ANSI C second edition), and they suggest that strcpy would typically be written as follows by an expert C programmer (p. 106):
And I wonder where I got the impression that strcpy does forward copies.
To their credit, though, waaaaay in the reference material in the back (Appendix B, p. 249), they mention that among all the memXXX and strXXX function, only memmove works with overlapping arguments. It's not in the explanatory body text, but rather buried in the man-page-like reference material at the back.
As for man pages and what to include: It's one thing to go into a complete dissertation on the foibles of string handling in a man page. It's quite another to point out a couple of gotchas with short mentions. Non-overlapping arguments and preventing overflows are worthy of mention. Explaining the concept of NUL-termination from first principles is not.
-- Program Intellivision and play Space Patrol!
And I wonder where I got the impression that strcpy does forward copies.
That's why you don't learn C properly just by reading other peoples' code. You need to consult the final authority on the language, which is the language specification.
As for what a man page should include, I would have to agree with you, to some extent, in your last paragraph. However, I do not like the BUGS section in strtok(3) on GNU/Linux systems. "Never use this function, but if you do ..." Might as well just say "Never use C because it allows you to shoot yourself in the foot".
It's contradictory from the start, and then they go on to say that "These functions modify their first argument." This should be bloody obvious since they take a char *, and not a const char *.
It should be pointed out that the man pages that come on Linux systems are not written by the people who implement the C library that those systems use. For the authoritative documentation you should refer to The GNU C Library, which can be read on the web or accessed via the 'info' program on your system (unless Debian has decided to not ship it, grrrrrr).
If you do use strcat or strcpy, then you instantly fail at being a programmer.
If you want to write standard C that works everywhere, you don't have much of a choice. You can use these functions correctly, you know. If you don't know how, then perhaps you should be doing something else.
That said, if your target platform(s) supports the l-versions, there's no reason not to use them.
>If you want to write standard C that works everywhere, you don't have much of a choice.
Or if your library doesn't provide those functions, you can be smart and copy the code from strl* functions (BSD license == no problem) in your own code, it's not as if these functions were big..
And reducing the risk of making an error is the smart way of programming: we're all humans, we *do* make mistakes.
If you can't be polite and
If you can't be polite and get along reasonably well with people, you shouldn't lead an open source project.
Oh wait...
leadership
The funny thing is, he manages to actually lead the project and get things done despite his personality flaws.
Programmers who can't grasp the basics of memory manipulation and management, on the other hand, frequently totally fuck up whatever project they're working on - and people like me get paid 1/2 as much come in last minute and fix things.
I totally understand the mindset of people like Theo.
However, I disagree with him in this particular case (a discussion about adding some documentation about the behavior of strlcpy and strlcat): there's no reason not to just add a note to the manpages about overlapping memory regions, even if the behavior should be obvious from the algorithm. You can't stop there from being people who don't understand the fundamentals of the tools they're working with, so anything you can do to help them learn those fundamentals is a good thing for everyone involved.
I don't see why not, thats
I don't see why not, thats the joy of open source freedom, if you're a fool, no one will suffer you gladly.
Shouldn't logic is flawed
Shouldn't logic is flawed with how the world really works.
Deontic idiots.
OpenBSD is priceless to OSS, along with *BSD.
Overlapping arguments
Wow, that's actually a new restriction to me, but I suppose it makes sense. A legal (but surprising) implementation *could* copy strings in the reverse direction.
I know I've written code in the past that does stuff like this (effectively):
/* Remove a prefix from a string */ len = strlen(prefix); if (!strncmp(prefix, string, len)) strcpy(string, string + len + 1);This should work for most obvious implementations of strcpy—pretty much any implementation that copies in the forward direction—although technically it can be considered an "overlapping string" if strlen(prefix) < (strlen(string) - strlen(prefix)).
I guess I question the rationale for leaving such a sentence out of a man page. It is one of the more obscure restrictions, and where it comes up can be subtle and non-obvious. Who are these man pages for, anyway? If you're so good that you can remember all the obscure restrictions, then why do you need to be reminded what the function prototype is? You can grep for that.
FWIW, the Linux man page does mention this restriction:
Of course, the man page also includes this rather, erm, professional bit of prose:
While it's true that many buffer overflows come from such programmer oversights, "stupid" and "lazy" aren't the only reasons. Incompetence, forgetfulness and just plain human error are additional reasons. Not exactly a model of how man pages ought to be written either, IMHO. It seems like it ought to be possible to warn programmers without insulting learners.
(I should add, one can be incompetent without being stupid. A smart person can be ignorant, especially about the peculiarities of one language when they're not fluent in it, vs. another that they might be fluent in. I would say that that person is not a competent programmer in the language they're not fluent in. I, for instance, consider myself a competent programmer generally, but I'm not competent in, say, LISP.)
--
Program Intellivision and play Space Patrol!
I ilke rude comments...
The ruder a comment the more likely it is that the reader will actually pay attention. If you are trying to learn and you see a comment like "that is, if the programmer was stupid or lazy", I would sure pay attention and think it would be important to follow the advice.
I don't want people thinking I am stupid. Lazy can be a virtue, but not in this case.
This kind of in-your-face comment provides a motivation that is better than a very polite and dry "The strings may not overlap".
I am in favor of the man page pointing out the nuances of the function, but the more interesting the man page, the more beneficial. Heck, we are all looking at it right now because of the "erm, professional bit of prose".
I have a hard time arguing with results.
There's a time and place
Rude (or at least unabashed and unreserved) comments have more place in forums, mailing lists and private messages than they do in documentation. I personally would rather hold man pages to similar editorial and stylistic standards as any other vendor-produced documentation.
You can still get results without insulting the reader. Strong language != insulting language. I personally would lean toward something like this:
--
Program Intellivision and play Space Patrol!
Documentation
Then you might as well go ahead and explain in strlen(3) and friends that you must pass a string, not just any char array. IMHO this has no place in a man page.
If you don't know what a string is, then you don't know C. If you don't know that you cannot modify string literals, then you don't know C. If you don't know the difference between memcpy and memmove, then you don't know C. You learn C by reading books, writing code, and reading the standard to get the finer details often left out in the books (and to write sweet portable code with well defined behaviour ;-) ).
man pages are verbose enough as they are.
Meh
Need some more straw for your strawman?
Some things are more obscure than others. For example, the overlapping string aspect of strcpy, I admit, hadn't occurred to me. I always think in terms of it doing a forward copy. I don't recall K&R talking about it when discussing strings, actually. And they know C.
Heck, some quick googling finds that this sort of error isn't so uncommon.
I actually went and dug up my K&R C book (the ANSI C second edition), and they suggest that strcpy would typically be written as follows by an expert C programmer (p. 106):
void strcpy(char *s, char *t) { while (*s++ = *t++) ; }And I wonder where I got the impression that strcpy does forward copies.
To their credit, though, waaaaay in the reference material in the back (Appendix B, p. 249), they mention that among all the memXXX and strXXX function, only memmove works with overlapping arguments. It's not in the explanatory body text, but rather buried in the man-page-like reference material at the back.
As for man pages and what to include: It's one thing to go into a complete dissertation on the foibles of string handling in a man page. It's quite another to point out a couple of gotchas with short mentions. Non-overlapping arguments and preventing overflows are worthy of mention. Explaining the concept of NUL-termination from first principles is not.
--
Program Intellivision and play Space Patrol!
man pages
That's why you don't learn C properly just by reading other peoples' code. You need to consult the final authority on the language, which is the language specification.
As for what a man page should include, I would have to agree with you, to some extent, in your last paragraph. However, I do not like the BUGS section in strtok(3) on GNU/Linux systems. "Never use this function, but if you do ..." Might as well just say "Never use C because it allows you to shoot yourself in the foot".
It's contradictory from the start, and then they go on to say that "These functions modify their first argument." This should be bloody obvious since they take a
char *, and not aconst char *.authorative reference
It should be pointed out that the man pages that come on Linux systems are not written by the people who implement the C library that those systems use. For the authoritative documentation you should refer to The GNU C Library, which can be read on the web or accessed via the 'info' program on your system (unless Debian has decided to not ship it, grrrrrr).
Instant FAIL for stringtards
If you do use strcat or strcpy, then you instantly fail at being a programmer.
That is all.
Instant FAIL for knuckleheads
Some of us know how to use strcat and strcpy.
Fail
If you want to write standard C that works everywhere, you don't have much of a choice. You can use these functions correctly, you know. If you don't know how, then perhaps you should be doing something else.
That said, if your target platform(s) supports the l-versions, there's no reason not to use them.
Or you can be smart
>If you want to write standard C that works everywhere, you don't have much of a choice.
Or if your library doesn't provide those functions, you can be smart and copy the code from strl* functions (BSD license == no problem) in your own code, it's not as if these functions were big..
And reducing the risk of making an error is the smart way of programming: we're all humans, we *do* make mistakes.
If you don't write
If you don't write everything in machine code, you are fail. C is for blouses.
Do I win the pissing contest ?
...not So Categorical
...Dear Theo, You Should Not Be So Categorical, I`am A Java
Programmer, for Example (-:
Best Regards,
OpenBSD rull!
Re: ...not So Categorical
QED