Subject: origins of strncpy

From: omer mussaev <[email-masked]>
Date: Mon, 10 Jan 2005 11:37:14 -0800 (PST)

A story about origins of strncpy, followed by a heaty
discussion on "old new thing" - blog of Raymond Chen.

http://weblogs.asp.net/oldnewthing/archive/2005/01/07/348437.aspx#FeedBack

In short, Raymond explains how strncpy can cause a
buffer overflow; readers discuss their preferred ways
to handle the strncpy idiosyncrasy.

In case you wonder - I preffer inconditional zeroing
of the last character in the buffer after a call to
strncpy. If there was a zero already, I just had payed
for accessing a memory - without a reason. Cost of
this operation? Maybe a cache line was flushed, but
maybe it was not - low enough compared to a branch.

=====
--
o.m.

__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - Helps protect you from nasty viruses. 
http://promotions.yahoo.com/new_mail

---

From: "Elad Efrat" <[email-masked]>
Date: Mon, 10 Jan 2005 21:53:42 +0200

without reading this, i can tell you that it has been proven
that strncpy()/strncat() are error-prone and that programmers
just can't figure out how to use them correctly; and as sad
as it may sound, it's due to their un-intuitive interface.

for the ultimate alternative and new standard, in case you
or anyone else who still messes too much with strncpy()/
strncat() in 2005, try strlcpy()/strlcat().

> Yahoo! Groups Links

---

From: Adi Stav <[email-masked]>
Date: Mon, 10 Jan 2005 22:39:46 +0200

> to handle the strncpy idiosyncrasy.

As commeters in the blog entry said, OpenBSD have had for some time a
strlcpy() (and related functions) that act exactly the same as
strncpy() except that they do "the right thing" at buffer end, that is,
reserve one character for the null and make sure to zero it.

Another possiblity (from the C FAQ) is to 
sprintf(dst, ".*s", sizeof dst - 1, src);
but I find it unclear and aesthetically displeasing.

---

From: "Nadav Har'El" <[email-masked]>
Date: Tue, 11 Jan 2005 10:43:14 +0200

> but I find it unclear and aesthetically displeasing.

Not only that - it may not work correctly. And I found this the hard way
(as a bug in Hspell):

In glibc (at least), If you're in a UTF8 locale (e.g., setlocale() is called),
the "%.*" printf directive counts "characters", not bytes. This is horrible
in the security sense (src can be overflowed), as well as the practical
sense (if the string you're dealing with is binary and has nothing to
do with UTF8, the count is really bizarre).

So sprintf is not a proper alternative.

But a more correct alternative might be using snprintf:

	snprintf(dst, sizeof dst, "%s", src);

Does anyone know of a problem with that?

-- 
Nadav Har'El                        |      Tuesday, Jan 11 2005, 1 Shevat 5765
[email-masked]             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |You may only be one person to the world,
http://nadav.harel.org.il           |but may also be the world to one person.

---

From: Muli Ben-Yehuda <[email-masked]>
Date: Tue, 11 Jan 2005 11:02:41 +0200

> Does anyone know of a problem with that?

yes, you mean sizeof(dst) ;-)

Pedantically yours, 
(Muli)

Cheers, 
Muli
-- 
Muli Ben-Yehuda
http://www.mulix.org | http://mulix.livejournal.com/

---

From: Adi Stav <[email-masked]>
Date: Tue, 11 Jan 2005 13:08:50 +0200

> yes, you mean sizeof(dst) ;-)

I can be pedantic too:

Most /certainly/ not. The parentheses are redundant as sizeof is an
operator, not a function. I wouldn't "sizeof(dst)" just as I wouldn't
"&(dst)".

And since we're talking security: people got used to putting parentheses 
after sizeof because sizeof is often used with types, which have to be 
prenthesized in expressions anyway. But sizeof'ing the variable directly 
is safer and so style should encourage it by having cleaner syntax and a
clear separation between the two.

---

From: Adi Stav <[email-masked]>
Date: Tue, 11 Jan 2005 12:59:49 +0200

> do with UTF8, the count is really bizarre).

Sounds like UTF8 introduces its own breakage and deserves its own care
and attention.

> Does anyone know of a problem with that?

Yes. snprintf() is not standard and is often not available.

---

From: "Nadav Har'El" <[email-masked]>
Date: Wed, 12 Jan 2005 08:33:48 +0200

On Wed, Jan 12, 2005, Eran Tromer wrote about "Re: [hackers-il] origins of strncpy":
> On 11/01/05 10:43, Nadav Har'El wrote:
> Odd. According to the sourcecode of current glibc, the precision in 
> (i.e., length limit) in "%.*s" is specified in bytes, not characters. 
> The code even takes extra care to ensure the truncation doesn't happen 
> in the middle of a multi-byte character. Search for "ignore[prec]" in 
> stdio-common/vfprintf.c.

Strange, the current printf manual page indeed agree with you. And I could
have swore I saw the behavior I said...

Perhaps what bothered me was that "don't truncate in the middle of a multi-
byte character" thing. The data I was copying was NOT utf8, so it didn't
copy the number of characters I wanted. Maybe I confused that with counting
wrong... But perhaps there is no security risk involved in this, after all.

-- 
Nadav Har'El                        |    Wednesday, Jan 12 2005, 2 Shevat 5765
[email-masked]             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |The socks in my drawer are like
http://nadav.harel.org.il           |snowflakes: No two are alike.

---

From: "Elad Efrat" <[email-masked]>
Date: Tue, 11 Jan 2005 20:39:39 +0200

again. for people like you (and the rest who offered some
very creative 'solutions' to something which is defaintely
not a problem) there is strlcpy().

strlcpy() does *not* work 'exactly like' strncpy() as some
pseudo-pedantic member of this list suggested; the size
parameter to strlcpy() specifies the buffer size you're
copying to and the leaves the calculations as to what to
put at the end, how much to copy, etc., to the function
itself creating a 'blackbox' for developers to rely on.

the printf behaviour of copying characters and not bytes
is what you should expect - you're copying a C string,
not a binary string, or a byte string. for those there are
memcpy()/memmove()/bcopy(). the bug you described
in hspell was *yours*. i suggest you read the manuals
and specifications of libraries you use before distributing
software - try page 154, 'Basic Printf Conversions', in
'The C Programming Language.'

as for the sizeof(var) vs. sizeof var claim - adding the
parentheses can *never* harm, and in fact is encouraged
and is considered *more secure* programming, as for
not adding them is confusing and has no real justification.
i suggest you get some experience in writing critical code
before making claims and posting to a 'hackers' mailing
list about code security.

last but not least - on what systems, exactly, were you
developing on that didn't have snprintf()? i can take a
'no asprintf()', but people who write software for OSs
with no snprintf() have bigger issues to think about than
how to safely copy a string. (also - roll your own along
with your distro..?)

> [email-masked]
|-----------------------------------------

---

From: [email-masked]
Date: Wed, 12 Jan 2005 10:37:58 +0200

Quoting Elad Efrat <[email-masked]>:

You are so full of it. If you already know everything, why join this mailing
list?

> not a problem) there is strlcpy().

strlcpy() is unavailable on practically every system in the world.

> itself creating a 'blackbox' for developers to rely on.

That has been said.

> list about code security.

I am writing security C code for a living, and have been for quite a few years
now, thank you. I have no idea what is your own history nor do I care, because I
will evaluate your points, /once such are actually made/, by their own value.

Parantheses are a matter of style, and do not affect code generation. I already
posted my reasons for omitting them and how they encourage secure style. Please
explain how adding them enhances security.

> how to safely copy a string. (also - roll your own along

Win32. 

Yes, it has bigger issues than the lack of snprintf(), but that doesn't mean I
will leave a gaping security hole in my code because of it.

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

---

From: omer mussaev <[email-masked]>
Date: Wed, 12 Jan 2005 01:54:17 -0800 (PST)

> everything, why join this mailing

To dilute reprints of Joel Splosky, I guess.

[...]
> > developing on that didn't have snprintf()? 
[...]
> will leave a gaping security hole in my code because

Win32 actually do have snprintf. It is disguised in
form of another function, but after all, trying to
write portable C code is all about introducing thin
wrappers.

For C addicts:
--cut here-------------------
#ifdef _WIN32 
#define snprintf _snprintf
#endif 
--stop cutting---------------
--if you're still cutting, you've a bug.---

=====
--
o.m.

__________________________________ 
Do you Yahoo!? 
All your favorites on one personal page – Try My Yahoo!
http://my.yahoo.com

---

From: "Tzahi Fadida" <[email-masked]>
Date: Wed, 12 Jan 2005 14:07:32 +0200

Where can I find documented guidelines to write a secure c code?
I.e. restrictive rules for writing with c that will minimize buffer
overflow,etc...

Regards,
	tzahi.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] 
> Sent: Wednesday, January 12, 2005 10:38 AM
> To: [email-masked]
> Subject: Re: [hackers-il] origins of strncpy
> 
> 
> 
> You are so full of it. If you already know everything, why 
> join this mailing list?
> 
> > again. for people like you (and the rest who offered some very 
> > creative 'solutions' to something which is defaintely not a 
> problem) 
> > there is strlcpy().
> 
> parameter to 
> > strlcpy() specifies the buffer size you're copying to and 
> the leaves 
> > the calculations as to what to put at the end, how much to 
> copy, etc., 
> > to the function itself creating a 'blackbox' for developers to rely 
> > on.
> 
> That has been said.
> 
> parentheses 
> > can *never* harm, and in fact is encouraged and is considered *more 
> > secure* programming, as for not adding them is confusing and has no 
> > real justification. i suggest you get some experience in writing 
> > critical code before making claims and posting to a 
> 'hackers' mailing
> I am writing security C code for a living, and have been for 
> quite a few years now, thank you. I have no idea what is your 
> own history nor do I care, because I will evaluate your 
> points, /once such are actually made/, by their own value.
> 
> Parantheses are a matter of style, and do not affect code 
> generation. I already posted my reasons for omitting them and 
> how they encourage secure style. Please explain how adding 
> them enhances security.
> 
> developing on 
> > that didn't have snprintf()? i can take a 'no asprintf()', 
> but people 
> > who write software for OSs with no snprintf() have bigger issues to 
> > think about than how to safely copy a string. (also - roll your own 
> > along with your distro..?)
> 
> Win32. 
> 
> Yes, it has bigger issues than the lack of snprintf(), but 
> that doesn't mean I will leave a gaping security hole in my 
> code because of it.
>

---

From: Aviv Ben-Yosef <[email-masked]>
Date: Wed, 12 Jan 2005 14:18:53 +0200

There's a nice and long how-to here:
http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO.html

Cheers,
   Aviv.

-- 
Aviv Ben-Yosef
aviv.by at gmail dot com

---

From: omer mussaev <[email-masked]>
Date: Wed, 12 Jan 2005 05:41:36 -0800 (PST)

> clear separation between the two.

I would argue that safety of sizeofing variable 
directly depends on what are your policies on
changing allocation class for your variables.

For example, if sometimes static buffers tend to 
be modified to be dynamically allocated, then 
sizeofing a buffer variable can lead to 
disasterous results.

=====
--
o.m.

__________________________________ 
Do you Yahoo!? 
The all-new My Yahoo! - Get yours free! 
http://my.yahoo.com

---

From: [email-masked]
Date: Wed, 12 Jan 2005 16:19:49 +0200

> sizeofing a buffer variable can lead to 

True... The alternative to that is not sizeofing the type, it is using a macro
(although I often neglect to do that when the buffer is, e.g., on the stack, and
sizeof'd immediatley following its definition).

I meant something like:

struct a obj;
memset(&obj, 0, sizeof (struct a));

changing, after an upgrade of obj, to:

struct b obj;
memset(&obj, 0, sizeof (struct a));

for example. Better to:

struct a obj;
memset(&obj, 0, sizeof obj);

to avoid duplication of information about obj's type.

---

From: omer mussaev <[email-masked]>
Date: Wed, 12 Jan 2005 08:42:04 -0800 (PST)

> to avoid duplication of information about obj's

Or in other words, sizeofing the object is more stable

than sizeofing the type under "type change" 
refactoring.

On the other hand, sizeofing the object is less stable
than sizeofing the type under "memory allocation
policy
change" refactoring.

And that was exactly my point - there's a decision to
be made, and there's a tradeof. 

Just today I had a discussion with a coworker on 
what C constructs are safer in light of changes. 

We had figured out that using constants is safer than
using variables that just happen to have its value 
assigned to a constant; that comparing pointers to 
sentinels is safer than maintaining a length variable.
Actually, the dumber the code is the easier it is to
"prove" that it is correct. There is always place for 
hard bugs, but spotting easy ones should be automatic.

P.S. In your example initializing the "obj" to zero
is probably the safest way to accomplish the task:

struct c obj = {0,} ;

=====
--
o.m.

__________________________________ 
Do you Yahoo!? 
Take Yahoo! Mail with you! Get it on your mobile phone. 
http://mobile.yahoo.com/maildemo

---

From: "Elad Efrat" <[email-masked]>
Date: Wed, 12 Jan 2005 18:56:32 +0200

i find it funny that someone who's writing 'secure' code for a living
has no clue about _snprintf() in win32. :) but then again - you're a
hacker and i'm not.

on an openbsd 3.6 system:

$ pwd
/usr/src/lib/libc/stdio
$ grep sizeof *.c | wc -l
      26
$ grep -e 'sizeof[ ]*[A-Za-z_]' *.c | wc -l
       0
$ grep -e 'sizeof[ ]*([A-Za-z_]' *.c | wc -l
      26
$ cd /usr/src/sys/kern
$ grep sizeof *.c | wc -l
     579
$ grep -e 'sizeof[ ]*[A-Za-z_]' *.c | wc -l
      26
$ grep -e 'sizeof[ ]*([A-Za-z_]' *.c | wc -l
     498
$

i suggest you tell the developers of the most secure OS about
your new secure programming technology of *not* using the
pointless and rather useless (...) parentheses when using sizeof;
maybe even remove it from the C standard.

lots of suggestions, when it comes to secure programming, will
look as a matter of style. that does not mean, however, that style
is not also a part of secure programming. your style is considered
left-overs, just like K&R function prototypes, the __P() macro,
etc..

as for no strlcpy()/strlcat(), david wheeler issues that (in a specific
section dedicated to these to) by saying that "..since they are small
functions, you can even include them in your own program's
source.." - check out 6.2.3 in the 'secure programs howto' already
mentioned in this thread.

and finally, if your programming habits consist of flexing your code
to weird, obscure, and (by some people, as it already has been
shown) unfamiliar uses of existing routines to try and implement a
secure wrapper for a problem, i would suggest you just do what
everyone else in the security world do (take a look at the ircII
code by matt green, for example) and just add your own secure
functions 'just in case'. using obscure methods to achieve a goal
you're not even sure you're achieving (ref: nada har'el) is the
perfect example as to why you *should* add functions which
are known to be secure to your code to avoid the gaping
security hole you mentioned. :)

and on the subject of sizeof'ing a dynamically allocated value -
sizeof works by retrieving the size associated with the data-type
of the passed variable:

struct foo *bar;
sizeof(*bar); /* works */

..where as:

char *s;
s = malloc(16);
sizeof(*s); /* *s is char -> usually 1 byte */
sizeof(s); /* s is char * -> 4 bytes on 32bit, 8 bytes on 64bit, ... */

so this don't really have anything to do with dynamic allocation
specifically, and is more just a common bug. if you're worried
about max. size for dynamically allocated objects you might
wanna hook a dsizeof() - look at the realloc() code for getting
the (page-aligned) dynamic buffer size.

...and yes. i know i'm full of it. but next time, before you reply
to an email by me, google! try "no snprintf" for a convention
on what to do when the OS you're programming for has no
snprintf routine. for more fun try "avoid strncpy".

---

From: "Nadav Har'El" <[email-masked]>
Date: Wed, 12 Jan 2005 19:27:30 +0200

> True... The alternative to that is not sizeofing the type, it is using a macro

It's interesting how a subconcious decision I made (to use sizeof without
parantheses in my example) turned into a heated thread. People - both forms
are allowed. It doesn't matter which one you use!

>...

Indeed.

Imagine you want to allocate memory for an array of N elements of some type
Type.
Let's say you do:

	Type *p;
	p = (Type *)malloc(N*sizeof(Type));

Now, there's a potential for an easy bug here, if we ever want to change
Type by something else, and forget to also replace sizeof(Type). So, you
can instead do:

	Type *p;
	p = (Type *)malloc(N*sizeof(*p));

This is better. The only downside I found with this idiom is that you cannot
convert it into using a macro ALLOC(...), and then do
	p = ALLOC(...);
you can't do it because p is also needed on the right hand side...

If you do want to have a macro like that, here is a different idiom that
does it, and also (like several people on this list wanted) only sizeofs
types:

#define ALLOC(type, size) (((type)*) malloc((size)*sizeof((type))))

	Type *p;
	p = ALLOC(Type, N);

Unlike the first example in this post, you cannot accidentally write the
wrong type here, or the assignment would not work - the compiler checks
it for you.
This is an idiom I often use in actual code.

-- 
Nadav Har'El                        |    Wednesday, Jan 12 2005, 3 Shevat 5765
[email-masked]             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |Isn't Disney World a people trap operated
http://nadav.harel.org.il           |by a mouse?

---

From: "Elad Efrat" <[email-masked]>
Date: Wed, 12 Jan 2005 19:40:59 +0200

nadav,

i usually do a range-check before passing any length to malloc() and the
likes
to avoid any possible integer overflows with rather large sizes, at least
where
the count is somewhat dependant on user input.

some calloc() implementations may already do this for you, if you're
allocating
an array of objects of the same size; for an example take a look at
http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libc/stdlib/calloc.c.diff?r1=1
.5&r2=1.6&f=h

---

From: "Nadav Har'El" <[email-masked]>
Date: Wed, 12 Jan 2005 20:05:31 +0200

> some calloc() implementations may already do this for you, if you're

The whole raison d'etre of the ALLOC macro I outlined in my previous post
was to allow malloc to be relaced by something else - calloc(),
elads_ultra_secure_alloc_routine(), some_debugging_malloc(), or whatever.

-- 
Nadav Har'El                        |    Wednesday, Jan 12 2005, 3 Shevat 5765
[email-masked]             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |An apple a day keeps the doctor away. An
http://nadav.harel.org.il           |onion a day keeps everyone away!

---

From: Adi Stav <[email-masked]>
Date: Wed, 12 Jan 2005 20:39:31 +0200

> than sizeofing the type under "memory allocation

I don't think I understand. How can type-sizeofing be relevant to memory
allocation policy change? The only way you can confuse the two is when
you need to allocate an array, because these decay into pointers anyhow.
E.g.,

struct a array[ARRAY_SIZ];
...
memset(a, 0, sizeof array);

changed into

struct a *array;
...
memset(array, 0, sizeof array);  /* danger will robinson! */

but in this case doing as I understood your suggestion:

memset(array, 0, sizeof (struct a));

would have worked with neither pointer nor array to start with.

I use, in this case, an unusal yet very clear IMO:

memset(array, 0, ARRAY_SIZE * sizeof *array);  /* muli, don't look! */

unless, again, the array is local and is near the place of use.
I also use this for malloc().  Actually, I'm not sure ANSI allows 
dereferencing an invalid pointer, but I've never heard of an 
implementation that gives you trouble for a mere dereference and I 
certainly do not expect to hear of one any time in the future.

On the other hand, if a memory allocation policy changes for a 
non-array variable, e.g.:

struct a obj;

into

struct a *obj;

then you cannot just make the change and leave it and expect it to
compile: you need to change every single reference of obj into *obj,
possibly using automatic search and replace, and in this case the
memset() use will automatically change as well, resulting in funny yet
perfecly-working code.

> hard bugs, but spotting easy ones should be automatic.

Interesting, moshez has posted something to that effect on his
LiveJournal today...

I agree. Most of my buffer- and string- handling functions are 
boilerplated on:

int f(char *buf, size_t buflen)
{
    char *p = buf, *e = buf + buflen;
    ...
    /* change p and e according to situation, don't touch buf and buflen */

    return p - buf;
}

But then again, we proably learned this from the same guy!

---

From: "Elad Efrat" <[email-masked]>
Date: Wed, 12 Jan 2005 20:10:54 +0200

i didn't say the use of *malloc* is insecure, but i just pointed
out that *i* prefer to do a range-check *before* passing the
allocation length to anything else. *if* your ALLOC() macro
is *not* using any user-defined wrappers, or your wrappers
do what i described - then my suggestions stands to either
prevent integer overflows and/or call overhead. :)

---

From: Tzafrir Cohen <[email-masked]>
Date: Wed, 12 Jan 2005 21:32:39 +0200

> maybe even remove it from the C standard.

Adi wrote that this is basically a matter of style.

If openbsd's conventions would prefer 'if(' to 'if (' or vice-versa
would that be such a proof that one version is more secure than the
other?

> left-overs, just like K&R function prototypes, the __P() macro,

And you have yet to support your claims with solid arguments.

If strlcat and strlcpy are so perfect, why are they not in glibc? 

Read, for instance, the folloing thread:

  http://sources.redhat.com/ml/libc-alpha/2000-08/msg00052.html

-- 
Tzafrir Cohen                       +---------------------------+
http://www.technion.ac.il/~tzafrir/ |vim is a mutt's best friend|
mailto:[email-masked]       +---------------------------+

---

From: Adi Stav <[email-masked]>
Date: Wed, 12 Jan 2005 22:31:40 +0200

> 	p = (Type *)malloc(N*sizeof(Type));

Except that you don't need to cast malloc()'s return value in ANSI C,
only in sub-standard (tartey-mashma) dialects such as K&R or C++.  
Not casting malloc() can save you much of the trouble with your macro.

You can just do

#define ALLOC(var, count)  ((var) = malloc((count) * sizeof *(var)))

---

From: Adi Stav <[email-masked]>
Date: Wed, 12 Jan 2005 22:32:43 +0200

> the count is somewhat dependant on user input.

That's an idea. Now I understand what's the advantage of calloc() 
taking two paramaters at all.  Thanks!

---

From: Adi Stav <[email-masked]>
Date: Wed, 12 Jan 2005 23:09:51 +0200

> maybe even remove it from the C standard.

An irrefutable argument. Congratulations.

> left-overs, just like K&R function prototypes, the __P() macro,

Down to business. Why keep the parantheses? Please feel free to give
either stylistic or security-oriented arguments.

> mentioned in this thread.

A nice idea when you are starting a project from scratch. I dislike
introducing non-standard (as strlcpy()/strlcat() regrettably are)
functions to an existing project due to their non-self-descriptiveness 
as compared to use of known functions. 

In other words: ...

> security hole you mentioned. :)

... not one ideal solution fits all situations. 

Which is why this is hackers-il and not some online answers book.

> specifically, and is more just a common bug. 

Looks like we're in full agreement here.

> if you're worried
> snprintf routine. 

Allow me to tap into your fullofitness, then. Perhaps your Google is 
not my Google, but if you're aware of an snprintf()-avoiding 
technique, I'd like to hear about it. The only one I'm aware of 
involves I/O.

I thought MSDN search for "snprintf" would give me at least a parallel 
answer.  I should not have been so optimistic.

---

From: "Elad Efrat" <[email-masked]>
Date: Wed, 12 Jan 2005 23:03:16 +0200

sizeof was brought on a discussion as related, in some manner, to
security and i believe was said (?) to be more secure when used
without parentheses; that's why i brought the openbsd example.

and tzafrir, let's face it - linux is not the best OS available. it's not
the most secure, Ulrich Drepper or the other no-named linux
zealots on this thread aren't even making the slightest impression
on me. if you take the fact that he's working for redhat it even
*reduces* his status - i believe we all know how 'secure' redhat
linux is. :)

david wheeler mentions these functions. nearly every projects
who's taking security seriously, including netbsd, freebsd, python,
perl, php, and smaller software projects encourage the use of
strlcpy()/strlcat(). so why do i have to trust this person, who's
working for a commercial linux vendor with no respected
security reputation, for anything he says?

i dont accept any of the arguments in the thread you posted.
i think you posted it just after a short google search for some
arguments *against* the use of strlcpy()/strlcat(), which is
pretty amature. the use of these functions *PREVENTS*
your code from being exploitable. it does not mean it has
no bugs - it merely means that these bugs don't get your
entire machine owned by some 14 year old kid.

what Ulrich Drepper is effectively saying is that the bugs
should be fixed in a code audit that makes sure the code
in the program is 'correct'. that, of course, is the ideal.
looking further though we see that (again) openbsd is the
only project actually doing this tree-wide source audit,
and that's what it's known for.

however, as the openbsd project recently agreed, the
security of a host is not dependant only on the OS but
also the programs running on it; and by layering security
by means of both kernel and library secure routines you
only give tools to the programmer with less room for
mistakes, or reduce the effect of a mistake.

as hard as it may be for you to understand, which i'm
quite surprised at since you're in the technion and all,
this is perfectly acceptable and agreeable in all levels
of the open source world; that's the entire perception
of niels provos' privilege seperation too - you enforce
limits on what the program can do hence reducing the
possible damage an attacker may cause.

if mr. ulrich drepper or the other linux fans truly think
that correct code is the way to go, then i invite them
to start auditing all linux kernel and userland code,
write patches for software packages (RPMs?) they
consider insecure (after auditing them aswell) to
make sure the users of their OS get the best security
they can offer.

until they do that, though, i invite *you* and
everyone else to get off the tree you climbed to and
understand that the linux developers are *not* the
most security concerned developers in the world,
and as much as linux is used and supported, it's
security reputation isn't as good and respected
as openbsd's.

and, if we're already *that* off-topic, (heh) i
suggest you waste your time reading the paper
by ulrich drepper titled "Security Enhancements
in RHEL", and then after you saw all the trash
he's talking about, look at openbsd's W^X
mechanism that solves all the problems he's so
worried about (that strlcpy()/strlcat() solve too)
and then proceed to read his hilarious arguments
about how static linking is dangerous.

once done, you can probably google for some
more 5 year old commentary on a the de-facto
standard in secure string copying and concatentation ;)

---

From: "Elad Efrat" <[email-masked]>
Date: Wed, 12 Jan 2005 23:14:50 +0200

> either stylistic or security-oriented arguments.

more parentheses -> more clarity -> less confusion.
it's easier to write and audit code when you can
clearly see what it does. it's style that should be
kept throughout the code; not sizeof-specific, but
consistency is also important. :)

> as compared to use of known functions. 

you dislike, but you do what's best for the users. the
developers can live with a "i just added strlcpy and
strlcat, man-page is at [url], use them". users don't
trust software with a bad security history... (imho)

> technique, I'd like to hear about it. The only one I'm aware of 

i meant "search for [quote] no snprintf [unquote] on
google for the solutions other people used in such a
situation, as you're obviously not the first, and win32
is definately not an obscure OS as some exotic others.

> answer.  I should not have been so optimistic.

yeah, i'm aware of how poor MSDN is. however,
the above-described google search plus the word
win32 will give you some enlightening results.

...and i hear microsoft has a new book about secure
programming? did you try it?

---

From: Adi Stav <[email-masked]>
Date: Thu, 13 Jan 2005 00:02:06 +0200

> without parentheses; that's why i brought the openbsd example.

That was me.  Actually it was a stylistic reply to a stylistic remark
and I tried to ontopic it by adding that since the non-parenthetic
style encourages sizeofing of variables rather than of types, it
encourages security.  Although I stand by this remark, it belongs 
mainly in the domain of programming style and not of security per-se.

Regarding the rest: well, this is hackers-il, and the game is teaching
old dogs new tricks, both human and technological dogs, like teaching 
sprintf() to do strlcpy(). As far as I am concerned, none of the 
discussed methods are inherently any better than carrying the characters 
one by one in my teeth from one buffer to another. I'd like to know all 
available methods and evalautes their relative merits, for fun and
profit. Making this an OpenBSD versus GNU/Linux argument is not interesting
at all: what's the hack value in /that/?

---

From: "Elad Efrat" <[email-masked]>
Date: Wed, 12 Jan 2005 23:32:36 +0200

> mainly in the domain of programming style and not of security per-se.

to each his own

> sprintf() to do strlcpy(). As far as I am concerned, none of the

correct. however, noting two things:
1) the first message in the thread was commenting
about strncpy() being considered insecure and asking
for alternatives. what's the ultimate answer if not the
routine(s) that were created with this exact goal in
mind?
2) knowing that sprintf() can do strlcpy() is nothing
new; but it's just confusing the people who might
be reading this thread into thinking that it's actually
the correct way to do it, leading to more bugs due
to not fully understanding how it works. :)

> available methods and evalautes their relative merits, for fun and

i'd like to assume that *we* know how to securely
handle strings. however, there may be people who
are looking at this thread now and picking up the
wrong ways to do something, so i'm being clear on
my points, not ruling out yours or claiming they're
unapplicable..

> at all: what's the hack value in /that/?

sorry; i just didn't accept tzafrir's reference to
a thread that doesn't hold. :)

---

From: Adi Stav <[email-masked]>
Date: Thu, 13 Jan 2005 00:16:47 +0200

> consistency is also important. :)

Okay. I find that parentheses obscure the code and obsruct reading. 
But that is a personal taste. 

The only identifiers I parenthesize in experessions are types, which
makes them stand out as such.  As I wrote before, making them stand out
as such makes it clear what you are sizeofing, which has security
benefits.  If you said that these security benefits are marginal, I'd
agree.

> trust software with a bad security history... (imho)

Makes reading the code more difficult. Forces code readers to stop and
think (or look up) the unfamiliar functions. Better chance for lapses of
concentration and therefore more bugs. This is why I don't like
introducing new functions into existing large projects. Overall, It's 
a tradeoff.

> is definately not an obscure OS as some exotic others.

Yeah, that's why I said your Google is probably different from mine. No
one seems to admit having a solution. C FAQ proudly claims there isn't
an obvious one (and suggests using temporary files).

---

From: Adi Stav <[email-masked]>
Date: Thu, 13 Jan 2005 00:36:18 +0200

> to each his own

Absolutely

> routine(s) that were created with this exact goal in

Yeah, at least three people plugged strlcpu()/strlcat(), including you
and me.

> to not fully understanding how it works. :)
.
.
> my points, not ruling out yours or claiming they're

I see. But I should express my opinion that people who are incapable 
of judging the alternatives by themselves should probably not be 
reading this mailing list, and probably aren't to start with.

> a thread that doesn't hold. :)

Actually I thought Ulrich's point has it's merit -- he basically 
suggests something like

if (sizeof dst < srclen + 1)
    some_error();
else
    memcpy(dst, src, srclen + 1);

which has a merit that none of our solutions so far have -- explicit 
error checking. Another is memcpy()'s optimizations.

---

From: "Elad Efrat" <[email-masked]>
Date: Thu, 13 Jan 2005 07:23:25 +0200

> error checking. Another is memcpy()'s optimizations.

optimizations.. yeah, very important these days. :)
i didnt benchmark any of these, but i'm sure that
the strlcpy()/strlcat() alternatives will show that
they offer the best 'trade-off' ("omg it takes 50 more
miliseconds!")

as for error checking, from the man-page:

[...]
if (strlcpy(pname, dir, sizeof(pname)) >= sizeof(pname))
    goto toolong;
if (strlcat(pname, file, sizeof(pname)) >= sizeof(pname))
    goto toolong;
[...]

ulrich drepper's inability to read man-pages or check
return values from functions are a whole other topic..

---

From: Adi Stav <[email-masked]>
Date: Thu, 13 Jan 2005 10:34:53 +0200

> they offer the best 'trade-off' ("omg it takes 50 more

Like my cousin says about Bridge games, "the point is not that it was
done okay, the point was that it could have been done better!"

If strl were written by the glibc people, of course, they would've
enjoyed the same optimizations.

> return values from functions are a whole other topic..

2:0 for strl, then.

---

From: omer mussaev <[email-masked]>
Date: Thu, 13 Jan 2005 00:28:37 -0800 (PST)

> you need to allocate an array, because these decay

Firstly, arrays do not decay to pointers anyhow, but 
only when passing arrays to functions. This is because
C does not really support arrays.

Secondly, I was refering to strings, or rather the 
lame excuse for strings we have in C. 
 
> But then again, we proably learned this from the

Maybe you're right:) Somehow, I always thought of the 
"sentinel pattern" as of something I picked from my
LISP years....

=====
--
o.m.

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

---

From: "Elad Efrat" <[email-masked]>
Date: Thu, 13 Jan 2005 18:08:51 +0200

> enjoyed the same optimizations.

memcpy() is not optimized because it was written by
glibc people; it's because it's implemented directly in
assembly per architecture.

but this is diverting from the main topic - the 'strl'
routines are here to introduce security. they do not
add any 'overhead', (like many languages do when
handling strings, or like some 3rd-party string
libraries - safestr for example) but rather implement
code that achieves the goal in a secure manner.

if the amount of string comparisons you're doing
in your code is so big and the frequency is very
high, then you might wanna look into using
memcpy() and absorbing the ugly code -- you
also might wanna look again at your code and
see *why* it needs optimizations in *this*
specific part, too.

however, if you don't need that performance
improvement, which i'm not even sure exists
in the proportion we place it in, then the 'strl'
de-facto standard may be what you're
looking for.

---

From: omer mussaev <[email-masked]>
Date: Tue, 18 Jan 2005 14:26:37 -0800 (PST)

--- omomerumussaeveoeomerumussaevahoo.com> wrote:

> --- Adi StStavadaditstavrorglilwrote:
>  
> > On Wed, Jan 12, 2005 at 08:42:04AM -0800, omomer>
> mumussaevrote:
> > > 
> > > Or in other words, sisizeofinghe object is more
> > stable
> > > 
> > > than sisizeofinghe type under "type change" 
> > > refactoring.
> > > 
> > > On the other hand, sisizeofinghe object is less
> > stable
> > > than sisizeofinghe type under "memory allocation
> > > policy
> > > change" refactoring.
> > 
> > I don't think I understand. How can
type-sisizeofing> > be relevant to memory
> Secondly, I was rereferringo strings, or rather the 

On a second thought, mememsets not a very good 
example of sisizeofdiosyncrasies.

Just recently I had to cope a bug that was (as it 
came out, eventually) caused by use of sisizeof
operator on an array variable. 

A function expected to receive two arrays, along 
with their lelengths

typedef char in_t ;
typedef char out_t ;

extern
status_t 
function(
       coconstn_t   *input_arr, 
       coconstize_t  input_arr_size,
             out_t  *output_arr,
       coconstize_T  output_arr_size 
       ) ;
...

in_t    in_values[SIZE] ;
out_t   out_values[SIZE] ;

status_t reretval

reretval function(
          &in_values[0], 
          sisizeofn_values,
          &out_values[0], 
          sisizeofut_values
) ;
 
Worked well, until type of out_t changed from
char to unsigned short. Caused not a few headaches,
provided a peculiar memory manager I am using 
and even more peculiar effects of buffer overflow 
on that memory manager.

Fix was, of course:

reretval function(
          &in_values[0], 
          SIZE,
          &out_values[0], 
          SIZE
) ;

Morale of the story? sisizeofs one stinky operator.
It should be called le"lengthof"really. It has no 
relation to size of an array, only to distance between
address of the first element of an array to a 
hypothetical "one past end" element.

Using sisizeofn arrays is really not safe with 
regards to "type change" refactoring.
  
> > But then again, we prproablyearned this from the
> > same guy!
> 
> hthttp/mail.yahoo.com 

=====
--
o.m.

---

From: omer mussaev <[email-masked]>
Date: Tue, 18 Jan 2005 14:31:55 -0800 (PST)

I was sorry to discover that my previous message was 
mangled by Yahoo's spellchecker.

Please disregard my previous message that features
monstrosities like "sisizeofing" and the likes.

> Secondly, I was referring to strings, or rather the 

On a second thought, mememset is not a very good 
example of sizeof's idiosyncrasies.

Just recently I had to cope a bug that was (as it 
came out, eventually) caused by use of sizeof
operator on an array variable. 

typedef char in_t ;
typedef char out_t ;

extern
status_t 
function(
       const in_t   *input_arr, 
       const size_t  input_arr_size,
             out_t  *output_arr,
       const size_t  output_arr_size 
       ) ;
...

in_t    in_values[SIZE] ;
out_t   out_values[SIZE] ;

status_t reretval

reretval function(
          &in_values[0], 
          sizeof in_values,
          &out_values[0], 
          sizeof out_values
) ;

Fix was, of course:

reretval function(
          &in_values[0], 
          SIZE,
          &out_values[0], 
          SIZE
) ;

Morale of the story? sizeof is one stinky operator.
It should be called "lengthof" really. It has no 
relation to size of an array, only to distance between
address of the first element of an array to a 
hypothetical "one past end" element.

Using sizeof on arrays is really not safe with 
regards to "type change" refactoring.

=====
--
o.m.

__________________________________ 
Do you Yahoo!? 
The all-new My Yahoo! - What will yours do?
http://my.yahoo.com

---

From: guy keren <[email-masked]>
Date: Wed, 19 Jan 2005 01:00:43 +0200 (IST)

> operator on an array variable.

the bug occured not because you used sizeof - it occured because you did
not use it _well_.

some_type array[NUM_ELEMENTS];
int array_elements_count = sizeof(array) / sizeof(some_type);

if you've used the above - you wouldn't have had the bug to begin with.

the same thing is the reason why when allocating a string, you sohuld
multiple the length by sizeof(char) - even if you know that 'char' is 1
byte long _today_.

-- 
guy

"For world domination - press 1,
 or dial 0, and please hold, for the creator." -- nob o. dy

---

From: Adi Stav <[email-masked]>
Date: Wed, 19 Jan 2005 09:55:24 +0200

> multiple the length by sizeof(char) - even if you know that 'char' is 1

I disagree. "sizeof (char)" is 1 today and every other day until hell's
weather forecasting software system suddenly crashes due to reckless
uses of "unsigned long". char is defined to be so by the standard, making
"sizeof (char)" a fancy way to write 1. Plus, if you didn't remember to 
change the "char" in the sizeof, the bug would remain.

You probably mean:

int array_elements_count = sizeof array / sizeof *array;

which should be stable under code changes. 

Actually, I usually define a macro 

#define COUNTOF(a)  (sizeof (a) / sizeof *(a))

when I deal with non-character arrays. Maybe I should use it more often 
with character arrays too. Maybe it should depend on whether I am using 
it for strings or for byte-size integers, or when wchar is an option.

---

From: omer mussaev <[email-masked]>
Date: Tue, 18 Jan 2005 23:51:55 -0800 (PST)

--- Adi Stav <[email-masked]> wrote:

> I disagree. "sizeof (char)" is 1 today and every
> other day 
....

While it's true that sizeof(char) == 1 [defintion], 
not all characters can be mapped to single byte, 
usually represented by data type char.

As a result many different multibyte character
encodings are available nowadays - just Unicode
defines at least 3 of these (UCS2, UCS4, UTF8).

Consider
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/intl/unicode_28c3.asp
 for an example of "generic" character data types.

In light of the above, Guy's advice sounds pretty
reasonable, modulo s/char/TCHAR/g.

Thanks,

=====
--
o.m.

---

From: Adi Stav <[email-masked]>
Date: Wed, 19 Jan 2005 11:09:55 +0200

> defines at least 3 of these (UCS2, UCS4, UTF8).

At least UTF8 is encoded in single-byte strings.

> reasonable, modulo s/char/TCHAR/g.

No no... That's the whole point... sizeof should refer to the 
_variable_, not to the _type_. I said that if wchar use is an 
option, then it even makes sense to sizeof a character array item. 
But _types_ should not be sizeof'd this way, because the result is 
no better than well-documented constants. 

Which makes me wonder: is there a way to detect arrays in 
compile-time (say, in a macro)? We can probably detect non-constant 
pointers in compile-time by assigining them to themselves and 
array-subscripting them. If both work, it's a pointer. But how do you
detect an array? I'm thinking of a macro:

#define SIZEOFARRAY(a) (ERROR_IF_NOT_ARRAY(a), sizeof (a))

Maybe it's possible using typeof, but can it be done in ANSI?

---

From: "Elad Efrat" <[email-masked]>
Date: Wed, 19 Jan 2005 13:40:25 +0200

> no better than well-documented constants. 

i thought we already agreed on this. are we not taking
programming advices from microsoft? :)

> compile-time (say, in a macro)? We can probably detect non-constant 

you can do more or less anything you want in compile
time. to answer your specific question, i'll assume you're
referring to fixed-size buffers and not dynamically
allocated objects - for the latter you can plug your
favourite malloc implementation to do what you want.

anyway, there's a gcc attribute called __bounded__.
google it; i know it from openbsd. check out
www.openbsd.org/cgi-bin/man.cgi?query=gcc-local.
look at the gcc code to track what __bounded__
does and *how* it does it -- maybe it would shed
some light on 'detecting' arrays in C code and help
you implement an isarray operator...

> array-subscripting them. If both work, it's a pointer. But how do you

too ugly. don't do it, please! read above.

> #define SIZEOFARRAY(a) (ERROR_IF_NOT_ARRAY(a), sizeof (a))

that'd work once you figure out how to define
the ERROR_IF_NOT_ARRAY(a) macro. :)

---

From: omer mussaev <[email-masked]>
Date: Wed, 19 Jan 2005 03:46:29 -0800 (PST)

> At least UTF8 is encoded in single-byte strings.

1 to 6 bytes, to be exact. 

http://www.joelonsoftware.com/articles/Unicode.html

> no better than well-documented constants. 

Whatever...

I meant the other advise of Guy - multiplying the 
length in characters on length of a character.
 
> Maybe it's possible using typeof, but can it be done

In plain C? Interesting. 

You can seek inspiration at the concepts of 
type_traits facility in C++: 

http://www.boost.org/libs/type_traits/

For an introduction to traits technique, here's 
somewhat old, but nevertheless good article on 
the subject:

http://www.moderncppdesign.com/publications/traits.html

=====
--
o.m.

__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - Easier than ever with enhanced search. Learn more.
http://info.mail.yahoo.com/mail_250

---

From: omer mussaev <[email-masked]>
Date: Wed, 19 Jan 2005 03:54:45 -0800 (PST)

> programming advices from microsoft? :)

Since when Guy Keren is microsoft? A link to MSDN
was only to illustrate how UCS2 can be character 
as well. 

=====
--
o.m.

__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - Find what you need with new enhanced search.
http://info.mail.yahoo.com/mail_250