Subject: RE: [hackers-il] Maintenance of 10,000,000 lines long application s?

From: Chen Shapira <[email-masked]>
Date: Sun, 6 May 2001 13:44:11 +0200

> what do you mean with 'setup read too'? also, since you 
> probably ahve a
> lot of baselines, i'd gather that when a baseline changes, then all
> baselines that use that baseline have to be recompiled as well?  the
> ammount of noise would most likely change based on the deepness or
> shalowness of the dependency tree.
> > We do have very small baselines (Like those belong to 
> DBA's) with less than
> > 5000 lines, larger ones take around 30,000. We try not to 
> have more than
> > 50,000 per baseline.
> 
> ok. then how many lines of code does your baseline (if there are few -
> pick an average) _depend_ on (i.e. code in baselines your 
> baseline depends
> on) ?

I was only talking about individual baselines.
I don't have much information about the dependency trees, mostly because
such information isn't centeralized.

I know that my baseline depends on 1 baseline, and 2 baseline is depended on
me. I could look at the component I depend on, and see what they depend on,
and so on, but I won't.

I do suspect that the tree is extremly deep, and we survive because the most
basic components aren't rebuilt often, if at all.
 
> and when you checking (until a build is make) - other 
> programmers can keep
> on compiling their code, or they have to wait until you compile (cause
> having a header file inconsisent with the matching class's object file
> causes useless binaries) ?

This is when we have croaks. 
We do all kind of things, either they wait for my compile, or if I prefer
not to compile at this time, I can give them the binaries and objects I
produced and have them use those until the next compile.

> exactly what? this was a 'choose the option' question. which 
> option in the
> two options above (full copy vs. mostly symlinks to public 
> baseline) is
> correct?

sorry. missed the symlinks part :-)
We have full copies.

> these teams are made of how many programmers? 50?

no 50 people is a full project. a team is around 3-8 people.
When they broke the compatibility however most teams were affected, so it
was around 50 people losing time.
 
> how often do you create new baselines? and how long does it 
> take to create
> a new baseline?

We create new baselines typically every 3 month (after we ship a product,
start a new one, etc.)

A new baseline that continues an old one (new product from the same
codebase) will take around an hour or two to prepare. (this is the most
common case). It involves mostly system work, and some small changes to data
files.

A new baseline for a new project that uses a familiar language may take a
day or two, and is usually part of the spec writing phase of the new
project. We decide on dependencies, directory structure, installations and
so on. (This isn't exectly rare, but isn't something every programmer did)

The worse case is when we try to adapt the system to a new langauge (Me and
two other programmers did it for Java, it was probably the first time it was
done in few years).
This takes a week or more, to decide on compilation proceedure, get a
compile server, and similar things.

> i also seen a place where it took them a long time to compile 
> the system
> after changes, but no one complained - they used this time to go have
> another looooong cup of coffee. hope that's not the case there ;)

A programmer is as productive as his compile cycle :-)

---

From: mulix <[email-masked]>
Date: Sun, 6 May 2001 14:35:00 +0300 (IDT)

> A programmer is as productive as his compile cycle :-)

i found in the last few years that the more i code (ergo, the more
experienced i become), i spend less time compiling (or debugging eith a
debuger) and more time thinking about the code. after thinking, i
proceed to write entire segments of code (be they function, classes or
even subsystems) and only then compile.

i also find that beginning programmers often compile after absurdly
small changes, even as small as one line of code.

so, a question, o' il'y hackers: how often do you compile? approx, per
how many lines of code?

(for the sake of the discussion, let us ignore compiling for failed
compilations due to typos or incorrect syntax, where you fix the typos
and recompile)
-- 
mulix
http://www.advogato.com/person/mulix

linux/reboot.h: #define LINUX_REBOOT_MAGIC1 0xfee1dead

---

From: "Nadav Har'El" <[email-masked]>
Date: Sun, 6 May 2001 15:25:56 +0300

> even subsystems) and only then compile.

I find that my coding has two major "modes": one mode is indeed the one
you describe: there's a new functionality I want to add, and I start
coding, sometimes 100 lines before I compile. But even in this mode
I don't write 1000 lines before compiling - I like the compiler to find
errors for me before I copy them around too much ;) I also like to test
what I do incrementally, so that I'm not suddenly be faced with a code with
50 new bugs.

But there's a second mode: after you already have a program with the needed
functionality, and you need to fix bugs and/or change small things. In this
case you typically wind up with many cycles of tiny changes followed by
checking whether this new change helped. A good debugger (and knowing how
to use it smartly) can save some of these cycles, but not all of them.

> so, a question, o' il'y hackers: how often do you compile? approx, per

I'm not sure it is a big "gluck" to be able to write long programs
and only then compiling them. I saw one example where a (rather beginner)
programmer wrote some function, compiling it only after the entire code
was done. When she finally compiled it and tried checking it, it was
HORRIBLE. The function did not work correctly in ANY case. It had many
cases and sub-cases, and when it was all written it was almost impossible
to understand what was wrong, or be sure if all the problems was fixed.
So I rewrote that function from scratch. I wrote the function case-by-case,
testing each case (by compiling and actually running the function) after I
wrote it, and thus made sure that all the cases worked correctly. All-in-all
it didn't take me much longer to write the function, but the difference was
that mine worked :)

If you are able to write large sections of codes without compiling/testing
it in the middle, and you DON'T make a horrible mess out of it, then it
might be a sign of hackerdom - but I don't agree that in order to be a good
programmer you must work that way.

Another method that good programmers use to speed up compiling and testing
(rather than compiling only after you wrote 500 lines) is to try to work
(as much as possible) with seperate test programs, rather than trying to
integrate your work with the entire project all the time. For example, if
you are writing some communication library for the entire project, try it
first and debug it using some simple main(), not the real program. This will
let you compile it dozens of times while you're experimenting, without
disturbing everyone else. Only after you finish checking your library on the
test program, you should make the real project use that library (you'll
probably find more bugs then, but not as many as you had initially).
But this refers only to the first mode I described above; In the second
mode of programming (debugging) you usually can't seperate the bug from the
entire project, and need to work on the entire project.

-- 
Nadav Har'El                        |       Sunday, May  6 2001, 13 Iyyar 5761
[email-masked]             |-----------------------------------------
Phone: +972-53-245868, ICQ 13349191 |Early bird gets the worm, but the second
http://nadav.harel.org.il           |mouse gets the cheese.

---

From: Dan Kenigsberg <[email-masked]>
Date: Sun, 6 May 2001 16:04:49 +0300 (IDT)

> small changes, even as small as one line of code.
I disagree. From my experience, a beginner programmer tends to write long code
before even trying to see if it compiles, let alone does the work.

The working programmers (I'm not speaking about few geniuses) will try out
everything they write before it grows to gyganticly unmanageably sized wrong
code.

A very helpful tool for little-by-little productive development is a visual IDE,
but I'm not crazy to say such a thing in this list.

Dan.

---

From: mulix <[email-masked]>
Date: Sun, 6 May 2001 20:17:27 +0300 (IDT)

> to use it smartly) can save some of these cycles, but not all of them.

agreed, to both accounts. on a tangent, the use of the debugger, i would
be happy to have your opinions on the following post by linus:

http://lwn.net/2000/0914/a/lt-debugger.php3

i find that when i try to avoid the debugger, the resulting code is
higher quality, because i have to reall _think_ about the code i'm
writing. however, in some cases (a deep call stack, an obscure bug
which only happens every n'th iteration), a debugger is very, very usefull.

> it didn't take me much longer to write the function, but the difference was

actually, nadav, the difference is that you wrote the code...

> integrate your work with the entire project all the time.

of course, modularity is a corner stone of good design and code code.
the question, i guess, is what is the smallest "atomic" chunk of code
you write without compiling.
--
mulix
http://www.advogato.com/person/mulix

---

From: guy keren <[email-masked]>
Date: Mon, 7 May 2001 10:07:42 +0300 (EET DST)

On Sun, 6 May 2001, Chen Shapira wrote:

> and soon, but I won't.

so you only make a copy of your own baseline, while using the public
copies of the header files + libraries + binaries of all units that your
unit depends on? and you haven't seen problems due to that? that's odd. it
could stem from teh fact that you're doing java coding, and there aren't
so many java units in your system yet, cause most of the code base was
written using other languages?

> > on compiling theircode, or they have to wait until you compile (cause
> We do all kind of things, either they wait for my compile, or ifI prefer

how do you do this process? by manually copying files (file manager), or
using some automated tools from your configuration management environment?
can you guestimate how much time you and other prorgammers there spend on
these coordination tasks?

> sorry. missed the symlinks part :-)

full copies of your baseline only? or yours and the one baseline your
baseline is directly dependant on, but not of those that baseline is
dependant on? or... ?

> compile server, and similar things.

since adapting to new langauges is not soemthing that is done very often,
then as long as you're given the time to do it properly, taking a week to
do that is a short time. i also assume that someone needs to take the time
to maintain these procedures, since as projects evolve, people most
likely find they need improvements and fixes to these procedures.

> A programmer is as productive as his compile cycle :-)

that's a rather false statement, IMO. its good for a poster to put on the
copany's bouletin board, but only in extreme cases it makes sense :0

--
guy

"For world domination - press 1,
 or dial 0, and please hold, for the creator." -- nob o. dy