Subject: Pattern for building several variants of a software module

From: Omer Zak <[email-masked]>
Date: Fri, 29 Apr 2005 01:56:03 +0300

In a project I have to build several versions of a long configuration
file for a certain software package.
The variants of the configuration file will then be tested by a focus
group of potential users of the software in question.  They will then
choose the best version and it will be developed further and shipped
with the final software build.

A configuration file is built using two-step process.  The first step is
to process a set of text files using cpp (the C preprocessor).  The
second step is to convert the resulting text file into a binary file,
which is the one actually used to configure the software.

In the set of text files, one of the files (file A) #include's the other
files (files B and C).

As it happens, files A and B are the same for all versions, and file C
varies from version to version.

The build environment is Linux and all the standard Free Software Linux
based tools are available.

Since cpp does not process macros inside #include statements, and so I
cannot pass to it a filename using the -D flag, I need another way to
modify file A for each version of file C.

There are two approaches using standard tools:

1. Use a keyword (say %WILL_BE_FILE_C_NAME%) and run sed to replace this
keyword by the actual name of file C before running cpp.
Makefile fragment (where fileA contains
#include "%WILL_BE_FILE_C_NAME%"):

%.cfg: %.cfgtxt
	compile_cfgtxt.py < $< > $@

%.cfgtxt: file_C_variant_%.txt fileA.txt fileB.txt
	sed s/%WILL_BE_FILE_C_NAME%/$*/g < fileA.txt | cpp - $@

2. Have file A #include files B and D, where D consists of the single
line:
  #include "file_c_name"
Then have the makefile create a D file for each version.
Makefile fragment (where fileA contains
#include "file_C_redfirect.txt"):

%.cfg: %.cfgtxt
	compile_cfgtxt.py < $< > $@

%.cfg: file_C_variant_%.txt fileA.txt fileB.txt
	echo '#include "$<"' > file_C_redirect.txt
	cpp fileA.txt $@

(DISCLAIMER: I did not actually test the makefile fragments, and they
may contain minor spelling or quoting errors.)

Both approaches work (or are supposed to work), but seem to me to be
inelegant.  I feel that there must be an elegant way to accomplish this
objective.

Since writing my own macro preprocessor (which allows me to specify
names of include files as command line arguments) is a trivial and
elegant solution to the above problem, I am explicitly forbidding it.

The challenge is, then, to find and recommend standard Free tool/s which
can accomplish the above objective in an elegant way.
                                                   --- Omer

-- 
MS-Windows is the Pal-Kal of the PC world.
My own blog is at http://www.livejournal.com/users/tddpirate/

My opinions, as expressed in this E-mail message, are mine alone.
They do not represent the official policy of any organization with which
I may be affiliated in any way.
WARNING TO SPAMMERS:  at http://www.zak.co.il/spamwarning.html

---

From: Oleg Goldshmidt <[email-masked]>
Date: 29 Apr 2005 10:29:38 +0000

> can accomplish the above objective in an elegant way.

Use any tool - awk, perl, s?ed, whatever - to do one of the following:

1) generate file C (or portions thereof) as necessary:

config: A B C
        <CPP rule>

C:      C.template
        $(AWK) -f $(SCRIPT) $< variant=$(VARIANT) > $@

2) generate file A as necessary:

config: A B $(C_VARIANT)
        <CPP rule>

A:      A.template 
        $(SED) 's/PLACE_HOLDER/$(C_VARIANT)/' $< > $@

Both are similar to what you propose[1], but are marginally cleaner in
terms of make (IMHO). UNTESTED, of course, and may contain mistakes.

IMHO, writing a custom tool when existing tools work well and are
guaranteed to be available is *not* an elegant solution, however
trivial it may be.

[1] Logically, you should not look for much else: your problem is that
CPP does not process #include directives, so either fix what is
included or rewrite the directive using another tool.

-- 
Oleg Goldshmidt | [email-masked] | http://www.goldshmidt.org

---

From: Omer Zak <[email-masked]>
Date: Fri, 29 Apr 2005 11:30:44 +0300

> included or rewrite the directive using another tool.

Is there another text files preprocessing tool, which knows to process
file inclusion directives before executing them?
                                                   --- Omer
-- 
Every good master plan involves building a time machine.  Moshe Zadka
My own blog is at http://www.livejournal.com/users/tddpirate/

---

From: Muli Ben-Yehuda <[email-masked]>
Date: Fri, 29 Apr 2005 11:43:17 +0300

> file inclusion directives before executing them?

Not sure what it is you're trying to do, but gcc has the #include_next
extension that might be useful to you, or completely irrelevant.
-- 
Muli Ben-Yehuda
http://www.mulix.org | http://mulix.livejournal.com/

---

From: guy keren <[email-masked]>
Date: Fri, 29 Apr 2005 15:06:09 +0300 (IDT)

> file inclusion directives before executing them?

i don't think it matters - it is better to use commonly-used tools, then
to fetch some rare tool that'll now be required on the compilation
machine. using the tools already found on the system is better for things
that they can easily do.

-- 
guy

"For world domination - press 1,
 or dial 0, and please hold, for the creator." -- nob o. dy

---

From: Oleg Goldshmidt <[email-masked]>
Date: 29 Apr 2005 11:12:56 +0000

> modify file A for each version of file C.

But A can contain

#ifdef FIRST_VARIANT
#include "C1"
#endif

#ifdef SECOND_VARIANT
#include "C2"
#endif

etc, can't it?

---

From: Omer Zak <[email-masked]>
Date: Sat, 30 Apr 2005 09:20:09 +0300

> > modify file A for each version of file C.

This can work, but requires me to modify both the makefile and A if I
want to add another variant of file C (when I want to build a file from
A and its #include'd files, and C - one of its #include'd files - has
several variants, and I want to build a separate file from A with each
variant of C).

An elegant way would allow me to modify only the makefile to add another
variant of C (or just drop in the additional C variant, if I use GNU
make's $(wildcard PATTERN) feature).
                                            --- Omer
-- 
MS-Windows is the Pal-Kal of the PC world.
My own blog is at http://www.livejournal.com/users/tddpirate/

---

From: Oleg Goldshmidt <[email-masked]>
Date: 30 Apr 2005 17:03:24 +0000

> several variants, and I want to build a separate file from A with each

Strictly speaking, no - you can do

make VARIANT=FIRST_VARIANT

at invocation time.

Frankly, I don't understand what you are trying to achieve here. All
the variants starting from your own seem workable to me. Take a look
at /usr/include/features.h for inspiration?