Subject: The Data-Driven Programming Paradigm (Part 1)

From: Shlomi Fish <[email-masked]>
Date: Fri, 1 Jun 2001 05:49:59 +0300 (IDT)

Following the discussion we had about programming paradigms, I decided to
take this space to present a paradigm I thought about myself. I
encountered uses of its techniques before, but I did not see it presented
as one integrated form yet.

This is not going to be the final forum in which I'd like to place this
text or a similar one. But I'd like to post it here for receiving initial
comments and feedback.

Data-Driven Programming (or DDP for short) deals with how programs that
manipulate a lot of data (such as database tables) can themselves be
written using Meta-Data as the guide to what it has to do. For instance,
consider a query or a set of queries that has to be performed on a single
(flat) database table. One could perform these queries by using generic
interfaces for manipulating _any_ flat table, while supplying them with a
special SPEC.

A SPEC is in essence a multi-level data-structure that describes the
table, its fields, their types, their constraints, and other information.
A SPEC would look something like that (I'm using Perl syntax):

my $spec = 
{
	'fields' =>
	[
		{
			'name' => 'id',
			'type' => 'int32',
			'flags' =>
				{
					'hidden' => 1
				}
		},
		{
			'name' => 'private_name',
			'type' => 'varchar',
			'flags' =>
				{
					'length' => 100,
				}
		},
		{
			'name' => 'family_name'
			'type' => 'varchar',
			'flags' =>
				{
					'length' => 100,
				}
		},
		{
			'name' => 'birth_date'
			'type' => 'date',
		},
		#
		# Other Fields snipped.
	],
	'primary_key' => [ 'id' ],
};

Now we can use a SPEC like that or a similar one as input to the table's:

1. Iterator.
2. Reporter (that in turn uses the iterator)
3. Editor
4. Insert/Add Entry Form.

As well as possible other manipulators which I forgot about. 

Naturally, I did not include all the possible data structure fields that
can relate to the table fields. There could be such things:

1. Input constraints .E.g: 

'range' => { 'min' => 1900, 'max' => 2050 }

2. Presentation fields. E.g:

'presentation' =>
{
	'title' => "Private Name",
	'tool_tip' => "Enter your private (=first) name here:",
}

3. Valid options, or a choice of them:

'options' =>
{
	'type' => 'mandatory',
	'values' => qw(Sun Mon Tue Wed Thu Fri Say),
}

In a similar way to that, one could write SPECS for a tree query (where
one field (or more) is designated as the node ID and one as the parent
ID), and a multi-table query.

By using such SPECs one could more easily modify the query as such
operations as removing fields, adding them, changing their types etc. need
to be done only in one place of the program - the SPEC. This is as opposed
to revamping a great deal of SQL queries, adding new variables in lots of
places, etc. etc.

I used this concept in several programs which I have written so far, but
have seen (and previously - also written) code that actively does not make
use of it.

There are more techniques which I belive fall under the DDP umbrella, but
I'd like to get some initial feedback regarding the SPEC concept, so I'm
leaving this article as is, and will get back to you with more.

Your comments and opinions are welcome.

Regards,

	Shlomi Fish

----------------------------------------------------------------------
Shlomi Fish        [email-masked] 
Home Page:         http://t2.technion.ac.il/~shlomif/
Home E-mail:       [email-masked]

A more experienced programmer does not make less bugs. He just realizes
what went wrong more quickly.

---

From: Chen Shapira <[email-masked]>
Date: Fri, 1 Jun 2001 07:54:13 +0200

> interfaces for manipulating _any_ flat table, while supplying 

Shlomi,

I remember seeing your SPEC on the Gamla project, I liked it so much, I
immidiatly used the idea in 3 running projects.

SPEC can be nicely extended to support star-schema, which you'll need in a
normalized database.

On the other hand, if your queries get so complex that SQL is becoming a
burden, SPEC won't help you. You still need to think SQLish inorder to work
with it. And some things just don't fit, so always have a generic query
interface in your layer.

BTW, have you found a way to nest queries in a similar SPEC?

It also works nicely as a layer above ADO.

Thanks,
Chen.