Subject: Perl 5 TNG

From: Shlomi Fish <[email-masked]>
Date: Mon, 11 Feb 2002 13:48:33 +0200 (IST)

Yet another mutiny, but rather language fork that I start. Forgot to
mention proper Garbage Collection and proper tail recursion (in the nice
to haves), but I'll fix it.

Regards,

	Shlomi Fish

I asked my good friend and fellow Linuxer Mulix if he reads
the Apocalypses and Exegesis of Perl 6. He said he does but
he do not understand them. Strangely enough, I realized that
I don't understand everything there anything. Apocalypse 4
was the worst - so many constructs which I could not understand
or care for too much.

I believe most good languages in use today were based on a previous
language: BCPL begot B which begot C which begot ANSI C which begot C++
which begot Standard C++. LISP and ALGOL and Lambda Calculus begot Scheme.
Pascal was based on ALGOL, too. Python was intended as a better Perl, and
Ruby was derived from Python, Perl and Ruby. Likewise, Perl 1 was based
on C, Sed, Awk and Shell. The other perls were based on the others until
Perl 5 came out.

Now there's Perl 6 which aims to be derived on Perl 5, but also on over
300 RFCes which suggest every feature imaginable or existent today. From
what I read until now, Perl 6 would be to Common Lisp what Common Lisp is
to Scheme. I.e: a complex, complexifiying and complexifiyied language. There
is a saying in Hebrew: "Tafastha Merube Lo Tafastha", which means : "If you
try to catch a lot you catch nothing at all".

There are many possible features which can be thrown into Perl, but Perl
has its particular niche, and cannot be expected to be the best language
in all cases. To quote Chen Shapira: "How can one expect to be a
programming language that will be suitable in all cases, if people
can't even make such a screwdriver". All the other languages I know fail
in some cases, and cannot be expected to be useful for 100% of the problems.
I don't think the next version of Perl should be the same.

Luckily, being an Objectivist and a Jew, I tend to be an individualist and
doubt the external authority of anyone, including language designers. I decided
to start a mutiny, and start to design a language dubbed "Perl 5 TNG"
(short for "The Next Generation") which will:

1. Be based on Perl 5 with a few cleanups.
2. Be a better Perl.
3. Not be much bloated than Perl 5.x.

I do not claim to be familiar or expert with half the computer languages (much
less half the human languages) that Larry Wall is, but I do know quite a few.
(look at my resume). Therefore, I think I know where perl 5 is flawed, and
what needs to be done to fix it.

Here goes:

1. The -> operator and the . operator will remain the same.

"." may be the industry standard in Java, Python, etc., but I believe -> as
an accessor is more cognitively sound. I.e: it make you realize that the left
hand side is a reference (or pointer in some languages terminology). It very
shape makes it so.

"." may be a strange choice for string concatenation, but "_" is worse, since
it can be part of an identifier. My personal favourite for a string
concatenation operator is "$+", which I think is a special variable in perl 5.
I suggest we stick to ".", because it's as good a choice as any.

2. Several Objects in the same file:

Associating Objects with modules was a bad idea. I plan Perl 5 to have
a fully non-magical object system, which one can implement in user-land. (CLOS
anyone?). The only "magical" element would be a special function named
"on_destroy" (name can be negotiable) that will accept a reference to a
function and return a special reference that will call it when it gets
out of scope. E.g:

sub hello
{
    my $num = 100;
    my $p = sub {
        for(my $i=0;$i<$num;$i++)
        {
            print "Hello\n";
        }
    };

    my $destructor = on_destroy($p);
    $num = 5;
}

Will print "Hello" 5 times. on_destory() will be used for implementing class
destructors.

Perl 5 TNG will have a default object system, so the situation of Scheme where
there are several proprietary and non-compatible ones will not repeat
themselves. Nevertheless, the user will be able to hack something on his own,
in case he does not like it too much.

3. In file namespaces.

I suggest adding a "namespace mynamespace { ... }" or similar construct to perl
or to be able to define "sub Hello::World{ ... }" whereever. It will
make symbol lookup a bit harder, but will be able to create scripts that
are self contained.

4. Globs are dead. Don't miss them. Don't miss them.

The new syntax for open would be "my $I = open("<file.txt");". and of course,
"print {$I} "Hello there!\n"." It would be nice if people won't type
"print $I;" and get an ugly reference dump on the screen. Perhaps, file handles
can be magical objects that will throw an exception if printed.

5. local is gone.

Long live my() and our()! Enough said?

6. Distinction between => and ,

At the moment creating hashes is bug prone, because => and , are exactly
the same thing. I suggest to distinct them and use => for a key+value atom.

7. Better and easier Bindings language and integration glue.

Parrot is probably going to be a solution, assuming it is designed right. I
don't mind it being the back-end of Perl 5 TNG, since it aims to be such a
back-end.

---------

Then there are nice to have things, which I don't need but may like:

1. Binding a variable to a type.

Perl is a dynamically typed language, but obviously some people will find
it nice if it will behave somewhat statically-typed sometimes. This will
be a problem with having more than one such type in a combined expression:
"$myint + $mystring" or "$myint . $mystring".

2. Defining new infix/prefix/suffix operators a la Haskell.

Will make parsing much harder, but would be a very cool feature. Can be
used to rationalize bulk operations on array. I.e: "@c = @a @+ @b;" would
be just syntactic sugar to:

<<<
@c = (map { $a[$_] + $b[$_] } (0 .. min($#a,$#b)))

It may be optimized for further speed.

3. Bulk operations on arrays:

Due to the fact that I'm an Electrical Engineering student, I know that
Matlab contains an inflation of operators to work with matrixes and tensors.
I'm not seeing Perl replacing Matlab as far as Engineers are concerned (at
least not until PDL becomes mature enough). Then there's APL, which I'm not
familiar with but whose programs look like operator line noise. (Matlab is
actually quite readable, usually)

Do we need Matrix multiplication? (not just @@*) Or a sin/tan/log function
that will take all the values of a tensor that is passed to it at once?
The user may actually pass it as an accident. One can distinguish it with @^sin
or @^^log of course.

If you want PDL like functionality to be an integral part of Perl 5 TNG and
its syntax, go for it. I on my part am neutral, because I like to use Matlab
were appropriate and Perl were it is.

---------------

Do you know of any other urgent problems in Perl 5 that need to be addressed
in Perl 5 TNG? If so let me know.

But I don't want it to be a catch all (= catch none - ;-)) language that
Perl 6 seems to be. If I don't understand those features now that I read
them, how can I understand them when I have to comprehend other people's
code?

The philosophy of Perl 5 TNG is to take Perl 5 and clean it up so it will
be more consistent, trustworthy and usable. I don't want it to be a language
that will integrate all possible features from every other language
in the world.

Like I said, languages should evolve, rather than revolve. Obviously, in
one day, someone may find some other features that need to be addressed
in Perl 5 TNG. At which point I or whoever make choose to make a language
that is derived from it. But Perl 6 will have too many features that I find
redundant and don't need.

Yes, Haskell's lazy lists are cool ([(i*i) | i <- [0 .. ]] anyone?), but
I could live without them anyday. I'd rather have Perl without lazy evaluation
than Perl with lazy lists. And naturally, most of the other suggestions that
Larry put in Perl fall into that decision too.

Regards and think about it,

    Shlomi Fish

"All language designers are arrogant. Comes with the territory".

    Larry Wall

----------------------------------------------------------------------
Shlomi Fish        [email-masked]
Home Page:         http://t2.technion.ac.il/~shlomif/
Home E-mail:       [email-masked]

"Let's suppose you have a table with 2^n cups..."
"Wait a second - is n a natural number?"

---

From: Shlomi Fish <[email-masked]>
Date: Mon, 11 Feb 2002 13:49:48 +0200 (IST)

Oh and most of those pesky global variables will be replaced by functions.

Regards,

	Shlomi Fish

---

From: Adi Stav <[email-masked]>
Date: Mon, 11 Feb 2002 21:54:14 +0200

On Mon, Feb 11, 2002 at 01:48:33PM +0200, Shlomi Fish wrote:
> 
> Python was intended as a better Perl, 

Nitpick: Python was intended as a better ABC, with Modula-3 
influences. It was written in 1989 when Perl 1.0 was still mostly 
an unknown language in the world.

---

From: Shlomi Fish <[email-masked]>
Date: Mon, 11 Feb 2002 21:20:08 +0200 (IST)

> an unknown language in the world.

Actually, when Python came out in 1990, Perl 3 was out. Check:

http://history.perl.org/PerlTimeline.html

I don't know how well known it was at that time. In any case, the
statement is a bit incorrect because Perl and Python have influenced each
other in some kind of spiralling way. At least from my impression. But as
long as Guido von Rossum, Moshe Zadka and you, think Python is a superior
language to Perl, and me and several others disagree we are set.

I think that in the Perl culture, people don't care too much if one uses
Perl, Python, Tcl, etc. as long as he knows the joy of using a
full-featured, fully integrated scripting language. Python people OTOH are
too occupied with converting people _from Perl_ to Python. But that's just
my impression.

Being the maintainer of a rather complicated Python script, it seems to me
that I'll always prefer Perl over Python. And if my Perl 5 TNG has its
way, then I'll be fully content with Perl. The thing is, there are always
times when I think a language should have been designed differently. No
matter which language I use, and I actually like to program very much. I
don't think the perfect programming language existed or can exist.

Regards,

	Shlomi Fish

>
> To unsubscribe from this group, send an email to:
> [email-masked]
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>

---

From: Adi Stav <[email-masked]>
Date: Tue, 12 Feb 2002 00:59:47 +0200

> http://history.perl.org/PerlTimeline.html

Python was originally written in Christmas 1989, which is the time
most important for deciding what it was originally intended for. 
According to Guido van Rossum, it was originally intended as the
native scripting language for Amoeba OS, building on his experience
with ABC and Modula-3. Looks like Perl 3.0 indeed had just been 
released, though. See Python FAQ.

> language to Perl, and me and several others disagree we are set.

I never said it was superior. I've said in the past I thought it 
was better as a first language, though. I haven't worked enough
with either language to be able to make such an encompassing 
statement as "a superior language".

> too occupied with converting people _from Perl_ to Python. But that's just

Looking at the Python mailing list threads from some to time, it's 
sometimes surprising how little they speak of Perl, least of all 
critically. When they do look at other languages it's more often 
Java and Scheme. I think Perl's influence on Python, although 
significant, is overrated.

I don't know how significant was Python's influence on Perl. I've 
heared it said that Perl borrowed Python's object model but I
don't know how accurate that is.

---

From: Omer Zak <[email-masked]>
Date: Tue, 12 Feb 2002 01:12:18 +0200 (IST)

> to haves), but I'll fix it.

<TONGUE_IN_CHEEK>
I can just see you (Shlomi) starting a new flavor of Judaism 10 years from
now.  Based upon your usage of Jewish concepts and quotes, you must now be
"chozer betshuva".  And given your suggestions for all kinds of mutinies,
after few years of immersion in Jewish life, you'll find a new way to
reform Judaism.
</TONGUE_IN_CHEEK>

> I don't think the next version of Perl should be the same.

Then, the first priority will be a mechanism for interlingual binding.
Something like what they claim that MS .NET claims to support.
Such a mechanism had better be able to figure out how to call a C++
procedure just by reading the corresponding header file.

> I suggest we stick to ".", because it's as good a choice as any.

Then, go for $+ (for generalized scalar additions)!
Array additions can be @+.

You are building your own variant of Perl, no need to be compatible.  At
worst, develop a script which will translate Perl 5 TNG into standard Perl
5.

> Will print "Hello" 5 times. on_destory() will be used for implementing class

I don't think that it is a bad idea to associate objects with files.  Java
does the same, too.  But the idea of on_destroy() is cool.  Of course, it
works cleanly only with automatic variables.  With dynamically allocated
variables (especially in garbage collected environments), destructors are
awkward - you can't be sure when does the object really get destroyed
(unless you invoke do_garbage_collection_now()).

> "$myint + $mystring" or "$myint . $mystring".

I love this idea.  How about a gypsum() operator, which freezes a variable
to the type which it has when operated upon by gypsum()?  It would be good
idea also in other dynamically typed languages.  This is the perfect
trade-off between not wasting time on unessentials like declaring variable
types when prototyping things, and making software crash-proof when it
goes out of prototyping and gets feature-stable.

                                             --- Omer
There is no IGLU Cabal.  Ask my neighbor over there about IGLU Cabal TNG.
WARNING TO SPAMMERS:  see at http://www.zak.co.il/spamwarning.html

---

From: Shlomi Fish <[email-masked]>
Date: Tue, 12 Feb 2002 08:55:33 +0200 (IST)

> after few years of immersion in Jewish life, you'll find a new way to

I believe the Jews are a people above everything else, and Judaism is
their "Peopleship". The Jewish Religion is a very small part of it. I am
influenced by many historical Jewish stuff, but also by a lot of
"Gentile" culture. I am an atheist however, and do not believe in God, or
think that any true Jew should.

> procedure just by reading the corresponding header file.

Parrot (http://www.parrotcode.org/) will have a common way to define
bindings to it. That does not mean those bindings will be suitable or
well-formed for any language front-end implemented above it, but they will
be usable. Assuming Perl 5 TNG will use Parrot (which I don't see a reason
why it should not), It will enjoy this binding stuff.

> worst, develop a script which will translate Perl 5 TNG into standard Perl

That's not what I aim for. No, I don't want P5TNG to be fully compatible
with Perl 5, but I do want it to look and behave much the same. Naturally,
assuming it will have bulk operations on arrays, than if $+ is for string
concatenation, and @+ is for array addition, the array string concation
should be something like @$+. According to the apocalypses array stuffs
are no ^+, ^^+, etc, but I can specify that they are @+ if I want.

Another issue is accumolators. If anybody is familiar with matlab, he
knows that sum, max, min and other accumolators operate on one dimension
of a tensor. Should we have sum($my_tensor, 2) to sum its second
dimension? As you know a multi-dim array in perl may be something like:

[ [1,2,3,4,5,6], [3,4,5], [2,9], [1,2,3,4,5] ]

I.e: not of the same length. I'm not sure I'd like to put PDL right in the
core language.

> (unless you invoke do_garbage_collection_now()).

The problem is that Python does not associate objects with files, and that
way an OOP program can be self-contained, which is not the case for files.
Likewise for Scheme and Stklos. I believe the Python/Scheme approach fits
Perl more than Java's.

> goes out of prototyping and gets feature-stable.

Sounds nice. Actually, I decided that statically-typed variables are not a
top priority, but I'll keep this gypsum idea in mind. But why not call it
freeze_type() ?

> There is no IGLU Cabal.  Ask my neighbor over there about IGLU Cabal TNG.

Check the TINIC of:

http://groups.yahoo.com/group/hackers-il/message/1513

Regards,

	Shlomi Fish

---

From: Adi Stav <[email-masked]>
Date: Tue, 12 Feb 2002 12:17:22 +0200

> Likewise for Scheme and Stklos. I believe the Python/Scheme approach fits

I'm not sure what you mean by "associate objects with files". The 
Python language has no special construct for files, they are treated
as objects, but they are built-in types, so in pre-2.2 you cannot
inherit from them (like string, list, dictionary, and so forth).

---

From: Omer Zak <[email-masked]>
Date: Tue, 12 Feb 2002 09:24:00 +0200 (IST)

> why it should not), It will enjoy this binding stuff.

But will this mechanism be friendly to users?
Suppose I find that COOL library, and it has only C language bindings (in
the form of header files).  How can I bind to it from my new Perl 5 TNG
scripts without doing things more complicated than writing declarations
like:

foreignlibrary "/usr/include/coollibrary/CoolLibrary.h";

> are no ^+, ^^+, etc, but I can specify that they are @+ if I want.

The meaning of regular '+' is that if it is given string arguments, it
tries to convert them into integers and then sum them.  $+ would convert
any integers into strings before addition.  For example:

        1    +   1    == 2
        1    $+  1    == "11"
        "1"  +   "2"  == 3
        "1"  $+  "2"  == "12"

@+ would, similarly, convert its arguments into arrays (if necessary)
before concatenating them.  Regular + or $+ would perform
element-by-element additions.

> dimension? As you know a multi-dim array in perl may be something like:

Yes, use sum(@my_tensor,2) and the like.  For any missing elements (due to
short rows), substitute the null value and use whatever value defined for
addition of null values.

Also, generalize the notation (I don't have a suggestion at the moment) to
other operators, which can be applied to tensor elements.

> top priority, but I'll keep this gypsum idea in mind. But why not call it

Due to the same reason people grep strings, rather than search_matches
strings.  Use "unexpected" words, to spice up the language.
                                             --- Omer
There is no IGLU Cabal.  This has been a goal not worth pursuing.
WARNING TO SPAMMERS:  see at http://www.zak.co.il/spamwarning.html

---

From: Shlomi Fish <[email-masked]>
Date: Tue, 12 Feb 2002 09:48:10 +0200 (IST)

> inherit from them (like string, list, dictionary, and so forth).

I meant that there can be several class definitions in the same module. I
did not mean file handles.

Regards,

	Shlomi Fish

---

From: Shlomi Fish <[email-masked]>
Date: Tue, 12 Feb 2002 09:59:45 +0200 (IST)

> foreignlibrary "/usr/include/coollibrary/CoolLibrary.h";

Actually, there is a CPAN module that allows the users to call DLL
functions directly. I don't know how versatile it is. The problem is that
C is very different form Perl - it has pointers, null-terminated strings,
etc. Creating a completely user-land C -> Perl glue is very hard.

> before concatenating them.

I don't understand you - what's wrong with "," for array concatenation?
AFAICT @+ will try to sum two arrays element by element.

I.e:

@a + @b ===> an error
@a @+ @b or @a ^+ @b (whatever) ==>
	map { $a[$_] + $b[$_] } (0 ..  min($#a,$#b))

> Regular + or $+ would perform

This will turn Perl into a full-fledged Matlab clone with dollars. ;-) I'd
rather the operators make some basic type checking and that a regular +
will not add an entire array.

> other operators, which can be applied to tensor elements.

My problem with this approach is that Perl's data structures are nested
data-structures and not really tensors like Matlab's. I'd rather make sure
an API that supports tensors has a good foundation (i.e: enough operators
or it can define new ones), than start doing an element by element
addition of a perl data structure.

> strings.  Use "unexpected" words, to spice up the language.

A ha. Well, a rose by any other names will smell just as sweet. However, I
usually prefer names that are meaningful, traditional, and assuming it is
a commonly used idiom - not too long.

Regards,

	Shlomi Fish

---

From: Oleg Goldshmidt <[email-masked]>
Date: 12 Feb 2002 10:14:02 +0200

> bindings to it. 

Guile?

Oh, I forgot, Shlomi wants Perl-based system, not Scheme-based one ;-)

-- 
Oleg Goldshmidt | [email-masked] 
"If it ain't broken, it has not got enough features yet."

---

From: Omer Zak <[email-masked]>
Date: Tue, 12 Feb 2002 10:29:54 +0200 (IST)

> etc. Creating a completely user-land C -> Perl glue is very hard.

Perl does a lot with certain standard data structures.
The binding mechanism needs to provide a standard mapping between Perl and
C data types, with provisions for overriding the mapping for those
applications, which need specialized handling.

For example:  can map between C structure and Perl hash:
struct cstruc {
  int a;
  double b;
  char* c;
} struvar;

%struvar = ( a => 1, b => 3.16143, c => "flame my choice of pi" );

> 	map { $a[$_] + $b[$_] } (0 ..  min($#a,$#b))

There are arguments in favor of defining @+ (and regular + on arrays)
either way.  But since we are starting from Perl, let's just do what they
do (i.e. use , for array concatenation).

> will not add an entire array.

I don't believe this is the right trade-off for a scripting language.
This kind of notation is easy to work with.  And if someone needs to be
sure the software does what he want, he should insert appropriate assert
statements to confirm this.  Rather than rely upon type checking
properties of operators (which he may not remember anyway).

> addition of a perl data structure.

For Perl and its ilk, it would be more interesting (and profitable) to
define tree traversal operators rather than matrix operators, anyway.

Ideas, anyone?

> a commonly used idiom - not too long.

No problems.  If I don't like your choice of names, before executing my
scripts I'll just pass them through a preprocessor, which translates
occurrences of my_favorite_function_name into your_stinking_function_name. 

                                             --- Omer
There is no IGLU Cabal.  An errant preprocessor substituted 'Ka' for 'Ca'
in all text which passed through it - with disastrous results.
WARNING TO SPAMMERS:  see at http://www.zak.co.il/spamwarning.html

---

From: Shlomi Fish <[email-masked]>
Date: Tue, 12 Feb 2002 10:52:28 +0200 (IST)

> Oh, I forgot, Shlomi wants Perl-based system, not Scheme-based one ;-)

I beg your pardon? I realize they intended Guile to be a common back-end
for scripting languages. However, their plan was that every language will
translate itself into Scheme. Parrot has a common, relatively language
neutral but nonetheless high-level assembly-like bytecode. Then, one
compiles Perl, Python, Scheme, LISP, whatever, into this bytecode.

I'm not sure how well or smoothly P5TNG will compile into Scheme. Not to
mention that Guile does not give me all the system services I'd like P5TNG
to have, while Parrot will support everything Perl5 has.

Regards,

	Shlomi Fish