There's a new essay on my site
that discusses which programming language a non-programmer should start from.
It has been brewing for quite a while, and now I'm pretty happy with it.
Thanks to the many people who read it and gave useful commentary. The licence
is Creative Commons Attribution Licnce (CC-by) which means you can
redistribute it, modify it, make derivative works and relicense it, as long
as you give proper credit.
This was the first entry on the shlomif-tech community, and I hope you found
it useful.
This is an IRC conversation that took place on Freenode. I am "rindolf".
<bogdan> rindolf: so you like being selfish, eh?
<bogdan> and you wanted to find a reason to be selfish.
<rindolf> bogdan: http://xrl.us/h2kn
<rindolf> bogdan: Rand made a mistake by naming her book "The virtue
of Selfishness".
<rindolf> bogdan: she didn't mean it in the sense of "Complete
disrespect for others".
<spectei> GPL > BSD
<rindolf> spectei: GPL is more selfish.
<Dianora_> BSD > GPL
<rindolf> Dianora_: :-)
<Dianora_> BSD is free, GPL is not free.
<rindolf> Dianora_: not you too.
<rindolf> spectei: the BSD is free, even Stallman thinks so.
<Dianora_> The GPL is like finding a coupon for $20 for pants on the
street, the BSDL is like finding a $20 bill on the street.
<Dianora_> I would rather have the $20 bill
<rindolf> Dianora_: please lets agree to accept Stallman's definition
of Freedom, and call BSD-style licensed software by another
name.
<rindolf> Dianora_: I don't see the anlogy.
<Dianora_> even esr prefers the BSDL
<rindolf> Dianora_: yes, I know.
<rindolf> That's what I use for my software.
<rindolf> If I can help it.
<Dianora_> No Stallmans version of Freedom is a political
mis-labelling of true freedom.
<rindolf> Dianora_: I'm not saying he defined freedom.
<rindolf> Dianora_: but he coined the term "free softwae
<Dianora_> He can try to relabel English anyway he wants, but he is
not taking my language from me.
<Dianora_> BAH!
<rindolf> Dianora_: but different things mean different things in
different contexts.
<rindolf> Dianora_: you can call BSD-style software PD-software or
whatever.
<rindolf> Dianora_: they are nutjobs.
<rindolf> But they are good people.
<rindolf> They're on our side.
<Dianora_> BSD is FREE, GPL is a restrictive license, that is NOT free
* rindolf is a nutjob but tries to be a good person.
<rindolf> Dianora_: fine. The GPL is not free.
<spectei> it is free software!
<rindolf> "I want to be there, I want to be there".
<pnbeast> GPL is, in fact, a very non-restrictive license.
<rindolf> "fine you are there."
<Dianora_> No, it is NOT free software, the GPL is a restrictive
license. period.
<rindolf> "danny! What are you doing here?"
<spectei> Dianora_, it is free software as defined by the FSF
<rindolf> "Here? <crying"
<spectei> :D
<Dianora_> it restricts your right to use that software!
<rindolf> "did I say something wrong?"
<spectei> Dianora_, in a good way!
<rindolf> It's from Sesame street.
<Dianora_> the FSF can label black as white and I will still not
believe them!
<rindolf> Dianora_: the GPL is better than the MS EULA.
<Dianora_> rindolf yes ;-)
<rindolf> Dianora_: I've contributed to GPL software, and I cannot
improve MSIE.
<rindolf> Or windows.
<Dianora_> don't get me wrong, I use GPL software, I write in a couple
of GPL projects, but I disagree that GPL is free.
<rindolf> Or Excel.
<rindolf> Dianora_: like I said if we want to communicate we have to
agree on a terminology.
<Dianora_> then the GPL is just that, the GPL a restrictive license.
<pnbeast> I won't argue "free", but I can't imagine how you think the
GPL is restrictive. Can you explain that to me?
<rindolf> Most people accept the fact that free software is software
by the FSF FS definition.
<Dianora_> no
<Dianora_> I am afraid you would be wrong
<Dianora_> available source code yes, but free? uh no. never.
<rindolf> Let's avoid the term free software.
<rindolf> Let's call it open-source.
<Dianora_> as I said, GPL is like a $20 coupon
<rindolf> Dianora_: do you think the GPL is not open-source?
<Dianora_> open-source is a good compromise.
<rindolf> Dianora_: ok.
<rindolf> Free software can also mean gratis software.
<rindolf> And the GPL is sourceware.
<Dianora_> you could have sourcecode that was not licensed to be
changed or used by anyone else, but is available to be
looked at
<rindolf> Dianora_: I hope you're not using qmail.
<Dianora_> in fact, everyones favourite whipping boy licenses his code
that way.
<Dianora_> I'd use qmail if it was the best tool for the job, no
matter what I thought of djb
<spectei> opensource doesn't mean anything!
<Dianora_> Disliking a person is no reason not to use someones code.
<spectei> haha
<rindolf> Dianora_: http://www.shlomifish.org/open-source/anti/qmail/
<spectei> dan bernstein is a wacko!
<Dianora_> djb does not allow his code to be modified ;-)
<rindolf> Dianora_: I find Theo objectionable but at least OpenBSD is
open-source.
<Dianora_> naw, I will make up my own mind thankyouverymuch.
<rindolf> So I can take it , fork it, etc.
<rindolf> Dianora_: I cannot with DJB's software.
<Dianora_> OpenBSD is open-source *FREE* code
<Dianora_> *correct* both DJB and GPL are open-source but both are
non-free!
<rindolf> Dianora_: and I'm not saying it just because I dislike DJB.
Although due to the qmail licensing it is a problem.
<spectei> GPL is free :D
<Dianora_> gosh, you made my point for me. restricted licenses.
<rindolf> Dianora_: qmail is sourceware not opensource.
<Dianora_> naw rindolf
<rindolf> Dianora_: have you read the OSD.
<rindolf> ?
<rindolf> Or reveled source, or available source, or whatever.
<Dianora_> we both agreed that when you could look at the source, it
was open-source no?
<rindolf> Dianora_: no.
<rindolf> Dianora_: open source is software that complies with OSD or
the FSD.
<rindolf> Preferably both.
<Dianora_> but both GPL and DJB's license are restricted use. ;-)
<pnbeast> Come on, Dianora_, won't you define "restricted" for me?
<Dianora_> the OSD or FSD can call black white and I will still not
use their definition. ;-)
<rindolf> Dianora_: yes, but the GPL is more usable than the DJB
terms.
<Dianora_> pnbeast: you are defining it yourself.
<rindolf> Dianora_: I can fork GPLed code.
<rindolf> And I can link against an LGPLed library.
<rindolf> But I cannot fork DJB's code.
<Dianora_> rindolf: doesn't matter! it's still restricted!
<rindolf> Dianora_: it is restricted, yes.
<Dianora_> in fact
<spectei> Dianora_, bad things are restricted!
<rindolf> Dianora_: but it's not black and white.
<pnbeast> I guess that's a "no".
<Dianora_> the BSDL _is_ also restricted
<spectei> closed source proprietary software is bad
<Dianora_> pnbeast: You defined it yourself
<spectei> ergo it is good that it is restricted
<spectei> one can be free and not free to poison a well
<Dianora_> GPL poisons wells
<spectei> opposite!
<spectei> it keeps software free!
<Dianora_> GPL is a virus.
<rindolf> Dianora_: Windows < DJB < GPL < LGPL < BSD
I've
already
discussed the Ctrl-R commands in command line mode in this column. Well,
I recently discovered that one can use Ctrl+R % there to enter the
filename of the currently opened file. That way if you want to save it
under a different filename (for example), you can type: :w Ctrl+R %
and then edit it.
I started and finished reading the
"Big Ball of Mud" article, which was recommended to me by a fellow Israeli
Perl monger. It was interesting, but a bit disappointing, and I kind-of felt
that it didn't teach me anything I hadn't already known.
I started going over the
the
Catalyst tutorial in the Catalyst-Manual section, in trying to learn
Catalyst, which is a web development framework for Perl.
Today, I've also continued in reading
Aristotle'
"Categories", and reached the beginning of the third and final section. A
while ago I gave up on it, after I saw that its beginnign contained a lot of
"is predicated by" statements in the same sentences. However, later it becomes
more understandable and quite enlightening. Afterwards, I'd like to read
the rest of
Aristotle's
Organon.
Offline, I've been reading Ayn Rand's "The Virtue of Selfishness" (which I've
been neglecting lately), "The Pragmatic Programmer", which I found to be
very interesting, and I also returned to reading some parts of
Feeling
Good.
For a long time, I thought it was a KMail (= the KDE mail client) limitation
that it couldn't cut and paste messages. Well, as of KMail 1.9.7 (and perhaps
earlier) this was solved.
In the Edit menu, there is a Cut Messages (Alt+Ctrl+X),
Copy Messages (Alt+Ctrl+C) and Paste Messages (Alt+Ctrl+V) options that can
be used to do exactly that. That way one can easily move a few messages at
once to a different KMail window or a different mail folder, without drag and
drop.
One can use
Ctrl+X
in Vim to invoke several sub-commands for auto-completion. For example,
"Ctrl+X ; Ctrl+L" completes entire line. So if you have a lot of
my $self = shift; or my ($self, $args) = @_; in your
Perl code you can start writing that line and press Ctrl+X;Ctrl+L to
fill the rest of them.
Ctrl+X;Ctrl+F completes filenames, and there are other completions like that.
I found it while reading
usr_24
- inserting quickly which contains some other tips. I also realised
that Ctrl+U cuts to the beginning of the line in vim editing,
or in Emacs or GNU Readline.
Here's a vim tip on how to copy some non adjacent lines to a register
("r), but first the story that made me find out about how to
do it exactly. I need to add full POD (Perl's documentation format)
coverage to a Perl module I inherited. What it means is that every
top-level function (or at least those that do not start with a "_") should
have their own entries. In Perl the function definitions goes like that:
sub first_function {
}
sub second_function {
}
sub yet_another_function {
}
Etc. So I thought that I could look for all the lines that start with "sub "
, collect them and append them to a register, where I'll paste them. I recalled
the
:g/re/$cmd
that executes a command on all lines matching a certain regex, and the fact
that I could append to a register instead of setting its value using a yank
command.
So after a little research I tried: :g/^sub /"Fyy. :g
searches for the /^sub / and invokes the command on every occurence, and
"Fyy, appends the current line to the register "f". However, as it turns out
:g executes an Ex command. However, fear not - after a little
research, I recalled the
:normal
command that allows to execute normal mode commands from within the
command-line (or Ex) mode.
So the final line that works is :g/^sub /normal "Fyy. You may need
to do :let @f = '' before that to clear the "f" register.
Cheers.
A few days ago I ran into some problems when I tried to install and use Common
Lisp's iterate construct
for my Mandriva SBCL (= Steel Bank
Common Lisp) installation. Luckily, after I consulted the wisdom of
Freenode's #lisp channel, and an IRC buddy of mine (thanks!), I was able to
get it working. With some variations it might be useful for other Common Lisp
APIs or Lisp implementation.
So here is how to do it:
-
Download iterate using darcs using the
instructions
on this page.
-
Add the line:
(require 'asdf)
at the top of your .sbclrc file, in case it's not already there. Then,
add this line somewhere after it:
(push #P"/home/shlomi/Download/iterate/" asdf:*central-registry*)
/home/shlomi/Download/iterate/ should be the path where you downloaded
iterate.
-
To use iterate from your Lisp script write the following at the top:
(asdf:oos 'asdf:load-op :iterate)
(use-package :iterate)
(As I discovered the colon in (use-package :iterate) is
significant.)
Then you can write this to test it:
(iter (for i from 1 to 10)
(format t "Hello No. ~A~%" i))
This should hopefully work. Please comment here if you still have problems.
Here are some of the things I've been doing lately. Recently, I've been
having a spell of tiredness and lacked the will to do anything productive.
But lately, it got somewhat better and I was able to work
on some productive tasks.
I've been learning the
Prolog programming language.
I started with
this
tutorial, but was dismayed from the fact it did not have exercises
involving writing code, so I switched to learning from
this
tutorial.
I've also been doing some of the tasks from
Project Euler.
I originally had written a Perl script to combine the feeds of my blogs
for my
aggregated RSS feeds. Originally this script executed
an aggregator program for
every feed, which fetched the feeds from the network every time. So a few days
ago, I made sure the feeds are fetched only once, and then are combined and
outputted into the files, using one program.
However, later on, I also decided to do hard-disk cleanup, and accidently
deleted it. I restored it from the backup, but it was the copy before the
improvements. So today, I set to reproduce my actions (this time using
version control), and I think it went quicker this time.
When I gave my
presentation about the Joel Test to the Israeli Pythoneers, I noticed that
one problem was that all the points appeared at once. So starting from
a few days ago, I customised the slidy.js JavaScript code from
HTML Slidy and adapted it
to my needs for the presentation, to display the points one by one. I removed
most of the functionality there and changed a lot of the rest, but with my
shallow control of JavaScript it was good to start from a working code.
Firebug was a lot of help in getting
to the bottom of the bugs I encountered.
Finally, I placed
Darien's
"Everybody's Free (to Ping Timeout)
online on my homepage, after I extracted it from my web-feed aggregator. It
disappeared from the Net after the demise of Darien's blog.
For a long time, I had many hangups with
the open-source Firefox
browser, which happened all the time, but I could not reproduce. Two weeks
ago, however, I noticed that when I was using
YouTube, after I opened a window
with a new video, and closed it, then Firefox freezed. I tried to reproduce it
and was consistently able to do so. So
I reported
it.
Then, after I tried to reproduce the bug in a new profile, I could not
reproduce the bug. So per
Steve
England's advice, I tried to find out which extension is causing this
problem, by disabling and re-enabling them. After a lot of binary search,
I found out that
I
could only reproduce the problem using three extensions enabled in the
original profile: Firebug,
Greasemonkey and "Split Browser".
Then Oded
Arbel said he could not reproduce the problem on a clean install using my
current description. And indeed
I could
not reproduce it on a fresh profile either. So there were also some
problematic preferences.
I set out to discover it. My most likely suspect was something in the
prefs.js file, so I copied it to the new profile, and it indeed caused
the problem to re-surface there. Then I gradually removed more and more initial
settings from there, to find out the problematic ones. Each time I did a
change, I had to try and reproduce the bug. I kept some -bad-$idx and
-good-$idx files as a poor man's version control. After spending some
time on it, I found out that the problem was with a single preference:
extensions.firebug.disabledAlways. So what was needed was installing
the three extensions and toggling this preference.
Then using this information
I
was able to reproduce the problem on my Windows XP machine.
This was a hard bug to properly isolate, and it now seems very pedantic.
But it may indicate a more general problem with Firefox. I'll be grateful if
you can try to reproduce it on your Firefox setup (in a fresh profile, of
course).
I wrote
a
patch to implement a search dialog for the Linux kernel "make xconfig"
configuration applet, and sent it to the Linux kernel mailing
list (LKML) back in January. I did it because I got bit by the lack of a
search facility there and wanted to improve it. As it turned out, by the time
I submitted the patch
a
search dialog was already implemented, but it was placed in the wrong
place in the menus, and was quite inadequate.
So I wrote
a
patch to relocate the menu item to "Edit" → Find,
and it got accepted into the main kernel. Then I neglected working on it.
About a week ago, I started working on it again. I enhanced the find dialog
to have substring search as well as regular expression search, and made it
search through the description strings. Afterwards, I implemented keyword
search.
When trying to submit this patch to LKML, I discovered that my address
was "not liked source for email", at least according to the bounce.
(This is orobably due to
this
email.) I re-submitted my first patch using the shlomif-lkml@iglu.org.il
address, which would have worked if iglu.org.il was still on qmail, but since
the domain was moved to a server running postfix, caused some bounces for
people who tried to reply to me in private. Aaarrrggh!
The second version of the patch (almost final, as far as I'm concerned) was
sent from my original address, and CCed to two relevant developers, and bounced
again. I hope this issue will be addressed after the Linux kernel summit (and
posssibly linux.conf.eu) are over.
In any case, I placed
the
patches I'm working on, in a temporary location until they get applied.
Enjoy!
There's a new
petition to release MainActor, a non-linear video editing
software under an open-source software, as it will be discontinued. See
also
this
opendotdotdot post.
Please sign the petition and help spread the word.
I should note that when the vote is submitted, you end up at the donation
page, not at the list of votes. Don't worry - your vote was registered.
This post will explain how to over-ride (or overwrite or replace) a global
JavaScript function found on a page with a completely new one using
Greasemonkey. Also see
this
greasemonkey-users discussion where I originally asked the question.
To do that write a global JavaScript function in the Greasemonkey script
(ending with .user.js) with the same name as the page-defined JavaScript
function that you wish to override, and do the following:
function PageFunction() {
// SNIPPED
}
function embedFunction(s) {
document.body.appendChild(document.createElement('script')).innerHTML=s.toString().replace(/([\s\S]*?return;)
{2}([\s\S]*)}/,'$2');
}
embedFunction(PageFunction);
embedFunction() is taken from the
"Embed a function in the Current Page" snippet from the
Greasemonkey wiki.
Happy Greasemonkeying!
STAF stands for "Software
Testing Automation Framework" and is a framework for IBM for software
testing. I spent a large amount of the last two days trying to get it up and
running on my Mandriva Cooker Linux system, in order to fullfill
this
request for a Linux beta-tester.
Trying to get the binary to work with my Perl failed due to a segfault when
running it. So I opted to build it for source. I followed the STAF developer's guide which explains
how to build STAF, but still needed a lot of trial and error.
Eventually, I installed ActivePerl-5.8 and used the following shell script,
that needs to be sourced (or "."ed) into the current shell, to build
everything. You may need to customise it a little.
#!/bin/bash
# One needs staf to build the main things, connprov* to be able to
# run STAFProc on the local machine and perl because I needed it.
export OS_NAME=linux BUILD_TYPE=debug PROJECTS="staf connprov* perl"
export JAVA_LIBS="/usr/java/jdk1.5.0_09/" JAVA_VERSION="1.5"
export JAVA_V12_ROOT="/usr/java/jdk1.5.0_09/" JAVA_V12_LIBDIRS="/usr/java/jdk1.5.0_09/libs/"
export PERL_BUILD_V58=1
export PERL_BUILD_V56=0
export PERL_V58_ROOT="/opt/ActivePerl-5.8"
export PATH="/opt/ActivePerl-5.8/bin:$PATH"
PLIB="$PERL_V58_ROOT/lib/"
A="$PLIB/CORE"
export PERL_V58_INCLUDEDIRS="$A" PERL_V58_LIBDIRS="$A"
# This variable is completely undocumented in the relevant part of the
# STAF documentation, but with its default value it won't work correctly.
export PERL_V58_TYPEMAP="$PLIB/ExtUtils"
After doing it, type "make". Cheers. Now I can go on with my business.
One day, I woke up and started using my Mandriva Linux Cooker system which
I left on overnight, and noticed that I couldn't access the Internet host
hosting my homepage (which also
hosts my Email account). No POP, no HTTP - nothing. I couldn't find any way
to restore access there except reboot. Eventually, the problem repeated itself
a few times, each time involving a reboot.
Eventually, I
reported
my findings to Linux-IL, but couldn't get much help there.
I
asked netdev@vger.kernel.org for help, and was told the problem was with
the
TCP
window scaling problem, but doing
"echo 0 > /proc/sys/net/ipv4/tcp_window_scaling" after the problem appeared
didn't solve the problem after at least 30 minutes.
Here's what I know so far about this problem:
-
The IP that causes the problems is 212.143.218.31.
-
It is exhibited by kernels 2.6.23, 2.6.24-rc1 and 2.6.24-rc2. (At least)
-
A different computer on the same Home LAN connected via a NAT/router has no
problem with that IP, at the same time the Linux installation exhibits the
problem.
-
During one time this happened, I could connect using telnet to port 80
eventualy, but it took an awfully long time.
-
I have problem with both HTTP and port 80 and POP.
-
Restarting the network ("/etc/init.d/network restart") does not help - only a
reboot.
-
The network as a whole (Google, etc.) works fine.
-
Doing "echo 0 > /proc/sys/net/ipv4/tcp_window_scaling" after the problem
appeared didn't solve the problem after at least 30 minutes.
Now, in trying to resolve this problem, I decided to switch to my
distribution's kernel so I could at least report it in its bug tracker. So
I tried to boot my system using the new kernel and it wouldn't boot. After
a lot of experimenting and no progress, I decided to install Mandriva again
on a new kernel, so I can copy the new kernel settings and initrd. It couldn't
boot either.
So I filed a bug
to the Mandriva bug-tracker, and have done some further investigation.
Apparently, a configuration I created for the Linus kernel, which is similar to the Mandriva config, could not boot the Linus kernel either. On the other
hand, my old config did work. So I started bisecting, and eventually
reached this
conclusion, where I have a working configuration that has some stuff as
"Y", and a non-working configuration that resembles the Mandriva one more (that
was created by toggling a "Y" option into "M" in "make xconfig").
Bug-investigation is often kind-of like
serendipity (or
rather
Whack-a-mole), where one discovers bugs that prevent one from further investigating
the previous bugs. At the moment, I'm still using the vanilla kernel, and am
still getting the networking problem that forces me to reboot the computer
more often than I'd like to.
If you can shed some light on any of these problems I mentioned, please
comment below - any help would be appreciated. Until then, happy, and
hopefully bug-free, computing.
GIMP and GEGL
I've recently decided that I'd like to give my life purpose again by working
on a large project. I decided to work on
GIMP again, joined its channel, and asked
how to help in porting it to the GEGL and
Cairo libraries. I was told that the HOWTO just involved learning the APIs
of these programs and figuring it out for yourself. So I started reading on
GEGL. As I read the GEGL homepage and API documentation, I discovered some
errors and mis-phrasings, which I reported and are now fixed in the trunk
(either by a different committer or by me.).
Next, I realised that GEGL did not have any automated tests, and decided to
work on that. For that I'd like to use
Ruby and
TAP - the Test
Anything Protocol. I have to resort to using Ruby instead of Perl because
GEGL does not have Perl bindings yet.
Archive-Zip
I recently asked Adam Kennedy for a co-maintainership status for the
Archive-Zip CPAN module
(so I can close its bug reports) and he kindly granted it. This enabled me
to fix and close several of its many bugs. It was sometimes time consuming
because the bugs sometimes did not have regression tests. I had to
write them myself and make sure they fail before patching and succeed
afterwards.
If you're a software enthusiast and reading it, please take note for
this:
when reporting a bug against a program, please give a full reproduction
recipe, and very preferably, a patch containing an automated regression
test. Otherwise, you're wasting the developer's time who has to interrogate
you, figure out the missing data or code, and why it failed for you. So
please, be kind to us, and show us the testcase!
Test-Count
While resolving an Archive-Zip bug, I had to heavily rework a test that
someone sent me. Now I had to execute a group of assertions once normally
and once with a Perl global variable ("$\" if you must know) set. So I used
the Test-Count,
$n=0 and then $n=$n+1 paradigm (and finally added two $n
to the test count) in order to count the tests. But it didn't work, and the
script exited with a weird error. As it turned out when retrieving the value
of the variable, I checked for truth instead of whether it's not undef(),
and so failed. So I had to take a break from working on Archive-Zip and fix
Test-Count in this regard.
My Test-Count testcase, then failed again, and as it turned out it happened
because assignments were performed twice due to a weird parsing problem, which
I also fixed. After all that was done, I uploaded Test-Count-0.0103 to the
CPAN, and got a bootload of failures for it. Turns out that the specification
of the dependent modules was wrong, and I fixed it and released
Test-Count-0.0104, which
passed 27 times
so far, while generating one "NA" with perl 5.005 for which it's not intended
anyway.
Nouveau
Nouveau is an ongoing project
to create open-source UNIX drivers for Nvidia cards. I've been trying to
consistently use nouveau instead of the properietary "nvidia" or the limited
"nv" driver, but often encountered some problems that drove me back to
"nv".
The last time it happened was about a month ago when x11-server-1.4.0 was
released and caused various random rendering problems when used with Nouveau,
which were too annoying to stand. It also had other non-Nouveau-related bugs,
but these were mostly corrected by vendor patches.
So I switched to "nv" again. A few days ago, I've visited the #nouveau
channel on IRC, and was told to try Nouveau again. So I downloaded, compiled
and installed the latest Nouveau and restarted X with it, and it seems to
work fine. The rendering problems are gone, and as a bonus from previous
versions of x11-server, I'm no longer getting the visible wallpaper
tile-by-tile-redrawing that I experienced previously.
One thing I notice is that when using Nouveau, I can still
exhibit this bug,
which exists in "nv" as well. (So it doesn't matter too much that I'm using
nouveau, in this regard.)
So nouveau works again, and I can happily beta-test it by using it daily.
Ben Collins-Sussman (one of the Subversion developers) wrote
an insightful article
titled "Version Control and 'the 80%'" on his blog. It received many
comments and I wanted to comment on it, too, but:
-
His blog ate my comment.
-
When I tried to post it again, it refused to, claiming a similar comment has
already been posted.
Given I have no choice, I'll post this comment here, and allow Ben to respond
to it. Here it is, with all of its glory:
Nice article. However, are you sure most developers are working on Windows? Not
according to the article which I referenced
in
this Joel-on-Software discussion:
"Linux is becoming the development platform of choice; for example, it is
expected that more developers will be developing on Linux than Windows by
2004;"
Also can you please add a "Preview Comment" button? I am unable to tell if my
comment is good or not because the only button I have is "Submit Comment".
I should note that I can also recommend Ben's
"Welcome to the Food
Chain" entry, which isn't technical.
I've been meaning to try KDE 4 for a while now, and finally decided to do it
after RC 1 (The first Release Candidate) was announced lately. I decided to
install it from the Mandriva KDE 4 RPMs. As I discovered
by reading "The
Running KDE 4 on Mandriva 2008" linux-tip article, installing it
involved typing "urpmi task-kde4". It failed to install the
kdebase4-workspace package and I had
to fix and report a
bug of it (which has since been resolved in cooker), but after installing
the fixed package, and doing "urpmi task-kde4" again, I had KDE 4 installed.
By then it was too late in the night (past 1 AM) and I decided to go to sleep.
Next morning I looked for a way to start KDE from a virtual console, as I
was not using a graphical run-level (with kdm/gdm/etc.). The article did not
mention how to do it, and after a little research, I found out that it can be
done by typing "kde4" at the virtual console bash command line. This will set
up the environment apprporiately and start an X session running KDE 4.
So I had KDE 4 running. Here are my initials impressions:
-
The decorations of the inactive windows were light gray, which made them
hard to distinguish. I could not find a way to change it. (I was told on IRC
that it is work in progress.)
-
There did not seem to be any support for
virtual desktops,
and I was also told it is work in progress.
-
In Konsole, there was no "New session" button at its tabs bar to start a new
tab. That seems to be a developer decision, and I discovered that I could
still start a new tab from the menus, by pressing the keyboard shortcut,
or by double-clicking on an empty space in the tabs bar.
-
There doesn't seem to be a way to drag items from the main menu into the
desktop or the quick-launch bar (which doesn't seem to exist) for easy
access.
In short, it's not ready for prime time, and I could not find it usable
for my day-to-day-use. If it's a release candidate, then I'm a Jackalope.
See also Diego Iastrubni's
recent review of KDE 4 (in English, and with some comments).
I started working at a certain workplace downtown, at least temporarily. There
I was instructed to set up a Fedora 8
system (Fedora, being the Red Hat-endorsed community distribution)
with some networking services, including
SMTP
(Internet Mail). The installation and post-install configuration of
Fedora there, went pretty flawlessly, but then I encountered an annoying
bug when trying to monitor the SMTP traffic. Apparently, when using
Wireshark the recorded SMTP
conversations on eth0 only have one side of the conversation, with the other
side completely removed. Here's what I found out about it:
-
HTTP conversations are recorded fine.
-
It still happens with iptables being cancelled completely (all ACCEPT), and
with SELinux set to permissive.
-
It happens with all current Fedora 8 updates, including the most up-to-date
kernel, applied.
-
It happens with the Wireshark filter set to empty ones, which should accept
all traffic.
-
It happens with both the Fedora supplied Wireshark, and a Wireshark that
I compiled from source and installed under a different prefix.
-
The traffic originates from a remote computer on the same Ethernet network.
putty which is used for the conversation, indicates that the SMTP server
has properly received the messages I sent it, and that packets were sent
in both directions.
Now I've tried to reproduce the problem here on my home machine, so I installed
Fedora 8. At home, however, everything works perfectly. Grrrr...
This has been driving me crazy because it makes it much more difficult to
debug the SMTP service. If anyone has any idea to what could be wrong, please
comment here.
This has given me a chance to try Fedora again. The first problem I noticed
(at home - at work it was OK) was that I didn't have external networking.
Apparently, the /etc/init.d/network script was not placed in
the rc5.d scripts for some reason. Running /etc/init.d/network start
from the command line and using system-control-services fixed it.
I noticed that I could mount XFS partitions, and was able to configure the
same /home partition (with the same user) as my Mandriva installation. However,
using the Mandriva configuration, Konqueror would just display the code of
an HTML page, instead of rendering it. Plus, when I booted the Mandriva system
again, I noticed that my KDE screen was a little messed up, and I had to
restore its configuration directory from the backup I prepared before
installing Fedora. I guess $HOME/.kde/share/config was not intended
to be shared like that between Mandriva and Fedora.
One thing that has improved is the support for .mp3's in Amarok. I recall
being completely unable to play them properly there using the xine backend,
but now it worked after I installed the relevant package as specified in
the Fedora FAQ.
At home, I installed Fedora 8 from the Fedora Live KDE CD, so it is possible
some of the problems I encountered will not be encountered if one uses the
installation DVD.
I've been meaning to try out the
Archlinux Linux distribution
for a long time now. During the previous weekend I finally dedicated some
time to installing it. Here are my first impressions.
The first thing I noticed was that the installation procedure is rather
text based and not very friendly. I didn't want to risk it formatting my
common /home partition so I eventually opted to configure it
after the installation. I had to run the installer twice in order to set
a bootloader, so I can tell how to boot Archlinux properly using the bootloader
on the Mandriva parition. I wanted to configure the Mandriva bootloader using
the files on the Arch partition but couldn't figure out where is the kernel
and initrd. Apparently, it keeps the kernel under /boot/vmlinuz26
and the initrd under kernel26.img. Moreover, the mkinitrd
program is called mkinitcpio there.
After being able to boot arch, I noticed that my networking was not configured
properly, despite the fact I use DHCP for this computer. I opted to configure
it manually using ifconfig and route, but I was later told
I can add eth0=(dhcp) in the appropriate place in
/etc/rc.conf. This would enable DHCP after you do
/etc/rc.d/network restart.
Arch also configured my clock improperly and I had to set the timezone and
clock settings manually:
HARDWARECLOCK="UTC"
TIMEZONE="Asia/Tel_Aviv"
The time was still wrong, and remained so until I installed and ran ntpdate
manually. The documentation on the NTP protocol on the wiki is confusing,
and I have yet to figure out how to do it on system startup.
Then it took me a long time to figure out Pacman, the Archlinux package
manager. As I was later told, it has
a good cheat-sheet on
the Archlinux wiki. If you're going to install Arch, I suggest reading
it and then keeping it handy on the hard disk in a plain text, because you'll
need it, and without it you won't be able to install anything.
I needed the help of the #archlinux IRC channel on Freenode on how to
install X and a usable environment. You can install it using
pacman -S xorg, and then pacman -S xf86-video-nv (or whatever
is your video card), and then pacman -S icewm to install IceWM (or
whatever). To install KDE you are better off configuring and using
KDEmod than the built-in KDE and there
are instructions how to do it on their site.
I still don't know how to do a few things in Archlinux. One is how to
search for files inside remote packages (like urpmf on Mandriva
or apt-file on Debian/Ubuntu). And it seems that mplayer, xine (and
thus - Amarok) don't support playing
Module files, because
support for them has been configured out during compile time, which is a bit
annoying.
I was able to share my /home partition and my user between Mandriva and Arch
pretty easily. My KDE exhibited a few problems on Mandriva, after using it
with Arch, so it may not be very recommended to do so. Archlinux keeps KDE
under /opt/kde, which could cause incompatibilies with the
Mandriva /usr-bound KDE.
As a general rule, Archlinux gave me a rather "raw" and non-user-friendly
feeling, so I didn't end up using it a lot after the installation. I wouldn't
recommend it for beginners because hardly anything works there out of the
box. Still, installing it was a good experience so I can see things being
done a little differently than I'm used to.
This is a reminder that Israeli Perl
Workshop for 2007 will take place next week on Monday, 31 Decemeber 2007.
If you're planning to attend, please register at the site, and make the
payment.
I already registered and am planning to be there, and I'll hope to see any
Perl enthusiasts I know, and those who would like to meet me, there too.
Happy new year!
The ~/.ssh/config accepts the Host meta-configuration that
enables one to specify directives for only a specific host. With the
HostName directigve, and the User directive this allows
defining shortcuts and defaults. For example, this portion out of my file:
Host berlios
HostName shell.berlios.de
User shlomif
Specifies that by doing "ssh berlios" I'll login to
shell.berlios.de as the user shlomif (in equivalence to
ssh shlomif@shell.berlios.de).
One use I can think of for this feature is to define a common destination,
with variable remote usernames, in uploading scripts. Until now, I've used
something like: ${IGLU_USERNAME:-$(whoami)}@iglu.org.il:, which
retrieved the value of the $IGLU_USERNAME and defaulted to the
local user. However, now one can simply use iglu.org.il: or
mytask.iglu.org.il: and define the appropriate user in the ssh
config file.
Credit is due to a
blog post by Diego
Iastrubni, where he blogged about this feature. Thanks, Diego!
I found the following projects which I found of interest to me by reading
the new Freshmeat.net releases. I'd like
to share them here:
-
GIMP Scripts that work in
GIMP-2.4 - forward-ported scripts for GIMP-2.4.
-
An Open-Source Search Engine for Subversion
Repositories - seems to be very useful in some contexts. I submitted
a patch to add
it to the Subversion links' page.
-
VisionProject -
a non-free issue tracking/project collaboration tool with Subversion
integration. It may also be appropriate to add it to the links page.
-
Yudit -
a Unicode text editor for X-Windows - a Unicode text editor with
Bi-directionality support that I seem to have missed. When trying to
input Hebrew with it, I'm getting the 4-digit-hex-representations and not
the letters themselves - I'm probably missing a font. Furthermore, my
compose key doesn't
work there and so I cannot input accented characters. It has no menu - only a
toolbar, and the right mouse button does the same operation as the left mouse
button.
Still, after I sort
out all the problems with it, it may prove as a good alternative to KDE's
KEdit and to
Katoob.
-
WP Clipart is a
public domain, bitmap-based clipart that looks interesting.
-
BurgerSpace is a clone
of a game I used to play on DOS. Trying to play it with the version in Mandriva
Cooker, I found that after a while I could no longer throw pepper to attack
the enemies, even after I died. But I didn't know if it was a bug or a feature.
-
CrissCross is another
C++ portability library
-
Deliantra is an
open-source Multi-player Online Role-Playing Game (MORPG). One interesting
fact about it is that it's partly written in Perl. I did not test it yet
because I'm not using the proprietary nvidia drivers.
-
Another MMORPG is The
Mana World, which doesn't require OpenGL or 3-D acceleration. I
temporarily gave up on installing it after running into one of its
dependencies.
Enjoy!
In continuation to my
previous entry about Amarok and Last.fm integration, I found
this bug entry
regarding the main problem with it that bothers me, and I commented
about it and re-opened it. A net-friend who checked it got different results,
and commented as well.
One thing I noticed about the Last.fm player is that it allows one to play
any arbitrary Last.fm tag, while Amarok only has a set of pre-defined tags
for common music genres in its GUI. However, after experimenting with "dcop" I
found out one can enqueue any arbitrary Last.fm tag using the following shell
script:
#!/bin/bash
tag="$1"
shift
dcop amarok playlist addMedia "lastfm://globaltags/${tag}"
To use it save as "lasttfm-amarok-tag", make it executable (chmod +x)
and do ./lastfm-amarok-tag "new age" to play the "new age" tag. You
can also type a command line like
dcop amarok playlist addMedia "lastfm://globaltags/reggae" directly
from the command line, but that's not very convenient.
And finally, here's a recommendation for a puzzle game I saw
mentioned
on the Free Gamer blog:
Hex-a-Hop. I was
able to easily compile it on my Mandriva Cooker system, and played it, and it's
very nice.
This story has two lessons. The first is that I'm an idiot, but I guess we
all already knew that. I'm not going to tell the second one, but it will
be available under a headline, in case you want to skip. But let's start with
the beginning.
The Bug
It's all started when I did my backup yesterday, converted it into a CD-ROM
image (an .iso file) and so used
growisofs. But when
running it, it completely hogged my system, causing the Amarok music playback
to be flaky, and making everything more sluggish. It didn't happen before, so
after
reporting
this bug to the Linux-IL mailing list, I decided to investigate.
I went on the IRC's #kernelnewbies channel, and someone told me that since
it worked before, I should start from a working kernel and bisect it. And
indeed trying the 2.6.23 kernel made the problem disappear. So I decided
to bisect the
kernels in between to try where the problem occured first.
Bisecting using git
The history of the Linux kernel is versioned using the git version control
system. Searching for "git kernel"
yielded the "Kernel Hackers' Guide
to Git", which I've skimmed, and found that I should start by typing this
in the command line:
git-clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6 in the command line.
Note that one cannot effectively interrupt this command without needing
to start at the beginning, so be prepared to wait a while. (I had to wait
for it to finish before I could reboot to test another kernel).
Similarly one can use git-clone linux-2.6 new-lin-2.6 to clone the
repository on the hard-disk, which also proved to be useful in order to build
the latest kernel I used to see if I could still reproduce the problem there.
Then I was told to used git bisect to bisect the kernel. (The "lion in the
desert" method). Read the link for usage information. I copied my
/boot/config-2.6.23 kernel configuration file to .config in
the top-kernel-dir, and ran "make oldconfig" on it. From then on, I used "make
oldconfig" on the new kernels, while pressing Enter all the time to accept
the defaults (also doable with yes "" | make oldconfig).
This way, I built one kernel after another while rebooting to test each one.
As I discovered in the middle of the 2.6.24, something went wrong, and one
could no longer run "make modules" in the kernel source tree, so I often
had to reboot to a safer kernel. This also slowed me down, but it was resolved
since before -rc6.
All Good is No Good
As it turned out, every kernel I built except the 2.6.24-rc6 kernel
which exhibited the bug in the first place, did not exhibit the bug (and
marked by me and git as "good"). Eventually
I ended up with the last patchset reported to be the problem, but it only
contained a change of the EXTRAVERSION from "-rc5" to "-rc6" in the top-level
makefile. I couldn't understand why it should make so much difference.
So what was the problem? As I discovered later, the v2.6.23 configuration, on
which the configurations of most of the kernels I built were based, was
slightly different than the one I had for v2.6.24-rc6, and this is what made it
work. I originally made the changes to investigate
a
different bug (where I tried to boot my system using a kernel with a
configuration similar to the Mandriva one), and they persisted in the further
kernels.
After changing the kernels (including the old ones) to the new configuration,
I was able to reproduce the problem in all of them. So it was a bug with
my kernel configurations and not with the history.
The Second Lesson of this Story
So the second lesson is that if you're going to bisect kernels make sure you
have the same configuration or one that is derived from the other one.
Otherwise, the kernels may behave differently.
What I Achieved
So what did I achieve?
-
I learned more about the git version control system, and how to use it. I
could use the knowledge in the future.
-
I now have 32 entries in my boot-loader menu.
-
I rebooted my machine 25 times. That's the most I've done in such a short
period since I've encountered the problems with my ReiserFS partition back
when I started using Linux Mandrake (now Mandriva Linux) 7.1.
-
I was pratically neutralised throughout all this time. I was able to send
some emails, and chat a bit on the IRC, but I was mostly waiting for kernels
to finish compiling.
-
I discovered that this kernel configuration diff has been the root of the
problem
--- config-ok 2008-01-07 20:53:30.698008159 +0200
+++ config-not-ok 2008-01-07 23:13:17.894346993 +0200
@@ -1,7 +1,7 @@
#
# Automatically generated make config: don't edit
-# Linux kernel version: 2.6.24-rc5
-# Mon Jan 7 16:37:58 2008
+# Linux kernel version: 2.6.24-rc6
+# Mon Jan 7 13:19:02 2008
#
# CONFIG_64BIT is not set
CONFIG_X86_32=y
@@ -621,34 +621,31 @@
CONFIG_BLK_DEV_RZ1000=m
CONFIG_BLK_DEV_IDEDMA_PCI=y
CONFIG_BLK_DEV_AEC62XX=m
-CONFIG_BLK_DEV_ALI15X3=y
-# CONFIG_WDC_ALI15X3 is not set
-CONFIG_BLK_DEV_AMD74XX=y
+# CONFIG_BLK_DEV_ALI15X3 is not set
+# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_BLK_DEV_ATIIXP is not set
-CONFIG_BLK_DEV_CMD64X=y
+# CONFIG_BLK_DEV_CMD64X is not set
CONFIG_BLK_DEV_TRIFLEX=m
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
CONFIG_BLK_DEV_CS5530=m
# CONFIG_BLK_DEV_CS5535 is not set
-CONFIG_BLK_DEV_HPT34X=y
-# CONFIG_HPT34X_AUTODMA is not set
-CONFIG_BLK_DEV_HPT366=y
+# CONFIG_BLK_DEV_HPT34X is not set
+# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_JMICRON is not set
CONFIG_BLK_DEV_SC1200=m
-CONFIG_BLK_DEV_PIIX=y
+CONFIG_BLK_DEV_PIIX=m
# CONFIG_BLK_DEV_IT8213 is not set
# CONFIG_BLK_DEV_IT821X is not set
# CONFIG_BLK_DEV_NS87415 is not set
-CONFIG_BLK_DEV_PDC202XX_OLD=y
-# CONFIG_PDC202XX_BURST is not set
-CONFIG_BLK_DEV_PDC202XX_NEW=y
-CONFIG_BLK_DEV_SVWKS=y
-CONFIG_BLK_DEV_SIIMAGE=y
-CONFIG_BLK_DEV_SIS5513=y
+# CONFIG_BLK_DEV_PDC202XX_OLD is not set
+# CONFIG_BLK_DEV_PDC202XX_NEW is not set
+# CONFIG_BLK_DEV_SVWKS is not set
+# CONFIG_BLK_DEV_SIIMAGE is not set
+# CONFIG_BLK_DEV_SIS5513 is not set
CONFIG_BLK_DEV_SLC90E66=m
CONFIG_BLK_DEV_TRM290=m
-CONFIG_BLK_DEV_VIA82CXXX=y
+# CONFIG_BLK_DEV_VIA82CXXX is not set
# CONFIG_BLK_DEV_TC86C001 is not set
# CONFIG_IDE_ARM is not set
-
Hopefully I found a way to overcome the bug. Now that I've upgraded to kernel
version 2.6.24-rc7 (which was
released today), I based my configuration on the one that doesn't exhibit
the bug, and will test it to see if that's indeed the case.
-
And like I said earlier, I realised I am an idiot, which is the first step
on the path to enlightenment.
So mission accomplished? I guess so. Paul Graham said in What You'll Wish You'd Known
that one is happy when being relieved after overcoming worries, and I feel
very much relieved now, so I guess I'm happy. I also feel a kind-of-calmness.
Here's to a bug-less and flaw-less use of your favourite open-source operating
system. Well, at least we can hope.
In order to play sound / audio on your Mandriva (and possibly other
Linux flavours) system, after being logged
in an X session with one user, and you've su'ed to a different (also possibly
under-privileged) user in the shell, do the following. Just add all the users
that should be able to play sound at all times to the "audio" UNIX group
in the "/etc/group" file (as root, of course). Then, you may have to run
"sg" or "newgrp" for these changes to take effect globally.
Then you can play sound as all the users you've su'ed to that belong to
the "audio" group. Also see
this
thread in the Mandriva Cooker mailing list, where I asked the question
originally.
And a more obscure and less useful tip: to exclude dependencies ("requires")
on different RPMs (that are calculated implicitly) when editing an Mandriva
SPEC use a statement like that:
%define _requires_exceptions /bin/zsh /bin/csh
There's an example of this in the Mandriva xemacs spec.
As a follow-up to
my previous
post about a networking problem I've encountered on my Linux box I'd like
to make a follow-up and re-summarise all the currently available information.
Someone suggested the problem may be caused due to a bad Ethernet card, so
I borrowed an Intel Ethernet card, replaced my Ethernet card with it, and tried
it for a few days. After keeping the computer on for a while, it started
exhibiting similar problems as the old one: no connectivity to the host of
www.shlomifish.org, bad connectivity to Google, etc. Solvable only
after a reboot.
I've uploaded Wireshark pcap dumps of a
"lynx http://www.shlomifish.org/" command from both cards before and after
the problem is exhibited.
So here's what I know now:
-
The symptom is that after the computer is on for a while (two days or so), one
cannot connect using TCP to some hosts, and the connection times out.
-
The IP that causes the problems is 212.143.218.31, but it also affects
www.google.com and possibly other hosts.
-
It is exhibited by kernels 2.6.23, 2.6.24-rc1 and 2.6.24-rc2
and 2.6.24-rc8. (At least)
-
A different computer on the same Home LAN connected via a NAT/router has no
problem with that IP, at the same time the machine running Linux exhibits the
problem.
-
During one time this happened, I could connect using telnet to port 80
eventualy, but it took an awfully long time.
-
I have problem with both HTTP and port 80, POP and SSH.
-
Restarting the network ("/etc/init.d/network restart") does not help - only a
reboot.
-
Some other hosts in the network, like Yahoo work fine.
-
Doing "echo 0 > /proc/sys/net/ipv4/tcp_window_scaling" after the problem
appeared didn't solve the problem after at least 30 minutes.
-
The problem is exhibited by both a RealTek card I have and an Intel Ethernet
card. (both 100 Mbps).
-
pcap
dumps of an HTTP connection to the offending site before and after the
problem occurs are available for both cards.
-
I can't ping www.shlomifish.org, because it doesn't answer to pings at all,
but when the problem occurs again, I can try pinging www.google.com and see
what happens.
-
iptables is completely off:
[root@telaviv1 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
-
My household is connected to the Internet through a Sweex NAT/Router
that doesn't have any updates yet.
-
Quoting srlamb:
The difference in these two packet captures seems to be that in the "bad" case,
the www.shlomifish.org->you packets have a bad checksum and are ignored
(see it
retrying the SYN after it's already gotten a SYN/ACK?). I don't know why this
would be, but it seems worthy of mention to the mailing lists you are asking
for help.
Some extra discussion can be found at
the previous
entry.
This evening, after I returned from work, I decided to further my studies
of unknown programming languages. The first thing I tried to do was to play
the
Practical
Common Lisp video that I downloaded. However, neither mplayer nor xine
could play it due to a missing codec. It worked fine a few days ago, so I
don't know what's wrong yet (but I intend to find out in the future).
Then I decided to go over Pick
Axe Book instead, but as it turns out, at the time of this writing all the
chapters give a "File Not Found" error.
Then I decided to watch the "Practical Common Lisp" video this time online
using the Flash applet. However, towards the start, I pressed the "More" links
in the related videos frame, and instead of a nice AJAXy thing, I ended up
overwriting the current page. As a result, the video had to be reloaded from
the beginning, and I could not skip and load the middle of it.
So I feel a bit frustrated from all these things going wrong. Luckily, I
jogged on the way back from work, and so felt more calm and relaxed
previously. I still hope to do some programming languages-related learning
this evening, but it will have to be something else than what I tried so far.
A few days ago I noticed that some operations in
the Amarok music player became very
sluggish. For example, the track switching took a very long time. I recalled
that something like that happened some months ago, at which point I ended up
renaming away the KDE base directory ($HOME/.kde/share/apps/amarok/)
which solved it. This time per advice from someone on the #amarok IRC channel,
I simply moved away the SQLite collection.db file. After the
database was re-populated, the sluggishness indeed disappeared.
However, I wanted to solve the problem permanently without having to keep
resetting the database. So I decided to switch to a PostgreSQL backend,
which hopefully won't get congested. As it turns out, if you can select in
"Settings → Configure Amarok →
Collection → Collection Database" PostgreSQL or MySQL, then they
are supported. So I created a database, set up a user for it and set up
Amarok accordingly. If it's not supported on your system, you may wish to
consult this page
for Mandriva and Amarok on the Amarok wiki and similar pages there for
other distributions.
I also had a long-standing problem that when I double-clicked a video file, it
tried to load it using Noatun, which
caused
artsd to crash. As someone on #mandriva-cooker told me, Noatun was no
longer maintained, and I should use
Kaffeine instead. So a few urpmi
commands later, and after tweaking the file types, Kaffeine played my
video files beautifully.
I hope you found this entry useful or at least a bit interesting.
Using tcpdump to record TCP sessions (or conversations) to a file, use the
following command (as
root or a privileged user):
tcpdump -w dump-file.pcap 'host 212.143.218.31'
Replace the 'hsot 212.143.218.31' with any tcpdump filter. Then
you can run "wireshark" on the dump-file.pcap file. Note that I got
incomplete results when I tried it now on my home machine and would love
to be further enlightened.
I last took
a look at KDE 4 three months ago. Since then, KDE-4.0.0 and KDE-4.0.1 had
been released. Yesterday, an IRC pal of mine said he was informed us he
was going to compile KDE-4.0.1 on his Gentoo system and test it. This prompted
me to take another look at KDE-4.0 to see if and how much it improved since
my last review. I am writing this report on KDE 4.
As I discovered (possibly again) to run KDE 4 one needs to run kde4env
from the command line, and then type startx. Running it inside the
.xinitrc/.Xclients file is not enough.
The first thing I noticed is that double-clicking the icons on
the desktop does nothing, nor could I find any other way to invoke them.
It's probably a bug. On the other hand, one can resize the icons, which
is a pretty nifty effect. So you can have large or small (but useless)
icons.
Moreover, when browsing the file system using Konqueror, I couldn't find
the tree-view, which was one of my favourite views. I don't know if it was
removed deliberately, or is still not implemented.
The Amarok progress bar of a song does not advance. It used to happen with
videos, but now it happens with .mp3s and .oggs too. Amarok still doesn't
display the visual part of videos. I thought it was supposed to be added
in this version, but perhaps I had a mis-conception. Plus, I could not
drag-and-drop files from Konqueror to its playlist, and Amarok doesn't
enqueue a new double-clicked .mp3 file after it got opened by the first
double-clicked file. Today I discovered one can double click files in the
Amarok file-browser to open them, which makes it somewhat more usable.
One should note that Amarok 2.0 (the KDE 4.0 version) is pre-Alpha, so
such bugs are expected, and the developers have asked not to report bugs.
KMPlayer seems to play the videos I double-clicked on fine.
On my setup none of the Alt+Tab, Ctrl+Tab, Alt+Space and Alt+F4 keyboard
shortcuts worked, and the keyboard shortcuts applet in the configuration
centre did not display anything.
In KDE 4.0.1, the taskbar displays applications from all virtual desktops.
Moreover, I could not configure it to behave otherwise, because the
right-click on the taskbar does not work. I was told I'll have to wait
for KDE 4.0.2 for that.
The fonts in gtk+ applications in KDE 4 seems smaller than usual for
some reason, which is a bit annoying.
In short, KDE 4.0.1 is better than the -rc I tried last time, but still
too buggy to be ready for prime-time.
My Vim tips column could use a more comprehensive entry about window
management (a.k.a "split view") in Vim, but this is a small tip in this
regard, that bothered me for a long time and that I want to record for
prosperity and share.
When having a verticla split, one can use Ctrl+W,Left or Ctrl+W,H to move
to the left split, and Ctrl+W,Right or Ctrl+W,L to move to the right split.
However, it seems that for some reason Ctrl+W,Ctrl+Left (while holding the
control) does not work. To fix it, add the following lines to your
.vimrc:
" mapping to be able to move to the left and the right windows
" without needing to leave the Ctrl key.
map <C-W><C-Right> <C-W><Right>
map <C-W><C-Left> <C-W><Left>
I've been annoyed by this behaviour for a long time, and now the salvation
feels sweet. I should note that other Ctrl+W,Ctrl+Someting key-combinations
work perfectly, so this aliasing is not needed there.
Arc is a new dialect
of Lisp
by Paul Graham and others. Recently,
Graham has released a working codebase for Arc, which started a small
community of people who play with it and improve it. As part of it, there's
a
world-writable version control repository for Arc called "Anarki".
Seeing that a comprehensive test suite was absent, I decided to start writing
one. I decided to use the Test
Anything Protocol (or TAP for short), which is a simple-to-implement
STDOUT and STDERR based mechanism for test scripts to emit results of
their test assertions to the TAP harness. This was partly because I expected
the Arc test suite to be written in more than one language, and partly
because I have been involved with the development of
Test::Run,
which is a test harness for TAP, and other TAP efforts, and so wanted
to eat my own
dog food.
So I've written (and later on enhanced) a rudimentary implementation of a
TAP emitter in Arc. This TAP emitting module can be found in arctap.arc
in the repository, and is used by the Arc tests. Some tests were written
in Perl, primarily to test and fix bugs in the Arc TAP implementation.
The Perl tests use
Test::More, which
is a TAP-emitting module for Perl, and
Test::Trap.
The latter is used to trap the STDOUT and STDERR of Arc programs that are
run by the Perl scripts. This is done to test the sanity of the Arc
interpreter and the Arc-TAP implementation.
So far the work on the test suite uncovered several bugs in Arc and one bug in
Test::Run, and also resulted in some enhancements to Arc. Plus, I was able to
better familiarise myself with Arc, and naturally to write test suite code
which helps in preventing future regressions.
One feature of Test::Run that helped here was its
AlternateInterpreters
plugin (which is an idea I originally derived from
something Curtis "Ovid" Poe (also
a TAP worker) had blogged about). Currently, my interpreters.conf.yml
file reads:
---
- cmd: ./arc.sh
pattern: \.arc(?:\.t)?\z
type: regex
This uses ./arc.sh to run the tests by default. The exact configuration
is a bit more complicated
than that (see Test.sh), but still the prove utility that
is part of Test-Harness-2.xx, which is the older TAP harness,
would not do. This is because it is impossible to assign a she-bang at the
start of the Arc files, because it seems it confuses the Arc interpreter.
Currently 66 assertions in 7 files were written so there's still a long way
for a decent coverage of the Arc features. But so far it seems that using TAP
for the Arc test suite has been a sound choice, and I'm happy with it, and it
probably facilitated putting the test suite together.
(Thanks to infi, mauke and simcop for reviewing earlier drafts of this
post.)
I recently tried to install the proprietary Nvidia Linux drivers on
Mandriva. It took me about
two hours, but eventually the solution turned out to be very simple. Then I
discovered there was a better way.
First the better way: if you're running a distribution kernel, you can install
the Nvidia drivers by using the dkms package. To do this, go to
Easy Urpmi and configure the non-free
packages' source for your distribution. Afterwards run XFdrake, and select
the nvidia card you have, and it will ask you if you want to install
the proprietary driver.
If, on the other hand you wish to install from the Nvidia installer, as
I initially did, and you are using the distribution kernel, make sure you
installed the kernel-desktop-devel-latest or the equivalent
kernel-*-devel package for your kernel.
That's it. After I did that, I was able to run and tweak
REnouveau
in order to help the Nouveau project. I also was able to run Extreme Tuxracer,
and Compiz Fusion. The latter did not play nice with KDE and misbehaved badly.
So yesterday I decided that my
homesite's markup may have had too much extraneous whitespace. So I looked
at HTML Tidy to try to remove it.
I searched for a way to tell it to remove whitespace on the Net, but most
pages I found only said it was possible, but didn't say how. After some trial
and error I found that this tidy.rc file:
indent: 0
indent-attributes: no
vertical-space: no
wrap: 0
indent-spaces: 0
break-before-br: 0
tab-size: 0
And a command line that specifies it with -config worked the best. It
still left a lot of newlines, which could be removed, but I guess it's better
than nothing.
While running it, I discovered that it complained about having empty files.
And I checked that the files generated were indeed empty. Then I experienced
with Website Meta Language (or WML) on
Mandriva and found out that piping into it even simple documents resulted in
empty results.
I found out that the
update with the fix for the WML CVE issue broke WML. So I had to found
out what the problem was generate a better patch and submit it. As it turns
out the WML test suite still failed, but it didn't cause the RPM "%check"
target to fail for some reason. After my fix, all tests passed again.
Then I was able to process the pages again. So back to tidy. Tidy still
complained about some stuff like missing "summary" attributes in "table"
tags. Sometimes, it exited with an error status without emitting any warnings
or errors, so I had to tell my build system to skip these offending files.
So tidy ran, but it wasn't the end of my problems. When trying to build
the homepage to run tidy with a clean copy, I found out that
TTML
did not run. This was caused by the upgrade to perl-5.10.0, and I looked for
a way to prepare an up-to-date RPM out of it again. Both cpan2rpm and
Ovid (the Perl module, not
the Perl author) failed, and so I looked into cpan2dist. After experimenting
with it a bit I got it running using
CPANPLUS-Dist-Mdv
and it gave me a working RPM. Now I'd like to prepare an equivalent package
for constructing Fedora RPMs.
That was not the end of my problems. When I tried to clean-build my site,
I found out that the directory structure did not get built before some of the
targets. So I had to play with the make targets. Afterwards, everything
went fine. The difference that tidy made was:
Before size: 4903322
After size: 4781819
What next? On the site I still have many links that start with "./". This
prefix is completely redundant, and I can save some bandwidth by eliminating
them. I also contemplated converting the navigation menus to JavaScript,
which will read the menus' contents from the site in JSON format. Now
that I think about it, the problem with this approach is that it will
hinder navigating the site by web-crawlers, and may diminish the Page Rank
of the various pages. It also won't work on clients which have JavaScript
disabled. One thing I can do is also generate an HTML file with the current
view of the navigation menu for each page and link to it directly for
JavaScript-challenged clients. I still have to think about it.
Recently, I wanted to buy a USB Disk-on-a-key to replace my old 32 MB one.
I went to Zap and compared the prices.
The cheapest offer was from the
Pandas.co.il shop, so I went to buy it there. The site appears to
be written in PHP and I was able to fully operate it using the KDE
Konqueror browser, and to complete the buying. I assume the Firefox browser
will also work nicely there.
The 8 GB disk-on-a-key arrived a few days ago, and it works great. So I'd like
to thank Pandas.co.il for making their site compatible with non-MSIE browsers,
and I'll consider them for my future computer parts orders.
Windows
are one of Vim's most convenient features, and learning some commands could
prove to be useful. Some of the commands are available from the "Window"
menu in gvim, or using the mouse, but using them from the keyboard is often
more convenient.
To split a viewport in two, use Ctrl+W,S for a horizontal split or
Ctrl+W,V for a vertical split. To move between windows use
Ctrl+W followed by the arrow keys or h,j,k,l (but see
this
previous tip). To close a viewport use Ctrl+W,C.
Ctrl+W,r rotates the windows, and Ctrl+W, shift+R rotates
them in the opposite direction. Finally, Ctrl+W,+ and Ctrl+W,-
increase and decrease the height of the viewport by one line (in case you are
mouse-deprived). If that's too little you can do something like
10,Ctrl+W,+ to increase the height by 10 lines.
In this tip, I'll explain how to debug C/C++ programs in Eclipse using
a remote
GDB session (gdbserver). To do this:
-
Install CDT - The Eclipse C/C++
Development Tooling.
-
Set up a C/C++ project so it will build and compile.
-
Install gdb and gdbserver. The latter does not seem to be available in any
Mandriva package and had to be compiled from the gdb source, where it
is built by default.
-
Run gdbserver on the generated executable using a command like
gdbserver localhost:10900 ./my-exe
-
In Eclipse go to Run → "Open Run Dialog...", and go to the
upper debugger tab and select gdbserver Debugger in the Debugger
drop-down. Afterwards, go to the "Connection" tab under
Debugger Options and configure your connection. For my demonstration
I've chosen "Type: TCP" "Host: localhost" and "Port number: 10900".
-
Select the Run → Debug in Eclipse option and start debugging
the application.
The reason I needed it was that I wanted to use "remote" debugging so I can
debug a C++ program that must be ran as root, and would rather not run
Eclipse itself as root. I'm still not sure it can serve as a good debugger
front-end, but I became very annoyed with
ddd and plain gdb start
to seem to be inadequate on visualising the rest of the source code. So I
looked for an alternative and thought Eclipse might be OK.
Back when I worked on Freecell
Solver , I used to prepare binary
distributions of the command line solver. Now it acted as a filter, and when
invoked without arguments it just waited for standard input. As a result,
when the Windows users double-clicked the fc-solve.exe executable, it just
popped an empty console, which they couldn't understand what to do. And
a lot of them sent me a "How do I use this" program email with the exact
description.
Since I became annoyed of it, I decided to output the following to the
standard
error by default:
Reading the board from the standard input.
Type "fc-solve --help" for more usage information.
To cancel this message set the FREECELL_SOLVER_QUIET environment variable.
After the poor Winnies read it, it probably made them decide the
program was not for them and to look for an alternative, but at least most
of the emails I received about it stopped.
A year ago, Joel on Software published an article about having
remarkable
customer services, in which he said that one needs to fix everything two
ways: first by helping the user, and then by making sure other users don't
encounter and complain about the problem again by tweaking the behaviour.
As one can see, what I did for Freecell Solver was essentially what Joel
advised to do. (Albeit much later than I incorporated the fix.).
More recently, I started the Better SCM
site, which compares and advocates several version control systems.
However, git has been absent due to the fact a suitable maintainer did not
come forward. And as a result, I received many emails asking "Where's git?".
A few days ago I decided to add a
Frequently Asked Questions (F.A.Q.) to the site where I answered
this and other questions. However, today I received another email with "Git?"
in the subject about that.
Another thing Joel had said is that
"People
Don't Read" or at least read very selectively. So I added a
placeholder
page about git with explanation why it is absent and a call for help, and
added it to
the comparison
with the text "Unknown. Due to a lack of a maintainer for this system."
with a link to the F.A.Q..
This should be enough to get rid of that particularly repetitive stream
of emails, and hopefully will also yield me a contributor to add a better
solution. Often designing web-sites and doing "customer service" (or feedback)
requires designing the site properly based on the feedback one gets.
I didn't particularly think of what Joel said when I decided to do what I did
with the F.A.Q. and the placeholders, but now that I think of it, I guess
he was right.
On my homepage, I have many
<h2>, <h3>, etc. tags with id attributes in them so one
can link directly to the middle of the page. I have written a
Website Meta Language API that allows one to generate a table of contents for
the page based on them. However, recently I also looked for a way to have a
link to their anchors somewhere close to them.
I eventually decided to try doing it using JavaScript and jQuery. It took me a bit of information lookup, trial and error
and consulting people on IRC, but I ended up with:
<script type="text/javascript">
<!--
$("h1[id],h2[id],h3[id],h4[id],h5[id],h6[id]").each(
function(i){
$(this).append( ' <span class="selfl">[<a href="#' +
this.id + '">link</a>]</span>' )
})
-->
</script>
Now for some explanations:
-
The $("h1[id]...") construct selects all the headings with id's.
There may be a shorter way to do it (comments are welcome.
-
The each method iterates over all of them and calls the closure inside. The
closure sets the "this" variable to the current element, and accepts
its index there (the "i" variable). In our case, we're not making use of the
index.
-
$(this) constructs a jQuery object from "this".
-
.append() appends an expression to the inner HTML of the element. I add
a little HTML there. this.id may result in an XSS attack if you have
a really funky (and probably invalid) ID, but since I have control over
my ID's it's ok.
You can see the result, in the headings of
the presentations' page
(for example), as long as you don't have JavaScript turned off. jQuery seems
very nice, and I'm looking forward to making even more use of it where
appropriate.
Andy Lester has written
a
great article about some of the social and philosophical problems with the Perl
world. He says that while Perl 5 is a great technology, has a very
comprensive collection of reusable,
open-source code called "CPAN", and has a lot to show for, it has suffered
from concentration of effort and other such problems.
This, in turn, contributed to the fact that many programmers concluded it
was "dead", "dying", "not good enough", or other such negative hype and
negative myths. While Lester's article suffers from some problems, it
still makes a good read even if you're not an avid Perler.
2008-06-03: Next OSDClub Tel Aviv Meetings
On Tuesday, 3-June-2008 (tomorrow or today depending how you look on it),
on 18:30, the OSDClub of Tel Aviv, which is a joint venture of
the Tel Aviv Linux club,
and Perl-Israel (and some other
FOSS clubs who want to join the fun), will hold a social meeting. It
will take place at the Café of Tel Aviv University near the junction
of Einstein and Haim Levanon Streets. This social was scheduled because Zvi
is in Israel. (If you don't know who he is, then come to meet him.)
Much later, on Sunday, 15-June-2008, OSDClub Tel Aviv will meet to share
some Vim/gvim (= the text editor) Tips and Tricks. We'll meet in Schreiber
(Math & CS) building in Tel Aviv University. We have more meaty
presentations planned for July, but we hope these two meetings will be a
a useful start after a long neglect.
2008-06-04: Mozilla/Firefox Tip: Getting xml:lang to Work
XHTML 1.1 has deprecated the lang="" HTML attribute in favour of the more
standard xml:lang. I like to use XHTML 1.1 because I find the extra
strictness (like no <a name="...">) useful, but in order
to not completely alienate
Microsoft Internet Explorer
users, I'm serving it with a Content-Type of text/html.
However, it seems that Firefox as of v2 or v3 completely ignores "xml:lang"
when the XHTML is being served as "text/html". It took me a long time to
figure out a way to solve it, but here's what I did:
-
Loaded jQuery:
<script type="text/javascript" src="../../js/jq.js"></script>
-
Added onload="lang_load()" to the opening "<body>" tag.
-
Linked to a short jQuery-based
script that
implements lang_load(), which I've written.
What the script is doing is go over all elements and if the
"xml:lang" attribute is set, then set "lang" to its value. It's a kludge,
but it works. You can
see the results on my homesite.
2008-06-06: Ubuntu is Dead
As illustrated by
this bug (which got closed as "invalid"), the Ubuntu world has become
infested with red tape, abuse, unfriendliness and hubris. So my suggestion
is to stay a clear mile away of anything Ubuntu and to switch to Debian,
Mandriva, Archlinux, FreeBSD, Fedora, CentOS or any of the other fine
distributions with much healthier communities. Which one you choose depends
on your needs and the situation.
This is especially in consideration of all the horror stories I heard of
Ubuntu Hardy getting hang-up. I have
spoken
against Ubuntu before, but I may have missed the point, and it's possible
the problems I've discovered recently are more recent.
The bug in question was closed because it wasn't a software bug, but that is
also the case for the infamous
bug #1.
In short, Ubuntu is dying now, but with enough determination on the part
of its leaders, it can be resurrected. But this will require a huge inside
motivation, and a lot of serious soul-searching. In the meanwhile, don't
get near Ubuntu.
2008-06-16: Firefox 3 Slogan
I said it on IRC and on IM several times, but here it is on the Web for
the upcoming release of Firefox
3:
Firefox 3 Runs Like a Fox on Fire.
It really does. And it leaks less memory and is very stable. Thanks to the
Mozilla and Firefox developers for all the work they've placed there.
(This time I think the term "a fox on fire", may be inspired by the
tale of Samson and foxes in
the Old Testament, but who knows.)
2008-06-27: Summary of the Firefox-IL Meeting (from my POV)
This is the summary of the Firefox
users' meeting that took place today at Park Hayarkon in Tel Aviv. I slept well today starting
from 1 AM so was annoyed by the 8 PM clock. (I still felt energetic throughout
the day.) After waking up and buying
Bourekas (not enough time for breakfast - didn't even shave), I tried to find
a bus station with the bus going there, but couldn't. So I took a cab.
The cab driver was very friendly. He didn't have a computer at home, but I
was able to explain to him about software and open-source. One of his daugthers
had studied Maths+CS+EE in Bar-Ilan and the other one is a lawyer. We did not
talk about Computers exclusively, of course. I directed him to the Paz gas
station and we ended up at a different one, and had to do a detour. Oh well.
After I got there, I took a detour and slipped on some mud approaching the
Firefoxers. But I was ok (I suppose the grass was being sprinkled with water
shortly beforehand.). There were already a few people there, and we started
chatting. This was an anti-conference, and so we didn't eventually hold any
formal thing like presentations/talks or introductions, but rather chatted,
drank, ate and were happy. We also got some Firefox swag, but pitchefkes (= the
Yiddish word for "small stuff") are not really an obsession of mine.
We had some beers there, and we had water and sodas, and we had some
Croissants. I didn't eat anything, but I drank some water and 7'up. Lots of
fun was had.
Here are some highlights from the conversations:
-
We discussed the super-domains: "http://beauty/" which L'oreal wants
and why people are less likely to want "ugly". But then I said
"http://paris.hilton.is.ugly/" would be cool. Then we discussed
"http://hilton/" and "http://ibm/".
-
We discussed the fact that Israel had 100,000 downloads of Firefox so
far, and that Iran is one of the top-10 countries there. BTW, I've been talking
with a few Iranians on IRC and they've been very friendly to me in spite of
knowing I'm an Israeli. I suppose most of the people in Iran are not the
problem, but rather the regime.
On the ride back (with Ori), I said
that I felt that the Israeli politicians are a joke and do not actually run
Israel. Who does is a good question, but it's possible that Israel just runs
itself, with some entities, like the Military having more influence.
Looking back, I can say, that the Israeli Law still has many bad elements, and
should be changed because it's causing a lot of damage.
-
We discussed the "israeli girls"
search on Flickr and how it is dominated by female soldiers. People told
me it was Digg'ed too. I said that Flickr's diversification algorithm was
probably not as good as Google's, if there is one at all.
-
A bit less political and off-topic, I had a long conversation with a
bio-informatician from Ben-Gurion University. He is using Perl and "R" (because
they are free-as-in-speech and do what he need.) He said their budget is pretty
big and they have a program running on the server with Java and Oracle and
stuff. Then he said that he bought guitars instead of going on trips abroad
and so, like me, remained in Israel most of the time. He seemed nice, and
said he remembers my name and will MSN me.
We discussed some machine-learning algorithms. How he has to optimise over
500 parameters and decide which to include and which not. Naturally the sets
of all subsets for 500 paramaters is huge, and so cannot be done by
brute force. I told him about my greedy algorithm for finding a meta-scan
in Freecell Solver (hi Muli!).
-
I said that I think that non-x86 (= non-Pentium etc.) architectures are
going to make a comeback in servers, and that we'll see more peopple
increasingly buy machines running UltraSPARC, PowerPC, and other
non-x86 CPUs. (More about that later). Tomer noted
that he does PHP software development at work and that such architectures
were where all the big money was made. He said some companies have ancient
IBM PowerPC-based setups which they don't upgrade.
-
There were quite a few girls there: the
Pink Fakatsa, ailaG, and two new girls I didn't recognise. I also met Kobi
Zamir, who is known as the libhocr guy, and he was very nice. He is tall
and slender, and I told him most Kobis I know had a bigger build, and he
said that he felt so too but said he was an exception. We discussed people
with similar names.
Israelis have many common private names, but many names are very common.
The situation is much worse in Russian, where they have less than 20 common
masculine private names.
-
I met Meni whom I didn't
see for a long time in Telux and other FOSS meetings. He said that he's
now working in a different place, which is harder to come to Telux from,
and that he now has a car, which he didn't need when he worked in Hertzeliyah
Pituakh. We discussed the fate of Atelis, where he and other three esteemed
developers worked, and now has been split into pieces, due to bad investors.
I'll probably write some further stuff in comments, but I'm just doing my
duty to people who did not attend. In short - a fun meeting. About 20 people
came and left, and a lot of fun was had. Next time, I suggest doing it on
a weekday during the evening or afternoon, and optionally not at the summer.
A more formal schedule and place would be good too, and I suppose I can
make some arrangements at Tel Aviv University.
I returned home with Ori, and we discussed some stuff, including Shania Twain.
I thought I'd upload some of her songs online, but that's what YouTube is
for, no?
Knock
yourself out with Shania goodness! (or Shania
Badness if you happen to think so.
Stay on fire (foxes)!
2008-06-24: Predicting Three Comebacks
Well, it's a common belief that comebacks in technology don't happen often, but
this time I predict three comebacks. I cannot recommend that you base your
decisions (much less investment plans) based on it, but here they are.
Comeback #1: Non-x86 Machines
Once upon a time, companies spent a fortune on costy UNIX servers or even
IBM mainframes, just to be able to run their operation. Since then, x86-based
computers have become faster and more capable, enough to make people be
able to run Microsoft Windows NT or Unix-based Operating systems on them. This
is now the most popular setup on the Internet for servers (much less clients
and workstations).
However, these x86 machines are not as integrated as their non-x86 equivalents
(UltraSPARC, PowerPC, etc.), are still much slower (due to the limitations
of the x86 architecture vs. the superior RISC family of architectures). This
costs in a lot of time of maintenance, many hardware failures, and sub-optimal
lifetime. It is well-known that there are some PC XT machines which are still
running. However, finding a low-end desktop Pentium I machine that's still
OK is an exercise in futility. And even the high-end x86 servers suffer from
many problems.
Human time is much more costy than shelling some money for getting a good
RISC machine. And with the portability and stability of
the GNU system, and portable and free
operating systems such as Linux, or the BSDs, you can easily run a server
there at ease.
So what can Sun and IBM (and friends) do about it?. They should phase
out their own OSes in favour of Linux, as people have grown to dislike Solaris
(to say nothing of AIX, HP-UX, or greater brain-damages). Don't get me wrong
- Solaris has its place and I'm sure is a fine piece of engineering. But
GNU/Linux is better for trying to promote the hardware.
Sun should also make the OpenSolaris licence
GPLv2 and GPLv3
compatible, to allow it to borrow code from Linux and other GPLed-products
(and vice versa). And they should also start supporting Perl, PHP, CPython,
CRuby, Mono etc. etc. instead of just Java or whatever has been hyped. (Similar
to what Microsoft have been doing recently for its MS-Windows OSes).
All of this will allow selling more hardware, getting more support and ergo
- more profits. I think the transition to non-x86 machines will happen anyhow,
but that will make it faster, and much more profitable for them.
Note that it is not the end of either Intel or
much less the
end of Microsoft. I still see workstations dominated by x86 in the near
future, and both MS and Intel are insanely profitable and have enough time
to re-invent themselves. But I think (and hope) we'll be seeing more and
more non-x86 servers and to a lesser extent workstations.
Comeback #2 : Mandriva
Once upon a time, the
Mandriva Linux distribution (formerly Mandrake) was the de-facto standard
for home Linux desktops. Then vocal Debian people started spreading a lot of
pro-Debian and anti-everything-else FUD, and Ubuntu also became the over-hyped
distribution-of-the-moment. Meanwhile, naturally, Mandriva suffered from
lack of popularity and negative FUD, but also still continued to improve.
Right now, as Ubuntu Hardy (which I nicknamed "Hardly") tends to hang up or
worse so often, and
Ubuntu as
a general rule seems to be infested with red-tape, it seems that there's
an opening for Mandriva and distributions that are very good, but were not
as hyped as Ubuntu.
Here are some Israelis who converted away from Ubuntu lately:
These are all Linux experts and FOSS contributors. It's harder to tell what
the common joe who wants to install Linux at home will choose. But I think
that they will tend to hear a recommendation from the experts, too.
Will Mandriva become the hottest distribution? Maybe not. But it doesn't
matter - Mandriva was never about hype. Naturally, Mandriva has had and still
have its share of bugs and problems (mostly non-critical). But so do most
other distributions, including such whose users were spreading anti-hype
against it.
Writing this from a Cooker (which is the Mandriva equivalent of something like
Debian Testing or Unstable) system.
Comeback #3: Perl
And now we come to the third and final comeback - Perl. You can often here
that "Perl is dead", or that "Perl is dying", etc. It's a matter of image,
more than technology or mindshare. Many people would blame it on the anti-Perl
FUD we can hear from various sources. But I believe the problem has lied in
the Perl world itself. To quote an email I sent:
The Perl technology is in an excellent shape. It's fast, feature-rich,
powerful, easy to use and learn, under active development, relatively
bug-free, with many automated tests, and with many 1st party, 2nd party
and 3rd party enhancements (re CPAN).
However, the Perl community is sick. Not dying - but sick.
This sickness was not caused by external attacks. Instead it was created from
within the Perl community. By a leadership/intelligencia who were more keen on
writing and selling closed books (see
my essay about it)
than on promoting the core Perl technology, documentation and community. It
was caused by an "official" IRC network (which is still the first hit on a
Google search for "irc perl") which is full of ego, abuse and hostility - where
kicking, bannings (and seemingly K-lines) are not only common, but tolerated
and seem desirable. It was caused by administrators and leaders who became too
"busy" (and as a result heavily unproductive) to have time for Perl, which
is what have been placing food on their table. It was caused by central sites
whose source code and data were hidden from the public due to a lot of
unknown bad reasons, and for absolutely no good ones. (We advocate openness
after all).
But most of all it was caused by the many competent and active Perl
contributors who did not take the initiative to replace the existing and sick
infrastructure by a
more independent and decentralised effort.
(I had written
something
relatively similar in the pre-Web-2.0 era as part of my "Usability of
the Perl World for Newcomers Essay". Nevertheless perlbuzz.com deserves
credit for presenting it better, making it public knowledge, and making
it more obvious.)
But the situation is now getting better. There are more independent
developers who have started to set up sites and help resources of their own.
More people blog about their Perl experiences. I'm also going to continue giving
my share of the contribution by setting up
Perl-Speak (Please don't register yet -
the E-mail does not work yet due to problems with my hosting), by
contributing to the FAQs, documentation and wikis, and by helping with
coding. People like me would also like to complement the inadequate,
"official", and completely non-open *.perl.org infrastructure, which has
been suffering from bitrot and neglect.
Lately, I've had the pleasure of talking to many people on IRC who have
decided to learn Perl for various reasons, and some of them are young
or even underage. Along with CPAN and perl5, Perl offers some clear advantages
that no language has.
Again, Perl is also not about hype.
Some people have privately raved about Perl and recommended it to their
friends, but otherwise Perl was promoted quietly, bottom-up and mostly by
word-of-mouth. It's hard to know if Perl will become the "hottest" language
again. But I believe it will still become more popular, even among new
programmers.
2008-07-21: Hardware Log: ATI HD 2600 Pro
Up until this Saturday, my Linux workstation had an old Nvidia
GeForce 4 MX card, whose fan recently died. We decided to upgrade the card,
and I told my father that an ATI card would be preferable over Nvidia,
because AMD/ATI are more FOSS-friendly and released SPECs. So he bought an
ATI HD 2600 Pro card. It's r600, but quite low-end (but still not as much
as the GeForce 4.).
We delayed installing it, but this Saturday, after I had to reboot after the
new kernel update, we got to it. The hardware replacement went surprisingly
well, and then we booted the computer. Everything went well on startup,
and we got the login prompt. Then startx worked right away, as my
Mandriva Cooker installation detected the hardware change and set up all
the drivers accordingly. Mandriva++ .
Not everything worked flawlessly. I had to edit the xorg.conf file to re-add
the settings for the Xkb Hebrew keyboard, and to change the
Bits Per Pixel
from 16-bits to 24-bits. Then I was happy. I'm using the
RadeonHD open-source drivers,
which don't really do 3-D yet, but since there are SPECs for them, the
situation is expected to improve.
I'm so glad I've now escaped from Hang-vidia-land, and have a card from
a company that plays along with the open-source ideology. I hope to contribute
further to the open-source drivers by reading the SPECs and writing code,
and hopefully I would be able to understand what I need to do better than
with Nouveau. In any case,
since I'm not obsessed with "gamer" computer games and am not using the
high-performance 3-D/2-D effects in much, I can survive just fine using
the non-3-D enabled drivers. (Just as I used the "nv" driver before the
upgrade.).
2008-08-02: Hacktivity Log: CMake and JavaScript XSLT
In
an
email to the fc-solve-discuss mailing list, I detailed recent work
on Freecell Solver. I added an suite of automated tests, which in turn allowed
me to perform the conversion of its build-and-configuration system from
the GNU Autotools (Autoconf, Automake and Libtool a.k.a Autohell) to
CMake. I ran into a few problems in the
process of the conversion, but it took less than 2 days, which was probably
much less than its Autoconfisication of Freecell Solver took, excluding
the constant maintenance of its Autotools configuration and random breakages.
You can refer to the trunk
for the present state of the CMake configuration and the
conversion-to-cmake
branch for the gradual CMakification process.
In any case, now the source archive is smaller, configuring it is faster,
make runs faster, and I have a configuration GUI. In the process, I also
added support
for the Qt GUI to the Mandriva CMake. There are probably still some
quirks in the config+build system, but all in all, I'm very happy.
I also ran into a problem that Firefox completely refused to display the
https:// URLs of svn.berlios.de, without letting me add an exception. As
it turned out, the certificates were probably changed, and I had to manually
delete all *.berlios.de related certificates, and then add them
with exceptions.
Next - yesterday I read
about jdavidb's problem
of finding differences in two versions of a Fedora RPM, which prompted me
to
find
out about Fedora's packages' version control repository, which I
did. Then I ran into a comment in that page about
the Better-SCM
comparison, which inspired me to do further work on it.
I decided to finally experiment with
the Google implementation
of XSLT in JavaScript in an attempt to customise the output of the
XML in the client-side. I wanted to fetch the contents of the XML document
and XSLT stylesheet from the server which involved doing AJAX using
jQuery. It took me a long time to get right, because I didn't realise the
jQuery $.get(...) callbacks were asynchronous and executed at their
own time. I guess it's called "Asynchronous JavaScript And XML" (= AJAX)
for a reason.
Then I got it running, but then ran into a Google JS XSLT limitation of
"not implemented: key" (probably the XSLT "<xsl:key />" feature which
I'm using). Still, I gained a useful experience working with AJAX, JavaScript
and jQuery.
In regards to August Penguin - I don't have a lot to report, because I was
too tired due to lack of sleep in the previous days and so left early, right
after the opening sessions.
All that put aside, I should note that the Cooker version of Mandriva (which
is its development/experimental version), which
I'm using now is now relatively broken on this machine, and many applications
get hang for noticable periods of time or even completely. Part of it may
be due to the switch from KDE 3 to KDE 4.1.x, but maybe there's something
wrong on this machine's configuration, which I cannot get to the bottom
of.
2008-08-03: Review of KDE 4.1 on Mandriva Cooker
Since KDE 4.1.x was recently
released and was made available in Cooker, I decided to give it
for a test drive. So I created a new UNIX account logged in to the console
and ran startx. KDE 4 started, looking like I was used to, and I started
making it behave like I was used to.
Here are a few problems I encountered, with solutions or workarounds, based on
my memory and my IRC logs:
-
I had problems putting an icon of Konsole, the KDE terminal emulator, on
the panel. The problem was that the KDE "K" menu was configured in Mandriva
to be the "Classic Menu", where the right mouse button does not work.
After switching it to the Kickoff style, I could right-click the menu item
to add it to the panel.
I don't understand why the right-mouse button does not work in the classic
menu, or why I cannot drag&drop menu items to the panel.
-
Right-clicking on an applet or application launcher in the panel does not
give one an option to move it. The only way to do it is by clicking
the GNOME-logo-like icon at the right of the panel and entering into
panel-editing mode. There is a "Move" option in the context menu of
panel icons in KDE 3, so I wonder why it's absent.
-
There doesn't seem to be a system tray by default in the panel, and
after I added it its display of the akregator icon is a bit awkward.
-
There aren't any tooltips for any of the icons in the panel, so it's hard
to know what they are all about (One needs to click them to find out.)
-
I had problems finding the equivalent of the KDE Control Centre (kcontrol)
in KDE 4. Apparently it's called System Settings in the menus, and is invoked
using "sysetemssettings".
-
I had problems making sure the taskbar only displays windows from the current
virtual desktop.
Apparently, one has to click on the taskbar in a place where "there's no
application" (i.e: in the space between applications) and then choose
"Task Manager Settings", where the option is present.
This option is not present in the KDE 4 "system settings" because it's
relevant only to an individual taskbar, of which there can be many. Still,
I would like to see a button on the left of a taskbar invoking a context menu.
-
The "Expandable" property of the "Details view" of Konqueror does not get
preserved across invocations of Konqueror, despite the fact that I pressed
"Apply" and "OK". This seems like a bug.
-
Finally, on Mandriva, KDE 4.1 does not make use of KDE 3's data, so
KDE 4.1 applications like KMail, Akregator, etc. are useless to me.
Of course, KDE 4.1 still seems likable, and I enjoy using it. But I feel
like they need to work on better usability. I am not a newbie - I'm actually
a computer geek and an experienced software developer - and I felt
like I could not make sense of some of the new UI there.
I'll try filing bugs in the KDE bug-tracker about it next, and see where it
leads me. Hopefully these usability problems will be fixed in KDE 4.2.
2008-08-09: Tip: Fixing rpm hangup as
root on RPM-based Linux distributions
This is more of a memo-to-self than a tip, but it may prove useful to others.
When rpm (the
package manager for Linux) as root gets stuck right at the start without
doing anything, try the following:
-
Kill all rpm-related processes.
-
Move away /var/lib/rpm/__db.* to a temporary location.
-
Run rpm --rebuilddb.
That's it! It already fixed my wedged RPM database twice, so I'm putting
this advice here so I won't forget it.
2008-08-21: Recent Hacktivity Summary
Well, I added more tests to
Module-Starter,
and while I was writing the tests, I discovered some bugs in it. So now
there's
a
patch with the tests and the bug fixes and
another
one with only the bug fixes. Both of them are unapplied.
Having been trying to learn Ruby recently, I decided to write a
Conway's
Game of Life implementation for it. It took me most of one evening to get
it working and then part of the other day to make it more idiomatic with the
help of the people on Freenode. It felt good in a way.
Then I decided to finally see why my
Test-Run failed
many tests recently. As it turns out, this was due to broken
TAP-Parser
compatibility: the skip message in skip_all field now accepts only
"SKIP " and not something like "skipping: ". Fixing the Test-Run tests
fixed it and I released new versions.
I also noticed that many of the Test-Run-CmdLine plugin had the following
pattern:
sub _init
{
my $self = shift;
$self->NEXT::_init(@_);
$self->add_to_backend_plugins("AlternateInterpreters");
}
So I overloaded the construction function, only to add a plugin. I decided to
fix this by doing an accumulation of such plugin specifiers in every plugin,
and so got only:
sub private_backend_plugins
{
my $self = shift;
return [qw(AlternateInterpreters)];
}
Which is much cleaner and less intrusive.
Next on my plate was Freecell
Solver. I decided to work on make_pysol_freecell_board.py
which is a Python script that generates the initial boards of PySol. The
script was crudely written to begin with and it became even cruder as
time went by. I decided to give it a serious face-lift. So what I did was:
-
Convert the columns (of the card games) from strings to arrays. Each card
was stored there individually.
-
Convert the cards to a class, instead of integers and strings. Create similar
classes for managing columsn and boards.
-
Extract many functions and methods.
-
Create a class to manipulate the various game types, and forward the different
logic based on it.
-
Pythonised the script by employing some Python paradigms.
You can find
what
I have so far in the repository. It's much better than what I started
with two days ago. Writing such Python code now seems more fun than I recall
it, and I actually enjoyed it.
In regards to
SMOP, they
convinced me to install Ruby-1.9.x under a prefix, which I did, but then
it yelled at me for not having a Haskell cabal thingy. Turns out that
the Pugs Makefile.PL installs it somewhere under the home-directory, which
I didn't want to happen, because I want to keep it tidy. Again, this
reminded me of Drivl.com
recipe on how to make square corners, and I decided to give up on SMOP
again.
And I should note that I was able
resolve
a long-standing problem I had with XML-LibXML/XML-LibXSLT on my Mandriva
system, and now I simplified the XML-Grammar-Fortune build-system.
I also spent some time writing a backup system for some of the Israeli
MediaWikis that I manage. This involved a bunch of Perl scripts.
So - Perl, Ruby and Python - all in a few days work. Cheers everybody.
2008-09-10: Tip: Handling mailto: Links in Firefox 3
Resources such as this
one explain how to setup Firefox 2.0.x to use kmail to send email from
"mailto:" links. However, compatibility with this scheme was broken in
Firefox 3, as evident from
Firefox
bug #428658 - "network.protocol-handler.app.mailto" users need to
edit/remove mimeTypes.rdf".
The solution I found is to access
"Edit → Preferences → Applications → mailto" and select the appropriate
application there. This seems to work for Firefox 3.
2008-09-14: Recent Hacktivity Summary
Another hacktivity summary - what can I say?
-
WebGUI -
I decided to try installing
WebGUI again. After preparing
SRPMs
for all of its CPAN dependencies and installing them, I was able to tweak its
and Apache's configuration files based on the instructions there to get it
to run. However, then when I tried to configure it, I found that it go stuck
on the colours configuration. So I temporarily gave up.
-
Readline Replacement:
I began working on Term-Eatline, which was supposed to be a Perl prototype of
libeatline, which aimed to be an enhanced BSD-licensed GNU readline replacement.
After starting writing it using curses, I ran into a
"Curses
Problem in Detecting Backspace". I found a solution in the CPAN Curses::UI
module, and went on. Then I ran into another problem with detecting Alt+Key
combinations, and decided to see how readline is doing it. As it turns out,
it doesn't use Curses after all.
I decided to see if I can now use tcsh's code to bootstrap eatline, and
ran into a tcsh
bug which refused to even let it run. I wrote a patch for this based
on the discussion and some debugging effort, and posted it on the bug report.
Afterwards, I contacted a tcsh representative about the bug, and after
telling him my motivation for contacting him, he referred me to
libeditline which is a BSD-style
readline clone. Guess I now have something substantial to start from.
-
Browser bugs:
I ran into a few browser bugs on my current Mandriva setup. For once,
Firefox freezes/stalls
for many seconds when loading most pages (which seems to happen
only on my system), and
there are many
delays in the font resizing.
I'm disturbed by these problems a bit, and don't know what causes them.
-
Gringotts:
Rorschach contributed a minimise-to-a-tray-icon patch for
Gringotts. The patch still has
some issues, but he is working on it.
-
MediaWiki:
I decided to contribute to
MediaWiki, and decided
to start by running the test suite. It failed. As it turned out, the test suite
is currently broken, and the MW's developers method of ensuring nothing is
broken is by uploading it to the Wikipedia and testing it there.
I decided to fix the test suite. After fixing a few minor breakages and
submitting patches for them, I ran into a very puzzling bug. I decided to
isolate it by gradually removing more and more unnecessary code, while still
being able to reproduce it. I started doing that on the test suite, and
found out that I needed to incrementally copy more code from the MediaWiki
in order to trim it. After 174 such iterations (and intermediate versions
occupying 7.9 MB of my hard disk), I was able to find what the problem was
and write a patch. The problem was that the database handle was initialised
in an old way.
Now the test suite still fails especially with some tests in the
t/inc/Parser.t tests, but I'm hopeful. I am disappointed that the MediaWiki
guys don't take automated testing more seriously.
-
CPANPLUS-Dist-Fedora: Someone contributed a patch cleaning up CPANPLUS-Dist-Fedora
so I needed to test it on my Fedora VirtualBox VM. This turned out to be
harder than I expected. When updating to the latest version, I ran into
a missing ldap .so dependency. I found out I can eliminate this problem
by removing gnupg2 and all of its dependencies including kdepim. So I was
able to update it after all.
Then I decided to use icewm instead of KDE, and wanted to set the wallpaper.
And then it turned out that infuriatingly, xsetbg and xli were not available
in Fedora anywhere for some reason.
Eventually, I was able to test CPANPLUS-Dist-Fedora there. The patch added
a change that caused loading the package to fail on all system without
an "rpm" executable, so I released an update as 0.0.3, which also eliminated
the use of two unnecessary modules.
-
CPANPLUS Typo fixing: while working on CP-D-Fedora, I found out that
CPANPLUS had a line with "it's" instead of "its". So I grepped the source
for similar cases and submitted a
patch that hopefully corrects all of them.
-
I asked where
the previous YAPCs were on yapc.org and was replied that it wasn't
implemented yet and was referred ot the source. True to my mantra that
"One shouldn't complain unless one is willing to implement it himself.",
I wrote
a patch to create such a page. Most of the foundation for it was already
there.
-
Bug in Cwd.pm: I received a
strange failing report for one of my CPAN distributions with an error
that I could absolutely not reproduce on my machine. As it turned out, the
problem was that on that machine they used the pure-Perl version of
Cwd.pm instead of the binary one that I used, and they differed in
their behaviour.
I wrote a
patch, with a test case and a fix and submitted it.
-
Finally, I prepared
Acme-CPANAuthors-Israeli.
The first version didn't list the dependencies in the Build.PL, but this
is fixed in version 0.0101.
At the moment, I feel a bit disoriented and don't know what to do further.
I think I'll help some more with Firefox bug triaging.
2008-09-16: Tip: Dumping the History of Google Code's (or any Remote) Subversion Repositories
Consult
this FAQ on Google Code for how to use svnsync to clone a remote Subversion
repository so you can have a copy of its history. I was told on IRC that
it has the same end result as using svnadmin dump on the physical
repository.
This post was supposed to be a rant on why I'm not hosting my projects
at Google Code because they don't have a way to retrieve the dump/history
of the Subversion making it a case of vendor lock-in. While this had been
true in the past, I discovered it is no longer the case today, while looking
for the relevant FAQ entry.
So kudos for Google and the Subversion people for fixing it.
2008-09-18: Shell Variable Injection
GuySoft writes
in his blog about Unix shell commands to convert video. In the shell
commands, he expands variablse like -srate 22050 $infile -o
$outfile".flv", without putting them inside double-quotes. In this
post, I'd like to demonstrate why this is so dangerous.
When the shell expands a regular variable outside quotes it breaks into
words separated by whitespace and then evaluates every word separately. To
use a variable as a single token, one must enclose it inside double-quotes:
"$MY_VARIABLE".
Now, let's demonstrate what can happen when we don't do it properly. I'm in
the home directory of a user called "shelltest on my machine". Let's see what
we have there:
[shelltest@telaviv1 ~]$ ls
Desktop/ test-shell-injection.sh* tmp/
Freecell-Solver/ test-shell-injection.sh~
[shelltest@telaviv1 ~]$ du -s Freecell-Solver/
86M Freecell-Solver/
[shelltest@telaviv1 ~]$ cat test-shell-injection.sh
#!/bin/sh
mkdir -p $HOME/Backup
cp $1 $HOME/Backup/
rm $1
[shelltest@telaviv1 ~]$
We have the populated "Freecell-Solver" directory, and a shell script called
"test-shell-injection.sh" that uses $1 (its first positional argument)
unsafely.
Now let's abuse it:
[shelltest@telaviv1 ~]$ ./test-shell-injection.sh "-fr $HOME/Freecell-Solver"
[shelltest@telaviv1 ~]$ ls -l
total 8
drwxrwxr-x 3 shelltest shelltest 28 2008-09-18 14:14 Backup/
drwxr-xr-x 2 shelltest shelltest 6 2008-09-18 14:05 Desktop/
-rwxrwxr-x 1 shelltest shelltest 58 2008-09-18 14:11 test-shell-injection.sh*
-rw-rw-r-- 1 shelltest shelltest 60 2008-09-18 14:09 test-shell-injection.sh~
drwx------ 3 shelltest shelltest 28 2008-09-18 14:07 tmp/
[shelltest@telaviv1 ~]$
Voila! Freecell-Solver is gone. I could have also said
./test-shell-injection.sh "-fr $HOME" to delete the home directory. If
the script does something like for I in * ; do ... # Something
with $I .. done, then a malicious user can supply a file called
"-fr $HOME" (where $HOME is the value of the home directory) in the input to
delete the home directory.
In short: when writing shell scripts, always enclose variable expansion in
double quotes, unless you are certain that they don't contain whitespace
(which implies that you generated them yourselves), or you want to use them
as a list (which also requires that you generated them yourselves).
2008-09-18: OSDClub Tel Aviv Meeting: Ori Idan about the Semantic Web
The Tel Aviv Open Source Developers
Club (OSDClub) (formerly Telux) will hold a
meeting on 21/September (next Sunday). Ori Idan will deliver a presentation
about the
Semantic Web.
Ori is the director of the Israeli branch of the
World-Wide-Web Consortium (W3C), and is a very good presenter, so it is
recommended to attend.
The meeting will take place at 18:30 in the Schreiber
Maths and Computer Science building of Tel Aviv University, room 008.
Attendance is free of charge and everyone are welcome.
2008-09-19: Impressions from Ruby
Ahoy mateys! Arrrrrrrrrr! Happy Talk
Like a Pirate Day!. In the piratey spirit, I decided today to get my
act together, and start working on a project I've intended to write. Since
I've been learning the Ruby programming language lately, I decided to write it
in it, to learn more about Ruby and to experiment with it. In this post
I'd like to blog about some of the facts I discovered that surprised me in
Ruby.
First of all, I discovered that Ruby regular expressions don't have Perl 5's
"\G" and /g feature that can be used to incrementally parse a string using
regular expressions. The closest thing I could find was using .sub! (with the
empty string), which does an in-place subtitution of a regular expression
The second thing I discovered was that Ruby did not have ++ or
-- operators - one has to use += 1 and -= 1.
When Ruby tried to parse a line containing "++" it yelled at me with
an obscure error, that caused me to need to incrementally comment-out
code until I found out what the problem was.
Another fact that disappointed me was that the Ruby debugger (ruby -rdebug)
does not seem to have conditional breakpoints (i.e: break at a certain line
only if a condition is met).
Finally, when I tried to do .inject { BLOCK } ("inject" is Ruby's
list reduce / fold method), without a starting element, I discovered that
it used the first element as a starting point instead of 0 or nil or whatever.
Doing .inject(0) { BLOCK } in my case fixed it. This fact is
documented in the documentation.
Otherwise, Ruby seems likable and I've been feeling I'm productive in it. I've
been using RSpec for writing my automated
tests. It's a so-called "Behaviour Driven Development" library,
which is the
same as Test-driven development, but RSpec is still nice.
2008-09-27: Tip: Circular Symbolic Links in Fonts Directories Causing Slowdown
I noticed a slowdown
recently on my Mandriva Linux Cooker system where X GUI applications would
stall
for a long time and gdb showed that it happened in
fontconfig calls. It happened with all
user accounts on the system, including new ones and was not fixed by upgrades.
Furthermore, other people who used Cooker did not have it.
Well, today I finally
found a
solution. Apparently, some of my font directories, contained symbolic links
to themselves, which confused fontconfig and caused it to scan them times
and again.
To fix the problem, run
strace -e open -o fontconfig.strace fc-cache -v, and see if it
excessively scans more and more sub-directories under the directory
paths. Then move away the offending symbolic links to say
/usr/share/REMOVED-fonts.
Now konsole starts instantly, KDE 3 apps no longer become unresponsive when
the file with the unicode character, firefox stalls less, and font zoom there
is faster. I'm finally happy again.
2008-10-07: Autumn Cleaning
I like to keep my queue of incoming items under control. Recently I noticed
several vectors that I did tend to, and decided to deal with them as well.
After I dealt with them, I've been a little negligent, but now they're
fully under control again:
-
Home Email Inbox (KMail) - down to zero messages.
-
GMail's Inbox - down to zero messages.
-
GMail's folders - down to zero unread messages in all of them
except in the Mandriva Cooker and Mandriva Bugs folders, which I don't need to
keep at bay. I previously left many unread messages there, but now I decided
to keep them at bay as well.
-
Bookmarks in Firefox's Main Bookmarks Menu - down to zero items.
I often bookmark them there to read, sort or otherwise deal with later.
-
Bookmarks in Konqueror's Main Bookmarks Menu - down to zero items.
-
Unread web feed in my web feed aggregator (Akregator) - down to
zero items.
-
Files in the root of the home directory - down to zero.
Did I miss anything? See this Perlcast Interview with Tom Limoncelli on
"Time Management for System Administrators" for more information on
keeping your incoming items at bay.
2008-10-10: Firefox Tip: Disabling the Drag & Drop Previews
In Firefox 3, when one drags and drops images or text, they are given as a
large preview around the cursor. I had found it annoying and sought a way
to disable it.
This tip came to the rescue after a short Google search
- just go to about:config and set the
nglayout.enable_drag_images to false.
Cheers!
2008-10-11: Reply to fxn Regarding User Was Successfully Created
fxn writes
in his Advogato blog about the fact that many web apps confirm that
a user was successfully created:
Why so many webapps confirm to the user that some action was successfully done?
Of course it was!
Look at your desktop applications, your editor does not bug you saying "File
saved!" constantly, iCal is indeed almost completely silent. You warn the user
when the disk has run out of space, right?
I think those messages come from the insecurity the developer feels about the
amount of failure points between request and response. Perhaps some are just
repeating the pattern seen elsewhere. But that's not the user's business, you
warn when you fail.
I disagree with this. The web is a different user-interface medium than
desktop applications where different rules apply. If you don't give
confirmation on the web for successful actions, the user will wonder if they
were successful or not. So you do need to do so.
For example when submitting
this form,
I am brought back to the same form page. So I don't know whether it was
successful or not or if it got sent at all.
So a web application writer needs to write a confirmation that many
successful operations took place, and to error if there was a problem.
2008-10-11: The "I Have Nothing to Hide" Fallacy
A common pattern we hear on the Internet in regard to privacy or
security is "I
have nothing to hide" - no one will want to target me because they'll
gain nothing from doing so. Today let me tell you a story that shows why
this is not true.
A few years ago, I was using the same easy-to-remember password (which
was only 6-letters long) on most of the sites I had accounts on. One day,
I received an email from
Freshmeat.net asking me if the
fact that I had changed the description in the record of
Freecell Solver there to "Freecell Solver is a useless
100% ANSI program that automatically solves games of Freecell", indicated
that it was not worthy of inclusion there. This surprised me because I
naturally didn't modify it like that nor intended to.
After talking with the admins of Freshmeat, I realised that someone logged
in to my account, and submitted the malicious update for inclusion. They ended
up giving me his IP, which was in Israel's Netvision ISP (while I'm subscribed
to a different ISP). Now, this change
was pretty innocent, but naturally, now that he knew my shared password,
the possibilities for him were endless. As a result, I went on a concentrated
spree of changing that password to new, different ones in all the accounts
I created on the Net with it. I made a smarter use of
my password manager and eventually
discovered the auto-remember-passwords feature of browsers such as
Firefox and Konqueror, and solutions such as
OpenID.
There's no good excuse to compromise on security. Do you have a bank
account and access it online? If you're not careful enough, a malicious
attacker installing spyware on your PC might empty it. So you say to yourself:
"What does he have to gain from me? I only have $10,000 there.". Maybe you do,
but if he empties hundred or thousands of accounts like that by writing
a robot, he'll become rich, so he isn't likely to not to target you.
And some people are keen on doing random vandalism with your online presence,
like the one I mentioned, who may have also been
trolling
my blogs. Therefore, make sure you're as safe as possible. This
incident was all I needed to become more careful, and I hope you now realise
that, as well.
2008-10-19: Three Tips: Perl with Vim, Subversion Problem and mailto: links in Firefox
First of all, my home-site's back
and you're gonna
be in trouble! Otherwise, this entry contains three technical tips that
I collected recently:
-
Perl and Vim - sometimes one encounters a case where he needs to handle
many errors in his Perl script. I encountered a similar case, when I had to
work on a script that was written without "use strict;" and "use warnings;"
and shouted at me senselessly when I added them.
To greatly help with that install perl-support,
and look at its Perl → Run menu - there's "update, run script" (Ctrl+F9)
and "update, check syntax) (Alt+F9) there which place the errors in the
quickfix buffer and allow to quickly navigate them.
-
Subversion and Apache - I recently encountered this error when trying
to mkdir the first directory in an empty Subversion repository, that
was served using Apache:
shlomi:~$ svn mkdir -m "Creating TO-DEL trunk" http://localhost:8080/svn/TO-DEL/trunk
svn: OPTIONS of '/svn/TO-DEL': 200 OK (http://localhost:8080)
shlomi:~$
This surprised me because my client was up-to-date and the service used the
same libraries, and it worked with other repositories. As it turned out
the problem was that the Location entry in Apache's httpd.conf looks
like this:
<Location /svn/TO-DEL/>
DAV svn
SVNPath /var/svn/TO-DEL
</Location>
The problem here is that /svn/TO-DEL/ in the "Location"
tag has a trailing slash which confuses Subversion. Eliminating that issue
solves the problem.
-
Firefox and mailto - if "mailto:" and other non-"http:" links
stopped working in Firefox, make sure that
NoScript is upgraded to the
latest version. There were problems with some recent versions that broke
"mailto:", "ed2k:", etc.
2008-11-01: Why I Hate KDE 4
Here are the reasons why I hate KDE 4:
-
It looks like shit. Seriously a huge step backwards from KDE 3. The
colours are annoying. The icons are hard to distinguish and ugly. I have
no idea how to make it look better.
-
Plasma leaks memory (see this bug report),
making me having to restart the desktop after a few days.
-
The buttons order is not Windows-like. Yes, they reverted to the
GNOME/Mac-o-Sucks order by default. I have no idea how to change it to
the Windows order which I like.
-
The New Konqueror/Dolphin File browsers are slow and dysfunctional.
They are much slower to scroll than their KDE 3 equivalents, and
typing the first
letters of the filenames there to jump to it is dysfunctional.. I have
to use Konqueror 3.5.x to enqueue files into Amarok.
-
Amarok 2.x is annoying. It looks ugly, the file entries have a huge
height, the context tab is gone, and it is generally dysfunctional.
-
The Konsole tabs are now dysfunctional. There is no button to open
a new tab, and the right click on them doesn't do anything. Both of these
worked in the Konsole of KDE 3.5.x.
-
No different wallpaper on each virtual desktop. See
this. This is
actually a regression from KDE 3.
-
The New Kopete is Dysfunctional. Its icons are annoying and
hard-to-distinguish, I'm playing whack-a-mole with the popups of the MSN
plugin, it crashes on exit,
the
input box is left-aligned in Hebrew,
and the Jabber chat
is now unusable.
To sum up, KDE 4 is dysfunctional, ugly, buggy and a huge step backwards.
I temporarily switched to KDE 3 because KDE 4 made me irritated and edgy.
2008-11-04: Tech Tip: Finding the Email of a CPAN Tester
I'm writing it here so I won't misplace it in a "Know where
the ledge is..."-style. If you need to find the id of a CPAN tester based
on his report, use the
"Find a tester" form on cpantesters.org that allow you to enter the ID (
or the URL) and find the tester. It took me a while to find it today
after I misplaced it and needed to contact a CPAN tester, and Google
was no help.
2008-11-05: Tech Tip: Sorting RPMs by Installation Date
In order to sort the installed RPMs (RPM Package Managers) by installation
date on an RPM-based Linux distribution, use the following command:
rpm -qa --qf '%{INSTALLTIME} %{NAME}\n' | sort -n | \
perl -MPOSIX -lpe 'my ($t,$p)=split; $_=POSIX::strftime("%Y-%m-%d",localtime($t))." $p"'
You can change the format of the strftime format to something else if you want
more than the date.
2008-11-22: Tech Tip: Changing the Dialog Button Order (GNOME-style vs. Windows/KDE style) for KDE 4 Applications
On my Linux system, I had a problem that the order of the dialog buttons
on the dialogs of KDE 4 applications were GNOME-style (e.g: [Cancel] [OK])
instead of Windows/KDE-style (e.g: [OK] [Cancel]). The solution I found
was to change the Widget style (in
"systemsettings → Appearance → Style → Style → Widget Style") from Clearlooks
to Plastique. However, no one seems to know for sure what determines the
actual button order.
Refer to this bugs.kde.org
bug report and this
Mandriva bug report for more discussion.
2008-11-24: Tech Tip: Setting MSIE 7 as the Default Browser on Windows
I should note that in the case of this tip, I don't know why it works whereas
everything else I tried didn't, but it worked for me. Your kilometrage may
vary.
At my home, we have a Windows XP computer with Microsot Internet Explorer
7. After I installed Firefox, and made it the default browser for my account,
it became the default for other accounts as well. Furthermore, setting Internet
Explorer as the default in its configuration tab
(under "Tools → Internet Options") did not work, as Firefox remained the
default browser.
The way I resolved it (eventually) was by also enabling the checbox that
instructs Internet Explorer to warn on startup if it detects that it's
not the default browser. Then and only then, it became the default browser
again.
I have no idea why it worked for me, but it did.
2008-11-24: Report on the Last OSDClub Tel Aviv Meeting
Yesterday, the Tel Aviv Open Source
Developers' Club held the first meeting in the 2008 Welcome-to-Linux
series for the "Mini-Intro" presentation. I arrived to the place early as
usual, and there was no one in the class-room except for some students who
came for the previous class. They asked me what is going to be there, though.
The attendees arrived shortly afterwards. Then some minutes before the
presentation, I called Eddie, who was going to give the talk on the cell-phone
and asked him where he was. He said he forgot all about the talk, and asked
if I could find a different lecturer instead of him. Luckily, Ori was there
and we went over the slides in preparation.
Ori gave a good presentation, in my opinion, and some people from the
audience asked him questions, and I and other people gave some commentary. Then,
some people arrived while the talk was in progress.
During the break some people asked me about the possibility on soliciting
some presentations that they were interested in. One of them told me he came
for a presentation about the command line, and was less interested in all
the GUI stuff which he was already mostly familiar with. Another asked if
we could give a presentation about SELinux, NIS or similar topics related
to system administration, because he was studying for his RHCE. We'll see
what we can do about these requests.
I forgot the camera at home, so I couldn't take pictures.
Then I ushered everyone out of the room, and those that were interested went
to the café. I went with two other guys, who turned out to be Biology students
- one of them talked on his cellphone all the way and while we waited for the
others to come, but I could talk with the other guy. He asked me which
programming language he should learn first and I told him the conclusions
from my Introductory
Progrmmaing Language essay. We also discussed distributions.
Then Galia and Kobi (who went to re-park Galia's car) found us, and said they
had thought we already were sitted in the café. We found a table, but then
decided it was too low to be effective, and so switched places. Then we
ate, drank and talked. We exchanged many jokes about Apple and stuff
like the iRack
and "iLand", which is Apple's CEO's private island. (One of our guests
made the latter one on the spot.)
In short, a lot of good time was had. Next presentation is next Sunday
about the Installation Process. Hope to see you all there.
2008-11-06: The Seven Deadly Sins of Arc
Arc is Paul Graham's dialect
of LISP,
that aimed to finally create a usable dialect of Lisp. It was released some
months ago, and I played with it and even contributed some code to the
Anarki Arc effort. In this post, I'd like to note why I think it has proved
to be mostly irrelevant and so quickly went into obscurity. People who
wish to create their own new popular programming language would be able to
use this list to learn what they shouldn't do.
The Seven Deadly Sins of Arc are:
-
Not Many Exciting Features and Paradigms - Arc may be more brief
than other Lisp dialects, and removed a lot of extraneous syntax, but it
doesn't seem to sport many exciting features and paradigms. It seems
like a Scheme-like-language only slightly better and slightly worse at
the same time.
Graham made a call for advice on his site, looking for ideas no how to improve
Arc, but it seems very few of them made it into the final language.
-
Standard State-of-the-Art Features are Missing - Arc has no
built-in Object-Oriented Programing (OOP) system and it also lacks
namespaces. Both of these features are part of the absolute minimum
expected of any language today.
-
No Version Control was Used for Arc - when Graham published
arc0.tar (more about it later)
on his site, it was just a tar. He didn't publicise the URL for a version
control system that he used for work on Arc. And
in this article
he claims he has recently been using a mostly-disconnected computer to do his
work, which indicates he's oblivious of the importance of using a remote
version control repository for constant work.
I don't have a lot of confidence in a program that's not kept under
version control, and neither do most professional programmers.
-
No Automated Tests Suite - Arc shipped without any automated tests.
If an obscure feature was broken, then Graham could not find out. Nor did
he have the specification-feature inherent in the automated test suite. Also
the lack of Test-Driven-Development (TDD) for its development is distressing.
Maybe that's what Graham calls exploratory
programming…
-
Basic Usability is Missing: there was no way to do ./arc
filename.arc to execute a file directly from the command line. I
had to add it to Anarki Arc, myself.
-
Weird Versioning - the first released Arc version was arc0.tar.
The second was arc1.tar and the third was arc2.tar. Normally,
one would expect that version
1.0 of an open-source project would be very mature, much less version
2.0, but
in Arc's case they were incremental releases, released after a short time.
So the versioning used by Graham is strange, not very conventional and
confusing.
-
Brevity Taken to Non-sensical Extremes - Arc tries to be
as brief as possible, which is good, but it sometimes defies common sense
and convention. For example the (not) function in Arc is (no)
so the code reads (if (no a) (pr "a is false")) (is that even
English?).
-
Bonus! No line numbers in errors - that's right, when something goes
wrong in an Arc progam, then you don't get a line number where it occured. How
can one write anything but toy programs this way? Of course, given that
executing files from the command line was not supported, it may not be
very surprising.
In short, if you were hoping for a usable and modern dialect of Lisp, then
Arc is not the answer, and won't be without a lot of work. I was
told Clojure is much better than Arc, which
I have
little experience with, but may be better. (Although the fact that it targets
the JVM is somewhat of a turnoff).
But Arc, as it currently stands, is not the answer. Graham had a lot of time
to invest in polishing Arc before its release, but it's not polished at all.
The initial flurry of interest in Arc seems to be largely gone now, and it
seems unlikely that it will ever mature into something professional and
usable. Arc gave me some ideas on how to create a more usable Lisp, but it
otherwise lacked essential feature and is still lacking. I'm going to look
elsewhere for a usable Lisp dialect.
2008-12-20: Tech Tip: Enabling Audio Notifications in KDE 3 Applications
For a long time, I didn't have any audio notifications in KDE 3 applications,
including not in such applications as Kopete, where people keep sending me
nudges. Today, I was finally able to fix it. Here's how I did it.
Run kcontrol, and go to "Sound & Multimedia → System Notifications". Now
notice the "Player Settings" button on the bottom-right corner of the screen,
and press it. Switch the option to "Use an external player" and enter
"/usr/bin/mplayer", or a similar capable non-interactive audio player. Press
the "OK button" and say "Apply" in the main KControl window.
Sound notifications should now work, or at least they do on Mandriva Linux
Cooker, where the combination of PulseAudio, artsd, and other curses like that
messed up KDE completely. Hope it works for you.
2008-12-26: Eventful Week
The week before this one (starting on 14-December) was eventful, with one
event on each day. On Sunday, I had a
Tel Aviv Linux
Club/Welcome-to-Linux meeting. Ori Idan gave his now regular "Living in
the Open-Source Community and How to Get Help" talk. There were about 10 people
there (less than previous times we held the series) but it went well. After
the meeting, three of us went to the local university café. At the end of
the meeting Galia spent a lot of time showing off the "cool factor" of her
Mac computer. There were some cool things there, but not enough to convince
me that Apple
wasn't Evil. One thing she showed was that you can take a photo using
the built-in-camera and drag-and-drop it to an MSN conversation to send it.
(As opposed to saving it to a file and then sending the file - the "Windows
way".) This seems sensible enough to have been done in Linux and Windows
already (didn't check, though), and on the other hand, I couldn't open
the photo that she sent from the Mac on my Linux Machine.
In any case, the next day, Monday, I gave a Perl lesson on MSN Messenger
to my Perl pupil. I find it hard to interact with her remotely without
the proximity, and due to the fact I don't have access to her computer.
Another thing that made it more difficult was the fact that I didn't have
sound notifications on Kopete which I solved since then.
On Tuesday, I went to the Open University center near Tel Aviv University
to meet my pupil for paying for the MSN lessons, and then went to
the doctor to get presecriptions for medications.
On Wednesday, I had a job interview. It went pretty well, but I still don't
know if I got the job.
On Thursday, I went to a meeting of the Herzeliyah Linux club. I got there
early and spent some time chatting with people who came to see the
presentation. I didn't even notice the talk started late, until the organiser
and the lecturer arrived. The talk was about the various ways to boot
a Linux system, and was nice. I returned home after the talk, because no
one wanted to go to a café or restaurant.
I hoped I could rest during Friday, but instead ran into several bugs in the
comptuer. Kopete of KDE 3 stopped working for some protocols, due to a missing
qca1, which turned out to be gone for good from Mandriva Cooker (as part
of their general depracation of KDE 3). I had to
download the old .rpm from the Mandriva repository, and tweak it to build
qca1. I should note that now KDE 4's Kopete is much more stable than it used
to be, and is actually usable, so I'm using it.
Then after I tried Amarok 2.0, I discovered that its album cover in the
middle pane, was too
large, and that it won't play
Module
files. On Saturday, I discovered that they dropped support for
out-of-process scripts, which means it'll be less trivial to port the Perl
script I wrote to Amarok 2.x. If I would want to do so, I'll need to
re-implement it in QtScript/JavaScript, which I heavily detest and which
will require a lot of extra research.
Finally, I had problems building perl-XML-SAX-ExpatXS, which doesn't
exist on Mandriva. On Saturday, I finally got it working, after looking
at other XML-SAX packages on Mandriva. This reminds me that I still need to
upload the new RPMs I created.
That's it, I guess. This week was less eventful, and I even got some coding
done (more about it later hopefully).
2008-12-26: Recent Perl-Related Hacktivity
It's been a while since I programmed intensively until a few days ago, but
that's changed recently. Here are some things I worked on recently:
-
I worked on my Perl/XS
tutorial-of-sorts. You can find it in its repository. The
first thing I did was re-organise the repository so there won't be a several
sub-dir in the trunk for each XS function I added to the module, which I
use to incrementally introduce the person to XS. Then I added a function,
which I
encountered
some problems with, and resolved them.
-
I worked on
Config::IniFiles, and uploaded a new verstion to CPAN. The good people at the modules@perl.org
mailing list resolved the problems we had with its maintainenace status, and
the upload got indexed. There are more fixes in the trunk as of now.
-
I worked on File-Find-Object-Rule which is my spin-off of File-Find-Rule
that uses
File-Find-Object
instead of the core (and philosophically problematic) File::Find module.
I'm breaking the File-Find-Rule API because it relies too much on
File::Find'isms, so I would have to rename it.
While I was working on File-Find-Object-Rule, I found a bug in File-Find-Object
which I fixed, and released a new version to the CPAN. It will be sometime
before File-Find-Object-Rule is ready for release, because while the tests
now pass, it still requires a lot of work.
2009-01-09: Problems With SVK
Many months ago, I discovered
several
huge problems with SVK (a distributed version control system based
on Subversion) on my system: it no longer propagates the local
changesets to the remote repositories, and I can no longer mirror a new
remote repository. However, there was no way I could get any help with it.
The chief SVK maintainer, CLKAO has been extremely unresponsive. I could
almost never reach him on IRC or on email, and when he does respond and I
respond to him, he doesn't respond back. I met someone on IRC who seemed
somewhat knowledgable, but he could not offer anything better than a workaround
where I'd lose my entire local history.
I offered a bounty of 50 USD to the first person who will fix this
problem, but no one took the offer. In the svk-users mailing list, many
(most?) questions go answered and unresolved. A new version of SVK (2.2.1) was
recently released, and I tried to prepare an RPM for it on my Mandriva Cooker
system, hoping it will fix the problem, but the tests failed there.. Fearing something
is broken, I decided against proceeding.
The
SVK
bugs queue contains 61 (!!) open bugs that weren't addressed or closed.
Best Practical, the company that hired CLKAO to further his work on SVK,
are offering
commercial SVK Suppot starting at 5,000 USD per year. (Which I cannot
afford) The original reason they hired CLKAO was because they were using SVK
and wanted to promote it and sponsor its development. However, it now seems
that they just decided to capitalise on it, and kill the pro-bono/community
channels of support.
In short, I'm stuck with a lot of data on my hard disk that I cannot propagate
to the outside. I'll be appreciative of any way to fix SVK so I can propagate
it to the outside, or a recommendation of a different tool that can do it for
me based on the SVK local Subversion repository. In the meanwhile, my advice
to you is that until the attitude of the core SVK developers changes, and the
bugs in SVK are fixed, then you should stay away from SVK, and not depend on
it.
If you're looking for a way to decentralise a remote Subversion repository,
then you can look at
git-svn ,
bzr-svn ,
and hgsubversion . But
you should not depend on SVK.
2009-01-10: Reports on Recent Open-Source Meetings
I attended four open-source-related meetings this week. On Sunday,
Gabor Szabo, gave a presentation about
Padre, the Perl IDE, which he
described as a "Notepad with a big ego". The presentation was originally
titled "wxPerl programming and Padre, the Perl IDE", but for better or for
worse Gabor did not talk about wxPerl at all. The presentation was mostly
focused about what Padre can do now, Goals for next releases, and a call
for volunteers. I found the most impressive part to be the fact that it had
some rudimentary Perl 5-syntax-aware refactorings, like lexically change
a variable name.
10-20 people came to the meeting, but many of them left before it was done.
(don't know/remember why), and only 4 of us went to the cafe afterwards.
Still, a lot of fun was had.
For another review of the meeting see xsawyerx.
Next on Wednesday, I attended the Perl-Haifa meeting. I took the train to
Haifa, and after I arrived at Matam, I met a regular
Haifux attendee on the way, whom I
haven't seen in a long time and we talked about stuff. Not many people came
to the meeting, but one or two ended up late.
Uri Bruck presented about Perl in Software Art. He presented a lot of
unorthodox art which was software based. I daresay the projects he presented
there were more strange than I expected it to be. One problem we had was with
the intermittent Wireless LAN connection on his computer, but that wasn't
too bad.
After that, Shmuel Fomberg gave
a presentation about
Data-ParseBinary.
The story of the module is interesting. I originally attended a presentation
about PyConstruct
given by one of its developers on a Python-IL meeting. Then, when I helped
Alan Haggai Alavi with the
(Perl/CPAN) Archive::Zip TPF grant, I suggested that as part of the grant,
he'll write a PyConstruct-based parser for .zip files, to help in analysing
different archives and see where Archive::Zip goes wrong. Now Shmuel
saw it mentioned in the grant, and liked the idea, so he ported it to Perl
as Data-ParseBinary. And then, Data-ParseBinary was good enough that Alan
Haggai decided to use it as the parser for the .zip format for his Archive::Zip
grant.
Before the presentation, I talked with Shmuel, and thanked him on behalf
of Alan. He said that he knew of Alan's use of Data-ParseBinary based on a
web-search and that he was his only known "customer". Even Shmuel himself
did not make an active use of Data-ParseBinary in production. (But he said
he's planning to soon).
In any case, in the talk, Shmuel gave a relatively comprehensive overview
of the module's features, and how to use it for many use-cases. One attendee
kept asking him questions about why he did things this way instead of the
other, and Shmuel answered with the right way to do things. He said that
Data-ParseBinary's interface was originally based on Construct's, and so many
of the problems the attendee found with it were derived from there. We
also had some fun correcting the pronunciation of certain English words.
Data-ParseBinary seems very cool and useful in case you need to analyse or
modify binary formats.
On Thursday, I went to a Herzelinux (Linux in Herzeliyah) meeting. Eitan
Isaacson gave a presentation about accessibility in GUI software in general
and about Accersiser, which is a tool he wrote in order to help test
the accessibility of programs. Accersiser basically lived at the lowest
layer of the accessibility API and acted as an accessibility aid that shows
what is available there. The presentation was interesting, but from my
impression, it seems that Accersiser is too crowded with information so
one cannot see the forest from the trees.
He mentioned KDE/Qt applications, and said that they don't yet interface with
the accessibility framework used by GNOME, Java and Mozilla because it is
based on CORBA, and they don't want to have to depend on it. They said they're
planning to add a DBUS-based transport, which will hopefully enable Qt and KDE
to interface with it as well.
After the Herzelinux talk, someone drove me to the Israeli Ruby meeting
at a pub, nicknamed "Beer on Rails". Since I ate a lot of refreshments at
Herzelinux, I only ate some spicy fries there (which were very good). I engaged
in a few talks there, but was soon left out because I sat in the corner
between two conversations that I could not hear well. One talk that I remember
was that someone told me he tried to use Vim instead of Emacs, but found it
frustrating that while in Insert mode he could not move to the previous or next
characters without pressing Escape. I told him to use the cursor keys, but
he said he'd rather not because they are too far away. Then I told him that
he can bind certain key bindings to do something else while in Insert mode,
but he said "Yes, but that would no longer be vi.".
I'm personally not too bothered by this, because I have no problem using the
arrow keys for certain tasks, and because I find that when working on editors
on other systems, I only do minimal tasks until I can set up an environment
that I like like I am used to. But I can relate to such problems.
In any case, I eventually got too tired, paid, and took the bus home. I hope
the next meetings will be in a different format, and somewhat earlier during
the day.
So it's been a busy week, and a lot of fun was had. Bye for now.
2009-01-14: Shavin' Another Second
I've been working on File-Find-Object
for quite some time now. File-Find-Object provides an alternative to
File::Find, which is a built-in module in the perl 5 distribution, with
several major philosophical limitations. What both modules do is traverse
a directory tree and allow the programmer to do something with all the
results, similar to the Unix
shell's "find" command.
My obsession with working on File-Find-Object started
in
this perl5-porters thread, where I also claimed that File::Find was
probably IO-Bound, and so making it iterative and capable of instantiation
(which is partially what File-Find-Object is all about), will not harm
its performance much. As it turns out, I was not entirely right.
About two weeks ago, I decided to finally benchmark File-Find-Object. The
original motivation for this was a small change I did and perceived as an
optimisation in which I reduced the number of hard-disk-reading system calls
that it used to a minimum. I wrote a few shell scripts that timed the
old version of F-F-O versus the new one on my ~/progs directory (which
contains many sub-directories), after it was in turn traversed by the
shell "find" command.
Both version took somewhat over 2 minutes to complete the scan, which seemed
pretty good to me. However, when I benchmarked a similar script using
File::Find, I found out that it finished the scan in under 5 seconds, making
it 24 times faster than File-Find-Object. I didn't want F-F-O to be so much
worse, so I decided to work on optimising it.
My methodology in optimising File-Find-Object was to profile it using
Devel::NYTProf
(the Perl profiler with the best reputation, and one which I can now
recommend as well), find problematic spots, try to optimise them, and
then benchmark the old code against the new to see if there were any
improvements, and if so - commit.
Among the notable changes I performed were:
-
The first thing I noticed was that a lot of time was spent in Class::Accessor
code. To remedy this, I switched to
Class-XSAccessor,
which resulted in saving 40 seconds of runtime. The other changes I made
were less substantial individually in terms of run-time saving.
-
Added a flag to $self (the object instance) with whether or not
the stack of the directories through which the traversal is in progress
is full. It was used inside several dynamically-generated functions where I
delegate to top and non-top versions.
To save some extra speed, the flag is checked using
exists which
should be a bit faster than checking for the truth of the value.
-
Saved 2.5 seconds of run-time by implementing a commonly used function
(that was reported to consume a lot of time) in a direct manner, instead
of the top/non-top delegation, and using some judicious optimisation using
the "||" operator.
-
Saved many seconds on eliminating calls to _copy methods that
flat-copy array references (and that are also generated dynamically). Often,
the copying was not needed at all.
-
Eliminated excessive calls to
File::Spec's
functions by caching the results.
-
For each node, File-Find-Object performs two actions in a specific order
depending on the "depth" instance-wide parameter. Previously, handling them
involved weird returns of arrays, and indexes-mangling (which I admit that
I originally implemented in my endless refactoring). This was changed
to a simple array of method names, from which values are extracted.
-
Re-arranged the order of operations to prevent an
if (_is_top()) { ... } conditional, that was done in every iteration.
-
Originally, the main object served as the controlling object for the top
directory, and dir_stack->[0] was the directory below it. So
when checking for fatherhood of a directory, one had to return the main
object for the father of dir_stack->[0]. In a series of commits, I
moved it to be a separate object, as well as occupying the first step
in the directory stack.
As a result, I was able to cache the "_current" object inside an accessor,
because doing so in for the main object before would have resulted in a
circular reference, which would have caused a memory leak.
Thanks to all these optimisations, File-Find-Object now runs at
about 30-40 seconds.
Yesterday I also benchmarked the first version of File-Find-Object
before I took over its maintenance, and started refactoring it, and
it runs at 17 seconds for the same test.
So what are the Conclusions?
-
Accessors in Perl may incur a large overhead if Class::XSAccessor (or similar)
are not used.
-
Extracting methods excessively can incur a heavy penalty on performance of
Perl programs.
-
Traversing a directory in Perl can be very CPU-bound.
-
Many small optimisations can together yield a huge benefit. Bill Raymond says
in
this
message to the fc-solve-discuss mailing list
that "I achieved my fast times by multitudes of 1% reductions". I can
attest to it here.
As indicated by the benchmarks, the road to a faster File-Find-Object
is not over, and I still have some ideas for improvements in mind. But
it's still much better than it used to be, and I can be proud of it.
The title of this post was inspired by Ido Trivizki's
"Shavin' another
bit" presentation given at YAPC::Israel::2004, and which I attended
and enjoyed.
2009-02-05: Tech Tip: Resolving JDBC+MySQL exception.
I recently tried to build a Java-based project that is using MySQL. I got the following error:
BUILD FAILED
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
Last packet sent to the server was 1 ms ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1074)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2104)
at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:729)
at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:46)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:302)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:283)
at org.apache.tools.ant.taskdefs.JDBCTask.getConnection(JDBCTask.java:319)
at org.apache.tools.ant.taskdefs.SQLExec.execute(SQLExec.java:429)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:357)
at org.apache.tools.ant.Target.performTasks(Target.java:385)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1337)
at org.apache.tools.ant.Project.executeTarget(Project.java:1306)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1189)
at org.apache.tools.ant.Main.runBuild(Main.java:758)
at org.apache.tools.ant.Main.startAnt(Main.java:217)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at java.net.Socket.connect(Socket.java:469)
at java.net.Socket.(Socket.java:366)
at java.net.Socket.(Socket.java:209)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:256)
at com.mysql.jdbc.MysqlIO.(MysqlIO.java:276)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2027)
... 27 more
After many experimentations, I discovered the problem was that MySQL on my
local machine listened only on a UNIX-domain socket and not on TCP, which is
what JDBC tried to connect with. After commenting out the "skip-networking"
option from /etc/my.cnf (which is the location of the MySQL
configuration file on my system) and restarted mysqld, everything worked.
2009-12-24: Web Application Frameworks
Ido Kanner
writes in his blog (in Hebrew) about web-application framework.
I am replying here, because his blog does not provide a Preview plugin
for comments (due to limitations of wordpress.com).
Aside from the alternatives he mentioned for Perl, I am aware of the following:
- CGI::Application - very basic
functionality, but also very flexible.
- CGI-Application-Plus - backwards compatible with CGI-Application, with some
improvements.
- CGI-Builder
- inspired by CGI-Application, but not backwards compatible. (by the same
author as CGI-Application-Plus).
- CGI-Prototype
- another lightweight framework based on prototype-based OO (like Self,
JavaScript, Io, etc.)
- Catalyst -
a web framework that supports many different view/model backends and
has many plugins on CPAN. Seems to be the most popular.
- Jifty - should be more
RoR'ish than Catalyst, and also have a concept of continuations. Only
supports HTML::Mason and Jifty::DBI.
- Gantry - don't know
a lot about it.
- AxKit - Perl and XML-based for
Apache
- WebGUI - a Perl-based
framework mixed with a content management system. (I had problems getting
it up and running on my Mandriva Cooker system, after installing it from
source, and it also made my Apache instances consume much more memory, so
I disabled it.)
-
Mojolicious - started as a fork of
Catalyst. Don't know too much about it, but you can find more from the site.
I also had some experience with the PHP-based
Symfony (review in the link).
Ido also mentions some JavaScript based "frameworks" like jQuery, or Mootools,
but naturally they are not an alternative to server-side frameworks, but
rather complementary.
Moreover, sometimes Content-Management-Systems can be used in a
similar fashion to web-development frameworks, and there are many of
those too.
2009-02-05: Tech Tip: Getting Rid of Visual Artifacts in KDE 4
I recently (or not so recently) discovered that the KDE 4 desktop on my
Mandriva Linux Cooker system displayed some bad visual artifacts when loaded
as the default desktop. It didn't happen on fresh UNIX accounts. As I
discovered, even moving away the entire home directory (/home/shlomi)
to a new place, and creating an empty directory did not resolve the problem.
Nor did, copying the files from a good account.
I finally discovered that the problem was in
/var/tmp/kdecache-shlomi ("shlomi" is my username), which after
exiting from KDE 4 and removing solved the problem for good. Of course, all
of this took many hours of trying to incrementally move away files and see
if it solves the problem. I cannot guarantee it is the only cause of
problems similar to that one, but it helped me.
2009-02-10: Oppose the Patent Encumbered IETF TLS Authorisation Patent
Glyn Moody reports on
an FSF
call-for-action for rejecting a patent-encumbered standard. It needs
to be done today, and this
is what I wrote to ietf@ietf.org:
Please don't accept the patent-encumbered standard for TLS
authorisation. Doing so will allow a single entity to control the use
of the standard and will weaken the authority of the IETF.
Please follow the instructions in the links and send them an email. You
also need to approve of the confirmation email.
2009-02-11: "Windows Refund Howto" on the Tel Aviv Open Source Club
Hi all!
The Tel Aviv Open Source Club
will host a talk by Zvi Devir about "Windows Refund HOWTO" - how to get a
refund for a copy of Microsoft Windows that ships
as part of a new computer - on Sunday, 15-February-2009 (next Sunday).
The meeting will take place at Tel Aviv University, at the Schreiber Maths&CS
building, room 008 on 18:30. So mark your calendars. For more information can
be found the club's homepage
and on the meeting's wiki page.
With any other questions or problems, feel free to
contact me.
2009-02-13: New Essay: "Optimising Code for Speed"
I published a
new essay about optimising code for speed on my site, which you are
welcome to read. Quoting from it:
We've all seen the fact that while computers got faster, software has
often become slower to run unless the hardware is upgraded. The so-called "Gates'
Law" claims that commercial programs decrease in speed by half
every 18 months, due to various reasons. It is well known that the various
versions of the DOS
operating system ran adequately on a PC XT's and 286's and that a Intel 386
was a "lean and mean DOS machine" as a certain journalist claimed back
then. On the other hand, Microsoft Windows 3.0 and Microsoft Windows 3.1
already required a fast 486 computer to be ran comfortably, while Windows
95 was barely usable there and needed a Pentium computer. Windows XP
already ran slowly on a Pentium machine and required a high end Pentium III
or Pentium 4 computer. Windows Vista
requires even more hardware resources than Windows XP, up to the point that
many computers in use today cannot run it comfortably.
Now, while software simulations that run directly against the CPU and
memory (and possibly hard-disk) are still running much faster than before,
the responsiveness of the system itself does not seem to improve much.
I used Wikibooks to work on this
article. Except for a few minor changes from other people, I was the only
contributor. One thing that disappointed me was that wiki2xml (a MediaWiki
extension that can convert the text to DocBook/XML and other formats)
blissfully ignored all internal or wikipedia links ([[…]]),
rendering many links in the article useless. This required that the article
be extracted from the wiki by cleaning up the HTML of the page.
2009-02-16: מדוע אתה מפתח את זה בחינם?
ארתיום שואל "מדוע אתה מפתח את זה בחינם?"
ואני החלטתי לענות לו כאן (ייתכן שאתרגם זאת לאנגלית יותר מאוחר). מכיוון שיזמתי ו/או תרמתי
לפרוייקטים רבים
הרי שלא אוכל לכסות את כולם ברשומה זאת, ואתמקד רק בבולטים שבהם.
התרומה הראשונה שלי של תוכן פומבי וזאת שהשקעתי בה את כמות הזמן הרבה ביותר היא
אתר הבית שלי.
אם לדייק, הרי שאני מרוויח ממנו מעט כסף משום ששמתי שם פרסומות. בכל מקרה, נכון לעכשיו, שני הצ'קים של כ-100 דולר כל אחד שקיבלתי מגוגל לא מתקרבים אפילו לכסות על כמות ההשקעה שהשקעתי באתר, ואף שאני אשמח שבשלב מסוים אצליח
להתקיים מהפרסום באתר בלבד, הרי שנכון לעכשיו, זה לא קורה. בכל מקרה - מדוע אני עובד עליו? הסיבה הראשונה היא שאני
נהנה לעבוד עליו ואני לומד מכך הרבה. הסיבה השנייה היא שאתר עשיר כל-כך מקנה לי מוניטין רב והרבה אנשים לומדים עליי כך ומתרשמים מיכולותיי. הסיבה השלישית היא שאני לפעמים מקבל פידבק (בדרך-כלל אוהד) מאנשים על דברים שפרסמתי באתר, או מתפתחים דיונים מעניינים בעקבותיהם. והסיבה הרביעית היא שאני לומד מכך הרבה.
בנוגע ל-Freecell Solver הרי שזה התחיל מנסיון שעשיתי כדי לראות
אם מספר רעיונות שהיו לי כיצד לפתור את פריסל אכן יעבדו בפועל. הם עבדו עד כדי אפסילון גדול מאוד והייתי כל-כך
מרוצה מהעובדה הזאת, שהחלטתי לפרסם גרסה ראשונה (0.2.0) של התוכנית באתר שלי בטכניון וכתבתי הודעה עליה ב-Freshmeat.net. המשכתי לעבוד על הפותר מתוך עניין, מכיוון שקיבלתי עליו משוב רב מאנשים שניסו להשתמש בו, מכיוון
שלמדתי ממנו הרבה, ומכיוון שאני יודע שהוא שימושי לאנשים רבים וגם אני עושה בו שימוש. בנוסף Freecell Solver סיפק לי השראה
רבה וחומר רב לדיון במאמרים ובמסות שלי כך שגם בזה הוא עזר
לי.
ישנם גם
המודולים שלי ב-CPAN,
שזה קצת חטא להכליל את כולם בקבוצה אחת כי הם משמשים למטרות שונות ומגוונות. את רובם כתבתי בעצמי (עם מעט תרומות מאנשים אחרים), וחלק קטן מהם אימצתי מאנשים אחרים. מדוע אני עובד עליהם? כי אני אוהב לכתוב פרל, כי אני מוצא את
עצמי מעורב בכל הקהילה השלמה והתוססת של פרל בארץ ובעולם, וגם מזה למדתי בעבר. חלק גדול מהמודולים היו כאלה שהייתי צריך לשימושי האישי והחלטתי לשחרר לציבור, כך שממילא הייתי צריך לכתוב אותם.
תרמתי בעבר גם לפרוייקטים כמו
סאבוורז'ן, ו-גימפ.
הסיבה שתרמתי לסאבוורז'ן היא שהתחלתי לעשות בה שימוש, אחרי שלדעתי האלטרנטיבות
שהיו לה אז (Arch, Aegis וכו) היו מסובכות, מבלבלות ומעצבנות ואחרי
שהיה
לי נסיון רע עם BitKeeper הלא חופשי. כאשר עבדתי על סאבוורז'ן התרשמתי לטובה מהיחס שלהם לתורמים, שזה דבר
שפרוייקטים רבים של קוד פתוח יכולים ללמוד ממנו. הפסקתי לתרום לסאבוורז'ן מכיוון שנוכחתי שהם היו במצב כזה טוב שהם
לא זקוקים כל-כך לעזרתי, וגם נוכחתי שסאבוורז'ן לא גורם לי לבעיות מיותרות, אז החלטתי לעזוב בשקט מבלי שאיש ירגיש. במידה שתהיינה
לי בעיות איתו בעתיד אני לא שולל שאשקיע זמן בלתקנן, אבל כרגע אין לי כוונה לחזור להיות תורם פעיל.
לגימפ תרמתי מפני שהיא לדעתי תוכנה מדהימה, ורציתי לקדם אותה.
הטלאי הראשון שלי עבר מספר גלגולים ושכתובים עד שהתקבל
בסופו של דבר, אבל זה היה לגיטימי מבחינתי. אחר-כך כתבתי טלאי פצפון ל-gimpressionist ומבט קצר על הקוד שלו הביא
אותי להיווכח שמצבו בכי רע, והשקעתי זמן רב בלנקות אותו. מה שמשך אותי היה הרצון לקדם את התוכנה.
אז זהו בעצם. אם אתם רוצים להצטרף לשרשור אז בשמחה אקרא את מה שתכתבו.
2009-02-16: Should We Include a Better Alternative to File-Find in the perl5 Core?
chromatic asked for some
link/follow-ups love to his
"Modern Perl Books" blog,
so here's a post inspired by one of his
Modern Perl posts where he complains about how inane the File-Find
interface is. I personally share chromatic's sentiments about File-Find,
and have for many years. After posting
to perl5-porters about its shortcomings, it was agreed that fixing them in
File-Find may hurt its performance, and so was not acceptable without forking
the code or rewriting it. At the end of the thread, I mentioned that I found
File-Find-Object
on CPAN, and that I intend to contribute to it and improve it. That's what I
did, and I eventually took over its maintenance from the originator.
File-Find-Object solves all the major shortcomings of File-Find that I'm
aware of. But this post is not specifically about File-Find-Object. If someone
will come up with a better alternative to File-Find than File-Find-Object
is, then I'll support the new code.
The point is that I think that, as a statement of intent, and of general
disgust from File::Find, a different module with a better interface
and a saner and more modern philosophy should be shipped as part of
the perl 5 core. It's not the first time that a core module in perl 5 was
"complemented" with a better alternative: CPANPLUS.pm was added to the core
in addition to CPAN.pm , Module::Build was added as an alternative to
ExtUtils::MakeMaker, and recently Test::Harness was augmented with
TAP::Parser/TAP::Harness/Test::Harness-3 . I don't see why File::Find should
be any different considering the fact it has several philosophical
usability problems which cannot be easily overcome (it cannot be instantiated,
cannot be easily interrupted in the middle, and does not have an iterative
interface).
I don't suggest we change the File::Find API. That will wreck havoc. But
I think we should include another module and put a big and visible notice
at the beginning of File::Find saying
"File::Find is deprecated and should no longer be used! Use
$better_file_find , that ships with perl as of perl-5.12.0 or available from
CPAN for earlier versions, instead."
What do you think?
2009-02-21: How I Recovered Data from a Hosed SVK Repository
Many months ago, I noticed that
SVK, a version control
system based on Subversion, stopped propagating changesets to the remote
Subversion repositories. Furthermore, I was unable to mirror other remote
repositories. I wrote about it, asking for help:
All of this has been left unsolved until yesterday when I decided that I had
enough and wanted to finally resolve this problem. My first idea was to
try to write a script that will propagate the changesets using the Subversion
APIs, so I looked at the SVK code trying to see where exactly it happens.
I found the SVK code hard-to-follow, so I first contemplated fixing what
caused it to report an error, but was also unable to.
Then I joined the Freenode IRC network
and went to the #svk channel. There was only one person there who tried
to help me, and he was intermittent in helping. Then I joined #perl and casually
mentioned that I thought SVK sucked (just to lose some steam), which prompted
mst to try to help me. After
interrogating me he suggested that in order to recover the changesets from SVK,
I should follow the following steps:
-
Find out what is the last SVK revision which still propagated changesets to
the Subversion repository. This can be found using the commit message in the
svn log.
-
Revert (using a selective svnadmin dump and then an
svnadmin create ; and svnadmin load) to the older version of
~/.svk/local that corresponds with the revision number
of the commit. (While naturally keeping a copy of ~/.svk/local/ and
its dump in a safe place.)
-
See that a simple change in the svk checkout (like
svk mkdir to-delete-`date +%s`) will propagate to the remote
repository.
-
Revert that change using the Subversion client directly.
-
Revert to the working-but-old copy of the SVK repository, and apply a
filtered stream of the dumps (using svndumpfilter) that will
exclude changes from the //mirror part of the SVK repository.
-
After all that is done, checkout the trunk and do svk push
from there.
Of course, the devil is in the details, and it took me a lot of experimentation
until I got to something working. Facts I've discovered:
-
svndumpfilter needs a dump that was generated without the
--deltas flag (and possibly without --incremental either.
Otherwise it will complain.
-
I couldn't seem to get svndumpfilter exclude mirror to work
and instead ended up using svndumpfilter include local
-
The --drop-empty-revs and --renumber-revs flags to
svndumpfilter which seemed like a good idea confused svnadmin load,
and everything worked after I removed them.
-
Using tar -czf and tar -xf can really speed up restoring
older versions of a Subversion repository because svnadmin load is pretty slow.
-
I ended up taking a range of repositories from the pristine revision to "HEAD"
and then manually editing out the first revision. Maybe it was unnecessary.
After all that, "svk push" worked and propagated 800 changesets to
my homepage's
trunk in 8:20 hours (must be a record). Since I only have an ADSL
connection with 25 KBps upstream, and the Subversion service is located in
Taiwan and I'm living in Israel, it took a while and I left it overnight
to run.
Today, after I woke up, I saw that the operation ended up successfully. I was
able to build an up-to-date version of my homesite from the Subversion sources,
and am mostly ready to stop using SVK.
Again, I'd like to thank mst for
suggesting this ultra-cool idea for recovering my data from SVK. It
took me the whole of one day, but now I'm finally free, and my data is out
there and safe. I'm going to use plain-old-Subversion to work
against it, until and assuming I'll get the hang of hg/bzr/git/whatever.
But at least I no longer have to use a mostly-unusable SVK.
Hackfully and happily yours, Shlomi Fish.
2009-03-02: CPAN Hacktivity Report
After my
last post about File::Find alternatives,
chromatic
responded, and said that it was opinion that the core should not be
bloated further and that he would prefer if File::Find did not exist in
the core in the first place. I brought the relevant links
on perl5-porters,
and David Cantrell wrote a patch to add a warning to File::Find, while
disabling it from all the places in the core that use this module. As other
people noted, we should practice what we preach, and so if we decide that
File::Find is deprecated, then we should include a different alternative
in the core, and adapt all the existing code to make use of it instead.
Someone even suggested implementing File::Find in terms of the new module.
Eventually, I realised that the best solution would be to include a note
in the File::Find documentation about its limitations, and refer the interested
people to the better alternatives on CPAN.
In any case, since my
"Shavin' Another Second" post, I implemented several more small
optimisations (and general code cleanups) to File-Find-Object, and it now takes
about 27 seconds to run in my test run. One thing that surprised me was the
fact that after a relatively large optimisation and overhaul to the code,
all the tests pass immediately after the first clean run. Usually, the code
required more persuasion to make it work.
I also began File-Find-Object-Rule,
which is the File-Find-Object-enabled-fork of File-Find-Rule, which provides
a more convenient interface above File::Find. The first release of F-F-O-R
had missing dependencies, and so consistently failed all CPAN testers'
reports. Version 0.0101 was better, but its tests failed on MS-Windows, due
to the fact that the File-Find-Rule tests excepted UNIXish paths (which
File::Find happily provided, but File-Find-Object doesn't). This was fixed in
File-Find-Object-Rule version 0.0200.
Version 0.0200 is still a mostly "smallest-effort" port of File-Find-Rule, so
now comes the interesting part of truly making use of File-Find-Object's
advantages, which may involve extending the latter API. I also would like
to create a documented API for File-Find-Object-Rule, so the code won't
be tightly coupled with File-Find-Object.
Speaking of File-Find-Rule, you may find
this recent discussion
on use.perl.org interesting.
Except for that I also did some work on XML::RSS, like integrating some
patches that someone else sent me, doing QA, and answering an email someone
sent about one of its limitations (which I'd like to fix.).
I also found
a bug in ack, and submitted a patch fixing it.
Finally, after my home-site's remote repository good condition was restored,
I decided to try to build it from scratch and correct the process that has
gathered a lot of rot since I last did it. I discovered a lot of problems
with installing the necessary CPAN modules that I wrote in part
for the homepage, but I'm waiting on a reply to
this
question about Module-Build on its mailing list, before I can correct
the build.
2009-03-11: A New "Perl 6 Tricks and Treats" Newsletter by Gabor Szabo
Gabor Szabo
announced
a new newletter he
started for "Perl 6 Tricks and Treats". I am subscribed to his
"Test
Automation Tips" newsletter, and am very happy from the quality of
his submissions to it, so I can recommend Gabor's writing first hand.
Please subscribe if you're interested in Perl 6, and please help publicise
the announcement of this newsletter by sending a message to your local Perl
mongers group, other FOSS-related groups or on your blog.
2009-03-23: Tech Tip: Installing CPAN Distributions without .pm Files
I recently needed to install
cpan_bot from the CPAN
(the Comprehensive Perl Archive Network). However, it does not contain
any .pm's (Perl Module files) - only a script. All my attempts to install it
failed until I tried this:
perl -MCPAN -e 'CPAN::install("ZOFFIX/cpan_bot-0.11.tar.gz")'
So you need to use the author's ID followed by a slash and the tarball's
filename.
2009-03-26: Fixing the Amarok 2.x Playback of Module Files
This is a behind-the-scenes story of a patch I worked on. The problem
I encountered was that Amarok (
a KDE-based music player) version 2.x could not play
Module files, such
as ".mod", ".s3m", ".xm" or ".it". The symptom was that dropping them
in the playlist did not get them enqueued. Talking about it on IRC, I got
a lead that led me to investiage the following code in Amarok's
src/EngineController.cpp:
// Filter the available mime types to only include audio and video,
// as amarok does not intend to play photos
static QStringList mimeTable =
Phonon::BackendCapabilities::availableMimeTypes().filter(
"audio/", Qt::CaseInsensitive ) +
Phonon::BackendCapabilities::availableMimeTypes().filter(
"video/", Qt::CaseInsensitive );
const QString mimeType = item.mimetype();
const bool valid = mimeTable.contains( mimeType, Qt::CaseInsensitive );
So when I ran it using the gdb debugger, while using
kde-devel-gdb
(a set of gdb macros to facilitate debugging KDE applications), I dumped
"mimeTable", and, after grepping it, found that it indeed did not contain
the MIME types associated with module files. Then I investigated why I
could still select and play module files in KDE 4's Dragon Player, and
realised that while it used the mime types list as a file-chooser dialog,
it did not enforce a file that was not associated with any of them from being
played.
Having realised that, I set out to see why the list of mime types was not
complete. I first tried to see what is the list that phonon returns as is.
So I wrote (or more actually copy and pasted) the following code:
#include <iostream>
#include <Phonon/VideoWidget>
#include <Phonon/BackendCapabilities>
#include <KAboutData>
#include <KApplication>
#include <KCmdLineArgs>
#include <KLocalizedString>
static KAboutData aboutData( "test-phonon", 0,
ki18n("Dragon Player"), "0.1.5",
ki18n("A video player that has a usability focus"),
KAboutData::License_GPL_V2,
ki18n(
"Copyright 2006, Max Howell\nCopyright 2007, Ian Monroe"
),
ki18n(
"IRC:\nirc.freenode.net #dragonplayer\n\n"
"Feedback:\nimonroe@dragonplayer.org"),
"http://dragonplayer.org" );
int main(int argc, char * argv[])
{
aboutData.setOrganizationDomain( "mpris.org" ); //for DBus
aboutData.addCredit( ki18n("David Edmundson"),
ki18n("Improvements and polish") );
aboutData.addCredit( ki18n("Matthias Kretz"),
ki18n("Creator of Phonon") );
aboutData.addCredit( ki18n("Eugene Trounev"),
ki18n("Dragon Player icon") );
aboutData.addCredit( ki18n("Mike Diehl"),
ki18n("Handbook") );
aboutData.addCredit( ki18n("The Kaffeine Developers"),
ki18n("Great reference code") );
aboutData.addCredit( ki18n("Greenleaf"),
ki18n("Yatta happened to be the only video "
"on my laptop to test with. :)") );
KCmdLineArgs::init( argc, argv, &aboutData );
KCmdLineOptions options;
options.add("+[URL]", ki18n( "Play 'URL'" ));
options.add("play-dvd", ki18n( "Play DVD Video" ));
options.add("debug",
ki18n( "Show Additional Debug Output" ));
KCmdLineArgs::addCmdLineOptions( options );
KApplication application;
QStringList mimeTypes = Phonon::BackendCapabilities::availableMimeTypes();
QStringListIterator mimeIter(mimeTypes);
while (mimeIter.hasNext())
{
std::cout << mimeIter.next().toLocal8Bit().constData()
<< std::endl;
}
return 0;
It has a lot of KDE verbiage, but I hope you get the idea. The most important
lines are at the end. And indeed it did not print the associated mime types.
Reading a bit about Phonon, I tried to see what the backend that it
was using was. I believed it was
Xine but wanted to be sure.
Eventually, what I did was to read /proc/`pgrep amarok`/maps
which contains a list of all the shared libraries that it used. A bit
afterwards, I discovered that the Phonon backend was configurable from
"Settings → Configure Amarok → Playback → Sound System Configuration"
and indeed had Xine as the most prioritised backend. After switching it
to gstreamer, I realised that Amarok could now play files ending with
.mod, but not with say .xm or other Module files' formats,
so it indicated that the problem was with Xine.
Looking at the code of the Xine Phonon backend (after some investigation), I
found the following:
QStringList Backend::availableMimeTypes() const
{
if (m_supportedMimeTypes.isEmpty())
{
char *mimeTypes_c = xine_get_mime_types(m_xine);
QString mimeTypes(mimeTypes_c);
free(mimeTypes_c);
QStringList lstMimeTypes =
mimeTypes.split(";", QString::SkipEmptyParts);
.
.
.
}
}
So it seemed that Phonon returns its results based on xine_get_mime_types().
Therefore, I wrote the following test program to see what xine returns:
#include <xine.h>
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char * argv[])
{
xine_t * xine_h;
char * mime_types;
xine_h = xine_new();
xine_init(xine_h);
mime_types = xine_get_mime_types(xine_h);
printf("%s\n", mime_types);
free(mime_types);
xine_exit(xine_h);
return 0;
}
And it didn't return the appropriate mime types for the MOD files, either. So
the problem was with xine-lib.
Next I decided to look at xine-lib and investigate. Searching for the
xine_get_mime_types() functions yielded a multiplexer that collected
the mime types of the various plugins. So I looked at
src/demuxers/demux_mod.c and found the following:
static const char *get_mimetypes (demux_class_t *this_gen) {
return NULL;
}
So no wonder no Module files-related mime types are returned. After fixing
it by writing
a patch that just added
the mime types as one-big comma-delimited string
(as seen in other demuxers), and rebuilding xine-lib, both of my test
programs returned the appropriate mime types, and I was able to play files
in Amarok.
I submitted the patch to
the xine-devel mailing list after subscribing to it. I have yet
to receive a reply, to say nothing of getting my patch applied, but at least
I now know the problem has been fixed and I (and others) will be able to play
Module files in Amarok to my heart's content.
2009-03-31: Tel Aviv FOSS Meeting on 5-April-2009: Intro to Perl 6 - Part II
The Tel Aviv Open Source Club
will host the second part of a series of talks by
Gabor Szabo about "Introduction to
Perl 6" - on 5-April-2009.
The meeting will take place at Tel Aviv University, at the Schreiber
Maths&CS building, room 008 on 18:30. So mark your calendars. More
information can be found on the wiki page for the meeting.
Attendance is free, it is not necessary to RSVP and everyone are welcome.
With any other problems, feel free to contact me.
Abstract
Introduction to Perl 6.
A series of presentations on learning and using Perl 6 from the ground up to
the special features.
Many would think that Perl 6 is just a new version of Perl and that it might
only be interesting for Perl programmers. However, Perl 6 is in fact a
compiled language running on a virtual machine that embraces many new concepts
not found in most programming languages.
The presentations will be equally interesting for Perl, Java and C#
programmers.
During the series of talks we will start by learning the basics of the
language and will get to various high level concepts. For now we have
planned 2 sessions but if we need more time we'll schedule more meetings.
Note:
After the talk we will go to the café at the main entrance where we can
continue the discussion. If people bring portable computers, we can get the
off the ground on the spot. VirtualBox images will be provided with everything
that is needed for playing with Perl 6 set up inside. So you may opt to bring
a computer with VirtualBox installed.
2009-04-01: New Book Coming Soon
It's official! I'm going to publish a new book soon, and I placed
a new interview
with me about it. Read it so you'll understand why "Mastering
cat" would be an essential addition to your collection of technical books.
2009-04-02: The PONIE Subversion Repository
While cleaning up an old hard-disk, I found a Subversion working copy of
PONIE - the Perl 5
port to Parrot, and found out the
URL of its long-lost Subversion repository. Now I see it's the first
hit on a Google search for "ponie repository", but still I wondered what
it was for a long time.
2009-04-05: Escaping Special Characters in MySQL Regular Expressions
MySQL sports the
REGEXP to
enable search using a regular expression pattern. However, it does not contain
a built-in function (such as Perl's quotemeta) to escape the special characters in such
patterns in order to prevent injecting malicious regular expression code.
Some time ago, I needed to search for a string within word-boundaries in MySQL
in a PHP site I help maintain. Here's the solution I came up with for
constructing this search:
function enbackslash($matches)
{
return '\\' . $matches[1];
}
$safe_word = $mysqli->real_escape_string(
preg_replace_callback('/([^\\w\\s])/', "enbackslash", $word)
);
$sql .= "text regexp '[[:<:]]${safe_word}[[:>:]]'";
Basically what happens is that I prefix a backslash to any non-alphanumeric
and non-whitespace character, and then escape the string again using
the standard RDBMS-specific escaping function.
I would appreciate any ideas for improving upon this code, but it seems to
work for me.
2009-04-14: Recent Hacktivity
This is another log of my (= Shlomi Fish) hacktivity.
-
xine-lib patch got applied
I
mentioned my xine-lib patch for fixing the MOD/S3M/XM/IT/etc. mime
types, and after pinging the mailing list
discovered that it
has been applied, and later on corrected.
After one of the recent Mandriva updates, I can now listen to Module files
on Amarok 2.x comfortably. Thanks to the Amarok developers for applying and
correcting my patch.
-
Building the KDE 4.x trunk - I decided to try the
KDE 4.x-trunk to
help with the wallpaper-per-virtual-desktop feature that was incorporated
there. It took me quite a long time to build it, because I did it wrong
and the KDE build process did not warn me about it. Here's the executive
summary of how to do it right: you need to build the following
modules from trunk (in order): kdesupport, kdelibs, kdepimlibs, and kdebase,
using a script such as the following:
#!/bin/bash
k4d="/opt/kde4-trunk"
k4sd="$k4d/share"
k4bd="$k4d/bin"
PATH="$k4bd:$PATH"
test ! -e build && mkdir build
cd build
rm -f CMakeCache.txt
export CMAKE_PREFIX_PATH="$k4d:$CMAKE_PREFIX_PATH"
cmake -DCMAKE_INSTALL_PREFIX="$k4d" \
-D"PYTHON_SITE_PACKAGES_DIR:PATH"="$k4d/lib" \
-DCMAKE_BUILD_TYPE=debugfull \
..
In kdebase, you may see that "make install" tries to install some python
files under /usr. I workarounded it by editing the appropriate .cmake
files and deleting the installation instructions for them, but there's a
patchset in the works for that.
-
KDE trunk bug - In any case, I noticed a few bugs in kde4-trunk (as
expected) and
reported a bug
about the picture names in the wallpaper selection. I spent the rest
of the day searching the code and running it under the debugger, in order
to try to fix the bug, and was eventually able to supply a patchset for that
(in the bug report). The patches were not committed yet.
-
First-Come First-Served Readers/Writers Lock - A Chinese
programmer contacted me by email and reported the fact that
the First-Come First-Served
Readers/Writers Lock did not implement the _destroy() function, which
was documented there. He was right, and it was funny because I
haven't touched this code for many years (2002 according to Freshmeat), and
didn't expect it to contain
any problems. Apparently, they were one of the first to want to use it in
production.
So I went to work. I converted the build system to CMake, because GNU
Autohell gave me a weird error (again), that I couldn't care to fix. I fixed
this problem, and also fixed some more issues with the code and API. Then
I uploaded a new version and published it on Freshmeat.
I guess Murphy was right by saying that every computer programme contains
one more bug than has been discovered so far.
-
CPAN - In the
XML::RSS realm, I applied a
patch that Simon Wistow sent me for a test file, and corrected it for
style, and then uploaded a new version (1.44) to the CPAN on 1-April-2009.
I also corrected some bugs in
HTML-Links-Localize
and
WWW-Search-MSN.
-
Freecell Solver - I investigated
a
build problem of recent Freecell Solvers with CMake-2.6.2, and
discovered that it was a bug in CMake-2.6.2, which was corrected in 2.6.3.
I also applied some cleanups to the code,
and extracted some incredibly complicated C-preprocessor-macros into
functions, and afterwards wrote a makefile for
targetting LLVM bitcode, and
implemented a working "profile" CMAKE_BUILD_TYPE.
-
GUI for a Ruby Library - as an exercise in learning the Ruby programming
language, I started writing a library, and then I needed a GUI to
visualise it. My first thought was using wxRuby, but Mandriva didn't have
a package for it. After contemplating on using gtk+, I decided against it,
and started prepared an .src.rpm for wxRuby, while trying to adapt an
existing wxPerl GUI to use the Ruby library using
Inline::Ruby. I
discovered that the CPAN version was incredibly broken and completely
non-functional on recent perl's, but looking at the rt.cpan.org bug-report,
I found a an up-to-date
version on gitorious, which I was able to use.
So I started adapting the wxPerl library to use the Ruby code. It worked
very nicely at first, but then I ran into a strange crash. I reproduced this
with a simple non-GUI Perl Inline::Ruby script that used my code, while
the equivalent Ruby script ran flawlessly. So I reported this as a bug in
the gitorious Inline::Ruby. Meanwhile, wxRuby failed to build, and I
have some bugs to report.
I realised I needed to write the GUI in Ruby, and so, decided to use QtRuby.
I started from the final tutorial example, and now have a mostly working
code. All of this wasted a lot of time that could otherwise be invested
on the library itself. But at least I uncovered some bugs.
2009-04-20: The Linmagazine Events Calendar Has Become Unreliable
This is a small warning for open-source enthusiasts in Israel
not to rely on the Linmagazine
Events Calendar. It's no longer actively maintained, and my two
submissions of Tel
Aviv Open Source Club meetings were not published. For one of them,
I sent a request to the editor to accept my event via Drupal's messaging
mechanism, and he didn't.
For alternatives you may consider the
Whatsup.org.il post calendar (
available on the front page) and hopefully we'll be able to arrange an
aggregated Google Calendar/iCalendar for FOSS events in Israel soon. But
if you've relied on Linmagazine.co.il's calendar alone - you shouldn't.
2009-04-20: My Review of The Perl Review, Winter 2008 Issue
For all those who are interested, I have written
a
review of the Winter 2008 issue of the journal "The Perl Review" on the
Israeli Perl Mongers wiki. I was given a copy of this issue as a present
by szabgab in one of the recent
Tel Aviv Open Source club
meetings, and felt that I should review it.
2009-04-25: No Wiki Spam Lately
I'm administrating several instances of
MediaWiki. Up to several months ago,
it had been the case that every few days, there was a spam attack on one
or more of them, which required my attention. Lately however, they haven't
been spammed at all. So I wonder why.
The anti-spam measures we employ are:
-
We do not allow anonymous commits, and instead require registration.
Nevertheless, the registration does not require an E-mail-handshake, which
means it can be implemented more easily by pure-HTTP bots.
-
We utilise the
MediaWiki
SpamBlacklist plugin, which blacklists edits based on URLs matching
certain regular expressions. We use both the Wikimedia-maintained blacklist
and our own blacklist, where we add URLs that we were spammed with.
Lately, our custom blacklist has not been updated, since we didn't encounter
any spam.
-
All external URLs in the wikis are using
rel="nofollow"
to make sure they do not gain page rank. This did not prevent wiki spam, but
it makes sure that spammers don't benefit from spamming.
Possible explanations to why there was no wiki spam lately is that the
wikimedia-blacklist has been more promptly maintained and prevented most
spam, or that spam-bots check for the rel="nofollow" attribute and in case
they find it, do not spam. That, or something changed in the configuration
of the wikis, which I wasn't made aware of.
I'm happy that I have less work to do to combat the spam, but am worried
that I may become negligent. But I guess I should enjoy it while it lasts.
2009-04-25: Tech Tip: using iotop to monitor the disk activity.
Today I discovered iotop (
on Freshmeat). It's a
(Linux-only) tool
to find which processes are doing the most hard-disk activity at present.
Useful for the cases where your hard-disks' LED is active and you
don't know why.
The 0.2.1 version (which is the latest) is available in the Mandriva Cooker
repositories, and it was a single urpmi command away from installation. I
assume installing it on Debian, Ubuntu, etc. should be just as easy.
I learned about iotop in this blog post
(in Hebrew) by Ilan Shavit, which I read today. Thanks Ilan, for the tip!
2009-04-30: Several Vim Tips
I have collected several vim tips in the
past weeks, so here they are:
-
One can use the
:sp(lit) command to split a window into two. :new splits and
starts editing a new buffer. Finally, both of these commands followed by
a filename stats editing the specified file in the new viewport.
-
This tip is long overdue. You can make Vim behave more like a Windows-like
editor by adding source $VIMRUNTIME/mswin.vim to your .vimrc.
Reportedly, it causes many problems and is not recommended, but I still
like to use it, afer all these years.
-
One can
match
any char including newline using \_.. The "\_" construct works for
other character classes, and allows you to match all of them as well as
newline.
It took me a long time to find that, back when I did, so I'm documenting
it here.
-
After one
indents
or unindents several lines using
"<", ">", "<<", ">>", etc. and the indentation level
needs to be promoted or demoted further, one can use the "." (= repeat)
command to repeat the shifting again (and again) until it is right.
I discovered this trick by accident, but immediately found it useful.
Hope all Israelis had a nice Independence Day.
Happy Vimming!
2009-05-10: Recent Hacktivity
I worked a lot on Freecell Solver,
doing refactoring and optimisations. On my Birthday (5-May, Tuesday) I tried
to do several optimisations, and most of them did not work, so it was a bit
frustrating. But some of them did work, and at least I know not to repeat
the other ones.
After using it for Freecell Solver, I had decided to adopt the
Statistics-Descriptive
CPAN module, and eventually got the necessary permissions. I uploaded version
2.7 which featured some bug fixes, cleanups and a conversion to Module-Build.
This release still failed some CPAN Testers tests, mainly on 64-bit
architectures, due to rounding problems, so I uploaded a new release - 2.8 -
yesterday, with more robust tests and a few other improvements.
I worked a bit on the Perl Beginners'
Site, fixing some broken links, adding more external links, and fixing
some other problems. While working on it, I found a validation problem caused
by
HTML-Widgets-NavMenu, which I corrected on its trunk, but did not upload to CPAN yet.
One day, I did not have a lot to do, so I decided to convert
Test-Run
from the old and deprecated NEXT.pm to
MRO::Compat. I ran into
a few problems because my inheritance graph was incompatible with the C3
guidelines, but mst helped me (again).
Thanks! So I ended up uploading Test-Run along with all of its plugins with the
converted code.
I also worked on a
solution for this KMail bug that a friend referred me to. It took me
a lot of experimentation to get a Perl script to be able to properly
filter the Postfix outgoing e-mail, and after everything seemed to work,
my ISP's email-server rejected the message. Either I'm doing something wrong,
or I need to use a different outgoing SMTP server.
Happy Hacking and happy birthday to
Alan Haggai Alavi!
2009-05-12: פגישת קוד-פתוח-תל-אביב: יישומי הקוד הפתוח של פרינג
מועדון הקוד הפתוח התל-אביבי (תלוקס)
ייפגש שוב כדי לשמוע את
הרצאתם של אלכס נרסט ומיכל גבע אודות יישומי
הקוד הפתוח של פרינג.
ההרצאה תערך ביום ראשון, 31 במאי 2009, בשעה 18:30, בחדר
008 של בניין שרייבר (למתמטיקה ומדעי המחשב) באוניברסיטת תל אביב. פרטים נוספים ניתן למצוא
באתר
ובויקי.
הנוכחות בהרצאה היא חינמית ולא נדרשת הרשמה מראש.
בהרצאה זו נכסה את הדור הבא של פיתוח יישומים לטלפונים ניידים.
פרינג היא יישום לטלפונים ניידים וקהילה עם מיליוני משתמשים
ברחבי העולם שמאפשרת למפתחים לבנות יישומים ניידים באמצעות
יכולות פיתוח הווב שלהם (כמו PHP, Java וכו).
אנו תמיד מחפשים מרצים שיתנדבו לתת הרצאות בנושאים שונים הקשורים לקוד-פתוח ולמחשבים.
במידה שאתם מעוניינים לתת הרצאה, או יש לכם הצעה להרצאה שמעניינת אתכם, נשמח לשמוע ממכם.
2009-05-14: Source for the "E Text Editor" Was Released
Apparently,
the
source for E Text Editor, a multi-platform Textmate clone was
released under a not-entirely-open-source licence. This is good
news despite the fact that it is not a fully-open-source licene, because
now Textmate-like functionality is available for Linux and other
platforms, and not only on
the Mac.
I'm still sticking with Vim, though,
which is fully cross-platform, open-source and even GPL-compatible.
2009-05-28: Tech Tip: Kopete: Recalling Previous Messages
When using Kopete, a unified
instant-messaging client for the KDE desktop environment, one can recall
previous messages using Ctrl+Up and Ctrl+Down. A friend, who is a KMess
developer, told me it was also a KMess
feature.
This wasn't very discoverable, and I discovered it only by accident. But
it seems useful.
2009-05-31: Tech Tip: Perl Debugger: Preserving the History Between Invocations
One thing that always bothered me about
the Perl debugger is the fact that
it didn't rememeber / preserve the GNU readline command history from previous
sessions like a shell does for example. Today I discovered how to enable
the preservation of the command history from previous sessions of the
debugger. To do so, add something like the following the the file "~/.perldb"
(under your home directory):
&parse_options("HistFile=$ENV{HOME}/.perldb-history HistSize=5000");
I explictly set the history size to a large value, but this is optional. You
can also set the path to the history file to a different value.
Now when you start perl -d it will recall all the latest values from previous
invocations.
2009-06-01: Recent Hacktivity
Before I start with the entry itself, I should note that
someone
copied my "Recent Hacktivity" meme. So I am going to tell my team
intimidating lawyers to send him a "Cease and Desist" letter for
Copyrights/trademarks/patents/trade-secrets/whatever violation. (Just kidding)
OK, I did a lot of work on Perl-Begin.
I went over all the pages and updated all the out-of-date information I could
find, and then also implemented a decent "Testimonials" sidebar instead of the
half-assed "These companies/projects are using" Perl that there was there
earlier. Next, I plan to have to create a tools/IDEs/etc. section.
I also did
some work on
the Better-SCM site. Most of it was done by going over the emails in
my relevant folder and implementing their suggestions.
I did a lot of work on Freecell Solver. I implemented
Simple
Simon support in Games-Solitaire-Verify, and verified all the
FCS-generated Simple Simons solutions up to PySol deal No. 5,000. I also
prepared
updated Simple Simon statistics.
I also did
some Perl and PDL work on creating a meta-scan that minimises the
average solutions' lengths generated by the solver. This worked out
pretty well. Finally the code in the Freecell
Solver trunk
is
now worth over a million U.S. Dollars (at least according to SLOCCount
by David A. Wheeler).
I did some extensive work on refactoring and cleaning up
the
Statistics-Descriptive CPAN module, and released version 3.0000 to the
CPAN. Next I'd like to create a perltidy theme to correspond with my
personal style, so I'll be able to reformat the code there properly. At the
moment, a lot of it is still using 2-whitespace-wide indentation levels, which
is annoying.
Next I tended to this akregator bug, where the "Copy Link Location" did not copy to the middle-mouse
button paste. Fixing it was pretty easy because there was a line doing it
commented-out. The patch was accepted and is now in both KDE-4.2 and KDE-trunk.
I helped sort out
the
Perl-related mailing lists page at the Perl 5 Wiki. The effort was
started by szabgab, and I did most of the sorting. In the process, I learned
about many interesting mailing lists, and of many almost dead ones. After
I was through, I did not know what to do with the rest of my life.
I ran into this bug
with Amarok 2.x on my system, which I fixed, by removing some weird
junk from the end of /etc/fstab. The fact that I could not use Amarok
irritated me a lot, and I was glad that I resolved it.
I released a new version of the Config-IniFiles CPAN module with some fixes
and a t/pod.t and t/pod-coverage.t files to prevent POD problems.
Finally, one evening after an intensive biking session,
I worked on forward-porting my
Amarok
per-song-volume script to Amarok 2.x. Now, the original Perl script
could not be used because running arbitrary UNIX process scripts is no longer
supported by Amarok 2.x, which now requires all scripts to be written in
JavaScript using QtScript. Porting it took all evening, but I had it
working eventually. I still resent the Amarok developers for only allowing
JavaScript there now, which I don't hold in high esteem.
2009-06-14: New Version of HTML-Widgets-NavMenu
Just for the fun of it, I decided to join
the Iron Man
Blogging Challenge, so here's my first post.
I recently released a new version (1.0300) of
HTML-Widgets-NavMenu,
my CPAN module for generating navigation menus. This release fixes a bug
in the "HeaderRole" type of navigation menus (which I believe is not widely
used) and also cleans up the internals. Instead of
my ($self, %args) = @_;, there's now my ($self, $args) = @_;,
which is cleaner and should also be a little faster.
You can see HTML-Widgets-NavMenu in action on
my homepage. By going to various pages,
you can see the navigation menu expands or unexpands in certain places, and
the breadcrumbs' trail and next/previous/up links updated.
The story of how I created this module is interesting. It started with a lot
of ugly and crufty
Website Meta Language logic that evolved
to generate my homesite's navigation menu. I didn't really like it so I look
for an alternative. I looked for one on the CPAN and found two modules that
didn't seem too inviting. Then when I mentioned my quest on
Perl-Israel, a fellow Perler said he
created a hack based on his module for generating JavaScript menus to do
just that and uploaded it to CPAN. That module seemed to have the most
user-friendly interface, so I decided to use it. However, I patched it,
inherited from one of its classes, and created a lot of code of my own.
Eventually, I ended up with only original code, and so I wrapped things up,
licensed it under the
MIT/X11 licence
and uploaded it to CPAN. Since then, it was improved to some extent, including
adding 100% Automated Test coverage at one point (which didn't prevent
the HeaderRole bug).
So far I received
two
five star reviews of the module on CPAN Ratings, and also heard from
other people using the bug tracker and on E-mail. As a result, I believe
this module is moderately popular, and fills a nice niche.
If you're looking for a module to generate navigation menus for your site,
then please take a look at it. And if you have ideas for improvements, I'll be
glad to hear from you.
2009-06-21: Tel Aviv FOSS Meeting on 28-June-2009: Programming Red Flags
The Tel Aviv Open Source Club
will host a talk by Yaron Meiry ("Sawyer") about "Programming Red Flags")
on 28-June-2009.
The meeting will take place at Tel Aviv University, at the Schreiber
Maths&CS building, room 008 on 18:30. So mark your calendars. More
information can be found on the wiki page for the meeting.
Attendance is free, it is not necessary to RSVP and everyone are welcome.
With any other problems, feel free to contact me.
Abstract
Some programming habits should be avoided at all costs; some should be
considered with care and some are just divine. Red flags tell us what to try
and avoid and what to inspect more closely. They are a sign of code that might
become problematic or straightforwardly dangerous. The talk will be
Perl-oriented (with Perl examples) but the programming practices are generic
and can be applied to most programming languages, so if you program in Ruby,
Python, PHP or other languages, this talks to you as well.
Hopefully by the time we're done, you'll be able to write cleaner, more
maintainable, less error-prone, and better code. If not, your money back
guarantee!
2009-06-28: Introducing CPANHQ and a Response to Limbic Region's "Improving search.cpan.org (Advanced Search)"
Limbic Region recently wrote a post on his use Perl journal
titled "Improving
search.cpan.org (Advanced Search)". The post concludes by saying that
"Feel free to add your own [ideas] - I know someone who may actually be
motivated to implement some of them as an alternate to search.cpan.org is
reading :-)". Since I talked with L~R about CPANHQ, a seacrch.capn.org
alternative in question, that I have been contributing to, I can assume
meant me.
Before I reply to L~R's points, let me introduce
CPANHQ. As
documented in the first
and only wiki page it has so far (which I wrote two days ago),
"CPANHQ aims to be a community-driven, meta-data-enhanced alternative to
such sites as http://search.cpan.org/
and http://kobesearch.cpan.org/.
Currently, the functionality is very basic, but we have great ambitions."
I learned about CPANHQ by overhearing a conversation on Freenode's #perl
channel. I joined the effort, and made several enhancements to the
Catalyst and
DBIx-Class-based
code, which was a useful learning experience. The functionality is still very
basic, and there's a lot of work to do, so any help would be appreciated.
Just fork the github repository ( or
my repository,
which I tend to keep somewhat more up-to-date ), and play with the code.
Now back to L~R's suggestions. At first I was a bit sceptical of them
and they seemed like an overkill, and not really useful. When I'm using
search.cpan.org to search for a distribution, either I know the name of the
module that I'm looking for and just perform a search to be directed
to the location of the exact page, or I usually find the order of the
search sufficient for my needs.
However, on a second reading, it seems like what Limbic suggests (like
being able to sort results by Kwalitee, creation date, number of downloads,
last update date, rating, etc., providing a code search, or recommendations)
actually has some merit and should be considered. One thing I'm sure about
is that the default CPAN search should remain simple, while only giving
an option for an advances search using a different form, where it might
be possible to formulate a more complex query using a specialised language.
Otherwise, if we complicate the existing search, it could prove confusing
to many users.
L~R's suggestions remind me of
my CPAN
Module-Rank thought experiment, that aimed to define an automatically
calculated metric for the relevance of CPAN modules.
L~R later referred me to a
post to Perlmonks.org titled "Advanced CPAN Search?" where someone
asks a similar question.
To sum up, please consider monitoring or even contributing to CPANHQ if
you want to improve the future of finding stuff and interacting with CPAN.
It seems that some people have issue with the current CPAN search
state-of-the-art, and would like to see it improved. We'd be happy to hear
what you think.
2009-07-01: Tech Tip: Finding CPAN Distributions that only have a Build.PL
A few times in the past, I wanted to find perl 5 CPAN distributions that only
had a
Module-Build-based
Build.PL file and not a fallback Makefile.PL file. Yesterday, after some
trial and error, I was able to formulate
Yahoo Search query, to do just that.
Here it is - the shortened URL with a hyperlink to the full one:
http://xrl.us/bezbkx - Yahoo Search to find Build.PL
only distributions
It works by looking for specific phrases in the /dist pages, looking for
"Build.PL" and specifically excluding "Makefile.PL". I hereby place this
URL and whatever associated techniques under
CC0 / Public Domain.
Enjoy!
2009-07-07: A Perlbuzz Feature by Me about CPANHQ
W00t! Andy Lester published
a feature I wrote about CPANHQ on Perlbuzz.
This is in continuation to my previous
blog entry about the topic, but much more focused.
This publicity seemed to have work, because we've been visited by two
new Perl people on #cpanhq,
who tore my hacky Catalyst code to pieces. Anyway, check the article out,
and if you like it, upvote it on reddit.
2009-07-12: הזמנה להשתתפות בכנס אוגוסט פינגווין 2009
כמידי שנה, קהילת התוכנה החופשית וקוד המקור הפתוח בארץ בחסות
עמותת המקור
יקיימו את
כנס אוגוסט פינגווין.
השנה יתקיים הכנס ביום שישי, ה-7 באוגוסט (2009), החל מהשעה-08:30 בבוקר בבניין
אבנר של מכון ויצמן. הכניסה לכנס עולה 30 שקלים כאשר חברי וידידי עמותת המקור ואיגוד
האינטרנט הישראלי לא צריכים לשלם.
אז מה יהיה בכנס?
2009-07-17: What you can do with File-Find-Object (that you can't with File::Find)
I've written about
File-Find-Object
before, but I've intended to write an entry demonstrating its philosophical
advantages over the core File::Find module. Today, I'd like to get to it.
As opposed to File::Find, File-Find-Object is:
-
Has an iterative interface, and is capable of being interrupted in the
middle.
-
Can be instantiated and be used to traverse an arbitrary number of directory
trees in one process.
-
Can return result objects instead of just plain paths.
I'd like to demonstrate some of these advantages now.
Case Study #1: Looking for a Needle in a Haystack
Let's suppose you have a huge directory tree containing many directories
and files, and you're looking for only one result (or a few ones). Once you
found that result you wish to stop. This question was raised in
this Stack Overflow
post.
So how can you do it with File::Find? Not very easily. Either you can throw
an exception:
sub processFile() {
if ($_ =~ /target/) {
die { type => "file-was-found", path => $File::Find::name };
}
}
eval {
find (\&processFile, $mydir);
};
if ( $@ ) {
my $result = $@;
if ( (ref($result) eq "HASH") &&
($result->{type} eq "file-found")
)
{
my $path = $result->{path};
# Do something with $path.
}
elsif ( $result ) {
die $result;
}
}
else {
# be sad
}
This is incredibly inelegant, and abuses the Perl exception system for
propagating values instead of errors. But there's even a worse way, using
$File::Find::prune:
#! /usr/bin/perl -w
use strict;
use File::Find;
my @hits = ();
my $hit_lim = shift || 20;
find(
sub {
if( scalar @hits >= $hit_lim ) {
$File::Find::prune = 1;
return;
}
elsif( -d $_ ) {
return;
}
push @hits, $File::Find::name;
},
shift || '.'
);
$, = "\n";
print @hits, "\n";
Here, we prune all the levels from the results up to the root to get out
of the loop.
So how can you do it with File-Find-Object? In a very straightforward manner:
#!/usr/bin/perl
use strict;
use warnings;
use File::Find::Object;
sub find_needle
{
my $base = shift;
my $finder = File::Find::Object->new({}, $base);
while (defined(my $r = $finder->next()))
{
if ($r =~ /target/)
{
return $r;
}
}
return;
}
my $found = find_needle(shift(@ARGV));
if (defined($found))
{
print "$found\n";
}
else
{
die "Could not find target.";
}
The find_needle() function is the important thing here, and one
can see it doesn't use any exceptions, excessive prunes or anything like
that. It just harnesses the iterative interface of File-Find-Object. And
it works too:
shlomi:~$ perl f-f-o-find-needle.pl ~/progs/
/home/shlomi/progs/Rpms/BUILD/ExtUtils-MakeMaker-6.52/t/dir_target.t
shlomi:~$
Case Study #2: Recursive Diff
Let's suppose an evil djinni has removed the -r flag from your
diff program, making you
unable to recursively find the differences between files in two directory
tree. As a result, you now need to write a recursive-diff program in
Perl that will run diff -u on the two copies of each equivalent path
in the two directorie.
Since File::Find cannot be instantiated two times at once, then when using
it, we will need to collect all the results from both directories, and then
traverse them in memory. But with File-Find-Object there is a better way:
#!/usr/bin/perl
use strict;
use warnings;
use File::Find::Object;
use List::MoreUtils qw(all);
my @indexes = (0,1);
my @paths;
for my $idx (@indexes)
{
push @paths, shift(@ARGV);
}
my @finders = map { File::Find::Object->new({}, $_ ) } @paths;
my @results;
my @fns;
sub fetch
{
my $idx = shift;
if ($results[$idx] = $finders[$idx]->next_obj())
{
$fns[$idx] = join("/", @{$results[$idx]->full_components()});
}
return;
}
sub only_in
{
my $idx = shift;
printf("Only in %s: %s\n", $paths[$idx], $fns[$idx]);
fetch($idx);
return;
}
for my $idx (@indexes)
{
fetch($idx);
}
COMPARE:
while (all { $_ } @results)
{
my $skip = 0;
foreach my $idx (@indexes)
{
if (!$results[$idx]->is_file())
{
fetch($idx);
$skip = 1;
}
}
if ($skip)
{
next COMPARE;
}
if ($fns[0] lt $fns[1])
{
only_in(0);
}
elsif ($fns[1] lt $fns[0])
{
only_in(1);
}
else
{
system("diff", "-u", map {$_->path() } @results);
foreach my $idx (@indexes)
{
fetch($idx);
}
}
}
foreach my $idx (@indexes)
{
while($results[$idx])
{
only_in($idx);
}
}
( As a bonus, we do not need to sort the results explicitly at any stage,
because File-Find-Object sorts them for us. )
This program did not take me a long time to write, it works pretty well,
and does populate a long list of results of one or both directories.
Conclusion
If you use File-Find-Object instead of File::Find, your code may be
cleaner, your logic less convulted, and you may actually be able to
achieve things that are not possible with the latter. I hope I whet your
appetite here and convinced you to give File-Find-Object a try.
So what does the future holds? I recently ported File-Find-Rule to
File-Find-Object and called the result File-Find-Object-Rule . As a result,
"->start" and "->match" are now truly iterative, and I believe
you can iterate with them on several objects at once. As I discovered by
porting File-Find-Object-Rule-MMagic,
I unfortunately cannot maintain full backwards compatibility with the plugin
API of File-Find-Rule, because the latter exposes some of behaviour of
File::Find (in
a
leaky abstraction fashion).
I'm planning on porting more File-Find-Rule plugins to File-Find-Object-Rule,
and would appreciate any help. I also would like to look at the directory
tree traversal APIs of other languages to see if they contain any
interesting techniques.
2009-07-20: IRC Channel for Perl in Israel - #israel.pm on irc.perl.org
This Hebrew announcement will be followed by an English one.
We have started a new Internet Relay Chat
channel for Perl in Israel - #israel.pm on the irc.perl.org network. You are invited to come there and chat with us.
2009-07-25: Preparing a Screencast Under Linux
I wrote about
the screencast
of PySolFC solving Freecell automatically on the fc-solve blog. As
I promised there, I wanted to tell the story of how I ended up
with a ready screencast. The process was not completely free of problems,
but proved to be quite straightforward.
Googling for screencast and linux yielded this blog post at
liunxhaxor.net. The first program I tried from there was Istanbul, but
it proved to be too crashy, and could never save the video to the final
file. Wink is non-open-source so I prefered not to use it, and I had a bad
prior experience with vnc2swf. Then I noticed it also mentioned
recordMyDesktop,
which is a command line program that worked very well and was able to
produce a working video. (recordMyDesktop has two GUI front-ends, but I
did not bother to learn how to use them, but they may work for you.)
After I had the recording of the screen, I needed to edit it, because one
could still see the cursor starting from the window where I ran it, and then
me Ctrl+Cing it. And I also wanted to add some background music. So I looked
for a good video editor. I had little luck with either
cinelerra or
Kino, but someone on IRC recommended
avidemux, and it proved
easy enough and bug-free enough to use. Nevertheless, it refused to import
the Ogg Video (Theora) file
that I got from recordMyDesktop, but I was able to convert it to
AVI format first with ffmpeg.
Using avidemux, I was able to delete the beginning and end of the file, as
well as add some background music (from
Adam Singer's
album "Lifeforce" - available under a Creative Commons licence).
Afterwards I had a ready video file, which I uploaded to YouTube. While
I waited for the file to upload, I wrote some blurb for the description.
So far the video received 67 views and did not receive any comments. I still
found learning how to make a screencast highly enjoyable, and look forward
to preparing future screencasts. I hope I've given you some insights into
how to prepare it.
Happy screencasting and watching!
2009-07-27: Updates to Perl-Begin.org
After the last news item, a lot of work was invested into Perl-Begin.org, the Perl Beginners' Site, making
it even better than it used to be. Here's a summary of the changes:
We hope you enjoy the new Perl Beginners' Site and please recommend it
to your friends. All content on Perl-Begin is made available under the Creative
Commons Attribution Licence which allows almost unlimited re-use.
2009-08-03: More updates to Perl-Begin.org
I thought the Perl Beginners' Site
was perfect after
the
last update, but boy I was wrong. A quick review and critique of the site
by a certain prominent Perl developer proved me wrong, and afterwards I
had more stuff that I found lacking. So here's what has changed.
We're now mirroring
the
public domain "Perl for Newbies" tutorial as well as the GFDLed book
"Impatient
Perl" by Greg London. Their
presence on the site was done in order to preserve Perl-Begin's common look
and feel, and to make sure people would be able to access them without
having to go to a different site, which may be blocked by over-zealous
web-filters. We hope to mirror other freely distributable material that
may prove useful to beginners on Perl-Begin in the future.
The "call-for-action" notice at the beginning was changed to
"Learn Perl Now!" and now appears on all the pages.
We also fixed several look and style glitches.
There are also several corrections to text and links.
Stack Overflow was added to
the web-forums page.
There are now mentionings of
two new topical books -
"Perl & LWP" and "Programming the Perl DBI".
Finally, the
"About this site" page
was updated and made more modern.
We hope you enjoy the new site. If you like it, please recommend it to
your friends - if you don't - let us know and we'll see what we can do.
2009-08-10: Perl readdir segfault.
I was working on the 5th part of
my
Perl for Newbies tutorials when I wrote some example Perl code like
the following:
my $dir_path = shift;
opendir my $dir_handle, $dir_path
or die "Cannot open $dir_path - $!."
my @files = readdir($dir_path);
closedir($dir_handle);
(Can you spot the bug?)
Much to my surprise, this code segfaulted when I gave it "." as an argument.
I realised that the problem was that I was reading from the path variable
instead of from the directory handle, but it still shouldn't have segfaulted.
I asked a few people on IRC about it, and some of them were able to reproduce
it, but only with perl-5.10.0 on Linux. Someone was able to golf it down
to perl -e 'readdir($x=".")', which segfaults too.
So I opened
a
ticket for this bug, and Rafael (RGS) fixed it in blead, but without a
regression test case. So I submitted
a
patch to add it, and also pushed it to my
github clone of the
perl repository. I hope it get applied.
On a different note, I'd like to note that I have completed the transition of
the automated test suite of Quad-Pres
from Bash to Perl 5 and Test::More TAP-emitting tests. Now they are much less
verbose, and possibly run faster (though it might be just a feeling).
Now I'd like to continue its switch from GNU Autohell to something more
decent. I still don't know whether I should use
CMake or
Module-Build as
I depend on the wml executable from Website
Meta Language. Any insights would be appreciated.
2009-08-10: Tech Tip: listing all files without "." and ".."
Today I had to deal with some pesky dot-files again, and again ran into
the problem that ls -a listed them along with "." and ".."
which were useless at best. A common idiom in filtering them out is using
ls -a | grep -v '^..\?$' (or something similar using egrep or
grep -P if it's available). But being too lazy to write it, I did
a man ls and discovered a better way, right at the first page of
the manual.
One can simply use ls -A or ls --almost-all to filter out
the implied "." and "..". At least, this works with GNU ls - don't know about
BSD or the various proprietery UNIXes.
So for example, ls -a may give you:
. .. FCFS_RWLock_Scheme_RLE.txt FCFS_RWLock_Scheme.txt .svn
While ls -A may give:
FCFS_RWLock_Scheme_RLE.txt FCFS_RWLock_Scheme.txt .svn
Enjoy! I hope to write about
August Penguin soon, so stay
tuned.
2009-08-17: Source of the perl.org Sites Are Now Available
Thanks
to Matt S. Trout (mst) and his awesome "power of asking", the
sources of the perl.org
domains are now available. I've wanted to merge sofe of my work on
Perl-Begin to
learn.perl.org and
www.perl.org , and now my job will be
much easier.
So thanks to Matt, and the www.perl.org maintainers who made the sources
available.
2009-08-20: XSLT Facts - find out how truly evil XSLT is.
We were chatting on
##programming when
someone came and asked for help with using XSLT (Extensible Stylesheet
Language Transformations - an XML-related standard) from within
Perl. We helped him, but meanwhile
someone else prompted us to start an "XSLT facts" marathon similar to the
Chuck Norris Facts
and other similar collection of facts. While
it can be classified as geek humour, most of the uttered facts are more
universal than geek culture. (I am rindolf)
-
→xmler: has joined ##programming
-
xmler: Hi
-
xmler: How can I do <a href="$variable">blah</a> in XSLT?
-
rindolf: xmler: you can use <xsl:attribute>
-
rindolf: xmler: or use ${...}
-
rindolf: xmler: have you read the Zvon XSLT tutorial?
-
xmler: rindolf: I've read various resources; perhaps it's the fault of XML::XSLT on CPAN
-
rindolf: xmler: use XML::LibXSLT instead.
-
rindolf: It's faster and better.
-
Zuu: <insert mean statement about XSLT>
-
Zuu: :)
-
rindolf: "XSLT is the work of Satan"
-
rindolf: "It's the worst thing since non-sliced bread."
-
Zuu: Hahaha :D
-
rindolf: "Mothers used to tell their children about XSLT to scare them."
-
xmler: Haha
-
Zuu: That last one is good
-
xmler: XSLT seems pretty cool
-
Zuu: xmler, no no, that's not mean
-
Zuu: you're doing it wrong
-
Zuu: :P
-
xmler: XSLT is too cool to be mean to :p
-
Zuu: Hehe
-
rindolf: "XSLT is the number one cause of programmers' suicides since Visual Basic 1.0"
-
tommy_the-dragon: lol
-
Zuu: :>
-
Zuu: I could believe in that
-
rindolf: "The X in XSLT stands for eXtermination."
-
rindolf: "XSLT makes the baby jesus cry."
-
rindolf: "The only things worse than XSLT are Excel and sugarless tea."
-
tommy_the-dragon: rofl
-
* Zuu: kinda likes Excel
-
rindolf: Zuu: yes, it was a joke.
-
Zuu: Noooh!
-
Zuu: they are all true!
-
rindolf: Now I'm out of ideas.
-
Zuu: i havent even had a single idea yet..
-
rindolf: "XSLT is what Chuck Norris has nightmares of."
-
Zuu: Whoah!
-
Zuu: wait.. that would somehow make XSLT cooler than Chuck Norris...
-
Zuu: Chuck Norris dont have mightmares
-
rindolf: "Confucius e says: 'XSLT made me realise humanity was hopeless."
-
Zuu: "Even APL wont make friends with XSLT"
-
Zuu: yay, i maded one!
-
rindolf: "God considered using XSLT as the tenth plague of Egypt, but thought it was too evil."
-
rindolf: Zuu: :-)
-
Zuu: Haha :D
-
Zuu: that plauge one is awesome
-
rindolf: "In Soviet Russia, XSLT codes you. Badly!"
-
rindolf: "Satan condemned Hitler for a million years of writing XSLT"
-
rindolf: OK, back to work.
-
rindolf: C - not XSLt.
-
Zuu: :)
-
rindolf: "The KGB used to torture their victims by having them look at scrolling XSLT code"
-
xmler: rindolf: lol, love the KGB + XSLT one
-
xmler: rindolf: by the way, what's with the XSLT hate by the way, is it some kind of meme? :p
-
* Zuu: gives xmler an XSLT interpreter written in XSLT
-
Jck_true: I wanna make ASM code from XSLT
-
rindolf: "My name is Inigo Montoya. You forced my father to write XSLT. Prepare to die! And be thankful I don't force you to write XSLT."
-
xmler: Ah for gawd's sake. Can't install XML::LibXML, it says I don't have libxml2 (I do)
-
rindolf:
xmler: do you have the -devel package?:
-
rindolf:
xmler: what is your distro?:
-
rindolf:
OS distro I mean.:
-
xmler:
rindolf: Debian 4.0:
-
xmler:
Ooh, perhaps I don't actually:
-
rindolf:
xmler: ok, then apt-get install libxml-libxslt-perl:
-
* Zuu:
have to go o/:
-
rindolf: xmler: maybe we should discuss it on #perl?
-
rindolf: Zuu: bye
-
rindolf: Zuu: and beware of stray XSLT code.
-
Zuu: I will :P
-
rindolf: Zuu: OK.
-
xmler: rindolf: sure, uno memento :)
Just for the record, I don't really hate XSLT, or think it's evil or
particularly scary, but I found generating such facts fun. This bit
was in fact converted to HTML using an XSLT transformation I wrote.
2009-08-23: Open Source Licences Wars
I published a new essay on my site titled "FOSS Licences Wars"
which "gives an overview of the various options in picking up a licence for
one's Free and Open Source Software (FOSS) project, and tries to give some
guidelines choosing one". Reading from the article:
When Joel Spolsky
("Joel on Software") wrote his notorious blog post
"Language
Wars", many people asked whether he has "jumped the
shark" and that his blog will go downhill from there. I too have read
the post, and agreed, that while it had a few good points, it was
too based on "feeling rather than knowing". Joel later on posted
many good articles and shorter entries on his blog, but many people
still recalled it as a very low-point in the blog.
Like Joel, I have a home-site and several blogs, where I post articles
and essays about my thoughts, and this time I've decided to risk
something similar to what Joel has done on an equally flamatory topic:
licences
of open-source software. I'm going to introduce the various
options, explain a little about their motivation and then
give some advice according to my own personal opinion.
Comments are welcome here or on
one
of the Internet forums where the story was covered. This essay is
made available under the CC-by licence, which allows full re-distribution,
and re-use as long as credit is given. Nevertheless, a donation or sharing
derived works under similar licences, would be welcome.
2009-08-27: A Valid Use of Ampersand-func?
I was talking on #perl one
day, when someone joined and asked:
foldr (\(a,b) -> (a:).(b:)) [] $ uncurry zip \
("Hv o edyu IPtdy","aeyura orSC oa?")
<- What would this be in Perl? (It outputs
"Have you read your SICP today?")
In case you don't know your Haskell, what it does is interleave the two
strings and have 1 letter from here and the other one from here. In Perl,
strings are not lists or sequences, so we understood we'd need to split
them first to do something like that. Furthermore, we recalled the
"zip" function from
List-MoreUtils
that may prove to be useful.
Eventually, someone came with this Perl 5 solution:
perl -MList::MoreUtils=zip -e 'my @former = split //, "Hv o edyu IPtdy";
my @latter = split //, "aeyura orSC oa?";
print join "", zip @former, @latter'
Assigning to array variables was necessary because zip has a prototype. Then
I figured out that we can avoid that by over-riding the protoype using
"&zip", and proposed this solution:
perl -MList::MoreUtils=zip -e 'print join "", &zip(map { [split//, $_] }
"Hv o edyu IPtdy","aeyura orSC oa?")'
Which was much shorter and clearer but used "&zip". What happens is that
we pass it the arrays directly as references. I was able to golf it somewhat
into:
perl -MList::MoreUtils=zip -E 'say&zip(map[split//],"Hv o edyu IPtdy",
"aeyura orSC oa?")'
So we got what appears to be a valid use of "&func" to over-ride a pesky
prototype. Now, Perl 6 hackers are invited to contribute a Perl 6 solution.
2009-08-31: MooseX-Getopt Hacking
It seems I didn't announce it here, but
Test-Run version
0.0120 was converted to Moose, the
postmodern object system for Perl 5. After I uploaded all 0.0120-related
modules, I was looking for more Moosification, and decided to try and
convert runprove to
MooseX-Getopt,
while still passing all tests.
After a lot of playing with the has attributes to get them I ran into
what appeared to be a problem which I could not overcome: I could not
configure it not to use
Getopt-Long-Descriptive, in case it found it was installed. I didn't want to use it,
because I wanted my help display.
After talking about it on #moose,
I was told that it is indeed a limitation. I decided to fix it and did it
by adding a new parameter. However,
t0m told me that I
should do it by extracting the non-GLD dependent functionality into a separate
role, and doing it. I did it, and he had some further comments which
I incorporated into my
MooseX-Getopt
repository. My enhancements are not in the CPAN module yet, but they
are going to be soon.
And I live you with a #perl
discussion about Chuck Norris:
<Su-Shee> 2010 is planned more or less as a refactoring year.
<DrForr_> A whole *year*? What physical plane of existence do you reside
on?
<Su-Shee> DrForr_: there's also bugs to fix and systems to care for and
things like that. it's not that we're locked into the closet
and a year later a new, shiny product is released. ;)
<DrForr_> Closet optional.
<Su-Shee> I'm not a wonderwoman refactoring half a million lines of perl
in a week, sorry. :)
<rindolf> Su-Shee: Chuck Norris refactors 10 millions lines of perl
before lunch.
<Su-Shee> rindolf: hm. that's the reason.. I'm not as hairy as chuck
norris and I don't have a beard...
<rindolf> Su-Shee: ah.
<rindolf> Su-Shee: Chuck Norris also wrote a complete Perl 6
implementation.
<Su-Shee> rindolf: I heard, he already wrote Perl 7.
<rindolf> Su-Shee: yes.
* rindolf wants to be as awesome as Chuck when he grows up.
<Su-Shee> rindolf: I envy you. I'll never be as awesome without a beard.
<rindolf> Su-Shee: heh.
<rindolf> Su-Shee: it doesn't matter if you're rigid on the outside as
long as you're rigid on the inside.
<Su-Shee> chuck norris doesn't make mistakes.
<rindolf> Su-Shee: Chuck Norris corrects God.
<Su-Shee> rindolf: I'll apply as his secretary.
<rindolf> Su-Shee: as Chuck's?
<Su-Shee> rindolf: yes.
<rindolf> Su-Shee: OK.
<rindolf> Chuck Norris doesn't code. When he sits next to a computer, it
just does whatever he wants.
<Su-Shee> I'll tell my boss tomorrow. Chuck is who he wants.
<rindolf> Su-Shee: Chuck Norris is his own boss.
<rindolf> Su-Shee: if you hire him, he'll tell your boss what to do.
<Su-Shee> good point.
2009-09-01: What Would you like in a CPAN Distro Manager?
I uploaded Dist-Man
(short for Dist-Manager) version 0.0.1 to CPAN, yesterday. It is based on
Module-Starter , and
its documentation still has some remnants from it, but I'm planning to take
it into new and exciting directions. The source code is maintained in
a Mercurial repository
at Berlios.de (called "web-cpan" after the
meta-project for most of my CPAN
modules - I'm only given one Mercurial/hg repository per-project there),
and there's also a bitbucket
mirror. Feel free to clone it, fork it, branch it, spin-it-off, or whatever
the cool kids are doing now with their distributed version control
repositories. I can also give commit bits. The reason I'm using Mercurial is
because I'd like to play with it a little more, and because git still often
throws me off when using it and still seems counter-intuitive. Like an old
email correspondent of mine said "You gotta move, or you stone.".
In any case, I'd like to share some plans and ideas I have in mind for a
distribution manager similar to Module-Starter and ask you for your
input. Note that my plans take an entirely
different approach from
Dist-Zilla. While Dist-Zilla generates your code from
templates, Dist-Man intends to deal with the code as is, and modify it
in place, assuming a change is needed.
So without further ado, here's what I'd like to see, or at least seems like
a good idea:
-
Provide scaffolding for more lib/*.pm t/*.t , etc. files. You can run
something like
pl-dist-man new --name=lib/MyPackage/NewModule.pm and it will
create lib/MyPackage/NewModule with all the POD and some sample code ready
to go, possibly with some tests under t/, etc.
-
POD at the end. - that's what
Perl Best Practices
recommends, and what Perl-Critic
checks for by default. It's not my personal style, but I heard someone
complain to me about it in private.
-
Keep a machine-readable configuration file with the settings - keep a
configuration file in YAML/JSON/Config-General/etc. format in the distribution
that will allow Dist-Man (and other programs) to determine configuration
options about the distribution. So, for example, it can use the licence field
to determine what to put in the licences of new modules, programs and test
scripts. And we can have Build.PL, Makefile.PL read this file and feeding
it to Module-Build or
Module-Install or whatever.
-
Hermetically changing configuration parameters in the code files - you
can change the licence in all the files in one step. Or bump the version
number in all files.
-
Support for more software licences - perhaps with some
integration with Software-License.
-
Make judicious use of Template-Toolkit - I'd like to make the
generation of the Perl code be based on
Template Toolkit so it can
be easier to maintain. The base Module-Starter code is at the moment modular,
but seems to crying for help from a good templating system.
-
Good integration with the underlying version control system - when
you do pl-dist-man add lib/MyPackage/File.pm it will add it
to the MANIFEST and the underlying version control system, and you can
move files or directories around, hermetically rename namespaces, etc.
-
A plug-in to add a diversity statement - see
what
Kirrily Robert wrote about it here. This plug-in will
allow the developers to add it automatically to all of their modules. I don't
rule out coming up with such a plug-in for Module-Starter too, but I just
wanted to mention it here.
While I have given Dist-Man as an example, this discussion should not be
specific to any particular implementation. So what would you like to see in a
manager of CPAN distributions?
Note: Yes, I'm aware that Dist-Man is a fork of Module-Starter, and I
tried my best to avoid it, but it seems I had no option. The reason for the
fork were the fact that there was a large round-trip delay when applying
my patches (usually several months), the fact that I could not get either
hgsubversion or git-svn to handle importing the history from its Subversion
repository
well (see this hgsubversion issue
entry), and the refusal of the Module-Starter maintainer to provide me
with a commit access, even after I promised that I will only work on a
separate branch, and not commit to the trunk directly (at least not without
his permission). I let him know about how I feel, and of my intention to
fork the project, and he implied he'd rather see me fork the code instead
of complying with one of my needs.
I feel bad about forking it but I feel like I had no choice, and
as Eric Raymond (ESR) says in the Magic Cauldron:
(It bears repeating here that nobody in the hacker community wants projects to
split into competing development lines; indeed (as I observed in Homesteading
the Noosphere) there is very strong social pressure against forking, for good
reasons. Nobody wants to be on a picket line, in court, or in a firefight
either. But the right to fork is like the right to strike, the right to sue, or
the right to bear arms—you don't want to have to exercise any of these rights,
but it's a signal of serious danger when anyone tries to take them away.)
So I'm not particularly interested in discussing it further. Please keep the
discussion on the topic of what you'd like to see in a manager of
CPAN distribution, be it Module-Starter, Dist-Man, Dist-Zilla or whatever.
2009-09-02: TelFOSS Meeting on 6-September-2009: Advanced GDB
The Tel Aviv Open Source Club
will host the talk "Advanced GDB" by David Khosid on
Sunday, 6-September-2009 (the upcoming Sunday), at 18:30 in Tel Aviv
University, the Schreiber Maths&CS building, room 008.
Attendance is free, it is not necessary to RSVP and everyone are welcome.
With any other problems, feel free to contact me.
Abstract
In general, I will be sharing my discovery with you: When the debugging of
modern software is required, basic GDB techniques are insufficient, but new
techniques can be created from the nearly 160 commands available in GDB.
"Modern software" refers to multi-threading, using STL and other libraries,
IPC, signals and exception mechanisms. In this lecture, I will explain
techniques for debugging large, modern software written in C++.
The presentation will be accompanied by vivid examples for all the topics.
Covered topics:
1. Preparing a program for debugging; starting the debug session
2. Controlling GDB: getting help, navigating, using breakpoints and watchpoints
3. Extending GDB through user-defined commands
4. Analyzing STL containers and algorithms
5. Dealing with multi-threading, C++ exceptions
6. Managing signals
7. Automating repetitive tasks on the almost unchanging code base
8. Remote debugging
2009-09-05: French Translation of the "Encouraging Women in FOSS" Howto Needed
Abstract: I'm calling for a volunteer effort to translate
the "HOWTO Encourage Women in Linux" document by Val Henson and others.
Background: I've recently been communicating and collaborating
with a young (16 years-old) French open-source enthusiast. His English is
pretty bad, but I can communicate with him with some difficulty. He's using
Linux, and has two computers in his room, but his younger 8 years-old sister
has none in hers ("Unfortunately" as he said). This reminded me of
this
section from the "encouraging" howto where it is said that:
The most striking example of a subtle bias against computing for women is that,
in the U.S. at least, the family computer is more likely to be kept in a boy's
room than in a girl's room. Margolis and Fisher give several telling examples
of this trend and its effects on pages 22-24 of Unlocking the Clubhouse.
Now, in Israel, the common practice among most modestly capable families is
to have a separate computer for every member of the family (whether
male or female), so I don't see why his family should be backwards here, given
that they can afford it, and that nowadays a powerful PC computer can be
bought for 400 USD (or less) and that can run circles around my original
PC XT machine, on which I've learnt programming and played many games.
The problem is that this kid's parents don't speak English, and so I need
a French translation for that. While some of the Linux documentation project's
howtos have been translated, I couldn't find any French translation of the
"Encouraging women in FOSS" howto. I suggest setting up a page for collaborating
on the translation for it on a public wiki, and that everyone contributes their
share at their spare time.
Translations to other languages will also be welcome, and I can give my share
of the work for a translation to Hebrew (though I feel it won't be necessary
because the vast majority of Hebrew speakers have decent if not good
English). If you're up for the task, please mention the wiki page with
your respective language below.
Hazal said that "Whoever saved a single soul - has saved the world entire.".
Note: while I have had
some
critique and reservations regarding the HOWTO, I still think it makes
a good read, and should be taken into consideration.
2009-09-11: Perl-Begin Updates: CSS, Page about Hashes, and More
Here are the new changes to the Perl
Beginners site since last time.
-
A New Page about
Hashes has been added to the site, with "Jerusalem Perl Monger"'s
excellent presentation called "The Hash Cookbook".
-
There are many corrections to
the "Perl
for Newbies" presentations, in part thanks to input from several
people who tried to learn Perl using them.
-
The CSS stylesheet of the page was compressed, minimised, transformed
to a fluid design and converted to
the compass CSS frameworks. Many
thanks to Alan Haggai Alavi, who
did the bulk of the work on the conversion of the page to compass.
-
The "Mailing Lists" page
was enhanced with a new introduction and some link fixes.
-
The IDEs page now
has some screenshots.
-
We've heard many complaints from experienced Perl users about a joke that has
appeared on the top of the
"Get a Job Doing Perl" page, so it was now removed.
2009-09-24: Hamakor Prize Winners for 2009
Hamakor is the Israeli Society
for Free and Open Source Software. Among the activities of the society, it
gives the "Hamakor Prize" for contribution to open source in Israel, whose
winners are determined by a vote by the society's friends and members. This
year, the prize was given for several categories.
Better late than never, we wish to announce the winners
in the Hamakor Prize for 2009, in the various categories:
-
The winner in the contribution to open-source category
is Gábor Szabó (szabgab). Gabor has jump-started
the community of Perl users in Israel; he set up an active mailing list,
organised monthly meetings, as well as several Israeli conferences. In addition,
he donated a lot of code and documentation for projects such as perl5, Parrot,
Perl 6. He has several modules
on CPAN and lately has initiated the
Padre project for implementing an
integrated development environment for dynamic languages
and is very active in its constant development and maintenance.
Gábor manages Perl Technology Israel (PTI)
which offers training courses in Perl, in Quality Assurance (QA) and in other
fields in Israel and Internationally.
-
The winner in the contribution to the Israeli community is
Shachar Shemesh.
Shachar is one of the founding membes of "Hamakor" and served as a board
member three times, including in the initiating board.
Shachar is the owner and manager of Lingnu
Open Source consulting Ltd., which provides services in the field
of open-source software. It administers several of the servers of Hamakor.
Shachar has also presented talks in the
Haifa Linux Club,
the Tel Aviv Open Source Club,
the Herzelia Linux Developers Club,
and the Jerusalem Linux club.
Furthermore, Shachar has founded and is managing several free software
projects such as
rsyncrypto
and fakeroot-ng and also
has contributed to other open-source projects including
WINE, a compatibility layer for
Microsoft Windows.
-
The winner of the Shmu'el Yaniv Prize for Open Source and Free Software
in the Localisation Category is
Tomer Cohen. Tomer is concentrates the
translation of Mozilla products in Israel, and is also active in
advancing and publicising the Mozilla products in Israel and making
sure web-sites are compatibile with them. He manages a blog on his site that
discusses Firefox, standards compatibility and other web-related topics.
Tomer heads Mozilla
Israel - mozilla.org.il, the Israeli community of the
International Mozilla project, which is developing a large number of popular
open-source applications.
-
The winner of the appreciation certificate for open source in education
is the department of education in
Petah Tikva.
It set up a Moodle system for
education as well as a an instance
of the Joomla content management system,
a WordPress-based blog,
a MediaWiki system
some classrooms and labs running Linux (LTSP
client/server) as well as setting up various open-source applications
on machines running Microsoft Windows.
Congratulations to the winners and the other candidates, who didn't win
this year, but whose contributions are still worthy of appreciation.
2009-09-27: Yak-shaving Perl Hacktivity
I recently asked the module-authors mailing list about which parser generator to use for
Text-Qantor. Eventually I opted to use
Damian Conway's new
Regexp-Grammars
as recommended
by Ovid. So I started converting the code from the broken Parse-Yapp
grammar to it. I had some tests and ended up getting it to work after
some trial and error and debugging.
Now, Qantor used to convert from its plain text representation directly
to XSL-FO,
but I preferred to convert it to an intermediate XML format instead. So
I worked on it and finished. However, then I needed to translate the XML
to XSL-FO, so I wanted to write an XSL transformation, and then realised I've
actually done such stuff a lot before:
XML-Grammar-Screenplay,
XML-Grammar-Fortune,
XML-CompareML, etc.
As a result, I decided to refactor all the common functionality into a common
XML-GrammarBase distribution.
Shouldn't be hard, right? Well, not so fast. When trying to create the skeleton
of the distribution using a pl-dist-man setup … command, (see
Dist-Man),
it died with an error - so I needed to fix it first. The fix was not hard,
but I needed to extract some common test functionality into a ./t/lib
module. However, the
Test-Count used some
Test-Count variables that now moved to the file. So I needed to implement
an include functionality. (Yak Shaving)++
Implementing this functionality in Test-Count was time consuming, and I did it
by changing many files. You can now do
# TEST:source "$^CURRENT_DIRNAME/lib/MyMoreTests.pm";, which includes
the file.
Then I went back to Dist-Man re-counted the text and fixed it, and now I can
work on XML-GrammarBase again. This is what is called
yak
shaving, but this time it yielded many benefits to my other projects.
2009-10-03: Perlsphere Implemented Using Perlanet
You may be familiar with Perlsphere,
which is a Perl-related feed aggregator, which is more open for membership
than "Planet Perl". I've encountered some problems with its web feed,
including some entries that displayed the HTML markup, and the fact that
web feed entries are not prefixed with the name of the Blogger or Blog.
As a result, I decided to experiment with implementing the same functionality
as Perlsphere, only using
Perlanet, which is a
lightweight PlanetPlanet-like web feed aggregator written in Perl.
So I present
perlsphere-using-perlanet.
There are links to the OPML file, the Atom feed, and the source
repository at the top. So far it seems to work pretty well, and it may be
a viable replacement for the Plagger-based Perlsphere in the future. Note:
the URL and everything about it are temporary, and I don't update the
feed peridiocally at the moment (but I can set up a cronjob). The OPML file
may be out-of-date too.. Anyway, comments and bug reports are welcome.
I should note that I've locally applied
the patch
in this XML-Feed bug report which also affected some of my CPAN modules,
and you may need it to get it to work.
2009-10-05: Pre-Announcement for the "Welcome to FOSS" Series of TelFOSS
In previous years, the
Tel Aviv Open Source Club has conducted the
"Welcome to Linux" series
along with other Israeli clubs, towards the beginning of the scholastic year.
This year the series will be in a different format of
Welcome to the World of Open Source
Software (also see
the wiki
page). We intend to introduce free and open source software to the users
in general, and among it many open source programs that can be ran on both
Linux and Windows, as well as introduce the ideology and philosophy of the
open-source code and their technological and social advantages.
Right now, we're looking for someone to give the demonstration of the
cool open-source programs, which is the first presentation which will take
place a few weeks after Sukkoth. Ori
Idan has already volunteered to give the presentation about the
philosophy of open source software. After these two presentations, we intend
to give several secondary talks. Among them would be
"Free Development Tools for Linux and Windows" (a general overview of them,
and not only those that are intended only for C or for Java), as well as
"Don't be afraid of the command-line" where we will show how nifty the
command-line can be. So we're also looking for volunteers to present these
talks.
Whoever wants to pitch in and contribute may
contact Shlomi Fish,
the coordinator of the series and the club and
subscribe to the
organisation mailing list (which doesn't have a lot of traffic at
the moment).
Be free and Happy Sukkoth.
2009-10-22: Code/Markup Injection and Its Prevention
I'm running out of time for a post to
Planet Perl Iron Man,
so I'm going to prepare something quick, but hopefully enlightening. I'll use
Perl as a demonstration language, but the practices I'm going to cover
tend to be more universal.
Starting from the early UNIXes and before, operating systems represented code
and its data as sequences of units of a certain number of bits called
bytes. Starting from Unix,
which was designed to run on such machines as the 16-bit PDP-11 and later on
the 32-bit VAX family of computers, this unit has generally become 8-bits.
Today, there are many 8-bit based text encodings (and many more binary
encodings for binary data), and the interested reader is referred to
Joel Spolsky's
introduction on the subject
and
Juerd Waalboer's
perlunitut or your language's equivalent document.
In any case, let's suppose we have a string where we want to embed a
variable containing a string. In Perl we can do:
my $total = "Hello " . $adjective " World!";
Or more simply:
my $total = "Hello $adjective World!";
So if we put "beautiful" in $adjective, we'll get
"Hello beautiful World!" in $total and if we put
"cruel" there we'll get "Hello cruel World!" there.
So far so good, if it's a plain string written in plaintext. However, what
if it's in a more well-formed format? Let's say HTML:
# Untested
my $input = get_input_from_user_somehow();
print <<"EOF";
<p>
$input
</p>
EOF
The alert reader will notice that $input was inserted as is into the HTML
output. And since we didn't check if it contains special characters or
escaped its special characters, a malicious user can insert arbitrary HTML
code and even JavaScript code there. This in turn can wreck havoc upon the
users of the page.
This form of HTML injection is called a
a
cross-site-scripting attack (XSS). If present in web applications or
web-sites, it may allow malicious crackers to set up traps to the unwary,
and possibly gain access to sensitive information on the site, such as the
passwords of users or administrators. And you did notice how easy it was
to write code that exhibited this problem, right?
Here are some other forms of code or markup injection:
-
Shell Command Injection - I've discussed it briefly
in
a different post about "shell variable injection" in Bash, but it also
exists in Perl. Imagine doing system("ls $dir"); or
as some newcomers are tempted to do `ls $dir`, which the latter
still has some legitimate uses. Now I as a malicious user can put in the
$dir variable some malicious shell code which will wreck havoc on
the system of the user that is running the script.
-
SQL injection
allows a user to inject malevolent SQL code that can do untold damages
in the database. It is very common in web applications and many
other applications that use SQL code. If you do something like
"SELECT id FROM users WHERE name='$name'" then by putting
single-quotes in the name, and using SQL syntax one can insert arbitrary SQL
there and do a lot of damage. There was also
a very nice xkcd comic about it.
-
Perl Code injection - let's suppose we want to construct an optimised
anonymous function ( sub { ... } ) on the fly. We can build
its code and then use
the
string eval - eval "". A lot of Perl programmers think it should be
avoided at all costs, but
metaprogramming
has some legitimate uses. Moreover, this can happen in other cases,
like when we construct a Perl program (or a program of any other language
on the fly and execute it).
In any case, if we insert a variable into the eval "" which was input
from the user without being escaped or validated, we can have an arbitrary
code execution.
-
Regular expressions' code injection - imagine you want to see if a
string is contained in a list of strings. One naïve way would be to concatenate
the strings using a separator that is unlikely to be contained in them
(such as \0) and then match this gigantic string using
$haystack =~ m{$needle}. However, if $needle contains special regex characters,
then the operation can take a lot of time to match or worse - yield an
incorrect result. One way to avoid that is to use
perldoc -f quotemeta or its \Q and \E regular expression escapes -
$haystack =~ m{\Q$needle\E}g. In this particular case, it is also
probably better to use a hash, but naturally this was just one example
where we'd like to embed some arbitrary (but plain) text inside a regular
expression.
These are the prominent examples I can think of now, but they are not
the only ones. Your program is in danger whenever it accepts text input from
the user and passes it directly to an output format that has some grammar and
syntax that can be influenced by this string.
So how to mitigate such code injection problems? There are many ways -
sometimes providing alternatives and sometimes complementing each other:
-
Make sure you have enough discipline to escape the input before it
is passed to the output venue. Write automated tests for that.
-
If you still want to allow some user input, then make sure that you
analyse it to make sure it doesn't contain any malicious code
that can abuse the system. For example, you may wish to restrict input
only to certain HTML tags and attributes.
-
Taint your data using unsafe typing or "kinding" and make sure that
it can only be output after either being escaped or being untainted. Joel
on Software recommends
making the
wrong code look wrong, which while desirable and important, is
probably less preferable than the wrong code to behave wrong
and abort with a huge "You suck!" error or something. This may not be
very possible given certain limitations of the programming language, but it
is a better ideal.
-
Use auto-escaping features of your environment such as SQL place-holders
(e.g: "SELECT * FROM mytable WHERE id = ?"), and the list argument of
"perldoc
-f system" (e.g: system { $cmd[0] } @cmd).
-
Perform frequent code reviews, black box tests, and encourage hackers to
find problems in your code.
-
Use complementary security measures that make sure that even if a problem
occurs, its damage is mitigated. As examples, you can try running the
script under an underprivileged operating system user, or as a database
user that lacks certain database privileges.
There are probably several measures that I'm forgetting, so feel free to
add them as comments or trackbacks. In any case, be careful when writing
code that may cause code or markup injection because the consequences may be
dire.
2009-10-24: "Criticisms of Perl" page
under work on the English Wikipedia
Due to the fact that the "Criticism" section was removed
from the English wikipedia
article about "Perl", I've decided to create
a
"Criticisms of Perl" page on the English Wikipedia under "User:Shlomif"
(which is my user-account). Nevertheless, the page can be edited by other
editors, and interested people can comment on the discussion page.
Reading from the page, it aims to be a coverage of the criticisms raised
for the Perl programming language which would be comprehensive, and written
from a Neutral Point of View, but still mostly concise.
. The article itself
still lacks many citations, and adding them would be appreciated. Also,
many important criticisms should be better covered. Be bold and edit.
2009-10-30: Follow-up to chromatic's "On Answers to Smart Questions"
On the Modern Perl Books blog,
chromatic wrote
a
post titled "From Novice to Adept: On Answers to Smart Questions"
where he mentions my comment to
a
previous feature of his there. I am flattered that chromatic decided
to expand upon my comment. Quoting my comment:
I agree with you about it. However, idioms are nice and dandy, but naturally,
when going on IRC, one can often open the Pandora box of the colour of the bike shed argument. For
example should it be:
Person->new(
first => "Sophie",
'last' => "Cohen",
birth_year => 1977,
);
Or:
Person->new({
first => "Sophie",
'last' => "Cohen",
birth_year => 1977,
});
(with a hash ref)
I tend to think the second option is better because it's probably a bit faster
(not that it matters a lot), and because it will warn about an "odd number of
hash elements" earlier, but there's a lot of code on CPAN out there that uses
the first option, because it involves somewhat less syntax. In one of my
modules, I supported only the second option, and a programmer who was after my T-shirt offer,
implemented convulted code to have it either with a single hash-ref or
flattened into @_, which I had to reject.
Naturally, this is just the tip of the iceberg. Brace indentation, placement of
HTML opening and closing tags, whether HTML should be nicely indented (even if
generated by a server-side language), or instead occupy as little space as
possible, whether you should sub-class sub new {... } or sub _init {....} etc.
etc. are all a matter of much debate, and they are pretty much a bike shed
colour's argument (while they often do have some points for and against).
I think it's important for a beginner (in Perl or in any other language) to
distinguish between such minor matters and non-idiomatic code, and that it is
even more important for an expert to see past his own preferences for the bike
shed's colour when helping newcomers.
chromatic discusses it in the page above, and gives some advice for newbies
who want to avoid getting confused from such discussions on personal style
and conventions. Often, matters such as those of indentation and whitespace
or placement of braces have some good and valid arguments either way, but they
are still a bike shed colour argument, and as such should be either enforced
globally by the project or workplace, or alternatively a programmer can do
what they prefer on their personal project.
2009-11-01: Why you should Upgrade to New Perl Versions?
Gabor Szabo wrote
about "Why bother upgrading perl?" and
announced it
on the Israeli Perl Mongers mailing list. guy keren
replied like so:
i have a different question: why do you care?
there are reasons for people to use the older software for any software
X you'll mention - you operating systems, office suites, programming
languages/compilers, external libraries etc. etc. etc.
what is the point of pushing people? this sounds like the method of
commercial companies - they want to sell the upgrades. what do you want
to sell - course-grades? [Editor's note: Gabor is giving commercial
training for Perl and other fields.]
the basic idea of open source is to give the power to the users -
including the power to stay with an older version of software if and
when they see fit.
if there was an assured (QA-wise) upgrade process that never broke
things and was trusted by people - people might have considered these
upgrades with a more favore. as long as this process has its cons -
you'll have people that will simply refuse to upgrade until they have no
choice - and sometimes even then.
Before I republish
my
reply to guy here, I should note that the "Why do you care?" attitude
is not very beneficial. Essentially, guy is implying that as human beings, we
should "mind our own business", be selfish and/or
self-centered and not
care what happens to other people. However, it's good that we care about
other people, because we are part of a community, and because helping
people we know can bring us a lot of pleasure. As much as I support
Ayn Rand's
"Objectivism" and
Rational egoism,
I still think that we derive a lot of short-term and long-term pleasure from
helping people we know, and that it is a healthy part of acting out of one's
best self-interest.
Selfishness
(meaning "having a complete disregard for others"),
is a form of undesirable behaviour, and should not be encouraged.
Now let's move on to my reply:
I think you're missing the point. Gabor is not trying to sell courses or
services for the newer versions of Perl (which as you note are free-of-
charge). Also, companies who want people to upgrade their software usually
have a better motivation than selling upgrades.
I blogged about it
in
my blog post - "The Broken Window Fallacy". Quoting from there:
I talked with a few people on IRC about how I hated the fact that most people
are still using Windows which is quickly getting inflicted with malware, or
Internet Explorer, which is lagging behind Firefox and other browsers in its
support for web standards, and makes the job of the web designer much harder.
So they told me that it is actually a good thing because that way someone has
to fight the malware (a very uphill battle), and they have more work as web
developers.
…
The problem is that tech workers can never run out of things to do. In the
time they have to adapt their web-sites for MSIE, they can better spend on
things like having better functionality, more web-sites, more pages. The time
people spend making sure their Windows systems are free of malware, is better
spent happpily using Linux and getting some actual, productive work done.
Many Perl contractors , consultants, workers and people who volunteer to help
people on online forums need to deal with old Perl versions or "Cannot use
modules from CPAN" constantly. Yes, it provides work, but these people would
and can rather do something else. See
the
"Parable of the broken Window" on Wikipedia.
Naturally, this is not only true for Perl, but for all software technologies.
I wouldn't want to support Red Hat Linux 9.0 (or older versions) or a 1996
vintage FreeBSD (which I happily used at the time), or Windows 95 or older.
And I've recently stopped caring about supporting Microsoft Internet Explore
(MSIE) 6 on my sites (which are not commercial, or else it may have
been more of an issue).
We are not making any money directly from such people who upgrade their Perls
or whatever to newer versions (unless they hire us to handle the upgrades for
them, which is a bit far-fetched.). But in the long run, we are happier
because we have an easier time supporting their systems and not having to
comply with their irrational whims.
As a result, we have a genuine, egoist, interest to make sure our customers
use recent versions of their programs and are not stuck on old versions. This
is because it prevents us from doing exciting and cutting-edge work during
the time that is spent in providing support for the old programs. And as I
noted in the original post, supporting old programs (e.g: adapting the sites
for Internet Explorer version 6), is causing a lot of frustrations and
bad feelings, which are also undesirable and have a bad net effect on
our self-esteem and happiness.
I hope you agree.
2009-11-06: New CPAN Package: XML-Grammar-Fiction
Earlier today, I uploaded version 0.0.1 of
XML-Grammar-Fiction to CPAN. It is a fork of my own
XML-Grammar-Screenplay,
which was heavily modified in the recent days, with quite a lot of coding
on my part (I felt encouraged by all this coding). The code is still pretty
hairy and only the basic functionality was implemented, and there may be
many bugs, but it passes all tests on my system, and I've already used it for
one of my stories, so
I'm happy.
So what is it about? XML-Grammar-Fiction provides the so-called Fiction-Text,
a lightweight
markup language of sorts to write prose (such as short stories, humorous
bits, novellas or novels) in, which in turn gets
converted to Fiction-XML, which is a dedicated XML grammar for representing
fiction, and which can also be used directly. Fiction-XML in turns can be
converted to DocBook/XML or
directly to XHTML.
I needed it because I found it troublesome to work on
one of my stories in
Hebrew ("The Pope Died on Sunday") directly as DocBook/XML, using either
gvim or
the KDE 4 Kate Editor, due to the
problem with the XML tags being in Latin whereas most of the text being
in Hebrew, so I sought a more clean text-based format. As a result,
XML-Grammar-Fiction was born, just in time for
NaNoWriMo (= (Inter-)National November
Writing Month), which I'm planning to spend some time investing
on my stories and screenplays (for which I already have
XML-Grammar-Screenplay).
So how does a sample Fiction-Text story look like? Here's the
markup for one straight out of the t/data directory of the
XML-Grammar-Fiction CPAN distribution.
<body id="index">
<title>David vs. Goliath - Part I</title>
<s id="top">
<title>The Top Section</title>
David and Goliath were standing by each other.
David said unto Goliath: "I will shoot you. I <b>swear</b> I will"
<s id="goliath">
<title>Goliath's Response</title>
Goliath was not amused.
He said to David: "Oh, really. <i>David</i>, the red-headed!".
</s>
</s>
</body>
You can see the mixture of some XML-like tags and plain paragraphs.
Naturally everything is still subject to change, and I suppose only
tech-savvy users will be able to install it and use it now, but I plan
to work on it some more in the near future.
Happy writing!
2009-11-11: Recent Hacktivity Log (11-November-2009)
For a while before, I felt like I was under-productive and that I had no
incentive to write a new recent hacktivity log. This has changed some weeks
ago, after a sprout of productivity where I did many small things. So here's
another boring recent hacktivity log, which you are free to skip.
In the Perl-Begin front, I've added
Matt
S. Trout's essay "But I can't use CPAN!" to
the
CPAN page of perl-begin.org. Thanks Matt (mst)!
After that, I finally
tended to the fact that the header image was too narrow and generally
resulted in a lot of padding in direct green on the side. I took
a photo I took
and placed on Flickr (under the Public Domain) and tweaked it using
the GIMP.
I've also done many CSS tweaks
and the people on #css were a
lot of help. I discovered that a co-maintainer added some JavaScript to
dynamically modify the CSS and which I've mis-tweaked, and had to fix it,
and convert it to jQuery. The people on #css said it was not necesssary at
all and could be done with pure-CSS, but I didn't like their solutions.
It also sparked a small discussion about our use of
Sass, and whether CSS should be generated
at all.
I've made some progress with the 5th part of
Perl for
Newbies,
and have started to cover the authoring of a CPAN-like module using
Module-Starter and Module-Build. I've been feeling quite unmotivated to work
on it, but I hope this feeling will pass.
I've discovered the
XML-Grammar-Fortune tests started failing, and realised it was because XML-LibXSLT
started outputting the XML with indentation. I had to workaround here by
trimming leading whitespace from both the expected and the received outputs.
Le sigh.
Later on in the past two days, I decided to make the
XML sources of the
fortune cookie files be served with a client-side
XSLT
stylesheet so
they'll be nicely displayed as XHTML in the browser. This turned out to be
hard because the XSLT I used internally in XML-Grammar-Fortune was not
accepted by either Firefox or Opera, and their errors left a lot to be desired.
Eventually, I was able to understand the Opera errors (which were more specific
than Firefox's) and while referencing the XSLT specification, was able to
fix the brokenness and get what I wanted to work.
I've also did some work on some of my screenplays and stories, among them
the
ideas page of the Blue Rabbit's logs, and
some
added text to "Star Trek: We the Living Dead".
Also in regards to my homepage,
I fixed the style of the breadcrumbs' trail at the top, and simplified
what I need to do to over-ride some stuff in the
Website Meta Language headers.
As I noted
in a previous entry, I also released
XML-Grammar-Fiction. I have some plans of creating a binary package for
MS-Windows with tools useful for writers based on the XML-Grammar-* work, but
this will require a lot of polishing of the code. Meanwhile, it would be
useful for people who are more tech-savvy.
I've done some work on
the
Catalogue of (program) optimisations over at the Algorithms wikia. I
should note that it is my initiative based on my call for that
in my
Optimizing Code for Speed essay, which I mostly wrote last November
(although published somewhat later).
By encouragement from someone,
I
did some work on my script to collect mailing lists from the Perl 5 Wiki,
and on normalising the format of the Perl 5 Wiki entries.
Recently, in the past day or two, all KDE applications broke on Mandriva
Cooker (Cooker is the Mandriva bleeding edge distribution) due to a Qt
problem. As a result, I had to work on my Debian "testing" partition. I
finally ran "apt-get update ; apt-get dist-upgrade" there again, and wanted to
work on my homepage. In order to do that, I need some CPAN packages. I first
tried to prepare and install a package of
CPANPLUS-Dist-Deb
using dh-make-perl, but the tests failed, so I did not want to risk installing
it. Furthermore, the "deb" line in the POD only had an entry for Debian
"unstable" - not for Debian "testing" which is what I have.
Then I tried to install
local-lib from the
Debian repositories, but it wasn't there, so I followed the bootstrapping
instructions on the POD, and was able to install it on my account and install
some modules using it - enough to build my homepage.
2009-11-12: הכרזה: מפגש פרל ב-17 בנובמבר ומפגש "מה היא תוכנת קוד-פתוח?"
הדבקה זו מהווה הכרזה ופרסום של אירועים קרובים של קוד-פתוח בארץ.
מפגש פרל ברחובות
ביום שלישי, ה-17 בנובמבר 2009, יערך
המפגש הראשון של Rehovot.pm
במסגרת חידוש מסורת מפגשי
שוחרי-הפרל הישראלים.
ראו גם את
דף
התיאום בוויקי.
מיקום וזמן: מכון ווייצמן ברחובות, בנין לווין (מס' 30) חדר 101 בשעות 18:00-22:00
על הפרק:
-
Matlab, שימושים וחסרונות ויתרונות מול פתרונות אחרים
-
PDL, The Perl Data Language
-
- אפשרות להדגמה קצרה של גרסת Padre העדכנית (עורך וסביבת הפיתוח לפרל)
-
לאחר מכן, בין השעות 19:00-22:00 יצוותו המשתתפים לזוגות או קבוצות קטנות ללימוד PDL בכלל וכתחליף ל-Matlab בפרט, תוך התמקדות במסמך
PDL למשתמשי Matlab.
הדוברים במפגש יהיו הדר לוי אהרוני, אבישלום שליט וגאבור סבו. ההשתתפות חינם. אם אתם מתכוונים לבוא נא
להודיע לגאבור למטרות הערכת כמות המשתתפים.
ברוכים הבאים לתוכנה חופשית: "מה היא תוכנה חופשית ופתוחת-קוד
ביום ראשון בתאריך 22/11/2009 בשעה 18:00 תתקיים הרצאה של
מועדון הקוד הפתוח התל-אביבי
במסגרת
סדרת "ברוכים הבאים לעולם של תוכנות קוד פתוח"
בנושא "מה זאת תוכנה חופשית ופתוחת-קוד ולמה זה טוב?". את ההרצאה יעביר אורי עידן והיא תתקיים במסגרת סדרת ברוכים הבאים ללינוקס בתלוקס ותתקיים באוניברסיטת תל-אביב, בחדר 007 במסדרון הבניין למדעים מדוייקים.
אנו נכסה את האידאולוגיה והפילוסופיה מאחורי תוכנות חופשיות ופתוחות קוד, נסביר מה מייחד את לינוקס ומערכות הפעלה פתוחות אחרות ממערכות הפעלה סגורות וכן את היתרונות הגלומים בהן בתור היותן חופשיות.
התחלנו להכין
כרזה
(פוסטר) לצרכי פרסום הסדרה
שניתן להדפיס ולתלות.
2009-11-15: Perl.org facelift , a new learn.perl.org Tutorials Page, and an IRC bit
In case you haven't heard
the
central *.perl.org sites got a facelift and are looking much better now.
Moreover, learn.perl.org finally has
a page with links to some
high-quality and recommended online tutorials, which is in turn
based on the corresponding page
from perl-begin.org. I'd like to thank Matt S. Trout and
apeiron for giving some
commentary and/or revisions of the initial patch.
And finally, a funny IRC conversation that took place on
Freenode's #perl
the other day. Enjoy!
- FreakGuard icke, how to start REPL?
- icke FreakGuard: looked in the docs?
- FreakGuard icke, no.
- icke you got to load it from CPAN
- FreakGuard icke, yeah, I've installed it.
- * Altreus sells FreakGuard perl docs at a reasonable price only $9.99 each
- icke perldoc Devel::REPL is the direct way to info then
- * Altreus wrings his hands and cackles
- FreakGuard icke, thanks :P
- * rindolf bests Altreus' bargain by 1 cent.
- rindolf Reminds me of what I learned in Game Theory.
- * Altreus offers free delivery
- * rindolf allows free download
- rindolf Of course quality > price.
- * rindolf offers a deluxe edition of the Perl documentation for 1,000 USD plus shipping and handling.
- huf diamond-encrusted?
- rindolf See http://perldoc.perl.org/ for a preview.
- rindolf huf: natural diamonds, too.
- rindolf huf: Canadian diamonds.
- huf well, if <> is natural... :)
- rindolf huf: heh.
- rindolf huf: not this kind of diamond.
- * Altreus compresses if () under several million tonnes of rock for a few aeons
- rindolf I also give free spaceships - <=>
- infrared heh
- rindolf Diamonds are heresy! We need PEARLs!
- rindolf /usr/bin/PEARL
- Altreus f00li5h: Seems like it!
- FreakGuard I prefer other gems :-)
- Altreus I suggested to Think Geek that they should do Perl necklaces but they didn't
- rindolf FreakGuard: Ruby gems?
- FreakGuard rindolf, correct.
- Altreus The rare PHP
- Altreus Darling I got you a PHP wedding ring
- Altreus oh it broke
- rindolf Altreus: as PHP tends to.
- * rindolf laughts maniacally.
- Altreus you maniac!
2009-11-18: דו"ח על פגישת הפרלאים הישראלים אתמול
טוב, אתמול הלכתי
לפגישה
של שוחרי הפרל הישראליים, בנושא MATLAB , PDL (Perl Data Language) ומה שביניהן.
למי שלא מכיר, מדובר בחבילות לחישובים נומריים, המספקות טיפול מהיר ונוח בוקטורים,
מטריצות וטנזורים מסוגים אחרים (שיכולים להגיע לגדלים ניכרים), וביצוע פעולות על כל
האיברים בטנזורים, כמו כפולה בקבוע, כפולה איבר איבר, כפל מטריצות, חיבור, חיסור,
חילוק וכו. מאטלאב כאמור התרחבה ונותנת יכולות רבות ומגוונות על סמך הבסיס הזה, אבל
זה הרעיון הכללי. PDL הינה תשתית בעלת תחום-עניין דומה לפרל.
בכל מקרה, התחלתי להתכונן לפגישה אתמול - הדפסתי את המפה, התרחצתי וחפפתי שער, בדקתי
את זמני
רכבת ישראל (האתר
החדש דרך אגב עובד מצוין לפי מה שבדקתי בפיירפוקס על לינוקס - כל הכבוד!), ארזתי תיק,
אכלתי ארוחת צהריים ויצאתי לדרכי. ברכבת פגשתי גברת שירדה באותה תחנה כמוני (רחובות) והיא הציעה לי להסיע אותי לפתח המכון. נסעתי איתה ובדרך דיברנו על החיים של שנינו. זכור
לי הרבה מהשיחה, אבל היא לא קשורה במישרין לנושא. בכל מקרה, ירדתי בעבר השני של הכביש
וראיתי קרוב שתי חומוסיות. מכיוון שהייתי רעב והיה זמן רב, החלטתי לקנות מנה פלאפל ב"פלאפל וייצמן" ואכלתי אותה בדרכי (פלאפל מצוין, דרך אגב). נכנסתי וקצת הסתבכתי בכניסה
עם האבטחה, מכיוון שלא זכרתי את שמו של חיים פרילוסקי שהיה המארח שלנו (להבא אדפיס את
הדף עם ההכרזה.), אבל בסוף הסתדרתי. כשהגעתי לבניין (בניין לוין, מס' 30 במפה) עדיין
היה זמן רב וחדר המחשבים בו נפגשנו לא היה פתוח, אז שתיתי ממטהר המים, וסתם הסתובבתי וחשבתי על כל מיני דברים.
בסוף אבישלום שביט (שהיה אמור
להרצות) בא ויכולתי לדבר איתו ומעט אחריו בא לאון, שלמד בטכניון ועכשיו
משרת בחיל האויר, והוא איש של פייתון בעיקר. בהתחלה חששנו שלא יבואו אנשים רבים, אבל
בסוף החדר היה כמעט מלא. חיים פרילוסקי לא היה יכול לפתוח לנו את החדר מכיוון שהרגיש
לא טוב, אז הוא שלח מאסטרנט בשם ערן במקומו, שפתח לנו את החדר ושאל אותנו
מספר שאלות.
מכיוון שהמרצה הראשונה עדיין לא הגיעה החלטתי לפתוח את ההרצאה ב-book report על הספר
remix
מאת לורנס לסיג,
שם דיברתי על הרשמים שהיו לי ממה שקראתי מהספר. המרצה אז הגיעה והייתי צריך להפסיק.
בחלק הראשון של ההרצאות עשינו סבב הכרויות בו כל אחד הסביר מעט על עצמו ומה הוא עושה
בפרל, מאטלאב ו/או PDL. רשמתי קצת על האנשים
בדף בויקי. אז התחילו ההרצאות:
הדר לוי אהרוני הסבירה קצת על מה זה מאטלאב והיתרונות שבו והראיתה קצת שימוש. אבישלום
שביט הראה באמצע ההרצאה מספר יתרונות של מאטלאב. דיברנו על כך שרק המאטלאב הבסיסי עולה
כ-5,000 דולר למחשב, ושתוספים רבים עולים עוד כסף, כך שכל הפונקציונליות יכולה להגיע
עד כ-25,000 דולר. אולם סטודנטים מקבלים אותו בחינם. יש מהנדסים רבים שמתרגלים כל
כך לעבוד במאטלאב ומשתמשים בו למשימות רבות כמו ניהול קבצים ועיבוד טקסט להן הוא
לא כל כך מותאם.
נאמר שם שאם רוצים לנתח קובץ טקסט המכיל בתוכו מספרים (parsing) הרי שבפרל
זה יהיה בד"כ מהיר בכפי 10 מאשר אם משתמשים במאטלאב לשם כך. למאטלאב יש יכולות
מדהימות של אפשרות יצירת גרפים והיסטוגרמות, אפשרות לחולל קוד למעבדי אותות דיגטליים
(Digital signal processors)
ולקמפל קוד מאטלאב לקוד בסי, יצירת מנשק משתמש בנוחות, אפשרות לכתוב הרחבות למאטלאב
ב-C, ודברים נוספים אחרים. כמו-כן סיפרו שמערכת העזרה של Matlab היא מקיפה מאוד והושקעו
בה זמן ומחשבה רבים וניתן ללמוד נושאים רבים במתמטיקה רק ממנה.
בהפסקה, גאבור הוציא את המאפים שקנה (szabgab++) ואכלנו לתאבון, ושוחחנו. יצא
לי לדבר עם
אמיר
שכרגע עושה תואר שני בבלשנות באוניברסיטה העברית, וסיפר כמה השימוש בפרל עזר לו בתואר,
ושהוא יכול בעזרתה לנתח ספרים שלמים ולהוציא מהם את מה שהוא צריך, והוא לא יודע
איך סטודנטים אחרים לבלשנות שלא יודעים תכנות, מסתדרים. לטענתו, קורס מבוא לפרל (או
לשפה דומה) צריך להיות קורס חובה בסמסטר הראשון של הלימודים. גם דיברנו על הוויקיפדיה
ועל "הוויקימילון", שלטענתו (ולדעתי) אינו מספיק מובנה ושהפורמט של וויקי פחות
מתאים עבורו. אז הזכרתי לו שאני משתמש בוויקיפדיה העברית בתור מילון אנגלי↔עברי והוא
אמר שגם הוא עושה בו שימוש ושהוא כתב תסריט בשביל לעזור בכך שהוא מחפש עבורו
מישהו שיעזור לו בלתחזק אותו.
גם ראיתי שם שוב את רינה
משוחרי הפרל הירושלמים
שלא פגשתי כבר זמן רב, ודיברתי איתה קצת. אני מקווה שהם יוכלו בקרוב להמשיך בפגישות
שלהם בירושליים (אם כי לי קצת קשה לבוא לשם מפאת המרחק והגישה הדפוקה לשם).
בכל מקרה, אחרי ההפסקה נמשכה ההדגמה של מאטלאב עם כל מיני הדגמות מגניבות, ואחר-כך
נאמר מה צריך כדי ליצור אלטרנטיבה טובה למאטלאב: סביבת עבודה טובה, תאימות טובה (לא
חלקית שכל הזמן (נתקעים), ודברים שאפשר לשפר מעבר למאטלאב. בכל מקרה, הודגמו שם
מספר דברים חביבים במאטלאב כמו סגורים (closures) - למשל ב-octave:
octave:4] f1 = @(x) @(y) { x+y }
f1 =
@(x) @(y) {x + y};
octave:5] f2 = f1(6)
f2 =
@(y) {x + y}
octave:6] f2(100)
ans =
{
[1,1] = 106
}
octave:7] f2(50)
ans =
{
[1,1] = 56
}
בכל מקרה, לאחר מכן גאבור עלה על הבמה, והדגים את ה-REPL (ראשי
תיבות ל-Read-Eval-Print-Loop) של
PDL
בשם perldl ודברים מגניבים אחדים בו (שככל הנראה רחוקי םלהגיע לרמה של MATLAB אבל
עדיין חביבים למדי), כמו עזרה אינטראקטיבית ודמואים ובכללן תצוגות תלת-מימדיות. כמו-כן
הסביר על כך שהחזון שלהם ל-Padre הוא שיהיה לו תוסף ל-PDL שיאפשר פיתוח יותר נוח ודומה יותר למאטלאב. כך למשל הגרסה הבאה של
Strawberry Perl
תכלול את Padre ואת PDL בפנים. הוא סיפר שכשנתן מצגת על Padre בכנס אחד באירופה הוא נתן רשימה ארוכה של תכונות
שלו ואמר "הן לא ממומשות עדיין. עכשיו אתם צריכים לממש אותן." ואכן חלקן הגדול מומש מאז.
אז היה כבר אחרי 20:00 והתכוונו להתחיל את ה-hacking session, אבל אני עזבתי עם
עוד תל-אביבי אחד בשם דותן כדי לתפוס את הרכבת האחרונה. בדרך דיברנו על דברים רבים כמו:
CGI-Application,
CGI-Prototype,
The Catalyst Web Framework,
JavaScript, jQuery, KDE 4.x, XFCE,
תרפיה קוגניטיבית-התנהגותית ועוד נושאים.
אז סך הכל - היה נחמד מאוד ונהניתי מאוד. זה היה שינוי מרענן בשגרה עבורי. עכשיו אם
תסלחו לי, יש לי קוד פרל לכתוב. אני מקווה לראות אתכם בפגישה של
מועדון הקוד הפתוח התל-אביבי ביום ראשון.
2009-11-22: Numeric Plans for Tests and Test-Count
Adam Kennedy (Alias)
writes in use.perl.org about "the unfortunate demise of the plan.". I've
read his post and all of the comments that were posted there up to now. I
agree with Adam that a numeric test plan is usually preferable to a
done_testing() one, because in one of experiences with a failing
test script (in a DBI-related module), I
noticed that it generated a different number of tests depending on the
version of one of its dependencies. So only a subset of tests run in one of
the cases, and this was enough for me to realise that I should make sure that
an exact number of tests run.
Eventually, I came up with
Test-Count, which
allows one to keep track of the number of tests inside a Test::More script
using a domain-specific-language embedded in comments next to the tests. This
way, one can have more faith that they have run the correct number of tests.
Personally, I found that some people take some time getting used to writing
the Test-Count annotations, but for me it's been a great boon. I do not
rule out that done_testing() has some uses or may be better in
some clear-cut cases, but since many of my tests are data-driven or use
loops or helper functions, then I always prefer to have a numeric plan
and to keep it up-to-date using Test-Count.
At the moment, Test-Count is tailored for my own needs and many important
features there may be missing. However, bug reports, feature requests and
especially patches are more than welcome.
2009-11-27: Changing the Seldon Plan
In Isaac
Asimov's book "Second Foundation", the second foundation are in charge
of the Seldon Plan,
the grand "psychohistory" plan for the advancement of the galaxy,
and one thing that is mentioned there is that it should by no means be
considered holy and that changing it according to the circumstances is
expected and even necessary. This mentality has an affect on us as open-source
programmers, who should do our best to remedy bad open-source code as we
find it.
Recently, in the Perl world there has been a continuous trend to shift
from
Ancient
Perl to Modern Perl, and often
people ask for our help on Perl 5 code they found (possibly on CPAN - possibly
elsewhere) with "Ancient Perl" paradigms. That doesn't necessarily mean that
Perl 5 code is very old, because often people who wrote it didn't know
any better, and there are plenty of Perl 5 (or even Perl 4) tutorials or
books still floating around the Net teaching bad practices. As a result,
we normally tell them to either abandon or convert the code completely for
a better trusted and more modern CPAN module, or alternatively to modernise
it.
So far, so good. However, some people who've asked for us help said something
like "I
do not consider myself capable of rewriting his module and [it] has
demonstrated robust stability over the years [so] I have
little reason to.". So essentially, all the comments that we have given
on it were dismissed due to apparent lack of competency on the part of
the code's end-user, and that Ancient Perl code will still linger in use.
The link I gave is not the only place where I saw it - I also saw it in at
least another place (though I think it was on IRC).
I think that this spirit stands in opposition to the spirit of open-source
and possibly even
the Hacker
Ethos. By all means, if we consider all the millions of lines of
open-source out there as the Seldon Plan which powers all the open-source
programs in use, and empowers us and end-users, then we should not
feel detrimental or afraid to change the code that we use everyday. And
people should not deploy code that they are not confident enough in changing,
modernising and adapting to their needs.
One cannot usually expect code to remain the same forever. As time progresses,
we can expect there to be code rot, features that needs to be added, tests
that need to be added, or as is often the case in Perl 5 and other languages,
paradigms that are considered to be bad and that should be changed to newer
and safer paradigms. If you're lucky, the original developer or a co-maintainer
can do that for you, but sometimes an end-user needs to stand up and volunteer
to do that.
The open source nature
of "Free and Open Source Software" permits everybody to create derivative
works of the software and distribute them. So we don't have legal
reservations from improving them and contributing our improvements to the
general public, and should not have any moral or ethical ones, either. "Hacker
sees bug. Hacker doesn't want bug. Hacker fixes bug.".
2009-11-30: Random Stuff: Word Count Vim Tip, Mirroring CPAN Using rsync, and Mozbot
This is a "random stuff" entry because I'm too lazy to write separate entries
for each of the topics.
First of all the vim tip: you can do
word count in Vim, by
selecting a text using "v" and then typing "g" and then "Ctrl+g". This will
display statistics on the text including word count. I always assumed it
existed in a form but didn't had to use it until recently when I wrote a scene
in a
screenplay and wanted to see how long it was. This tip can also be found at
":help word-count".
The second tip is that in order to mirror a mini-CPAN that contains only the
most recent releases using
rsync instead of minicpan's
HTTP or FTP download, one can use
mst's mirror.sh in conjuction with
the list of CPAN
rsync mirrors. I needed to modify mirror.sh slightly in order to mkdir
the "work" directory which it requires. I changed:
cd $LOCAL
to:
cd $LOCAL
if ! test -e "work" ; then
mkdir "work"
fi
Finally, a little "recent hacktivity log": at one point someone on
irc.mozilla.org asked me if I could help them with
their
IRC bot called firebot and yesterday I decided to take a look at its
core called "Mozbot",
and maybe try to improve it. I took a look and detected many vestiges
of
"Ancient
Perl" there, and other anti-patterns such as long subroutines or use
of Net::IRC and decided to work on improving it.
After reading the
"Using
Mercurial locally with CVS" document on the Mozilla site (first hit
in a Google search for "hg cvs") and setting up
a
"mozbot-shlomif" Mercurial repository to work on it,
I submitted my
first patch and have done more changes in the mozbot-shlomif repository.
The code is not perfect, but at least it has "use strict;", the "-w" flag
and the "-T" flag, so it could be much worse. I'd still like to move away
from Net::IRC to
POE-Component-IRC
and maybe even try to port the entire Mozbot to
buu's
buubot. But it will take some time.
2009-12-07: Studying Moose Top-Down
This is a post about Moose
the post-modern Object-Oriented Programming system for Perl 5. Until a few
days ago, I've learnt most of what I knew about Moose by reading some
presentations about it and experimenting. However, a few days ago, I've decided
that maybe there was a more-Moosey way of doing the following pattern that
I've been doing in my classes:
# Super-base class.
sub new
{
my $class = shift;
my $self = {};
bless $self, $class;
$self->_init(@_);
return $self;
}
# In derived classes:
sub _init
{
my $self = shift;
my $args = shift;
$self->_init($args);
# Initialise from $args.
return;
}
Turns out there was - you can use the BUILD method to provide
initialisation for each part of the class hierarchy (it gets calls in
a walk-method style - not only for the most qualified sub-class). Only in
my case in my en-Moosification of
XML-Grammar-Fiction,
I only needed a 'default' callback to the class members, and to
make them 'lazy' by default.
This made me conclude that I have to learn Moose top-down to become more
familiar with it. So I started reading the chapters of
Moose::Manual
and have already learned that I could encapsulate push() and
shift() out of an array ref using
its
delegation, which also proved useful in XML-Grammar-Fiction. After I'm
done with the manual, I'm planning to go over the Moose cookbook.
Hopefully, this will give me a lot of ways to avoid repetitive code in my
CPAN modules. Of course, now, I'm facing the classic dilemma of whether
to make something a base class or a role… decisions, decisions.
2009-12-11: Recent Hacktivity
Happy Chanukah, everyone! Lately, I've had little energy for blogging, and
have been coding a lot.
I suppose that's a good thing, but I'd like to keep up with my
Planet Perl Iron Man
statistics. So here we go - another recent hacktivity report.
As I said, I've went over the
Moose::Manual
and learned many nice new tricks to make my Moose-using Perl modules more
Moosey. I've implemented some of them in XML-Grammar-Fiction and also
on Test-Run,
which is a much more massive codebase.
While I worked on Test-Run I noticed
that some of the Test-Run-CmdLine tests failed, and realised the problem
was a MooseX-Getopt bug which I was able to supply a test for, and which
involved a very short fix. While I was in the neighbourhood, I also contributed
a test for handling the Getopt::Long no_ignore_case which I used as
well, and the results were released in MooseX-Getopt-0.26 by
Tomas Doran.
I also ran into an undesirable behaviour with MooseX-StrictConstructor which
I was able to workaround with some help from the
#moose people: when using it inside
a class which has a base class, one needs to do
BEGIN { extends ("BaseClass"); } instead of just
extends("BaseClass");. I didn't understand why, but it just works.
I'm now reading the chapters from the Moose Cookbook, which will take
me some time.
After that, I finally
did
the work I promised on the list of Perl mailing lists. There's still a lot
to fill in, but the YAML/DBI foundation is in place.
I worked on a Ruby program which I originally started writing in order to
learn Ruby. I've added some tests, and refactored it. I've got bitten by a
case where I overrided a variable called yx as a parameter to a
closure and it got clobbered afterwards. This is one reason why I prefer
Perl 5's (and Scheme's)
explicit lexical
scoping using "my", which provides better protection against such
mishaps.
I continued to work on
Mozbot, but
my original patch was not applied yet,
so I don't know if there's any interest in further work on the internals
of the robot.
I've made some progress with reading the book
xUnit
Test Patterns: Refactoring Test Code. One of the insights I gained so
far was that the failed tests should point what became broken without too much
debugging. The book is off to a slow start with many overviews and
introductions, which is kinda annoying, though.
Finally, I attended a meeting of PHP-Israel where there were a lot of
Bourekas,
and were someone gave a presentation about PHP Unit and writing unit tests.
There was a discussion of whether we should only test the external behaviour
of a class, or also its protected/private methods, and if the latter - how
(there was some way to do it in PHP). One thing I noticed was that some
of the PHPers used Eclipse for their development, and there was some discussion
about how to get it to behave properly on Linux.
A few of the PHPers also thought that they would love to use JavaScript on
the server. It's funny, but as a Perl 5 programmer (with experience in many
other languages) I look at JavaScript and see many horrible aspects,
and missing features. However, the PHPers seem to think it's better than PHP.
Maybe it's the Blub Paradox.
2009-12-16: #perl and ##programming fortune collections
I've split
a
separate file for fortune cookies made out of
Freenode #perl conversations,
and
another one for those taken from ##programming. The motivation for them
was to make my "shlomif" collection smaller and more managable, but at
least it's now in the same place. I should note that you can find the XML
version of the file by changing URL's the to .xml and the text
version by changing it to a blank one. (Memo to self: link to them from
the HTML page). The
main page of the
fortunes contains more information.
Enjoy!
2009-12-19: GNU Make Tip - Speeding Up By Using the -r / --no-built-rules Flag
By default, when GNU make
runs makefiles it keeps looking for many default targets like ",v" and a
lot of other junk. If your makefile has no default targets (especially if
it's a makefile for a non-C/C++ project), you can speed things up
considerably by saying "make -r $TARGET" or "make --no-built-rules $TARGET"
instead of the default target.
Here is a comparison of its performance on my homepage, after it was completely
built:
shlomi[homepage]:$trunk$ time make
make: Nothing to be done for `all'.
2.03user 0.07system 0:02.21elapsed 94%CPU (0avgtext+0avgdata 19344maxresident)k
0inputs+0outputs (0major+2025minor)pagefaults 0swaps
shlomi[homepage]:$trunk$ time make -r
make: Nothing to be done for `all'.
0.13user 0.03system 0:00.18elapsed 89%CPU (0avgtext+0avgdata 12256maxresident)k
0inputs+0outputs (0major+1582minor)pagefaults 0swaps
shlomi[homepage]:$trunk$
Very significant!
In order to avoid specifying it times and again you can make it the default
by setting the MAKEFLAGS environment variable (which isn't
documented too well) to "-r".
How I discovered the tip? Well, I occasionally used "make -d" for debugging
my makefiles, and noticed it was trying to apply a lot of implicit targets.
Having understood that they may be very time consuming, I ran "man make" to
see if there was a way to avoid running them. And lo and behold, they made
running make on my homepage much faster.
2009-12-21: Why You Should Not Use Python .pyc Files for Hiding Source Code
A few months back, I've spoken to a Python programmer (who used to program
in Perl) and among the many things we discussed, he said that one feature
he appreciated about CPython was that you could take the Python .pyc files,
which contains compiled Python bytecode, and ship them without the .py source
files in order to hide the source code from people he's distributing it to,
so it will be harder for them to reverse-engineer and study the code.
I told him that with a symbolic language such as Python, such .pyc files
do not provide adequate "protection" as they contain the names of identifiers
and other detailed information on the code. At one point, he even thought
that they are compiled C code, but I told him CPython can work pretty well
on machines without any kind of C compiler.
On Freenode's #python, people seemed to have agree with me (I am rindolf):
<rindolf> Another question: can I depend on compiled python
bytecode (.pyc) for "hiding" code? Doesn't it still contain all the
identifiers verbatim?
<lvh> rindolf: No. You cannot hide code, stop trying.
<kniht> rindolf: why are you trying to hide code?
<DeadPanda> rindolf, check the new IEEE Security and Privacy (if you can),
you can't hide Python code
<DeadPanda> rindolf, unfortunately, there's nothing you can do.
Otoh, you can probably do the bare minimum and make your boss happy.
<kniht> rindolf: because these final users are liars and cheats? if that's the
business plan, not sure what I can say
Python knows the identifiers of the variables at run-time. For example:
shlomi:~$ cat exec-test.py
#!/usr/bin/env python
import sys
a = "I am a"
b = "I'm b"
exec(sys.stdin.readline())
shlomi:~$ python exec-test.py
print a
I am a
shlomi:~$ python exec-test.py
print b
I'm b
$ python exec-test.py
print a.upper(), b
I AM A I'm b
shlomi:~$
Even if you're not using exec(), eval() or friends, python still has to
accomodate for them being potentially used and as a result keeps this
information in the bytecode. My partner said he doesn't use eval and friends
because they are "a bad programming practice" and as a result thought he
was safe. However, that's not the case.
He told his clients that Python bytecode was only marginally worse than
Java and .NET bytecode which "are used to protect the code of highly sensitive
IDF and US millitary applications - bytecode is sufficient protection."
However, according to this Slashdot thread "java is [a] cake to
reverse engineer".
So it's not adequate protection, and Python is even substantially less than
that.
I next suggested he may opt to use obfuscators to obfuscate his code, and he
said that:
They can't sue me, they'll have to sue whoever reverse engineered the code - I
believe it's illegal to reverse engineer commercial compiled bytecode - it
isn't however to reverse engineer obfuscated code far as I know -- it's not a
technical issue, it's a legal/business one.
I don't understand the distinction between obfuscated code and bytecode
in this case. And like I told him "Any sufficiently advanced obfuscation is
indistinguishable from bytecode.".
Talking with a different Python programmer, he told me that .pyc were
considered adequate "protection" for them, despite the fact there are
several .pyc disassemblers and decompilers present.
In short, depending on .pyc's for protecting your source code, only provides
fig-leaf protection. So it's not a substantial advantage of CPython over
perl 5, which doesn't cache the bytecode in ".pmc" files by default. (even
though it has a bytecode that it uses internally in memory).
As a postscript I should note that sometimes after that my conversation partner
said that to me:
BTW, I looked more info python bytcode and you're right - you can get something
almost identical to the original. It still requires some tikering but there are
commercial solutions for decompilation that produce something pretty close to
the original for 5 euro per 5kb for personal code or 100 euros per file for
commercial code
☺
2009-12-26: Directory Tree Traversal In Various Languages, C# and a funny IRC conversation
While looking for some sample C#/.NET code (more about it below) and doing a
search I stumbled upon
this
page in Rosetta Code that contains directory tree traversal codes in various
languages. I've been meaning to do a comparative blog post comparing
File::Find, File-Find-Object and File-Next to directory tree walking APIs
in other languages, but now it will be easier. Of course, the page is not
perfect - for Perl, it only gives an example using File::Find (guess I'll have
to add the other ones) and in many cases (such as C) it gives a custom,
recursive solution instead of using an API such as libfts.
Still it's a pretty cool page. Now I fear I'll get the feeling that "it'
s already been done and so I shouldn't do it" and so won't write that post.
The reason I was looking for C# information was that one of my Perl programs
ran too slowly. It was written to find a good heuristic for
Freecell Solver, and used
a
menial algorithm: check the most optimal solution for the first iteration
with starting quotas between 100 and 1,000; Then using the result, check them
for the second iteration, etc. The Perl program used
PDL (the Perl Data Language), but: 1. It
was wrapped in a lot of Perl OO code and 2. it had a lot of pure-Perl
algorithmic code that I did not figure out how to properly PDLise yet. As
a result, I was looking for a faster language to port it to.
I originally considered using C, which I know pretty well, but then thought
about trying out C#/.NET using its
open-source Mono implementation.
At the end I spent a day and a bit of the evening beforehand writing
409 SLOCs of C# code, and 113 SLOCs of testing Perl code for the task
(according
to SLOCCount by David
A. Wheeler) and that was after being delayed due to many web searches
to find out how to do a lot of stuff. The reason I wrote the testing code
in Perl was because I lacked the energy to learn NUnit, but now that I think
of it it may be useful to test the original Perl version (which is still
of some utility to me) too.
At the end of the day, the C# code worked and worked well. Mono programs
start up quickly, and the C# code I wrote ran much more quickly than the
Perl one, which saved me the time of waiting for so long. I think I like C#
so far. It seems much nicer than Java, and with
less protocols and some syntax niceties and useful APIs and it also
starts up much more quickly. I still find that writing Perl 5 code is quicker,
but if your Perl/CPython/CRuby/etc. code is under-performant, then C# and
.NET would be viable choices.
(I should note that I've still barely scratched the surface here. A
single-threaded 409-line-strong command-line program is hardly indicative of
any possible advantages or alterantively pitfalls you may run into with
C# and .NET. But so far what I tried worked and nicely.)
And finally, a funny IRC conversation. It's been a while since there was
something on Freenode's #perl that I found really funny, but now the wait is
over:
- Enl apeiron: because I write client side (Provider) which sends push messages to the phone and checks for new mail in perl
- rindolf Enl: a cellphone?
- apeiron No, a rotary POTS phone that's capable of receiving email, rindolf.
- Enl rindolf: iphone, yep
- rindolf apeiron: :-) apeiron++
- rindolf apeiron: I want a phone like that!
- rindolf apeiron: do you sell them?
- apeiron rindolf, Yes, and I have some oceanfront property for you, too.
- rindolf apeiron: would you accept some of my copious gold bars in return?
- apeiron rindolf, No. I only deal in Latinum!
- rindolf apeiron: Latinum.
- rindolf apeiron: gold-pressed Latinum?
- apeiron yes.
- rindolf apeiron: I only have silver-pressed Latinum.
- Enl rindolf: get an iPhone, lol
- rindolf apeiron: would you accept LeoNerd and nanonyme as substitutes ? You can sell them for mucho Latinum.
- apeiron heh
- rindolf Enl: iPhones are worthless.
- * rindolf conspires to steal the ApeironPhones.
- rindolf I didn't say I was honest.
- * apeiron jealously guards his G1
- Enl rindolf: now really
- rindolf Enl: you should get an ApeironPhone too.
- rindolf Enl: chicks love it.
- apeiron what
- Enl rindolf: pff, chicks dont matter
- rindolf I would kill for an ApeironPhone!
- Su-Shee ahaem?
- * rindolf kills Su-Shee and takes her ApeironPhone.
- apeiron ...
- apeiron wtf.
- go|dfish hahaha
- * rindolf uses his RindolfMindReading™ to see who else has an ApeironPhone.
- apeiron Put down the acid, Shlomi. o.o
- huf like that'll help
- huf he's still got ~8 hrs on it
- Su-Shee rindolf: I have an apeiron-phone?
- Su-Shee where did buu go anyway?
- rindolf Su-Shee: you had one.
- rindolf Su-Shee: before you died.
Hope you enjoyed this entry.
2009-12-29: KDE 4 Tip for KMail, Kopete, etc.: Right-aligning Text
For a long time, I've been bothered by the fact that on the Linux/Unix
KDE 4 desktop on applications such as
KMail,
and Kopete, the Hebrew text was
left-aligned. I expected it to be right-aligned when it encounters
a leading right-to-left text, like it was on KDE 3, or now is the case
in Kate (the KDE text editor). However, there is a way to align the text
to the right for display.
To do so, press Right-Ctrl and Right-Shift on the keyboard (possibly while
selecting the text). Like in Windows. It's much better in KMail than in Kopete,
where you have to right-align the text this way after every message. But at
least it makes life with KMail much more bearable. And please try to
reproduce and comment on my
"Hebrew
Messages are Displayed with Jumpy Margins" bug in KMail.
Cheers and a happy new calendar year.
2009-12-30: Vim Tip: Finding the First Non-Matching Line from the Cursor
For a long time I've been viewing and editing vim text files, for example
those of the UNIX find output or of archive contents, that contained many
consecutive lines of similar data. What I wanted to do is find the line where
these consecutive data end where they no longer end. Among the workarounds
I tried was jump to the end of the file (using G) and then search
backwards for the last matching line (which may not be very reliable). But
a few days ago I decided that I had enough and sought a better way, and
I found one.
You can do it using a negated pattern. If for example you enter
/ home\/shlomi\/\(Arcs\/\)\@! you will find the first line that
contains "home/shlomi" that is not followed by "home/shlomi/Arcs/".
For information see
:help \@!.
2009-12-31: Cygwin and Perl Save the Day!
In my house, we have a Windows XP desktop computer that serves the family
(while I'm normally using my Linux desktop machine.). A few days ago my father
complained that drive C:\ there, which contains all the programs is almost
completely occupied, and said we need to remove programs from it. Now this
drive is roughly 45 GB in size, so it seemed unlikely to me that it became
entirely full because of the executables.
I decided to investigate. I switched to my account, started a
cygwin shell,
and ran \du . > ~/du.txt from the C:\. What du does is report the
size of every directory starting from the input path recursively. Then I
ran my
du-output-analyser written in Perl to analyse the results, and eventually
found out that 20,000,000 (20 million) 1 KB blocks are being occupied by a
directory containing photos. Then I sent a report for my father with my
findings.
This just goes to show how much people who are not familiar with the UNIX
command line and with programming are missing and are much more helpless
than the more clueful UNIX power-users.
2010-01-01: Legal: ACTA and Software Patents in Israel
This post is about two legal matters. The first is about
ACTA,
the so-called "Anti-Counterfeiting Trade Agreement", which is a trade
agreement that aims to criminalise a lot of the openness of the Internet
under the guise of fighting counterfeited goods.
Michael Geist has an
analysis of the provisions of ACTA and
tracks
its progress. This is worrying and one should better take a look.
The other issue is Software patents in Israel
(see "Fight Software Patents"'s wiki
entry about Israel and the official call for opinions on the Israel
Justice ministry site). I started preparing
my
own statement document, which is still incomplete and there's
a letter that Jonathan Klinger (attorney in law) has prepared). Hope
we can stop this madness.
Happy new civil year!
2010-01-03:
פגישת קוד-פתוח-תל-אביב: Moose למתחילים - מערכת האובייקטים לפרל
מועדון הקוד הפתוח התל-אביבי (תלוקס)
ייפגש שוב כדי לשמוע את
הרצאתו של
ירון מאירי (Sawyer) אודות "Moose, מערכת תכנות מונחה העצמים לשפת פרל (למתחילים)".
ההרצאה תתקיים ביום ראשון, 17 בינואר 2010, בשעה 18:00 (שימו לב לשינוי בשעה משנה שעברה),
באולם הולצבלט, מס' 007 במסדרון הבניינים למדעים מדויקים (שימו לב לשינוי במיקום משנה שעברה) באוניברסיטת תל אביב. פרטים נוספים, מפות להגעה וכיוצא בזה, ניתן למצוא
באתר
ובוויקי.
הנוכחות בהרצאה היא חינמית ולא נדרשת הרשמה מראש.
Moose
הינה מערכת תכנות מונחה-עצמים פוסט-מודרנית לשפה פרל 5. היא נכתבה
מכיוון שההוגה המקורי שלה (סטיבן ליטל) קינא במה שפרל 6 סיפקה
בנוגע לתכנות מונחה עצמים, ולכן במקום לעבור לרובי הוא שקד על פיתוח
מערכת דומה לפרל 5. Moose שאבה השראה מיכולות ה-OOP של שפות רבות
כמו פרל 6, Smalltalk, ליספ, רובי, ג'אווה, OCaml ושפות אחרות כשהיא
נשארת נאמנה לשורשי ה-פרל 5 שלה.
ירון מאירי הינו מנהל מערכות ומפתח פרל. הוא מרצה על קוד פתוח, תוכנה חופשית,
אבטחה וסטנדרטים של תכנות. ירון העביר בעבר את
ההרצאה
על דגלים אדומים בתכנות עבור שפות עיליות ביותר במועדון התל-אביבי.
אנו תמיד מחפשים מרצים שיתנדבו לתת הרצאות בנושאים שונים הקשורים לקוד-פתוח ולמחשבים. במידה שאתם מעוניינים לתת הרצאה, או שיש לכם הצעה להרצאה שמעניינת אתכם, נשמח לשמוע ממכם.
2010-01-05: Next TelFOSS Meeting: Moose, the Perl OOP System on 17-January-2009
The Tel Aviv Open Source Club
will host the talk "Moose, the Perl OOP System" for beginners by
Yaron Meiry (Sawyer) on
Sunday, 17-January-2009, at 18:00 (note the change of time since last
year) in Tel Aviv University, Holcblat Hall No. 007 in the corridor of the
exact sciences building (note the change of place from last year). Further
details, maps for arrival, etc. can be found on
the web-site
and in the Wiki.
Attendance is free, it is not necessary to RSVP and everyone are welcome.
With any other problems, feel free to contact the organiser.
Abstract
Moose is a post-modern Object-Oriented
Programming system for Perl 5. It was written since its originator (Stevan
Little) was jealous of the capabilities that Perl 6 provided in regards to OOP,
and so instead of switching to Ruby, he worked on developing a similar system
for Perl 5. Moose drew inspiration from the Object Oriented Programming
capabilities of many languages such as Perl 6, Smalltalk, Common Lisp (CLOS),
Ruby, Java, OCaml and other languages, while remaining faithful to its Perl 5
roots.
About the Lecturer
Yaron Meiry is a systems' administrator and a Perl developer. He gives talks
about open source, free software, security and programming standards. Yaron
has previousl given the
presentation
about red flags in programming of very high languages in
the Tel Aviv Club.
We are always looking for people who will volunteer to give presentations on
various topic that are related to open source code and to computers. In case
you are interested to give a talk, or that you have a suggestion for a talk
that interests you, we'll be happy to hear from you.
2010-01-07: bzr is Dog-Slow and Generally Sucks
The
Bzr version control system (Bazaar) is a version control system from
Canonical Ltd. (the parent company behind the Ubuntu Linux distribution). While
being open-source, it sucks royally and this is a blog post about this.
First of all it is excruciatingly slow. Branching postr.dev (which is a bzr branch of
the very small
postr project takes 2:48 minutes. And that's not all:
shlomi:~/Download/unpack/net/www/TEMP$ cd postr.dev/
shlomi:~/Download/unpack/net/www/TEMP/postr.dev$ du -cs *
4.0K AUTHORS
20K COPYING
80K data
4.0K nautilus
20K po
4.0K postr
4.0K postr.doap
4.0K README
4.0K setup.py
256K src
4.0K TODO
404K total
shlomi:~/Download/unpack/net/www/TEMP/postr.dev$ echo "Hello" >> postr
shlomi:~/Download/unpack/net/www/TEMP/postr.dev$ time bzr di
=== modified file 'postr'
--- postr 2007-07-19 17:57:09 +0000
+++ postr 2010-01-07 18:00:06 +0000
@@ -42,3 +42,4 @@
for url in sys.argv[1:]:
p.add_image_filename(url)
reactor.run()
+Hello
Command exited with non-zero status 1
1.47user 0.08system 0:01.69elapsed 92%CPU (0avgtext+0avgdata 60080maxresident)k
288inputs+8outputs (0major+5673minor)pagefaults 0swaps
shlomi:~/Download/unpack/net/www/TEMP/postr.dev$ time bzr di
=== modified file 'postr'
--- postr 2007-07-19 17:57:09 +0000
+++ postr 2010-01-07 18:00:06 +0000
@@ -42,3 +42,4 @@
for url in sys.argv[1:]:
p.add_image_filename(url)
reactor.run()
+Hello
Command exited with non-zero status 1
1.47user 0.07system 0:01.61elapsed 95%CPU (0avgtext+0avgdata 60096maxresident)k
0inputs+0outputs (0major+5672minor)pagefaults 0swaps
So a simple "bzr diff" command takes over 1.6 seconds.
I reported a bug
in inkscape - "bzr branch lp:inkscape is very slow and emits errors and leaves
a broken local repository after exit" It took bzr 3 minutes to try and
checkout the Launchpad's Inkscape branch, and it finished with an error and
left the repository in an unsuable state. I tried it on my Debian Testing
VM and ended up with the same result.
To quote Sjors: "Bzr is slower than
Subversion in combination with Sourceforge." (and that says a lot).
Recently, GNU
Emacs switched to Bzr. Now the question is how many decades one will have
to wait to "bzr branch" its main repository.
Cheers!
2010-01-08: libtap-1.02 Was Released
libtap is a C library to implement the Test
Anything Protocol (TAP), which is used by the test suites of perl 5,
CPAN modules and other programs. libtap allows one to write C code that can
be analysed by TAP harnesses such as Test-Run.
I started maintaining Nik Clayton's libtap, which he seems to have abandoned
on my
homepage.
There's also a
Subversion repository.
Reading from the NEWS file:
tap-1.02:
-
Add the COPYING file.
-
Now disabling the thread-safety by default (in order to have it add
-DLIBTAP_ENABLE_BROKEN_THREAD_SAFE .
-
Changed configure.in to configure.ac.
2010-01-11: New Solver for "Black Hole" Solitaire in Perl
I wrote
a
new solver for the solitaire "Black Hole". Here's the
complete story:
Yesterday, I was going to play a game on
Brain Bashers when I
remembered the Penguin solitaire variant, which is similar to Freecell and
which I wanted to cover on the Cards Wikia. So I did a Google search for
"penguin solitaire" and saw it had a page on the wikipedia.
There I saw that it was invented by David Parlett and I saw there that he also
invented
a solitaire called "Black Hole"
So I looked for it in PySolFC,
read the instructions there and started to play. The game involves putting a
card that is one above or below the foundation (wrapping from kings to aces).
I noticed that there wasn't any over-populated talon or something like that
there, which meant that I could probably build a DFS-based solver for it. So I
set out to see if it was possible.
The first thing I did was to adapt my PySol/PySolFC game generator to generate
the initial board of the PySolFC deals. This turned out to be a very
complicated and frustrating task for me, because I had to understand what's
going on with the code. But after a lot of playing with it, I was able to get
it to generate the initial deals. The changes for that are in the Freecell
Solver trunk now.
Then I started working on the solver. I decided to write it in Perl 5 because
it's a good prototyping language and I know it well. When writing it, I
heavily optimised for speed and low memory consumption by using bit fields (
see the
vec() function ) and a gigantic hash lookup.
Then, after it was written, came the moment of truth: I ran it on the board
and it reported success. Great! But what's the solution? So I added some
solution tracing logic, to output the cards that should be moved. Then I was
able to play it and the game was solved. Yay!
I tried it on another game and it also worked. Then I ran it on the first few
games. Game #1 was reported as unsolvable, Game #2 was solved after a while,
and Game #3 consumed over 15% of my RAM and then was solved.
So it seemed to be working nicely. Today I wrapped the code inside its own
CPAN package and prepared a nice app-in-a-module and a script for it.
he solver's code is made available under the permissive MIT/X11 licence -
share and enjoy. And as I mentioned before
here
is its home-page.
2010-01-18: The Quest for a Good Window Manager / Desktop Environment
I'm using Linux (Mandriva Cooker) and have been routinely starting many
window managers and desktop environments as the need arises. Lately, I've been
working a lot with KDE 4 and I happen to agree with
OSNews.com's
assertion that it is slow, sluggish, and has frequent random crashes. I'd
like to move to something more usable until the KDE 4 programmers get their
act together. Now, I've traditionally used
IceWM but it no longer cuts it for me
because it does not have a different wallpaper
for every virtual desktop. XFCE fails on the same premise. Enlightenment 17
has a different wallpaper per virtual desktop, but its task bar is
dysfunctional, if not completely non-existent, and it has many other very
annoying quirks. GNOME is just plain annoying (and doesn't have a
different-wallpaper-per-virtual-desktop either), and KDE 3 is no longer present
on Mandriva. LXDE does not have it either.
Now I dislike the concept of
tiling
window manangers, of which there has been an inflation, recently, and I am
looking for a window manager that has stacking. And like I said, it also needs
to have a usable task bar, a different wallpaper per virtual desktop
and be responsive and stable.
Do you have any recommendations?
2010-01-20: Report on the TelFOSS "Moose for Beginners" Meeting
Last Sunday, Tel Aviv Open Source club
(TelFOSS) met to listen to Sawyer's presentation about "Moose for
Beginners". Despite the fact that the meeting was well-publicised, very
few people came. Sawyer did not prepare a lot of slides and since there were
not many questions he finished quickly. After all that, he went to demonstrate
how to implement several use cases that the audience has proposed as Moose
programs, and despite that we still finished early.
I asked about implementing an "XML parser", but in fact I wanted to implement
a processor for a certain XML grammar. In any case, he misunderstood and told
me to use Parse-RecDescent for that. I've grown to dislike P-RD for most stuff
though.
In any case, I'm getting tired of the low attendance in TelFOSS meetings. We
do so much work in organising and publicising the meetings, and most people
don't bother to come. Are the people too busy? Did they run out of steam? Why
don't they come? This makes me frustrating and unwilling to further organise
the TelFOSS meetings.
2010-01-21: Perl: Using Vim's snipMate for Perl 5 snippets
I noticed that there was some Perl 5 code that I had to type or copy-and-paste
again and again on many occasions. So I decided to find a way to put it in
one place and then recall it. I thought of writing it myself, but then recalled
the snipMate
snippets extension for Vim (which
Peteris
Krumins covered in a blog post), which allows that. After a little reading,
I was able to prepare the snippets.
I placed the following under ~/.vim/snippets/perl/_slurp.snippet :
sub _slurp
{
my $filename = shift;
open my $in, "<", $filename
or die "Cannot open '$filename' for slurping - $!";
local $/;
my $contents = <$in>;
close($in);
return $contents;
}
And now I can type "_slurp<TAB>" to recall it. The existing "slurp"
snippet in ~/.vim/snippets/perl.snippets is quite evil, with an
ugly inline line, typeglobs, and a two args open without a die statement.
Thanks, but no thanks.
Afterwards I added the following in
~/.vim/snippets/perl/_ltestb.snippet
local $Test::Builder::Level = $Test::Builder::Level + 1;
And now I can type "_ltestb" (short for "local Test::Builder") and put this
line there to create my own custom Test::More/Test::Builder tests.
Thus, using snipMate, you can create your own short-hands for commonly-used
snippets like that.
2010-01-22: The CPAN Dependencies of Website Meta Language
In the past few days I resumed work on converting
the Website Meta Language (WML) build-system
from the GNU Autotools (also known as "GNU Autohell") to
CMake. This involved implementing a lot
of the existing makefile and autoconf logic in CMake and Perl.
Yesterday, I converted the "wml_common" sub-directory which turned out to
contain some ancient versions of CPAN distributions with a forced-upon way
to install under the WML prefix. I decided to forego installing all of them
and just check for their existence in the perl version that wml uses using
some WML and "perl -MMyModule" code.
I was able to find all the needed modules in the Mandriva repository, except
one -
File-PathConvert
, which I've never heard about.
I checked it on CPAN, and saw that it was deprecated and superseded by
File::Spec and Cwd. So I grepped the tree for its use and found it was used
only in one Perl program which I quickly converted to File::Spec and Cwd.
Now, the future build system of Website Meta Language will be simpler
and it won't use deprecated modules.
Oh! And
Firefox
3.6 (Gamma) is out! I've been using the betas and RCs on Mandriva Cooker
for some time now, and now I've upgraded to the stable version and it's
still great.
2010-01-25: XML-Grammar-Fortune version 0.0200 Was Released
I recently released version 0.0200 of the Perl 5 CPAN Module
XML-Grammar-Fortune.
XML-Grammar-Fortune is a module to maintain a collection of Unix-like
fortune cookies as XML and convert them to XHTML and plain-text.
New in this release are a heavily fixed and enhanced code for rendering the XML
to plaintext and a more enhanced META.yml file. XML-Grammar-Fortune was
successfully utilised for
my collection of Fortune
Cookies.
2010-01-30: Project Euler Problem #10 in Haskell, Perl and C
zerothorder told
how
he found a solution for Project Euler's Problem 10 in Haskell. The
problem is "Find the sum of all primes less than 2,000,000". It is given
here below:
primes :: [Integer]
primes = 2 : filter isPrime [3, 5 ..]
where
-- only check divisibility of the numbers less than the square root of n
isPrime n = all (not . divides n) $ takeWhile (\p -> p*p <= n) primes
divides n p = n `mod` p == 0
result = sum $ takeWhile (< 2000000) primes
main = do putStrLn( show result )
He says "If this doesn't give you a nerdgasm, I don't know what will.". The
problem is that this nerdgasm will last a long time. Benchmarking this
program gives that 20 iterations of it run at 310 seconds - less
than - about 15 seconds each (on my Pentium 4 2.4GHz machine running
Mandriva Linux Cooker). So next I tried a better Haskell
implmenetation that I recalled from a thread I started in the Haskell Café
mailing list about implementing
a sieve of
Eratosthenes in Haskell:
import Data.Int
primes :: Int64 -> [Int64]
primes how_much = sieve [2..how_much] where
sieve (p:x) =
p : (if p <= mybound
then sieve (remove (p*p) x)
else x) where
remove what (a:as) | what > how_much = (a:as)
| a < what = a:(remove what as)
| a == what = (remove (what+step) as)
| a > what = a:(remove (what+step) as)
remove what [] = []
step = (if (p == 2) then p else (2*p))
sieve [] = []
mybound = ceiling(sqrt(fromIntegral how_much))
--main = print (length (primes 1000000))
main = print (sum (primes 2000000))
This does not involve costly operations such as modulo or division
and 20 iterations of it run at 135 wallclocks seconds - over two times faster
than zeroth's Haskell version.
Now how about Perl? Since Perl has assignment, we have the advantage that
we can create a vector of bits that we will mark with the primes by iterating
over all numbers up to the root of the limit. Here is the code:
#!/usr/bin/perl
use strict;
use warnings;
use Math::BigInt lib => 'GMP';
my $limit = 2_000_000;
my $primes_bitmask = "";
my $loop_to = int(sqrt($limit));
my $sum = 0;
my $total_sum = Math::BigInt->new('0');
for my $p (2 .. $loop_to)
{
if (vec($primes_bitmask, $p, 1) == 0)
{
$sum += $p;
my $i = $p * $p;
while ($i < $limit)
{
vec($primes_bitmask, $i, 1) = 1;
}
continue
{
$i += $p;
}
}
}
for my $p ($loop_to .. $limit)
{
if (vec($primes_bitmask, $p, 1) == 0)
{
if (($sum += $p) > (1 << 30))
{
$total_sum += $sum;
$sum = 0;
}
}
}
$total_sum += $sum;
print "$total_sum\n";
20 runs of it run at 95 walclock seconds - even faster than the Haskell
version. But it gets better. Since all the primes we encounter greater than
2 are not even, we can create a map of their pseduo-halves and conserve on
memory and iterations. This is the Perl version:
#!/usr/bin/perl
use strict;
use warnings;
use Math::BigInt lib => 'GMP';
my $limit = 2_000_000;
my $primes_bitmask = "";
my $loop_to = (int(sqrt($limit)))>>1;
my $half_limit = ($limit-1)>>1;
my $sum = 0+2;
my $total_sum = Math::BigInt->new('0');
for my $half (1 .. $loop_to)
{
if (vec($primes_bitmask, $half, 1) == 0)
{
my $p = (($half<<1)+1);
$sum += $p;
my $i = ($p * $p)>>1;
while ($i < $limit)
{
vec($primes_bitmask, $i, 1) = 1;
}
continue
{
$i += $p;
}
}
}
for my $half ($loop_to .. $half_limit)
{
if (vec($primes_bitmask, $half, 1) == 0)
{
if (($sum += (($half<<1)+1)) > (1 << 30))
{
$total_sum += $sum;
$sum = 0;
}
}
}
$total_sum += $sum;
print "$total_sum\n";
Running this 20 times takes 73 wallclock seconds, close to half that of my
Haskell version.
Then I wondered how long C will take. Here is a C implementation without
the halving:
#include <string.h>
#include <math.h>
#include <stdint.h>
#include <stdio.h>
#define limit 2000000
int8_t bitmask[(limit+1)/8];
int main(int argc, char * argv[])
{
int p, i;
int mark_limit;
long long sum = 0;
memset(bitmask, '\0', sizeof(bitmask));
mark_limit = (int)sqrt(limit);
for (p=2 ; p <= mark_limit ; p++)
{
if (! ( bitmask[p>>3]&(1 << (p&(8-1))) ) )
{
/* It is a prime. */
sum += p;
for (i=p*p;i<=limit;i+=p)
{
bitmask[i>>3] |= (1 << (i&(8-1)));
}
}
}
for (; p <= limit; p++)
{
if (! ( bitmask[p>>3]&(1 << (p&(8-1))) ) )
{
sum += p;
}
}
printf("%lli\n", sum);
return 0;
}
This was too fast to measure with 20 runs alone, so 500 runs of it took 15
seconds, two or three orders of magnitude faster than the fastest Haskell
or Perl versions. But naturally, we can apply the halving paradigm there too:
#include <string.h>
#include <math.h>
#include <stdint.h>
#include <stdio.h>
#define limit 2000000
int8_t bitmask[(limit+1)/8/2];
int main(int argc, char * argv[])
{
int half, p, i;
int half_limit;
int loop_to;
long long sum = 0 + 2;
memset(bitmask, '\0', sizeof(bitmask));
loop_to=(((int)(sqrt(limit)))>>1);
half_limit = (limit-1)>>1;
for (half=1 ; half <= loop_to ; half++)
{
if (! ( bitmask[half>>3]&(1 << (half&(8-1))) ) )
{
/* It is a prime. */
p = (half << 1)+1;
sum += p;
for (i = ((p*p)>>1) ; i < half_limit ; i+=p )
{
bitmask[i>>3] |= (1 << (i&(8-1)));
}
}
}
for( ; half < half_limit ; half++)
{
if (! ( bitmask[half>>3]&(1 << (half&(8-1))) ) )
{
sum += (half<<1)+1;
}
}
printf("%lli\n", sum);
return 0;
}
500 runs of it take 10 wallclock seconds - 54.35 times per second,
and 50% better than the previous C version. And I still haven't applied
platform-specific gcc optimisations.
I should also note that the executables generated by ghc are extremely
large in comparison to their C ones:
$ ls -l c_* haskell_*
-rwxr-xr-x 1 shlomi shlomi 6082 2009-12-04 07:46 c_mine
-rwxr-xr-x 1 shlomi shlomi 6103 2009-12-04 07:46 c_mine_half
-rwxr-xr-x 1 shlomi shlomi 6092 2009-12-04 07:46 c_mine_micro_opt
-rwxr-xr-x 1 shlomi shlomi 796825 2009-12-04 07:56 haskell_mine
-rwxr-xr-x 1 shlomi shlomi 571717 2009-12-04 07:46 haskell_zeroth
c_mine_half is less than 1% the size of haskell_mine (and
runs faster). When talking about this to other people, they said that Haskell
has a very optimised primes sequence generator, which I can try using (which
should be over 3 times as fast), and that it has two kinds of integers, which
the other type is faster, and that it has a better way to emulate assignment.
But the bottom line is that the naïve and intuitive way to write such
programs in Haskell is under-performant, even in comparison to Perl, and 100
or 1,000 times as much in comparison to C.
I've written this as a separate post and not as a comment to the original
blog post because I'm very limited with the markup in the commenting there
(I'm going to post a comment there with a link to this blog post, though).
I should note that you can find all the code I mentioned inside
a
dedicated Github repository, and you can experiment with it further.
2010-01-31: Escape from GNU Autohell!
I've set up a
page on my web-site titled "Escape from GNU Autohell!" about the evils
of the GNU Autotools (GNU Autoconf/Automake/Libtool and other GNU
auto-brain-damages) and why you should convert to a good alternative such as
CMake. The page converts a list of
the many disadvantages of the GNU Autotools, some refutation of common
arguments against CMake, and some links.
And lo and behold, some time after writing the page I spent hours dealing
with yet another Autohell problem:
"Mandriva's
libtool causes "make install" of xine-lib-1.1-hg to fail". I actually
had to build the Mandriva libtool several times to find the offending
downstream patch, and libtool's "./bootstrap" script was so slow that
I could finish running "make" and "make install" in xine-lib while waiting
for it to finish. And it says a lot because xine uses GNU Autohell
which slows down the build process considerably. Eventually, I found the
offending patch after several hours, and I wouldn't have finished on time if I
didn't cancel the "make check" test in the .rpm.
Autohell must die!
2010-02-02: פגישת קוד פתוח בתל-אביב: 14 בפברואר - vtiger CRM
מועדון הקוד הפתוח התל-אביבי (תלוקס)
ייפגש שוב כדי לשמוע את
הרצאתו של
רמי הדדי אודות "מערכת ניהול קשרי הלקוחות
vtigerCRM.
ההרצאה תתקיים ביום ראשון, 14 בפברואר 2010, בשעה 18:00, באולם הולצבלט, מס' 007 במסדרון הבניינים למדעים מדויקים (שימו לב לשינוי בשעה ובמיקום משנה שעברה) באוניברסיטת תל אביב. פרטים נוספים, מפות להגעה וכיוצא בזה, ניתן למצוא
באתר
ובוויקי.
הנוכחות בהרצאה היא חינמית ולא נדרשת הרשמה מראש.
vtiger CRM
היא מערכת ניהול קשרי לקוחות אשר פועלת בממשק Web דרך הדפדפן.
המערכת מכילה את כל הכלים הנדרשים לניהול מלא ומקיף של עסק: החל מעסק קטן
של אדם אחד וכלה בעשרות עובדים. במערכת ניתן למצוא ביטוי ללקוחות, יומנים,
הזדמנויות עסקיות, מכירות, שירות ועוד. המערכת מבוססת על קוד-פתוח וניתן
גם לבצע בה התאמות בקלות רבה. המערכת תורגמה לשפות רבות ובכללן עברית.
רמי הדדי הינו מנכ"ל אקטיבטק אשר מתמחה ביעוץ, פיתוח ואירוח של vtiger CRM.
אנו תמיד מחפשים מרצים שיתנדבו לתת הרצאות בנושאים שונים הקשורים לקוד-פתוח ולמחשבים. במידה שאתם מעוניינים לתת הרצאה, או שיש לכם הצעה להרצאה שמעניינת אתכם, נשמח לשמוע ממכם.
2010-02-02: New Page about Text Editors and IDEs for Programmers
After answering the questions "Can anyone recommend a good text editor?" and
"What is a good IDE?" a few times in the past, I set up
a
page listing some prominent text editors and Integrated Development
Environments (IDEs) for development on my homepage. Currently it includes
only cross-platform open-source text editors and cross-platform open-source
IDEs, but I'm planning to expand it as time and interest permits. Additions
would be welcome.
Vim is there and so are XEmacs and
Eclipse, and also
Padre, the Perl IDE and more
specialised IDEs such as the
"Eric" Python IDE.
2010-02-05: NYTProf-3 is Out!
Tim
Bunce writes on his blog about the new features in
Devel-NYTProf version 3. Devel-NYTProf is a profiler for the
Perl programming language, which has
put all the previous attempts in profiling in the dust, and now it's even
better than before. Enjoy! (Thanks to
Fred
Moyer's post on the San-Fransisco Perl Mongers mailing list).
2010-02-11: Article Recommendation: "What Every CSer Should Know about Floating-Point Arithmetic"
I've read the paper
"What
Every Computer Scientist Should Know About Floating-Point Arithmetic" and
can recommend it as it has many interesting insights. I had received quite a
lot of reports for bugs in
the
"Statistics-Descriptive" CPAN distribution that were caused by expected
floating-point behaviour. Some of them I was able to resolve using some
trickery, and some others possibly can be somewhat mitigated using the insights
I've learnt in the article.
2010-02-19: Solving gaal's Circuit Riddle Using Perl
gaal (= Gaal Yahas)
mentions a riddle on his blog:
There’s a box with three input signals: A, B, C. It has three output signals:
~A, ~B, ~C (that is, the negation of each input).
Design a circuit satisfying the above description. You have two NOT gates, and
as many AND and OR gates as you like.
(If you solve this, don’t tell me the answer — I haven’t solved it yet; I just
want to share the misery.)
I decided that I will try to solve this riddle on my own
and then attempt to write a solution using a Perl 5 program, which I've
also thought about how to write. As it turned out, I ended up exploring
and eventually completely solving the riddle using a series of Perl programs.
I won't mention the final solution here, but I'll tell which programs
I wrote to solve them and how I wrote them. The code can be found in
its
Mercurial repository under the MIT/X11 licence, as I mentioned
in a
comment to gaal's post.
Well, first of all I decided to write a program that will take two negated
expressions of AND and OR gates and see if one can arrive to the final solution
using them and the given inputs (A, B and C). To do this I represented every
such possible expression as its components of (A + ~A) × (B + ~B) × (C + ~C) -
8 in total which fit in an 8-bit datum, and which I indexed using an array
of 256 items to see which ones exist in the population. After doing the coding
I realised none of the expressions I tried worked.
Then I decided to see if I there are any two possible expressions that when
added to A, B an C will yield all the necessary outputs. I ran it and indeed
found a single pair (and only a single pair) of such expressions (given as
numbers) which I was able to analyse. I should note that the programs as
I've written it using some functions and closures was too slow, so I ended
up writing another Perl script to preprocess the code, and inline the
functions, which has made everything much faster. (You can find this
preprocessor in the repository).
Then I wanted to see if I can indeed construct these expressions from A
, B, and C using two negations. When I had thought about this, I believed
I needed to maintain a collection of expressions' populations, and to
keep how many negations were in each. But when I got to code it I thought
of a simpler way: marry all the inputs using AND and OR until no more can
be married, then go over the existing ones and negate each one in turn, then
add the negated vector to the population and marry it with all the rest, and
then do it again for all the existing inputs. Using this I was able to
find a single solution.
Then I had to output the solution. For that, I wrote a recursive expression
dumper, which ended up outputting with many parentheses, so I had to post-
process it a bit.
The solution I got to was very complicated and I believe it would have taken
me a long time to reach it without resorting to programming. But at least
it saved me a lot of time and head-scratching.
2010-02-22: New Perl Quiz-of-the-Whatever: Short Freecell Solutions
I published a new
Perl
Quiz-of-the-Whatever about finding short Freecell solutions. Hope you enjoy
thinking about it. Here is the Perl
Quiz-of-the-Whatever (formerly "Quiz of the Week") information..
2010-02-25: Review: Alternative (to Firefox) Linux Web Browsers
Since I've been blogging about Perl
a lot lately (trying to maintain and advance my
Planet Perl Iron Man
status), I've been neglecting to blog about other things. So here's a
new entry not about Perl, not even about programming - but about web-browsers.
I've been using Firefox (now at version 3.6) as my primary browser on my
Mandriva Linux Cooker system for a while now, and have been mostly happy
with it. Today I wanted to edit
this wikibook
titled "How to Write a Program". It had been written awfully
(see the
discussion page), but I was introduced to it after it was made
a
prerquisite of a different wikibook which I've written, and decided
that the dependency should be kept (assuming "How to Write a Program" would be
written better). Now, I could either throw away "How to Write a Program"
and start from scratch, or revamp it to submission. Right now, it seems like
it would be a combination of both.
In any case, after editing it, I've ran into a long-term
bug in Mandriva, or
in my Mandriva setup (which is exhibited in any Firefox that is running
directly on it.), which I've finally reported there (though I believe I had
reported it in the Firefox bugzilla too) and so had been unable to use Firefox
for it.
So I've been looking for an alternative browser. I ruled out
Konqueror because
KHTML has been suffering from a lot of bug report, and Konqueror is just
plain annoying. (WebKit has been forked from KHTML, and it proved to be
more popular.). I initially ruled out
Opera because it's not open-source. So
that left me only with WebKit-based browsers. I asked for recommendations
and people on #ubuntu-uk (great
channel, BTW) recommended Chromium and
GNOME's
Epiphany, and some other more obscure browsers. I've decided to check them
out.
I decided to go with
SRWare Iron as
my Chromium browser. The first thing I noticed was that the Linux download
link led to a web forum post - very unprofessional. After downloading and
unpacking the archive, it refused to run -
/home/shlomi/apps/iron-linux/iron: error while loading shared libraries:
libbz2.so.1.0: cannot open shared object file: No such file or directory.
A symbolic link to libbz2.so.1.0.0 (done by root) fixed that problem, and
then it crashed with an X error. However, it started the second time and seemed
to run fine (don't know what the problem was). I noticed Chromium
was indeed very fast, but then I ran into a few glitches:
-
No menu bar. WTF? All desktop applications, including web browsers had menu
bars for generations. And I'm used to invoke the File menu and select
"New Window" and "New Tab". But Chromium does not have any menus, and it appears
to be by intentional design (but probably a crappy one).
-
If this wasn't enough, there was another more minor but still irritating
annoyance: after clicking on the URL bar and pressing double click, only
the current word is selected as opposed to the Firefox behaviour of selecting
the entire URL.
So I decided to look at Epiphany. Slower than Chromium, but maybe it will
work. Or maybe it won't. The first thing I noticed was that it looked
ugly - very unaesthetic and several widgets in the main window were slightly
out-of-place. Its URL bar also behaved unlike Firefox in the double click
respect, which was also annoying.
So what can I do? I eventually decided to use Opera for editing Wikimedia
wikis until the Firefox-influencing bug is resolved. It is fast, it has a
menu bar, and double click in the URL bar selects the entire URL. Very nice.
It's not open-source, so I'm trying not to get myself used to it, but seems
like I'll need to use it.
If you haven't done so already, I suggest you read
the
"Joel on Software" "must-read" books recommendations page. I haven't read
all the books there, but the page itself is also good. Quoting from it from
our context:
A few months ago when we released CityDesk, I got an email from a customer
complaining that he was used to doing Alt+F, Alt+S to save files. Unfortunately
due to a tiny, unnoticed bug, that keyboard shortcut saved the file and then
closed it, irritatingly. I had never noticed because I'm in the habit of doing
Alt+F,S to save files, not Alt+F,Alt+S -- a tiny difference -- and Alt+F,S
worked fine.
Once you get into the habit of doing Alt+F,Alt+S to save, it becomes so
automatic you don't think of it as Alt+F,Alt+S. You think of it as save. And
when you push the "save" button in your brain and the file you were working on
goes away, it makes you feel like you're not in control of your environment.
It's a small thing, but about the fourth time that it happens, you're going to
be seriously unhappy. That's why I spent several hours tracking down this bug
and fixing it. In a bizarre application of Murphy's Law, this fix led to a
cascade of events that caused us to waste something like a week, but that's
neither here nor there. It was worth the time spent. This is what it means to
be concerned about usability. If you still think that something as small as how
long you hold down the Alt key when you activate a menu command doesn't matter,
well, your software is going to make people unhappy. These tiny inconsistencies
are what makes Swing applications so unbearably annoying to use, and in my
opinion it's why there are virtually no commercially successful Java GUI
applications.
If I can't go to "File → New Tab" in the menubar to start a new tab then I'm
going to be unhappy. If I have to triple click the URL bar in order to select
the entire URL, then I'm going to be unhappy. These tiny things are why I can't
use either Chromium or Epiphany. (And frankly, a lack of a menubar is a huge
UI screw-up, not a tiny one).
At the moment, I'm just ranting, but I am planning on reporting these bugs
to the bug trackers of the appropriate projects. I just hope Google won't
think the lack of menu-bar in Chromium is a feature. "Welcome to Google UIs".
(Welcome to Google Hell…).
2010-03-01: A Perl-based →with(...) method
Lately, I've been writing some PDL (Perl Data
Language) code and noticed that I've been doing the following:
my $temp_pdl = $pdl->…->final_expr();
my $result_pdl = $temp_pdl->where($temp_pdl < 0);
I've been thinking if I could somehow write it like that:
my $result_pdl = $pdl->…->final_expr()->with(sub { $_->where($_ > 0)});
Which seems cleaner and more elegant. I went on IRC asking about it and someone
suggested
rindolf: (sub { local $_ = shift; $_ * $_ * $_})-gt;( $obj->... );
(up to an episolon, but naturally this reverses the order and messes with
the flow of the code). Eventually
Matt S.
Trout (mst) proposed the following solutions:
sub UNIVERSAL::with
{
local $_ = shift;
my $sub = shift;
return $sub->($_,@_);
}
my $_with = sub {
local $_ = shift;
my $sub = shift;
return $sub->($_,@_);
};
(I had figured something like that was possible previously, but I was beating
around the bush.)
The $_with approach is covered in one of Matt's blog posts titled "Madness with
Methods".
I wondered if it will work with
autobox and mst said that
I don't know and I hope not since the UNIVERSAL approach is evil.
. But
it does, out-of-the-box:
#!/usr/bin/perl
use strict;
use warnings;
use autobox;
sub UNIVERSAL::with
{
local $_ = shift;
my $sub = shift;
return $sub->($_,@_);
}
my $_with = sub {
local $_ = shift;
my $sub = shift;
return $sub->($_,@_);
};
print +(10+1)->with(sub { $_ * $_; }), "\n";
print +(10+1)->$_with(sub { $_ * $_; }), "\n";
This prints 121 twice.
I now wonder what to do with this knowledge. I have some aspirations of
releasing it to the cpan as a mini-module. Still, it's really cool.
Finally, I should note that it seems like I'm going to lose my current
Planet Perl Iron Man karma due to inadequate blogging. Oh well.
2010-03-16: New Topical Pages and Pages for Perl Uses on Perl-Begin
This summarises the changes to the Perl
Beginners' Site since the last update.
We added a a page about using Perl for E-mail
processing, a page about Perl for
multitasking and networking, a some topical pages:
The IDEs and Tools page now contains
screenshots.
Many new links have been added and many typos have been corrected. Enjoy!
Perl-Begin continues to be the most comprehensive first-stop for Perl Beginners.
Moreover , its content is distributed under the Creative Commons Attribution
License (CC-by), and its source
code is available to facilitate contributions.
2010-03-21: Tech Tip: Getting AAC to Play in gstreamer
If you're getting a message like the following when playing a video or an
audio file with AAC sound in a
gstreamer-based and no sound is played:
** Message: Missing plugin: gstreamer|0.10|totem|MPEG-4 AAC decoder|decoder-audio/mpeg, mpegversion=(int)4, framed=(boolean)true (MPEG-4 AAC decoder)
Then you need to install the 'faad' gstreamer element from gst-plugins-bad or as
a separate pacakge. Thanks to "__tim" from Freenode for the insight.
2010-03-22: Report on the recent Haifa Perl Mongers Meeting
This is a report and some notes from the Haifa Perl Mongers meeting a
while ago.
I arrived there by train from Tel Aviv with
Sawyer, who was going to
give the presentation about Moose. We met in the Arolozerov train station, and
I was a little late since I noticed I had dressed too lightly for the cool
weather. We bought tickets and went on the train to Haifa.
On the way we talked about Perl, C, C#, bugs in CPAN modules, Chuck Norris
facts, and other stuff. We arrived at Haifa well ahead of time, and I led the
way to the meeting at Qualcomm. As it turned out, the road I took was
relatively sub-optimal because there was a way with more pavements going
there. But we got there.
We entered Qualcomm, and saw Shmuel there and we were given tags for entrance
(in exchange for our I.D. cards). We wanted to drink something and the cooler
was broken, so we ended up using the tap. We waited for everyone else to
arrive and in the meanwhile I munched on a croissant and a Rogalach.
Then the presentations started. Erez gave a good presentation about how to
talk to newbies, where he said that a lot of the things we say are irrelevant.
Asking people to add "use strict" will cause their scripts to stop running
completely whereas they previously only had a bug, so it's not always the best
strategy. The old "RTFM" mantra or "You can't just make shit up and expect the
computer to know what you mean, retardo!" were also mentioned.
He then gave a few links where people can visit to help newcomers. He
mentioned learn.perl.org but I was
disappointed that he didn't mention Perl-Begin
- the Perl Beginners Site.
In any case, then Sawyer gave his presentation introducing Moose. It was a
good presentation and the audience spent more time discussing it than last
time he has given it in Tel Aviv. We spent a lot of time discussing the
clearing methods which turned out to be equivalent to
delete($self->{'myfield'}) or so. Then we discussed the
compilation-time
overhead of Moose where someone said it was unacceptable because it would mean
his test suite composed of many scripts would run much slower. We then
discussed about whether most people use all of the features in Moose, and
whether it indicates code bloat and is worth the extra overhead.
Then the meeting ended and we walked to the train station and returned home.
Erez took the train with us and on the way home we discussed Java, Matt S.
Trout and Planet Iron Man, Lambda Calculus, the earlier discussions about
Moose and other stuff. I tried to explain to them about Lambda Calculus and
not sure I was successful, but you can read what I had written about it
in my "Scheme and
Lambda Calculus Lecture" and what Mark Jason Dominus wrote about it
in Perl and Lambda Calculus.
So it was a very nice meeting and I greatly enjoyed it. Also see what
Sawyer
wrote about it,
and what Erez
wrote about the meeting.
2010-03-27: My Mojolicious and KiokuDB Story
I was hired by Insurgent Software (one of whose directors is an old IRC/IM
friend of mine) to write
a Mojolicious and KiokuDB-based user authentication system.
I was naturally getting paid for it and the resultant code was going to be
open-source so it seems nice. I did not have any previous experience with
either Mojolicious (a web-development
framework) or
KiokuDB (an
object-based persistence system), both of which
are recent Perl 5-based technologies,
so I believed had quite a lot to learn. Which indeed was the case.
The first thing I noticed when starting to work on the Mojolicious project
was that it lacked a good testing module, as Test::Mojo hardly provided
anything I was used to from
Test-WWW-Mechanize-Catalyst.
So after some thought, I decided to create
Test-WWW-Mechanize-Mojo
based on it. This took a day of work to hack the first version,
and required some subsequent updates due to misunderstanding the dependencies'
APIs, or doing other things. Afterwards, I also added some XPath-related methods
in the
test program and then implemented some application-specific logic using
Moose in a further sub-class (this time Moose-based).
This eventually turned out to work pretty well for some needs, though
Test-WWW-Mechanize-Mojo still emits some warnings, and I needed to fix some
bugs in it.
Our Mojolicous::Lite code was very succinct, which I could appreciate, but
due to its design philosophy, Mojo/Mojolicious ended up re-implementing
a lot of CPAN in slightly different ways, which required learning and studying.
In regards to KiokuDB - it worked pretty well. When trying to deploy it on
my Mandriva 2010.0 laptop, I discovered that it gave me a lot of problems
with the Perl debugger (perl -d), which were resolved by upgrading
namespace-clean
(thanks to nothingmuch on IRC for that.)
I misunderstood the KiokuDB API a little, which resulted in some things I
needed to fix. I also eventually used the Moose "around" method-modifier
to abstract the new_scope() functionality (which
someone told me he did something similar with "before" which I'm not sure will
work because it will go out of scope, before the function call).
I've also written yet another form handler/generator because I was too lazy
to look at one of those on CPAN, and because mine kinda evolved.
The end of the story was that despite everything, the project was completed
successfully to the satisfaction of Insurgent Software, and that I was mailed
the cheque with the second half of the payment. However, this project tended
to have irritated me, and also consumed me and left me able to do very little
other FOSS-related activities. Still, I'm happy that I took it, and now that
it's over, I resumed my open-source activities and among other
things have released a new version of Freecell Solver.
2010-04-01: Announcing Freecell Solver™ Enterprise Edition
There's an exciting development today for
one of my programs. Reading from the
publicity:
Freecell Solver Enterprises™, Inc., on behalf of the
Freecell Solver™ development
team, is glad to announce the upcoming availability of
Freecell Solver™
Enterprise Edition. In its Enterprise Edition, Freecell Solver™ will be
enhanced to solve generalised Freecell, in which there can be an arbitrary
number of card ranks. Since generalised Freecell is NP-complete. This will
allow using Freecell Solver™'s ingenious, cutting-edge algorithms to
solve the previously hard, provably NP-Complete problems of
the
Travelling Salesman problem,
Subset sum,
Sudoku and
Enterprise
content management.
Read more on the announcement to learn more about this and
upcoming exciting developments.
2010-04-03: Why I Wanted to Quit Planet Perl Iron Man and How to Improve it
Note: I discovered that the Planet Iron Man/Woman people are already
planning some changes to the Planet, and will deal with my complaints here
as part of this. So I'm not going to quit blogging on the Planet eventually,
at least not yet. I'm still posting it as a general insight for people
who wish to set up successful online communities.
I decided to quit Planet
Perl Iron Man and I'd like to tell you why and how I think it should be
improved. The reason I quit is because of the fact that if one doesn't blog
frequently enough, their status gets demoted (possibly back to Paper Man
always). Now, while if I blog frequently enough I will increase my status
which is motivating, demoting or even resetting the karma is
demotivating. This is
the Carrot and stick
method.
Now, I don't want to feel obliged to blog in order to prevent my karma from
being gone. I used to blog about Perl and other topics because it was fun, not
because I felt I had to. As a result, the "stick" in this place is out-of-place.
I request that assuming the current system remains, my status be either kept
as "unknown" or even that my blog be currently taken out of the Iron Man/Iron
Woman aggregator. Everyone should feel free to aggregate my Perl feed (
or any of my other feeds
as you see fit), without asking me, but I don't wish to play the carrot and
stick status game.
So how to improve it? Simple - avoid the stick. Make the Planet Iron Man be
a game of infinite karma, similar to Stack
Overflow or Perl Monks. On
Stack Overflow for example, your karma never decreases. If 2 people upvoted
your comment or post and 6 people downvoted it (for a total score of -4),
you'll still get 2 points. Furthermore, your karma can grow indefinitely, and
you also get other motivations like badges etc. Perl Monks has a similar
system with its Experience Points. (I should note that I'm not fanatically
trying to improve my karma on either site, and my current status on both
is very low, because I'm not into being competitive in them, though I found
both useful at times when I was seeking help.)
Perhaps we can make a similar program for a competitive planet, where people
get points for every posts, every link from someone else's post on the planet,
every upvote on Reddit, Stumble Upon and other social bookmarking sites, get
extra points for making, say, 3 posts in 7 days, earn badges etc. without ever
needing to fear their karma being reduced.
I should note that I've noticed other social sites being undermined (at least
temporarily) by such "sticks". For example
Advogato used to allow people to rate
other people's blogs' quality from 0 to 10. Naturally, some people gave bad
marks for some blogs which was demotivating. Luckily, it was removed during
an update. On
use.perl.org you could specify people you
dislike as "foes" in your user page, which ended up showing in them as
"freaks" (= haters). Again, that's demotivating.
There's probably still a lot to be learnt about designing successful web
communities, but I think that if you want people to do things because they
enjoy it (and/or you don't intend to pay them), you should not demotivate them
by threatening the use of "sticks".
2010-04-10: TelFOSS Meeting on Wednesday 21-April-2010 - "Version Control Sytems"
The Tel Aviv Open Source Club
will host
the
talk "Version Control Systems - What, Why and How" by
Yaron Meiry (Sawyer) on Wednesday, 21-April-2010, at 19:00 in
Shenkar College in Ramat Gan (12 Ann
Frank St.) room 300 (note the change of time and place from the
previous meetings - we are now meeting in Shenkar college and on 19:00.).
There's
a
map with directions on the Shenkar college's site.
Further details, maps for arrival, etc. can be found
on the Wiki.
Attendance is free, it is not necessary to RSVP and everyone are welcome.
With any other problems, feel free to contact the organiser.
Abstract
Version
control systems allow us to keep track of an unlimited amount of
changes across any number of files. They are an essential tool for any
developer.
Using version control systems can help you restore lost data, examine
the differences between between multiple versions of your data, tag versions
and access them quickly, simplify cooperation with your peers, easily merge
changes, and maintain several branches of your data at once.
This presentation will explain what version control systems are, why you should
use them (even if you think you shouldn't) and how you should use them. Most
examples will be given using the
open-source version
control system Git.
About the Lecturer
Yaron Meiry is a systems' administrator and a Perl developer. He gives talks
about open source, free software, security and programming standards. Yaron
has previously given the
presentation
about red flags in programming of very high languages in
the Tel Aviv Club and a presentation about
Moose, the modern Perl 5 Object System
to TelFOSS and several other clubs.
We are always looking for people who will volunteer to give presentations on
various topics that are related to open source code and to computers. In case
you are interested to give a talk, or that you have a suggestion for a talk
that interests you, we'll be happy to hear from you.
2010-04-15: פגישת תלוקס: "מערכות ניהול גרסאות" ב-21 באפריל
מועדון הקוד הפתוח התל-אביבי (תלוקס)
ייפגש שוב כדי לשמוע את
ההרצאה
"מערכות ניהול גרסאות - מה, למה, וכיצד"
מאת ירון מאירי (Sawyer) ביום רביעי, 21 באפריל 2010 ב-19:00
במכללת שנקר ברמת גן (רח' אנה פרנק 12)
חדר 300 (שימו לב לשינוי במיקום ובזמן מהפגישות הקודמות - אנו ניפגש במכללת שנקר ובשעה 19:00)
ניתן למצוא
מפה
עם הוראות הגעה באתר של מכללת שנקר.
פרטים נוספים, מפות להגעה, וכו ניתן למצוא
בוויקי.
הנוכחות בהרצאה היא בחינם ולא נדרשת הרשמה מראש.
במידה שישנן בעיות, אל תהססו
ליצור קשר עם המארגן.
סיכום ההרצאה
מערכות ניהול גרסאות
מאפשרות לנו לעקוב אחרי כמות לא מוגבלת של שינויים במספר שרירותי של קבצים. הן מהוות כלי הכרחי
עבור כל מפתח.
השימוש במערכות ניהול גרסאות יכול לעזור לך לשחזר מידע אבוד, לבחון את ההבדלים בין מספר גרסאות של
המידע, לתייג גרסאות ולאפשר גישה אליהן בקלות, להקל על שיתוף פעולה עם שותפיך לפרוייקטים, לאחות
שינויים בקלות, ולתחזק מספר ענפים של המידע שלך בו זמנית.
מצגת זאת תסביר מה הן מערכות ניהול גרסאות, מדוע כדאי שתשתמשו בהן (גם אם כרגע אינכם חושבים כך)
וכיצד כדאי שתעשו בהן שימוש. מרבית הדוגמאות יועברו באמצעות
מערכת ניהול הגרסאות פתוחת-הקוד "גיט".
אודות המרצה
ירון מאירי הינו מנהל מערכות ומפתח פרל. הוא מרצה על קוד פתוח, תוכנה חופשית,
אבטחה וסטנדרטים של תכנות. ירון העביר בעבר את
ההרצאה
על דגלים אדומים בתכנות עבור שפות עיליות ביותר במועדון התל-אביבי ומצגת
אודות
Moose, מערכת העצמים המודרנית לפרל 5
במועדון התל אביבי ובמועדונים אחרים נוספים.
אנו תמיד מחפשים מרצים שיתנדבו לתת הרצאות בנושאים שונים הקשורים לקוד-הפתוח ולמחשבים. במידה שאתם מעוניינים לתת הרצאה, או שיש לכם הצעה להרצאה שמעניינת אתכם, נשמח לשמוע ממכם.
2010-04-26: TelFOSS Meeting Report: Sawyer about Version Control Systems
Here are some notes from
the
Tel Aviv Open Source Club's meeting last week, in no particular order.
They are not exactly a report, but are probably better than nothing. I
originally
posted
them to the club's mailing list
and forwarded
them to to the Perl-IL mailing list, where it sparked a small (but funny)
discussion. I hope to expand upon them in this blog post.
-
About 20-30 people came in my approximation, some of them a little late, so
it was pretty successful. So hopefully it was a good decision to find a venue
which can afford starting at a later hour. We'll see if the trend of
successful presentations can continue.
I think I recognised most of the attendees, but there were some new faces,
which is a good thing.
-
Sawyer gave a very good presentation in my opinion, and I've learnt some
new stuff about git there. There was some good involvement from the audience.
Next time, I'll try to pass the baton to a different lecturer, because I think
Sawyer has been giving too many presentations lately, and we need to see some
fresh blood.
At the moment we only have one proposed presentation for June (by cool-RR /
Ram Rachum ) and he still noted he may need to cancel. I've been considering
to prepare a presentation/demo about jQuery and also proposed to give a
"Bottom Up Subversion" presentation/tutorial. I'm not a good speaker so I'd
prefer if
someone can give it instead. We can also have a session of short, volunteered
presentations on the spot
(lightning
talks or somewhat longer.).
Nevertheless, it would be preferable if someone will volunteer to give a
presentation.
-
Some logistics about the room - there is a keyboard and a projector, but no
screen. One can connect a laptop. The computer runs Windows , and we're not
sure if it has OpenOffice.org. There's an Ethernet connection and also some
Wifi networks.
-
Sawyer mentioned the fact that from his experience creating a branch in
Subversion (using svn copy) was time-consuming for him ("I went to prepare
coffee"). This sounds strange to me, because branching and tagging in
Subversion are reportedly cheap, constant-time ( O(1) ) operations, and
I've verified it from what I've read about the Subversion internals. I often
branched the Freecell Solver's trunk which is 26M in size and it didn't take
more than a few seconds. svn.kde.org has many branches and each of them keeps
the entire KDE source code and is positively huge with 100s of megabytes of
files.
-
I mentioned the mini-repository Subversion pattern - each with its own
trunk, tags and branches which can be seen in action in the Web-CPAN repository. This is a single
subversion repository with several mini repositories - one for each project.
It can be done in git too by keeping each project in its own
sub-directory, out of the top-level sub-directory.
I should note that I also eventually decided against putting the files of the
program directly on the trunk, but rather always put them under a
sub-directory of the trunk, so if I need to add something else, they won't
clobber the trunk.
-
There was a short discussion about whether version control systems are using
databases as their repositories' back-ends. Someone said he wouldn't trust
a VCS that uses an off-the-shelf database, and I'd like to comment about that.
The first thing that one should note is that like
"Practical
mod_perl" notes many data storages are in fact databases without us really
thinking about them. Even the file-system is a database that maps filenames
to the file contents. Furthermore, Subversion requires a transactional database
for its use: it started from Berkeley DB (which is a database with some
accepted
and deliberate philosophical limitations), and then got the ability
to use a backend called FSFS, which is a specialised, ad-hoc, database coded
specifically for use by Subversion; and the Subversion development team
have a distant goal of also creating an SQL-based backend for the
repository.
That's not all: Monotone has been using
SQLite since its inception as its local storage backend, and reportedly
ClearCase
is using a database called
"Raima". And naturally,
I expect that even git, Bazaar, Mercurial and similar version control systems
implement a database storage of some sort internally.
-
We discussed the
Simple
Simon solitaire, which I've introduced Ido (ik) to and to which he became
addicted to lately. I've ranted a
bit about how "Solitaire" (or what the British call "Patience games") is
actually a generic name for single-player card games, and that what Windows
calls "Solitaire" is actually just some variants of Klondike Solitaire.
We compared and contrasted both PySolFC and KDE 4's KPatience.
The one who introduced me to Simple Simon was
Stephan Kulow of KDE
fame, who when I told him that I'd like to adapt Freecell Solver to solve
Klondike said that "I don't think the world is ready for that." and then
added that he could really use a solver for Simple Simon. After playing
a few games of Simple Simon, and experimenting a little with the
fc-solve C code, I was able to create a satisfactory solver, but then I also
started to like Simple Simon and played a lot. And it was contagious as my
mother and my sisters also started to play it.
Right now, the KPatience's Simple Simon demo ends up in many practically
infinite loops of no apparent objective. I hope to work on converting it
to use my solver instead (which does not exhibit this problem), but it
will take some work.
-
About 5 of us went to eat at Spaghetim (an Italian restaurant in Ramat-Gan)
after the meeting. Besides me, there
were two Perl programmers ( http://szabgab.com/ and Sawyer, who was the
presenter) and two Python programmers and I was appointed as the referee. We
were able to explain some stuff from both languages to the other party.
-
Szabgab mentioned that when he asked some people he gave training to if
they know Perl, they said "No, I can only read it.", which to him seemed to
imply that maybe Perl's reputation as a write-only language was unfounded.
-
When we discussed Ram's Garlicsim ( http://garlicsim.org/ ) he mentioned
that he is mostly a web contractor, but that he would prefer to be hired for
writing simulations. Then Sawyer mentioned that often an invention brings a
necessity, which reminded us of
the
book "Guns, Germs and Steal". It is highly recommended. Then we discussed
some other books by the same author and I
ended up saying that I've also read
the
book "The Origins of Conciousness In the Breakdown of the Bicameral Mind" by
Julian Jaynes, which I enjoyed and found of a
similar interest to "Guns, Germs and Steel" because of its philosophy of
taking a bird eye's view of human history.
-
Also in the after-meeting, we discussed automated tests and test coverage,
and someone said that it may not be indicative of how well all the edge
cases were dealt with. For example we can test that the tests cover
the expression "z = x / y", but it won't warn us that we dealt with the
case that "y" is zero, which we should. Someone noted he likes the green
indication that he has 100% test coverage. On Perlsphere
Ovid
recently wrote that a general piece of advice is that one should not strive
for 100% test coverage.
-
We saw a demo of GarlicSim (which is
LGPLed), and its Wx-interface (which is at the moment not open source,
but has the source available for download under restrictive terms). Ram
showed us how he started a simulation where the simulator calculated a line,
and then it was possible to branch it into a different line of execution
and continue from there. He intends to continue working on GarlicSim into the
near future and then hopefully be able to give some presentations about it
to the local open-source clubs.
To sum up, it was a great meeting, and I hope this trend will continue.
2010-05-03: The Quest for a List of Open Source Software for Windows
As I was editing
the wiki document I've started writing called
"How to start contributing to or using Open Source Software" on the
Tecaching Open Source wiki, I've reached the part where I instruct people
to try and use open source applications for Microsoft Windows, and was
looking for lists of FOSS for Windows. I remembered
the "Open Source Windows" site,
which is nice and aesthetic, but not very comprehensive, and noticed
they got rid of the second page, which contained some other programs. So I
had to find another list.
A google search yielded many lists that were either not maintained,
disorganised, not open for easy editing, or all of them. Eventually, I decided
to look for something in the wikipedia and got to
the List of free and open source software packages, which I was
familiar with, but was not Windows-specific. However, on its bottom
I found a link to
the
List of Open Source Programs (LOOP) for Windows on one of the Ubuntu
wikis, and this seemed like the best of breed and what I was looking for.
I was already able to login to the wiki, add some programs I was familiar
with, and do some comsetic fixes. I also added it to the document,
favourited it on StumbleUpon, and linked to it from my homepage. Now I'm
blogging about it here to publicise it.
2010-05-09: Recent Hacktivity Log
Here is another boring recent hacktivity log, feel free to skip it. It was
motivated by Perlbuzz's question: "What are you working on in Perl?" and the
comment I've posted there in reply, though it's not restricted to Perl alone.
So thanks to Andy Lester from Perlbuzz for inspiring this post.
First of all, a note about my general status recently: I've been lacking
physical energy, and feel like I often cannot exercise a lot, and would rather
not travel too much outside my neighbourhood (including not to many FOSS
events). I don't know the exact reason why. I am still motivated to work
on the computer. I also have been working primarily on code and a small amount
of humour stuff, and not too many essays, articles and blog posts, which may
not be a bad thing, because essays and philosophical stuff may be considered
"schema-driven" (as my psychologist had put it). Besides coding, I've been
playing Fillomino and other games on Brain
Bashers, playing some PySol (Freecell and Simple Simon primarily), dealing
with bug and problems in my Mandriva system as I run into them, chatting on
IRC and Instant Messaging, writing some E-mails, including some to answer the
questions of random people on various mailing lists, and doing some job
hunting.
I turned 33 (or 21h like the DOS
interrupt) on 5-May, and also attended my cousin's birthday party on
Friday, and had a lot of fun playing ball with a different cousin's son
at the party, and also enjoyed the food there. But on to the hacktivity:
Recent Hacktivity - for real this time.
I've worked on XML-Grammar-Fiction/Screenplay a lot recently. I've decided
to merge the two modules because XML-Grammar-Fiction started as a fork
of XML-Grammar-Screenplay, which was then improved, and I've started forward
porting its changes to X-G-Screenplay. Forward-porting the changes to
X-G-Screenplay proved to be very not straightforward due to the fact that I
converted the X-G-Screenplay parser from being procedurally recursive to being
more stateful and incremental with O(1) function call stack.
Then I had to merge the two codebases into a few base classes and two
specialised sub-classes. Then I merged different parts. While there were still
some things I wanted to do in my ongoing clean-ups, I decided to upload
version 0.1.0 to CPAN as it is, in order to get some CPAN testing, to
"release early, release often" and because perfect is the enemy of good. I
indeed got some CPAN testers failures indicative of bugs which I have since
hopefully resolved.
A bit off-topic from the question of this post, but I took a break from working
on Perl code per-ce and started working on incorporating a new feature, which I
nicknamed "flares" to Freecell
Solver. I was able to split the code into many commits, each one passing
all the existing and possibly new tests (some of which are written in Perl 5).
Back in the early Freecell Solver 2.8.x-or-below-days, when I was less careful
and methodical, I would have just written it in one swoop and then spend many
fun hours with gdb trying to isolate and correct many bugs. But now I know
better than to do that. I still had to use gdb and other debug mechanisms, but
this time I was able to detect the source of the bug and correct it relatively
quickly. Having automated tests rule!
I've had to write some text analysis logic in ANSI C, while using only mostly
standard libc/libm functions and really wished I could have done it in Perl. I
wrote and refactored some testing code in Python/PyTAP, because I needed ctypes
to interface with the binary shared library. In the course, I've written some
pretty complex Test-Count (=
a CPAN module to keep track of TAP tests' counts) code (and enjoyed the
fact that I can source variable calculations from common modules, which I've
added for a previous project (which I don't recall what it is now.)) I also
made use of a radix tree
building and compiling Perl 5 program that I wrote for managing Freecell
Solver's command line arguments.
Finally, I prepared
a
short presentation about the Mojolicious web-development framework,
which I've
blogged
about previously, and I end up explaining why I dislike it at the
bottom.
Not exactly my hacktivity but I'm glad that chromatic's suggesting of
porting Python's ctypes to Perl
has
materialised as a Google Summer of Code project. I hope to assist
Ryan Jendoubi in doing this (without being intrusive.), because it
would be very cool to have something like ctypes in Perl 5.
So that's it, I guess. Happy upcoming
Shavouth everyone!
2010-05-20: Perl 5 Anti-abstraction: _with_curr_line
In the old XML-Grammar-Screenplay code, I created a small _with_curr_line
method (unrelated to my previous post about
→with()) that read like that:
sub _with_curr_line
{
my ($self, $sub_ref) = @_;
return $sub_ref->($self->curr_line_ref());
}
What it does is evaluate a closure and passing it the reference to the current
line to "avoid" duplicate code. In one of my comments I also wrote
"# Now Lisp got nothing on us.", which as it turned out was completely untrue.
Furthermore, this abstraction, which was meant to get rid of explicitly
assigning the "curr_line_ref()" to a variable in many places made my code
much more messy. Some notable examples are:
# Skip the [
$self->_with_curr_line(
sub {
my $l = shift;
$$l =~ m{\G\[}g;
}
);
Instead of:
# Skip the opening square bracket - '['
${$self->curr_line_ref()} =~ m{\G\[}g;
And:
my $which_tag;
# We need this to avoid appending the rest of the first line
$self->_with_curr_line(
sub {
my $l = shift;
# Apparently, perl does not always returns true in this
# case, so we need the defined($1) ? $1 : "" workaround.
$$l =~ m{\G([^\&t;\[\]\&]*)}cgms;
$curr_text .= (defined($1) ? $1 : "");
if ($$l =~ m{\G\[})
{
$which_tag = "open_desc";
}
elsif ($$l =~ m{\G\&})
{
$which_tag = "entity";
}
elsif ($$l =~ m{\G(?:</|\])})
{
$which_tag = "close";
}
elsif ($$l =~ m{\G<})
{
$which_tag = "open_tag";
}
}
);
(As you can see, I assign to $which_tag inside the closure and have to declare
it beforehand).
my ($sayer, $what);
($sayer) = $self->_with_curr_line(
sub {
my $l = shift;
if ($$l !~ /\G([^:\n\+]+): /cgms)
{
Carp::confess("Cannot match addressing at line " . $self->line_num());
}
my $sayer = $1;
if ($sayer =~ m{[\[\]]})
{
Carp::confess("Tried to put an inner-desc inside an addressing at line " . $self->line_num());
}
return ($sayer);
}
);
Regex fun and declaring the two variable twice.
As a result, I decided to get rid of this anti-abstraction and do
a simple "my $l = $self->curr_line_ref". Now what about Lisp? In Common
Lisp, one can declare a macro to define methods that will already
have the "l" for the current line reference variable pre-declared
and pre-assigned. Or maybe one can also temporarily rewrite the method
definition macros or add hooks for the duration of the class. So I guess this
could have been much better there, and Perl 5 is not as good as Lisp is in
this respect.
And I want to hear mst give a talk called
Patches Welcome
(might as well pool my votes to a good cause - I hope the talk will be recorded
and there will be slides available.)
2010-05-31: Quirky Abstractions
Joel Spolsky had written an essay titled
"The
Law of Leaky Abstractions" where he discusses the fact that sometimes
one needs to be aware of details of the abstraction's implementation to know
how to properly make use of it. Today, I'd like to talk about an analogous
phenomenon which I came to call a "quirky" abstraction, in which
an abstraction over several technologies (probably two or more) fails to
expose certain features in one of the technologies that it abstracts.
I first noticed this concept in
a
Stack Overflow question of how to encrypt/decrtypt an SQLite database from
within Qt. Now, the Qt SQLite backend module links against the vanilla
SQLite and not its encrypted implementations, so in order to make use of them
one will have to recompile Qt and possibly expose some extra functionality in
the abstraction.
At the moment, I've also volunteered to help maintain
the Graph-Easy
CPAN module and the reason was that one of the features of GraphViz was
not exposed in its Graph-Easy backend, which brought the need to hack
something as a workaround.
Can quirky abstractions be avoided? I don't know. As the abstraced
technologies progress and gain more features, the abstraction
needs to keep up with them, which makes a situation of a quirky abstraction
more likely to occur.
I'm not sure quirky abstractions are as bad as leaky abstractions, but it's
still an phenomenon that we need to be aware of.
2010-06-14: Rehovot Perl Mongers meeting on 15-June and Hamakor Assembly on 17-June
Here are some upcoming open-source-related meetings in Israel:
The next meeting of the Rehovot
Perl Mongers will take place on 15th June
(tomorrow) between 18:00-22:00 in the Weizmann Institute in Rehovot
in Room 101 in the Levine Building. See the details and map.. The talk ("Introduction to Perl 6"
by Gabor Szabo) starts at 18:30.
Content: Introduction to Perl 6
I've just written and gave this talk 2 days ago at the LinuxTag in Berlin.
After a short background and a few words on Perl 5 vs. Perl 6 I'll go over
a number of examples that is part of the imminent Rakudo Star release.
People who bring a notebook and will have enough network bandwidth will be
helped to install Rakudo on their machine. It is quite easy to install so
you could do it ahead of the meeting by following the instructions on
http://www.rakudo.org/
On Thursday (17-June-2010), Hamakor, the Israeli NPO for free and open
source software will have a general assembly in Shenkar college, Frink
building, room 306 in 18:30.
Arrival instructions are available on Shenkar's site
Thanks to Yoram Gnat for the help in organising a place.
2010-06-24: Declaration against ACTA
ACTA,
the so-called "Anti-Counterfeiting Trade Agreement", which is a trade
agreement that aims to criminalise a lot of the openness of the Internet
under the guise of fighting counterfeited goods. The Free Software Foundation
has started
a declaration against ACTA, which everyone should sign.
Also please help spread the word.
2010-07-11: Tech Tip: Automatically Restoring the Wifi Connection on Linux
Having bought a new laptop and after installing Mandriva Linux on it, I noticed
that the wifi (wireless networking) connection tends to get disconnected after
a while. Eventually I wrote this script to automatically restore it. Feel free
to re use it under the terms of the
MIT X11 License. You
may need to adjust some parameters. Run this script as root in the background
(using tmux or GNU screen or
whatever).
#!/bin/bash
while true ; do
if ! ping -c 1 10.0.0.1 ; then
/etc/init.d/network restart
fi
sleep 5;
done
Hope it helps.
2010-07-15: Tel Aviv Open Source Club Presentation: "GarlicSim" by Ram Rachum
מועדון הקוד הפתוח התל-אביבי (תלוקס)
ייפגש שוב כדי לשמוע את
הרצאתו
של רם רחום אודות GarlicSim
ביום חמישי, 22 ביולי, 2010 ב-19:00
במכללת שנקר ברמת גן (רח' אנה פרנק 12)
חדר 300 (שימו לב לשינוי במיקום ובזמן מהפגישות הקודמות -
אנו ניפגש ביום חמישי, במכללת שנקר, ובשעה 19:00). ניתן למצוא
מפה
עם הוראות הגעה באתר של מכללת שנקר.
פרטים נוספים, מפות להגעה, וכו ניתן למצוא
בוויקי.
הנוכחות בהרצאה היא בחינם ולא נדרשת הרשמה מראש.
במידה שישנן בעיות, אל תהססו
ליצור קשר עם המארגן.
סיכום ההרצאה
GarlicSim
הינה תשתית עבור הדמיות (סימולציות) ממוחשבות הכתובה בפייתון. היא מהווה
פרוייקט קוד פתוח שאפתני בתחום המחשוב המדעי, ובפרט בזה של הדמיות ממוחשבות.
בכוונתה להטוות מחדש את הדרך בה אנשים חושבים על הדמיות ממוחשבות, וליצור
סטנדרט חדש לדרך שבה יוצרים ומשתמשים בהדמיות.
GarlicSim
הינה פלטפורמה לכתיבת, הרצת, וניתוח סימולציות. היא מספיק כוללנית
כדי לטפל בכל סוג של הדמיה: פיזיקה, תורת המשחקים, התפשטות מחלות,
אלקטרוניקה, וכו. ייתכן שאתם תוהים מה משותף לכל ההדמיות האלה.
אנו ננסה לענות על שאלה זו בהרצאה. אנו נראה דוגמאות של הדמיות אחדות
ב-GarlicSim ושל כלים שהיא מספקת להרצתן וניתוחן.
אודות המרצה
רם רחום
הוא מפתח פייתון. הוא עובד בעיקר על GarlicSim, אבל ידוע גם כמי שיצר את
PythonTurtle,
על מנת להפיץ את פייתון לאלפי ילדים ומבוגרים ברחבי העולם.
אנו תמיד מחפשים מרצים שיתנדבו לתת הרצאות בנושאים שונים
הקשורים לקוד-הפתוח ולמחשבים. במידה שאתם מעוניינים לתת הרצאה,
או שיש לכם הצעה להרצאה שמעניינת אתכם, נשמח לשמוע ממכם.
2010-07-30: Git Tip: git status
I've had to work with
the
git version control system extensively in the past few weeks (due to
the proliferation of projects I'm involved in that use it.). I
found the standard "git status" output to be far too wordy and annoying,
and sought a way to improve it and make it more subversion like.
"man git-status" helped with this: one can write git status -s to
get an output similar to svn status (or svn st for
short) and git status -s . to restrict the output only to files
below the current directory (and not get the ../… files). Enjoy!
2010-07-31: Recent Hacktivity Log
Another boring recenty hacktivity log. First of all, in case you have not
heard Rakudo Star,
a usable implelmentation of Perl 6 has been released after a lot of work.
Now on to the hacktivity.
File-Format-CRD
I released the first version of
File-Format-CRD
to CPAN. This is a CPAN distribution that processes
Windows Cardile (.CRD)
files (does anybody still remember those?) and allows to convert
them to other formats. I wrote it because the only program with source that
I found for them was written in C and converted them to invalid HTML. I based
my efforts on it.
After I wrote the program I was able to extract the contents of an old
CRD file I had with riddles, but then was appalled from the immaturity there
(as I wrote it during high school). Still, the module may prove useful. It
seems like the second CPAN distribution in the File-Format-* namespace after
File-Format-RIFF, but I could not find any place better to put it.
Test-WWW-Mechanize-LibXML
Test-WWW-Mechanize-LibXML
is another off-shoot of my project for Insurgent Software, that provides
some primitives to test web pages' content using
HTML-TreeBuilder-LibXML.
Right now, it requires some funky multiple-inheritance to work together with
Test-WWW-Mechanize-Catalyst because they both override the same function. But
it's there.
libfilefind
I've continued working on libfilefind, which is a translation of
File-Find-Object to C using glib. Translating the Perl code to C is very
tedious, and since I have a lot of Perl code, I decided to translate it all at
once, pray, and then test it. After that, I hope to create Perl bindings to
this library, and put them on CPAN in order to increase File-Find-Object's
speed.
catable
I've done a little work on
Catable, which is our
Catalyst-based blog engine.
Homepage
I've done some work on the homepage, adding yet-unannounced pages,
and also adding some more "<meta />" keywords to existing pages.
Project Euler
This weekend, I've made
a lot of progress with
solving Project Euler problems. One problem which took me a long time
to get right was eventually solved using
Test-driven
development (TDD) and then I made more progress including solving one
problem by hand.
Games-Solitaire-Verify
I released a new version of
Games-Solitaire-Verify
and this time took
Sawyer
X's advice and included a small command-line application inside, which
proved useful for me as well in my Solitaire research.
2010-08-07: TelFOSS Meeting: Kaltura, the Open Source Video Platform
The Tel Aviv Open Source Club
will host a presentation about Kaltura, the Open Source Video Platform
by Zohar Babin on Thursday, 12-August-2010, at 19:00 in Shenkar College in Ramat Gan
(12 Ann Frank St.) room 300 (note the change of time and place from
the previous meetings - we are now meeting in Shenkar College, on a Thursday
and on 19:00.). There's a map with directions on the Shenkar College's site. Further
details, maps for arrival, etc. can be found on the Wiki.
Attendance is free, it is not necessary to RSVP and everyone are welcome.
With any other problems, feel free to contact the
organiser.
Abstract
Kaltura (also see its Wikipedia entry) allows publishers of all sizes to easily, quickly, and cost
effectively enhance their web site with video and interactive rich-media
functionalities, including video management, searching, uploading, importing,
editing, annotating, remixing, sharing, and advertising.
Unlike the alternative proprietary solutions, Kaltura’s platform is
completely flexible, extensible and free! Web publishers,
value-added-resellers, and integrators use our software development kit to
customise a unique rich media experience that fits their specific purposes, and
seamlessly integrates with many popular content management systems. Moreover,
Kaltura’s reference implementations and growing library of applications,
extensions and plug-ins allow publishers to select off the shelf solutions for
rapid self-serve deployments that can be fully enabled within minutes.
Visit Kaltura.org for the community
site, forums and all the open source projects that are part of the Kaltura
Platform.
In the lecture
- We will cover all
major aspects of online video and why a platform to handle the various tasks is
required.
- We will learn about ways to install the Kaltura CE v2.x, and use it to deploy your own video
website/RIA.
- There will be an open discussion about using and extending Kaltura
for personal use, academic use and enterprise solutions for rich-media
knowledge management.
This lecture is an open invitation to
learn more, try out and become a part of this fast growing and exciting open
source solution.
About the Lecturer
Passionate about
people, ideas, technology and video; Always ready to chat about innovative
ideas or creative solutions, Zohar is a community technology activist and
programmer. Among his interests are the study of communities, collaboration,
open source, digital art, Video and 3D.
Zohar works at Kaltura where he
heads Developer Relations & Community.
He is a fellow manager of
flashoo.co.il, the Israeli Flash Platform developer community, founder of
WebiTalks the Israeli web developer community and frequently speaks at web
events worldwide.
Twitter: @zohar And his fresh new Blog
A
video artist by heart, developer by night. :)
We are always
looking for people who will volunteer to give presentations on various topics
that are related to open source code and to computers. In case you are
interested to give a talk, or that you have a suggestion for a talk that
interests you, we'll be happy to hear from you.
2010-08-09: Report on August Penguin 2010
Hi all. This is a report on the
August Penguin 2010 conference,
which I've attended and enjoyed. I decided to write it in English due to the
proliferation of Hebrew reports, out of convenience, and in order to publicise
the Israeli open source activity abroad. So here goes nothing.
First of all, an apology: I wanted to publicise the conference more
intensively, but I got distracted by the fact that the opening paragraph
in the site was supposed to be fixed (and I did not know how to fix it because
I'm not a Drupal expert and there was no edit button there), and that I started
working intensively on a for-pay project (and a Perl and
Catalyst one at that) shortly
before the conference, which turned out to be time-consuming and
also distracted me. So I apologise if it resulted in fewer people attending
the conference, and I'll try to keep it in mind for next year.
In any case, I posted an E-mail to the mailing list asking for a ride,
and Omer Zak (a.k.a "TDDPirate")
volunteered to give me a ride. After negotiating an hour, I realised I needed
to wake up at 06:00 AM, in order to catch the ride. The alarm woke up me
up and I had relatively little trouble to get up. I made some preparations,
ate something, and started playing a deal of Freecell in
PySolFC (which later on during
the day motivated me to
find a performant scan for Freecell Solver to solve it), and didn't finish
in time. Then I went to meet Omer.
He picked me up on Keren Kayemet Boulevard a long stretch before the
designated meeting place. Apparently he was early (or I was a bit late),
and had to diverge from the main road. So we drove all the way to
Weizmann Institute where the August Penguin conference took place. The ride
was relatively uneventful, there were no substantial traffic jams, but we saw
another car with some FOSS-related stickers on the way (probably going to the
conference as well). We arrived early before the doors of the hall were open,
but other people got there too and I've talked with them a little and also
posed for a photograph with my name.
After they opened the doors, I helped carry some food and stuff from the cars,
and then ate some of the pastries that were brought and drank some assorted
juices - all of which were good. (I think I had eaten too much during the
conference, because later on, I didn't have too much appetite for the midday
meal).
In any case, I met a few old and new people during the conference. One of
them told me he met me on Freenode's
#perl, where I guided him a bit and that he was on Freenode often. He
brought an Archlinux-based laptop, and also said he's been writing some Perl
scripts for various tasks. I've also met
Dor for the first time as
he was volunteering in the Ubuntu-Israel stand.
I should note that I had volunteered to guide a blind attendee and she
eventually arrived and I showed her where everything was in the hall. During
one of the breaks, I also showed her where
Ori was. She asked me to escort her
to the bus station in order, but eventually someone else volunteered
to escort her a short time before I went out of the hall to do so.
I enjoyed several of the presentations there especially the ones about
the 0AD game (which looked great
based on the screenshots and screencasts), and
the Open Knesset
(= the Israeli parliament) project. I stayed up to the giving of
the Hamakor prize, where I was invited to the stage to receive the third
place for the communal prize for the Israeli
Perl Mongers.
Omer eventually gave me a ride back home after wrapping up the transcripting
business there.
So the conference was very nice and I enjoyed it a lot. Here's to next
year's conference, and hope to see you in
the upcoming Tel Aviv Open Source Club meeting this Thursday.
2010-08-25: Perl Debugger Tip: A Session Startup File
Here's a small Perl debugger tip: in order to have a file whose commands
will be executed at the start of the debugging session (for example in order
to get to a certain point in the code) - a session startup file, similar
to gdb's --command=cmds.gdb flag - one can do the following:
First of all write a file with the debugger commands you want (let's call it
cmds.perldb) and then when inside the perl debugger say:
source cmds.perldb
This will execute all the commands.
I noticed that after a while the perl debugger stores it inside the history
(assuming you're using Term-ReadLine-Gnu), so you can recall it with
s and then pressing the history-search-forward
and history-search-backward that were set up (Mandriva assigns
them to Page Up and Page Down).
Enjoy and I'm sorry for having neglected this blog for a long while!
2010-08-31: Is the web becoming fragmented?
Peteris Krumins
writes:
So I participated in the 48 hour Node.js Knockout competition together with
James Halliday and Joshua Holbrook. Our team was called Dark Knights and we
created an online chess application called Node Chess.
…
Oh, and it works only [on] Chrome. Ancient-browsers-please-be-gone!
My question is: how can you call Firefox, whose
latest stable release was on the 23 of July
this year and which has perfectly usable and nightly builds ancient?
It is highly possible that due to the recent hype surrounding HTML 5 and its
mutually partial implementation by the different browsers, that we are entering
a situation where many sites or demos will only work on particular browsers.
This didn't start with Peteris' post - naturally. Previously, someone from
the Israeli Internet Society
referred me to some demos that said required a WebKit-based browser, and during
a presentation about HTML 5 in an "Alphageeks" meeting, the presenter had
to use three different browsers, because all the features he wanted to
demonstrate did not work on all of them. And this is without taking
the account
Apple's
block of non-"Apple Safari" browsers from its HTML 5 demos
and the fact that www.spice-space.org
obnoxiously redirects you to
"This site requires JavaScript" page if JavaScript is disabled, and many other sites do not function
properly without JavaScript enabled.
And as a commenter on Reddit for a web demo said, while the
demoscene people have
been trying to produce demos that utilise the most out of the computer's
resources, the web demos have come to waste a lot of resources in creating
anachronistic demos, whose only selling point is that they run inside
a browser. As Joel
on Software notes: Combined with the speed and responsiveness from Ajax,
FogBugz has almost reached the level of speed and fluidity of my dry
cleaner's DOS 2.0 character mode database application. And that's pretty darn
responsive for a web app.
Are we headed into another "Best viewed with Netscape 2.0", "Best viewed with
Internet Explorer 4.0", etc. era of web fragmentation, because we opened
the Pandora box of HTML 5? As for me, if I were a judge on that competition
that Peteris took part of, I would fail his project due to not being capable
of running on my ancient browser.
2010-09-12: Perl Debugger Tip: Viewing Using The Pager
Here's another Perl 5 debugger tip: if the output of a command is too long,
one can prefix it with "|" (a vertical bar / pipe) to display it using the
pager. So, for example
|x $my_object will display $my_object using the shell's pager (
GNU less or more or
whatever). Enjoy!
2010-09-23: Copying Ubuntu Bug No. 1
Tel Aviv, Israel: "Ubuntu can't have all the fun only for itself",
open source distributors are saying as they rush to copy its
Bug No. 1 titled "Microsoft has
a majority market share".
Debian, Ubuntu's parent distribution, has set up
Debian Bug No. 1 which also reads "Microsoft has a majority market
share". Mandriva,
another competing distribution has set up
Mandriva
Bug No. 1: "Microsoft has a majority market share and Ubuntu has a majority
market share on the Linux desktop". Fedora, Gentoo, Archlinux, Slackware
and other distributions are expected to follow suit.
Mandriva's Bug No. 1.
In the meanwhile, Microsoft Corporation (MSFT) have decided that they won't
stay out of the game, and have unveiled their brand new web-based bug-trackers,
with "Microsoft Windows Bug No. 1: Windows does not have a 100% market-share",
and "Microsoft Internet Explorer Bug No. 1: We are losing market-share to
Firefox and other browsers.". Among the enthusiastic responses from
commentators on the bugs one can find "Why is this a bug, FFS???" by
"MS-must-Die" and "If this is a bug then I'm a Jackalope, though you should
fix many of the f***ing bug dependencies." by "Slashdot Anonymous Coward".
Some prominent open-source projects such as
youtube-dl,
abcde (A Better CD Encoder)
and bashdb (the Bash Debugger)
have decided to duplicate Ubuntu bug No. 1 with all of its comments as their
own "Bug No. 1"s noting that "Microsoft's dominance of the market
threatens the very fabric of our existence, and it is our utmost priority to
fight it".
youtube-dl's Bug No. 1.
Ubuntu's Bug No.1 proved to be a good publicity stunt for Ubuntu and a
focus of active (and often tangential) disucssion. However, now it seems that
Ubuntu are not alone in this, and it remains to be seen how their competitors
will benefit from copying it.
Ubuntu's former CEO Mark Shuttleworth is pleased with the ongoing popularity
of the "bug No. 1" meme. "We believe that Ubuntu's bug No. 1 was the key
differentiating factor in the success of Ubuntu, and we are glad other open
source projects finally realise that. Hopefully, we will soon open more bug
No. 1s (different bugs with the same number) for 'Ubuntu has an equal
marketshare to distribution X'".
Meanwhile, some voices on Ubuntu bug No. 1 have called for converting
its status from "critical" to "release critical", which will make it block
a new Ubuntu release until the bug is resolved. "I am positive that we should
not release a new version of Ubuntu while Microsoft still has a majority
market-share", said an anonymous commenter, "it certainly must block new
releases, and Ubuntu releasing new versions with such a grave bug still
open only amplifies the problem."
Licence
/images/somerights20.png)
This
work is licensed under the
Creative
Commons Attribution-ShareAlike 2.5 (Unported) License (or at your option
any greater version of this license). Copyright holder is Shlomi Fish and year
of copyright is 2010.
To attribute, please mention my name, and link to this page and
to my homepage.
2010-10-02: Estimating the Size of the Perl 5 MojoMojo Wiki
There was an myth going on that
"you
cannot write large-scale programs in Perl". It's been a while since
I've heard it but it's possible that Perl just went out of the hackers' radar.
In any case, today I wanted to address it by estimating the size of
MojoMojo, a wiki written in Perl. MojoMojo
stands at the top of the CPAN Top "Heavy"
100 with dependencies on 330 direct and indirect CPAN distributions.
Here's the executive summary: MojoMojo and its dependencies contain
approximately 631,222 source lines of production code (what is under
the distributions' lib/ directory) - all of it Perl - and
approximately 272,027 lines of testing code (what is under the t/'s)
, about 265,833 out of which are Perl. The total is 897,055 SLOCs of Perl.
Here is the lib/ report by
SLOCCount:
Totals grouped by language (dominant language first):
perl: 631222 (100.00%)
Total Physical Source Lines of Code (SLOC) = 631,222
Development Effort Estimate, Person-Years (Person-Months) = 174.27 (2,091.23)
(Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
Schedule Estimate, Years (Months) = 3.81 (45.68)
(Basic COCOMO model, Months = 2.5 * (person-months**0.38))
Estimated Average Number of Developers (Effort/Schedule) = 45.78
Total Estimated Cost to Develop = $ 23,541,431
(average salary = $56,286/year, overhead = 2.40).
SLOCCount, Copyright (C) 2001-2004 David A. Wheeler
SLOCCount is Open Source Software/Free Software, licensed under the GNU GPL.
SLOCCount comes with ABSOLUTELY NO WARRANTY, and you are welcome to
redistribute it under certain conditions as specified by the GNU GPL license;
see the documentation for details.
Please credit this data as "generated using David A. Wheeler's 'SLOCCount'."
And here is the t/ report:
Totals grouped by language (dominant language first):
perl: 265833 (97.72%)
pascal: 5888 (2.16%)
java: 168 (0.06%)
ansic: 96 (0.04%)
cpp: 30 (0.01%)
sh: 12 (0.00%)
Total Physical Source Lines of Code (SLOC) = 272,027
Development Effort Estimate, Person-Years (Person-Months) = 72.01 (864.08)
(Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
Now for a longer story of how I collected these statistics. At first I wanted
to download all the dependencies. For that I needed to list them. I looked at
CPANDB, but could not figure
out how to do it properly using it, because its API still has many
SQLisms, and really needs a convenient high-level API. My SQL skills were
also too low to hack on the SQL directly. However, someone on
#toolchain referred me to
CPAN-FindDependencies, whose synopsis did exactly what I needed. I ran it and it generated
a file with all the dependent modules and their CPAN distributions. This
was filtered using this shell one-liner:
cat ~/mojomojo-deps.txt | ruby -ne 'puts $1 if / \(([^\)]+)\)/' |
sort | uniq > dep-dists.txt
(Note - I've used ruby instead of perl here, not because I don't like Perl,
but because I've been trying to learn Ruby lately and could use the practice.
I can do the same with Perl, too, by adding -l and replacing "puts" with
"print".)
Then I got a list of paths like
A/AB/ABIGAIL/Regexp-Common-2010010201.tar.gz ,
A/AB/ABW/AppConfig-1.66.tar.gz etc. After trying to get the CPAN
client to download them to the current directory, I've given up and instead
used the following script:
#!/bin/bash
cat ../dep-dists.txt |
(while read T ; do
echo "Doing $T" 1>&2
L="$(basename "$T")"
if [ ! -e "$L" ] ; then
lwp-mirror http://biocourse.weizmann.ac.il/CPAN/authors/id/"$T" "$L"
fi
done)
If you're using it, please replace the biocourse.weizmann mirror with a nearer
mirror - I used one suitable for Israel. Then I had a list of tarballs. I
cded to a different directory and ran a loop to unpack them all:
for I in ../dists/*.tar.* ; do tar -xvf "$I" ; done
Then I used the following loop for extracting only their lib directories:
ls -d ../unpacked/*/lib |
perl -lne '$x = $_; s{\A../unpacked/}{}; s{/}{-}; print "cp -a $x $_"' |
bash
(Note: it's not safe for paths containing special characters, but I didn't
have to worry about it here.)
I used a similar loop for t and then ran SLOCCount (which is written in Perl)
on the directory structures yielding the final statistics.
Some conclusions are:
-
Perl code can scale to large code-bases (over 600K lines of code). On the
original link I've pointed, the one who made that statement said after I
pointed him to some data disputing his claim that it was possible but it
was not easy. However, writing all this code was not extremely difficult
and I didn't hear that the maintainers of these CPAN distributions have
quit in despair because maintaining the distributions was too hard. So it's
possible and not too difficult.
-
It's probably a bit worrying that there is a relatively small percentage
of testing code out of the production code. I expected it to be higher.
Finally, I should note that having discovered that MojoMojo can also be
used as a blog engine, I'm considering to abandon my
Catable project, which aimed
to create a Perl and Catalyst-based blog engine (and
which still only has basic functionality) and simply use MojoMojo. Thanks
to EdwardIII from
#perl for
pointing me to that.
Cheers!
I should note that
my
previous post (which was a joke) took an ugly turn when many of the
commenters there thought I was serious and flamed me. I don't usually
receive so many comments, but their tone depressed me a little. This post
is serious, I promise.
2010-10-10: Bash Tip: Writing Numbers with Commas
One thing that vexes me a lot about computing conventions is that numbers
in most computer programs cannot be written with commas. Quick, is
60000000, six million or sixty million? Isn't 60,000,000 much better? Now in
Perl, one can use underscores in to separate the digits of numeric constants:
my $one_milliard = 1_000_000_000;
Here's how to achieve a similar effect in Bash:
MAX_DEAL='32,000'
freecell-solver-range-parallel-solve 1 "${MAX_DEAL//,/}" 500 -l by
We've declared the number constant MAX_DEAL with commas, and we got rid
of the commas before passing it to the program using the //,/
trick. The latter is part of
Bash's
parameter subtitution. Search for "${var//Pattern/Replacement}" for the
specific use, which does global replacement of a pattern - in our case - a
comma, with a replacement - in our case - an empty string.
Death to the unreadable numbers!
2010-10-27: Rehovot.pm Meeting on 9-November-2010
The Hebrew text will be followed by an English one.
מועדון שוחרי הפרל של רחובות
ייפגש שוב ביום שלישי ה-9 בנובמבר 2010, במכון וייצמן, כדי לשמוע את ההרצאות הבאות:
-
Mojolicious - דבק לווב של HTML 5. (אודות התשתית לפיתוח יישומיי ווב)
מאת דותן דימט.
-
POE - מערכת ריבוי-משימות מבוססת אירועים עבור פרל. מאת ירון מאירי (סווייר)
הפגישה תיערך החל מ-18:00 ביום שלישי ה-9 בנובמבר, בחדר 101 בבניין לווין במכון וייצמן ברחובות.
פרטים נוספים ניתן למצוא
באתר של שוחרי הפרל של רחובות.
The Rehovot Perl Mongers will meet
on 9th November 2010 (a week earlier than expected) between 18:00 and 22:00
in thethe Weizmann Institute in Rehovot
in the Levine Building. See the details and map..
The planned presentations are:
-
Dotan Dimet: Mojolicious - duct tape for the HTML5 web
Mojolicious is a new web development framework optimized to be "easy and fun", from Sebastian Riedel, the creator of Catalyst. I'll introduce the shiny pieces in the
Mojolicious tool chest and show how they fit together to make an application, and then we'll try to write a simple program and see how to test it using Mojolicious' built-in web client and XML parser.
Estimated time: 40 minutes talk, another hour to try things out.
-
Sawyer:
POE - Event-based, multi-tasking framework for Perl
Most programmers will eventually reach a point that will require or help them
to know event-based programming, such as writing a server that accepts multiple
connections (sockets, web, etc.), a file/diretory handling daemon or any system
background service.
POE has been the de facto standard in event-based programming in Perl for a
while now, thanks to its flexibility and incredible modularity. It might seem
intimidating but it's actually pretty simple and fun to work with.
I'll be giving an introduction to it with usage examples, helping those that
heard of it but were too scared, those who tried without success, those who
might be doing it wrong and lastly, those who don't even know what it is.
$rehovot_pm->on_event( poe_talk => \&attend() );
Hope to see you there.
2010-10-28: Tech Tip: Making Single Click on a URL Do the Right Thing on Linux
I had a long-time problem on my Mandriva Linux system where, after I set
Firefox as the default browser on KDE, it either made KDE first download
the page to the hard-disk and redirect Firefox to access it from
/var/tmp - or, if I set the default browser to "firefox %u", it
opened the link in a new window twice.
Today I decided to go to the bottom of it, after seeing that an upgrade of
KDE on Mandriva Cooker did not fix the problem (which I was told would). After
consulting some people on IRC, I realised that I could not reproduce the bug
on a new UNIX user account, so it was a configuration problem here. Then I
realised that /usr/bin/www-browser which is what the
BROWSER environment variable points to also opens it this way. After
looking through the source code of www-browser, and testing some
command-line invocation of capture-shells there, I suspected the problem was
with a stray firefox.desktop file.
Running locate found two such files inside my home directory:
/home/shlomif/.gnome2/panel2.d/default/launchers/firefox.desktop and
/home/shlomif/.local/share/applications/firefox.desktop. Moving away
them both resolved the problem and now if the default browser is set to
"firefox", the Firefox opens a link in a single new tab after a single click.
Cheers, and hope it helps.
2010-11-12: Recents Enhancements to Perl-Begin.org
This post summarises the recent enhancements to
the Perl
Beginners' Site.
Perl-Begin continues to be the most comprehensive first-stop for Perl Beginners.
Moreover , its content is distributed under the Creative Commons Attribution
License (CC-by), and its source
code is available to facilitate contributions.
2010-11-13: Recent Hacktivity Log
This is another recent hacktivity log. First of all, I should note that my
new day job (working from home for a Tel Aviv-based startup doing Perl
and Catalyst (a web-devel
framework for Perl)) has been consuming a lot of time and energy, which
has taken its toll on my pro-bono open-source work. But I still have a lot
to tell.
Freecell Solver
I've done
a lot of work on the Freecell Solver
trunk, including the long-planned RCS-like state storage of keeping only
the moves and the previous state in the states collection. Right now, what
I'm struggling with is constructing a good meta-scan that will yield short
solutions, based on the new scans I found recently, and fixing a nasty crash
after enabling some recent run-time command line flags. I've quite neglected
working on it recently due to being distracted with other things.
libfilefind
libfilefind, the C port of the File-Find-Object CPAN module is now passing
an initial test. It's much faster than the FFO Perl-based program, but still
slower than an equivalent Perl program written using File::Find. So I now need
to profile and optimise it further.
Perl-Begin
I did a lot
of work on the
Perl Beginners' Site. Today I completed adding all the items in the
TODO list that I wanted to incoroporate into the
"Perl
elements to avoid page", so now I have time to blog instead. :-)
Project Euler
I made a lot of progress with solving
Project Euler challenges, and in the
process discovered a bug in the "primes" program from the bsd-games package.
Reading from a bug report I sent to its maintainer and the Debian maintainer:
I discovered a bug in bsd-games' primes:
after downloading
http://ftp.de.debian.org/debian/pool/main/b/bsdgames/bsdgames_2.17.orig.tar.gz
and doing yes "" | ./configure (the interactive configure script
is very annoying, BTW) and "make primes/primes" on my x86-64 laptop
running Mandriva Cooker I'm getting the following:
{{{
[shlomif@lap primes]$ ./primes 7757777777 7757777778
7757777777
}}}
However, 7,757,777,777 is not prime:
{{{
$ factor 7757777777
7757777777: 65827 117851
}}}
This tripped me on this Project Euler challenge:
http://projecteuler.net/index.php?section=problems&id=111
KDE's Konsole
Konsole is KDE's console window. I recenty reported
a long time bug
in Konsole and after I was done, I triaged some bugs. I discovered
a crash bug when
triaging a different crash, and also tried to reproduce many other crashes.
perlipc
I sent a series of patches to
perlipc - the
Perl Interprocess Communication document in an attempt to modernise it.
This started a dicussion on perl5-porters, and eventually Tom Christiansen
posted an
"authorial perlipc edit", which was applied immediately. Some of my changes
were not incorporated there and now I'll need to start over, but I guess that's
life.
Kakurasu Solver
I've written a solver for
Kakurasu
using lp_solve and its Python
bindings. I
asked
for a code review on the Python-Israel mailing list, and got many comments,
which I've incorporated, and I think it's now ready for release.
CPAN Smoking
I've worked on setting a CPAN smoking environment on my Acer x86-64
laptop. I followed the instructions on
the CPAN Testers Wiki and
it went pretty straightforwardly. Then I started the smoking, but noticed
that Perl/Tk got stuck for a long time on its JP.t test. I'd like to find a
way to avoid it.
In the process, I tested the installation of my HTML-Widgets-NavMenu and noticed
it was still using Class-Accessor, so I converted it to
Class-XSAccessor,
which should make it faster.
I also received and applied a patch for
Config-IniFiles,
which I've incorporated into its 2.59 release. I had to quickly release a
2.60 release because the patch caused one of the test files to fail if
IO::Scalar was not installed.
2010-11-20: Tech Tips: Fixing KDE and GNOME problems
These are a few assorted tech tips in my tech tips column. The first one is
about how to make sure KDE's
Konqueror in file manager (or
Doplhin for that matter) will remember the state of its "Detailed" vs.
"Icons" view for particular folders. While there are many variables for
this equation, my problem in a particular place, was that I tried to do
it on a Samba SMB/CIFS share (a Windows-like
share) that was mounted with permissions of 755 and under my UNIX
user-ID/group-ID (but still read-only) which fooled konqueror and dolphin to
believe it could write its settings there. Reconfiguring samba solved the
problem, though it seems like I found two bugs - one in Samba and the
other in KDE.
The second issue also required a lot of experimentation on my part to
solve. My problem was that some X.Org applications such as KDE's Konsole, XFCE's
Terminal and the IRC client XChat had
larger fonts on XFCE and IceWM, than on GNOME and KDE. After a lot of trial
and error with stracing the sessions, and moving away dot-directories and
dot-files, I realised the GNOME problem was caused by a stray problem
in ~/.gconf/ which moving that directory away solved it (one can solve it
more hermetically by getting rid of the offending configuration problem, but
I did not bother with it yet). The KDE problem persisted afterwards, but
was easier to fix: I acked
~/.kde4/share/config for "dpi" and found share/config/kcmfonts
and a line in another file that set the dpi to a different value which I
fixed, and that made both the GNOME and KDE font size and DPIs (dots-per-inches)
to be the same as those of IceWM and XFCE. Now the relief has made me very
happy.
If you find these tips helpful, drop me a note at the comments below.
2010-11-28: Introducing Module-Format
This weekend I worked on
Module-Format,
a new CPAN distribution, and yesterday released version 0.0.1 of it to the
CPAN. In a nutshell, Module-Format allows one to convert strings representing
Perl modules (and their encapsulating distributions) to representation-netural
objects, and to render them into other formats of modules. To better
explain the motivation, let's start with a few use cases for this.
If you're working on Debian, Ubuntu or a derivative and would like to install
Perl distributions using apt-get, you can define the following shell function:
up()
{
apt-get -y install $(perlmf as_deb "$@")
}
And then inovke it like up XML::RSS XML-LibXML DBIx/Class.pm (note
the inconsistency - perlmf handles all those formats for you), and it
will install them.
Similarly in Mandriva using urpmi you can use:
up()
{
urpmi --auto $(perlmf as_rpm_colon "$@")
}
And then type the same command. (It will actually be better because urpmi
will fetch individual rpm provides modules such as perl(XML::RSS)
that may be contained in larger distributions).
Similarly to install stuff using cpan2dist, one can say something like:
function c()
{
cpan2dist --verbose --format=CPANPLUS::Dist::Mdv \
--install --timeout 0 $(perlmf as_colon "$@")
}
So this is what perlmf, which is the command-line program that ships with
Module-Format can do. Module-Format provides a more generic Perl-based
API for that, and replaces some ugly and ad-hoc shell functions that I defined
for that.
Module-Format is: 1. Free and open source software distributed under the
permissive MIT/X11 licence. and 2. Only dependent on core modules and
List::MoreUtils (a dependency which I'm planning to remove in a future
release). The source code is maintained in
the web-cpan Mercurial
repository, so feel free to contribute.
Happy module formatting.
2010-12-16: Tel Aviv Perl Mongers Meeting on 29-December-2010
The Tel Aviv Perl Mongers (part of
the Israeli Perl Mongers) will meet on
29-December-2010 at
Shenkar College,
Anna Frank 12, Ramat Gan, room 323. The gathering is going to be at 18:30 and
the talks will start at 19:00.
We are going to hear these talks:
Why Perl? (Sawyer X)
A short talk on what makes Perl such a compelling language to learn and work
with, while focusing on Modern Perl and innovative technologies.
What's new in Perl 5.10 and 5.12 (Gabor Szabo)
Find out what exciting features new versions of Perl offer.
Read only state for fun and profit (Yuval Kogman)
Even though Perl is highly optimised for modifying data in place, using read
only state can be very rewarding. What is immutability and when it should be
applied.
2010-12-17: Accentuate.us - a service for helping save human languages
I was requested to publicise this announcement.
Are You the Right Type?
As of today, about 7,000 languages are spoken in the world. More than 90% of
these are expected to disappear before year 2100, leaving a best case of only
700 languages. Language death is an irrevocable loss akin to animal or plant
extinction–it is the loss of a repository of culture, tradition, and world
view.
Nineteen-year-old entrepreneur Michael Schade, of Spearhead Development, has
teamed up with Dr. Kevin Scannell, of Saint Louis University, to revolutionize
computer input and help save endangered languages.
Due to inadequate keyboards, or clumsy input methods, speakers of hundreds of
languages often simply leave out the accents and other special characters when
typing, which leads to ambiguities and confusion. We believe strongly that
speakers of all languages have the right to type their languages quickly,
easily, and correctly.
We are proud to announce the free and open-source
Accentuate.us, a new method
of input for over 100 languages that uses statistical reasoning so that users
can type effortlessly in plain ASCII while ultimately producing accurate text.
This allows Vietnamese users, for example, to simply type “Moi nguoi deu co
quyen tu do ngon luan va bay to quan diem,” which will be automatically
corrected to “Mọi người đều có quyền tự do ngôn luận và bầy tỏ quan điểm” after
Accentuation.
To date, we support four clients: Mozilla Firefox, Perl, Python, and Vim, with
more to be added shortly.
Mozilla Firefox - Available today at http://accentuate.us/get/firefox through
Mozilla AMO, the Firefox add-on brings the power of the Accentuate.us service
to anywhere on the web: blogs, web mail, social networking, and anywhere else
where users input text. Please check out the short demonstration video at
http://accentuate.us/help.
Perl and Python - The Perl and Python clients available on
http://accentuate.us/
are command-line utilities that work either with argument flags or through
stdin. This versatility has proven powerful, allowing Accentuate.us to be
easily integrated into any program that can operate through such inputs, such
as vim.
Vim Provided by Bill Odom, co-founder of vim-geeks and previous President of
The Perl Foundation, and available today on http://accentuate.us/, the vim
plugin implements user-customizable maps to Accentuate over visual, motions,
and ranges. It harnesses the Perl or Python clients, exemplifying just how easy
it is to plug in to the Accentuate.us API.
Plans are currently underway for extending the Accentuate.us service, including
enhancements to the Firefox add-on and support for all 114 languages. Support
is needed especially for African languages, many of which use special
characters. There are, for example, over a million words of Lingála on the web:
blogs, news articles, Wikipedia entries, etc., but well over 95% of this
content is typed in plain ASCII, despite the fact that the standard spelling
system makes heavy use of diacritics to mark tone and special “open” vowels
such as ɛ and ɔ.
We appreciate any assistance that we receive to improve service and keep it
free, including translating the interface, hosting a server, or donating to
help us maintain existing servers. Find out how to help at
http://accentuate.us/contributing.
2010-12-26: Reminder: Tel Aviv Perl Mongers Meeting This Wednesday
This is a reminder that the Tel Aviv Perl Mongers will meet this Wednesday
(29-Decebmer-2010).
The Tel Aviv Perl Mongers (part of
the Israeli Perl Mongers) will meet on
29-December-2010 at
Shenkar College,
Anna Frank 12, Ramat Gan, room 323. The gathering is going to be at 18:30 and
the talks will start at 19:00.
We are going to hear these talks:
Why Perl? (Sawyer X)
A short talk on what makes Perl such a compelling language to learn and work
with, while focusing on Modern Perl and innovative technologies.
What's new in Perl 5.10 and 5.12 (Gabor Szabo)
Find out what exciting features new versions of Perl offer.
Read only state for fun and profit (Yuval Kogman)
Even though Perl is highly optimised for modifying data in place, using read
only state can be very rewarding. What is immutability and when it should be
applied.
2010-12-29: C Pointer Comparison Gotcha
Some days ago I integrated a new BSD-licensed balanced binary tree, based
on the red-black
tree of kazlib into Freecell Solver (fc-solve) and ported two
of its collections to it. I've written a comparator function for it and
asked my IRC pal Amadiro to test the new fc-solve with some memory
optimisations on one of his University's HPC machines which he had access to.
However, it crashed pretty quickly.
I was able to reproduce a similar crash on my x86-64 laptop, and tried to
investigate it. Then it hit me, because the comparator callback was this:
extern int fc_solve_compare_lru_cache_keys(
const void * void_a, const void * void_b, void * context
)
{
#define GET_PARAM(p) ((unsigned long) \
((const fcs_cache_key_info_t *)(p))->val_ptr)
return GET_PARAM(void_a) - GET_PARAM(void_b);
}
Do you see the problem? Subtracting two unsigned longs will always yield
a non-negative value, which will yield the comparison void. I considered
several alternatives to this including using longs instead of unsigned longs
(which will still be a problems on platforms where long is less than the
pointer size), or subtracting const char * (which may overflow
the int return) and eventually opted for
the safest version of:
typedef const char * fcs_lru_side_t;
extern int fc_solve_compare_lru_cache_keys(
const void * void_a, const void * void_b, void * context
)
{
#define GET_PARAM(p) ((fcs_lru_side_t) \
(((const fcs_cache_key_info_t *)(p))->val_ptr))
fcs_lru_side_t a, b;
a = GET_PARAM(void_a);
b = GET_PARAM(void_b);
return ((a > b) ? 1 : (a > b) ? (-1) : 0);
#undef GET_PARAM
}
This way I'm sure that the comparison works correctly, and indeed after that
Freecell Solver ran fine on the HPC machine, and would have appeared to
scale to up to 600 million positions within 64 GigaBytes instead of the previous
400 million positions.
If you're a sucker for pretty graphs, you may wish to inspect
the
OpenDocument spreadsheet (.ods) of the memory consumption of the solver,
though the data is quite estimative.
2011-01-02: Report on the Tel Aviv Perl Mongers Meeting
This is a report on the latest Tel Aviv
Perl Mongers meeting. During that day, I woke up relatively early (07:00
or so as far as I can recall), and did not go to bed before night time so I was
a little tired later on. Before the meeting, I spent some time trying to get
the game Aquaria, which
I got as part of the 2nd Humble Indie Bundle to work properly on my Linux
system, unsuccessfully. (A more recent update is that
someone
eventually helped me after I filed a bug report, and now I am able to
play the game nicely.). This delayed me a little, and I left relatively late
to the meeting.
After I arrived to Shenkar, I met Ido
and a friend of his, and guided them to the room, which was different than
the one where the Tel Aviv Open source used to meet. We met some people there,
including Sawyer and I decided to go to the nearest kiosk and buy some snacks.
I bought some chocolate chip cookies, Doritos and a chocolate and hazelnut
treat and returned with still a lot of time before the lectures started.
In the meantime, other people arrived there, and we talked about how to get
more people to contribute, about making use of CPAN modules in projects
and about which versions of Perl are used in the enterprise. Then the talks
began.
Sawyer X
gave an entertaining presentation titled "Why Perl?" with several jokes in
between that got the audience to burst into laughter. He mentioned
this page titled
"Your language sucks" where several of Perl 5's shortcomings are mentioned
(along with other languages), and concluded with bad things about Perl,
like the fact that it makes you lazy, and that you can't stand programming
in other languages. He also mentioned there several ways to start with Perl,
but I was disappointed that he did not mention
the Perl Beginners' Site (Perl-Begin).
After that, Gabor came and talked about
what's new in perl-5.10.x and perl-5.12.x. He covered most bases and then
when he got to perl-5.12.x's
while (my ($index, $value) = each(@array)) I asked
him if there was also keys(@array) and he said he doesn't know.
Today I tried it out and there is indeed keys(@array):
shlomif:~$ perl -E 'my @a=(50 .. 60); say join(",",keys(@a));'
0,1,2,3,4,5,6,7,8,9,10
There's also values(@array), but it's likely the same as "@array".
There are many small and large improvements in perl-5.10.x and 5.12.x and he
mentioned that the perl5-porters are now switching to a yearly release cycle,
which will make improvements materialise in production more rapidly.
After that, Ilan did a round of introductions where every person introduced
themselves and said which languages or technologies they work with and
why they came to the meeting. It was entertaining.
Then came Yuval's presentation
about "immutable state for Perl". He spent a short time explaining what is
mutable state, and that if we want to create a modified object, we need
to clone it with different constructor arguments. He spent the rest of
the time touting some of the advantages of immutable state, like easier
distributed NoSQL storage due to lack of cache invalidation, and he described
how he handled a problem in KiokuDB by creating a buffer for it, and also
focused on a recent problem we had at my (and previously his) work for
Reask where HTML::FormHandler misbehaved due
to modified state after reset, and we were unable to resolve it and had to
switch to a different form generator (HTML::FormFu).
Some people with whom I talked felt this presentation was a little bit
abstract and "floating in the air" and I tend to agree, but I still enjoyed
most of it.
I left shortly after that presentation because I was tired (as I had woken
up early that day) but based on
Sawyer's
post, some people went to dinner afterwards and had a good time. I walked
with mksoft to the bus station
and talked about games in Linux and the Humble Indie Bundle 2 which we both
had bought. Then a suitable bus had arrived and I took it home.
To sum up, I had a great time, about 20 people came, which meant the meeting
was a success, and I'm looking forward for the future meetings. Meeting at
Ramat Gan is much more convenient for me than the Rehovot meetings and I met
some people I didn't see in Rehovot like
Ran, who also gave me some money
in exchange for the refreshments I bought (Thanks!). I'm looking forward for
the next meetings and thanks to all the people who had organised and publicised
it.
2010-12-16: Tel Aviv Perl Mongers Meeting on 26-January-2011
The Tel Aviv Perl Mongers (part of
the Israeli Perl Mongers) will meet on
Wednesday, 26-January-2011 at
Shenkar College,
Anna Frank 12, Ramat Gan, room 323. The gathering is going to be at 18:30 and
the talks will start at 19:00.
This meeting will try to focus on web development and will (as any meeting)
include both beginner talks and intermediate ones.
We will also start a tradition of exhibition! Module exhibition, that is.
Pick a module you want to exhibit and take 5 minutes to talk about it.
If you want to exhibit a module, you can talk to me privately (if you need a
laptop, or funny hats) or you can just step up in between talks.
We are going to hear these talks:
What the hell is web development? (Sawyer X)
A nice, easy, hopefully interesting, understandable introduction to web
development.
The web stack (Ilan Arad)
A hands-on, discussion-oriented talk on advanced understanding of the web
stack, from server-side to client-side.
(advanced talk)
As always, there are a few good reasons to come to the Perl Mongers
meetings:
- Meeting more geek-minded people like yourself
- Learning new stuff, if you're a beginner
- Learning new stuff, if you're advanced
- Shlomi brings snacks!
- Laughs, jokes, fun, and anything else we can fake! :)
Please feel free (and do) invite anyone you want!
2011-01-29: Update and Recenty Hacktivity-ish Post
Well, I'm still alive and kicking, so I decided to put a small update about
what I have been up to.
First of all, I attended the FogBugz
and Kiln World Tour, which I did in order to see
Joel "on Software" Spolsky speak,
and was otherwise a blast. In the second part, they gave several good reasons
for why Mercurial and git are better than Subversion, and the demo of
FogBugz was also instructive, but seems like it contains many features that
won't be useful in my daily work as an open-source developer. They also said
that the reasons they prefered Mercurial over git were because it had a
somewhat friendlier command names (which are more familiar to people coming
from Subversion and CVS) and because it worked better on Windows, and the git
people were hostile towards fixing the bugs that git exhibited on Windows.
Otherwise, they said that git was a fine distributed version control system.
I also attended the latest Tel Aviv Perl
Mongers meeting, which was nice, and this time stayed for the after-party
dinner. I'd like to give a more detailed report about it later.
I'd like to announce that, in case you have not heard about it,
KDE 4.6.0 is out.. It
has many big and small improvements, including some speed boosts, and runs
like a dream on my Mandriva Linux Cooker system (though Amarok got broken
with the upgrade to MySQL-5.5.x). Here is also
the
OSNews.com coverage and
Slashdot.org
coverage). I should also note that I've recently downloaded and installed
the Amarok 2.x for Windows distribution and it ran very fine there, but
as opposed to Linux could not play the video files I tried at all.
After some length attempts to contact the originator of the
Text-Format CPAN module,
I was finally given a comaint status and I uploaded Text-Format-0.53 to
CPAN, its first release since 1998, and the first one that contains a dash
before the version digits (a small difference, but it confuses some packaging
systems). There's still some work to do there, but I haven't forgotten it.
Also in CPAN-land, I've uploaded a new version of
Config-IniFiles,
which contains a patch I applied from Peter Xu, who asked me to be his mentor
in his first steps in contributing to open-source. I've expressed my
willingness
to be a FOSS mentor for some time now and several people contacted
me about it, but this is the first time that a code contribution came out of
it, and I'm happy with Peter's patch. I guess good things come to those who
wait.
I've recently started reading
Kent Beck's "Test Driven Development: By Example", which I can recommend
so far, and also been working on some other open-source projects which I'd
rather not tell too much about now.
Hope you are also doing fine, wherever you are and happy hacking.