13 April 2015

Coding Super-Complexity

Young, intelligent developers often fail when they become accustomed to the following two things:

  • being 10x smarter than everyone around them; and
  • fitting all the code in their head 
When they join a team where everyone else is pretty smart - even if not quite as smart - they can still fail hard.

They are like younger, stronger kung-fu warriors that yet cannot best the wizened master!

The problem is that while they may be able to solve problems that are 10x more complex than other developers, this is only true when limited to a practice which requires one to remember and understand everything about the system under development. Note that this is typical experience in hobby projects and many university projects.

I think software engineering, definitively, focuses on solving problems which require software to be built that is generally much larger than anyone can fit in their head. 

In some ways intelligence can be a curse because it delays the lesson that systemic practices and superior tools (languages, architecture, frameworks) are necessary. Here I identify tools and techniques as superior primarily by their ability to massively magnify the effectiveness of our ability to deal with complexity. To tame it. To contain it.

Isolation, abstraction, automation - these things all help pump complexity out of the brain and into the software. It's true these systems can fail. Machines can fail and all software has bugs. But it's also true that we can often rely on them and that refusing to rely on them is a severe limitation and in some people who never learn this, even a personality defect!

22 October 2014

Open Source Needs Copyright

Copyright is not very well understood by many people. Open source likewise. Recently a friend of mine asked me to explain my statement that Open Source Software critically relies on copyright. I think there is a sense in the population outside of software development (and perhaps outside intellectual property law) that open source is an alternative or distinction to copyright.

In fact the opposite is true.

Open source software is protected by copyright. If it were not, then any company could co-opt it and make it their own. Depending on the license, “free software” which is a distinction frequently (and some would say tediously) insisted upon by open source leading lights like the inimitable Richard Stallman, this software may impose a restriction on the rules by which reproduction or distribution of the software may be done - including modified versions of the software.

This free software is freedom-enhanced or freedom-protecting. That's free as in speech, not free as in beer.

The famous GPL (GNU Public License) is like this - some say it is “viral”. If you or any company makes a product which is the result of modifying open source code which is licensed under the GPL, then the modifications you have made must not be published without the same affordances, indeed not without the same exact license. Notably this requires you to provide access to the the full source code without preventing anyone from further adding or modifying it.

Copyright law enables this because without such protections, software licenses would not be necessary. Failure to follow the terms of the license make one’s distribution of software a violation of copyright. Many companies have been busted because they have not followed the licenses and these days technology companies are familiar with meeting their obligations on this matter.

You may notice that most consumer electronic devices - computers, TVs etc. contain code (e.g. Linux), the license of which requires the manufacturer to tell you that you have the right to obtain the source code and that the manufacturer cannot deny you this right - nor can you deny anyone else the same rights should you decide to take that Samsung TV firmware and make a better version! Believe it or not, people do this sort of thing. You might get a bundle of open source licenses along with your equally impenetrable operation manual.

There are other licenses which are extremely permissive, such as the MIT license or the BSD license which do not place these viral restrictions on distributions. Some people believe this is more convenient, especially for software that is ideally suited to being incorporated in proprietary products as one of many components. However, just like GPL, whatever provisions the licenses do make - if these are the only terms under which the software is provided then they stick because copyright is an exclusive right to distribute unless it is explicitly assigned.
Lastly, copyright assignment is completely independent of the terms of a license. Typically copyright is held like title deeds and not given to users of the software. Exceptions are if the work is undertaken as "work for hire" where someone pays someone else to develop that software - typically copyright is assigned in exchange for the money - and another common case of copyright assignment is when an individual developer submits a modification to be included in an official open source project. Such open source projects (like Apache) have a foundation that exists to hold the copyright so that the stewardship of the software can be maintained without a messy consultation with hundreds of dispersed pseudonymous bedroom hackers who happened to have made a small contribution.

28 October 2013

Show Me Numbers! Automated Browser Performance Testing

Here's my talk from AtlasCamp 2013, Atlassian's annual plugin developer conference held in Amsterdam.

My topic is Automated Browser Performance Testing.

JavaScript developers often talk about performance, always tweaking to make the user experience faster. The good developers carefully and accurately measure this performance in multiple browsers. The great developers automate this measurement.

In this talk I tell the full-stack story of the design and implementation of User Experience (UX) performance tests in JIRA. Learn techniques critical to improve not only actual performance, but also perceived performance - and when they are not the same thing!

23 April 2013

Nice Syntax for Ignoring Exceptions in Groovy

Save your breath. I don't need to hear how we shouldn't ignore exceptions. I agree with you. We should use exceptions as a part of an API and we should accept exception handling as an important, fundamental design element in our code. Happy?

OK now, sometimes you know you need to ignore an exception. Often it's a narrow exception type, for example, the following exceptions are frequently ignored with good reason (usually the exception cannot happen in practice):


Groovy inherits Java's exception handling syntax, the try, catch, finally blocks. While writing some code that ignored the exceptions I wondered what a Groovier way to do things would be. The following Config class is a kind of bean containing validated values read from a properties file. The defaults remain if the config file cannot be parsed for the given value.

Notice how the try {...} catch {...} looks like Groovy's closure passing syntax, as if there was a try method which is being passed a block. In fact I think Scala's equivalent is literally this (also, Scala is awesome). 

But while it looks that way, it doesn't quite act that way. 

But could it?

What I really wanted to have was something like this:

ignore (NumberFormatException) {
    threads = Integer.parseInt(config?.threads as String)

This would ignore only a NumberFormatException if it was thrown, if something else was thrown, it would be thrown out of the present block.

As I expected this is pretty easy to implement in Groovy and after a brief search I couldn't really find anything in the standard libraries or the blogosphere at large.

Here's my implemenation with some example code:

Run this on the live GroovyConsole yourself.

What do you think? 

31 January 2013

Activate Remote Disc with Command Line on MAC OSX

On Macs without a DVD drive (i.e. everything these days) OSX is configured to easily allow use of a "Remote Disc" wherever another machine has enabled DVD/CD sharing.

The trouble is the "Remote Disc" device was not showing up on my old Mac Mini's (broken DVD drive) and although I can usually avoid it I recently wanted to install something from a DVD. This machine has a DVD drive which is presumably why the option did not show up in the Finder preferences under Sidebar. 

Fortunately these preferences can be turned on with a command line as follows:

defaults write com.apple.NetworkBrowser ODSSupported -bool true
defaults write com.apple.NetworkBrowser EnableODiskBrowsing -bool true

Enter that in the terminal and then Restart Finder using the Force Quit menu so the settings take effect.

Remote Disc icon now shows up in the Finder Sidebar under Devices

Then you should see the remote disc icon and can mount remotely shared DVDs and CDs. Note that you must also enable DVD/CD sharing on the machine with the disc.

19 October 2011

New Stuxnet Virus Beta Version Discovered

Stuxnet, the most advanced computer virus ever discovered (by a long way), generally believed to be the first sophisticated example of a virus being used as an international military weapon, just got an upgrade. Most likely from its original authors.

At least, according to Symantec's analysis lab where a recently found version has been dissected on the operating table, it looks like the versions found in the wild "in Europe" were targeted test runs of a new version that was partly recompiled from the original source code which has so far remained secret.

The new version carries a different payload to the original Stuxnet which, it is generally believed, was intended to shut down or disrupt Iranian nuclear facilities. In fact the new version's actual payload is pluggable. Modules can be loaded dynamically from a command and control server located in India.

Duqu, as the new version is called, can read and write files, execute programs, capture keyboard input (to collect passwords) and generally open back doors for later use by the unknown attackers.

Check out the great article on this by Read Write Web and for a quick recap on the original Stuxnet, check this nicely produced - if creepy - video by Hungry Beast in Australia:

09 October 2011

Why Programming is Hard

Programming is not hard because computers are fussy. Programming is hard, but not for that reason.

There are plenty of pesky surface details in managing programming projects and programmers regularly find themselves burdened with managing tools and configuration details. More so when using Maven (Zing!).

But this is just work. Annoying repetitive work. It's not what makes programming hard.

In fact, programming is hard because there are so few limits to what you can do.

Certainly there are limits to computability, and there are famous problems and algorithms that define the state of the art in computer science, but for many programmers, knowing about these limits is enough to prevent them from pushing up against the hardness of these limits on a regular basis.

Programming is hard because the only limit is our own mental capability. It's always going to appear to be hard. It doesn't matter how smart you are.

If programming is not hard, then you're doing it wrong. Your brain is not being fully utilised? You are being limited by your tools and you need to replace them.

Any time spent dealing with repetitive details, remembering lots of things or even managing huge amounts of code, this time waste generally can be eliminated with more programming! This assumes you're in a postion to change the system enough and perhaps this is where practical project tradeoffs impose. And perhaps "the business" isn't interested in funding improvements to the open source build tool, or switching from Java to Scala. (Double Zing!) But those are not programming problems.

For me, because software is so soft, the goals can be altered subtly, continuously, destructively. Especially when ill-defined. The hardest thing generally comes down to dealing with the confusion of some form of not being clear what I'm trying to do. Unravelling the "is it this or is it that" trail of mental plans that led to the curent state. When this happens in the large, a software project is almost certain to fail.

But maybe that's just me. Do you think programming is not hard? Do you think it's hard for a different reason?

30 September 2011

32 Bit 64 Bit - All up in your Bitness

Here's one for the logophiles. English usage nerds. I'm coming out as a bit of a language nerd officially, today, so consider yourself warned.

In film they talk of footage. In transport, haulage. Then there are words like passage, patronage and of course cleavage. At school my friends and I would regularly put "-age" after words. "Foodage?" would mean "are you hungry?". "Skateage" was time spent skateboarding.

So what is it when you want to refer to how many bits something is?

You know your latest laptop is 64 bits and your last one was 32 bits. You know about the 16 bit retro video games like Sonic the Hedgehog and maybe even the "8 bit era" of the Commodore 64. So what do call that ... bitness?

I've heard it called bitness, and tweeted about it a while ago and got some interesting alternatives:

  • bitness
  • bittage
  • bittitude
  • bitality
  • bitosity
  • bitch
I guess the last one is a little challenging, linguistically. Maybe German?

Of course I had lots of replies to my tweet saying well you're talking about "word length" or "machine architecture" or other term specific to the usage. Well what if I'm talking about sound samples? "Is that a 16 bit sample or a 32 bit one?". Or colour space in digital imagery? Is that 24 bit colour? Ignoring bittitude is just like saying you can't call it footage, you have to call it film length.

We need to agree on a word that covers bitness and I think I'm coming down on the side of bittage.

Of course this is a question of crucial importance to the world (not - but I did warn you).

What do you think?

image credit jeffrey onMr Breakfast

24 August 2011

Last Minute Usability

Usability, like scalability and security cannot be added to a finished product any more than flavour can be added to a finished meal.

Delightfully Delicious
delicious photo by Shutter Ferret

19 April 2011

This is NOT the 80:20 Rule You Are Looking For

The 80:20 rule is based on the assumption that there is (often) an 80% (ish) subset of the realised value for a project that only costs 20% of the resources (e.g time) to complete. The rule, actually the Pareto Principle popularised by Richard Koch and others, like all rules of thumb, can be applied incorrectly.

It's not a license for doing things half-arsed.