22 October 2014

Open Source Needs Copyright

Copyright is not very well understood by many people. Open source likewise. Recently a friend of mine asked me to explain my statement that Open Source Software critically relies on copyright. I think there is a sense in the population outside of software development (and perhaps outside intellectual property law) that open source is an alternative or distinction to copyright.

In fact the opposite is true.

Open source software is protected by copyright. If it were not, then any company could co-opt it and make it their own. Depending on the license, “free software” which is a distinction frequently (and some would say tediously) insisted upon by open source leading lights like the inimitable Richard Stallman, this software may impose a restriction on the rules by which reproduction or distribution of the software may be done - including modified versions of the software.

This free software is freedom-enhanced or freedom-protecting. That's free as in speech, not free as in beer.

The famous GPL (GNU Public License) is like this - some say it is “viral”. If you or any company makes a product which is the result of modifying open source code which is licensed under the GPL, then the modifications you have made must not be published without the same affordances, indeed not without the same exact license. Notably this requires you to provide access to the the full source code without preventing anyone from further adding or modifying it.

Copyright law enables this because without such protections, software licenses would not be necessary. Failure to follow the terms of the license make one’s distribution of software a violation of copyright. Many companies have been busted because they have not followed the licenses and these days technology companies are familiar with meeting their obligations on this matter.

You may notice that most consumer electronic devices - computers, TVs etc. contain code (e.g. Linux), the license of which requires the manufacturer to tell you that you have the right to obtain the source code and that the manufacturer cannot deny you this right - nor can you deny anyone else the same rights should you decide to take that Samsung TV firmware and make a better version! Believe it or not, people do this sort of thing. You might get a bundle of open source licenses along with your equally impenetrable operation manual.

There are other licenses which are extremely permissive, such as the MIT license or the BSD license which do not place these viral restrictions on distributions. Some people believe this is more convenient, especially for software that is ideally suited to being incorporated in proprietary products as one of many components. However, just like GPL, whatever provisions the licenses do make - if these are the only terms under which the software is provided then they stick because copyright is an exclusive right to distribute unless it is explicitly assigned.
Lastly, copyright assignment is completely independent of the terms of a license. Typically copyright is held like title deeds and not given to users of the software. Exceptions are if the work is undertaken as "work for hire" where someone pays someone else to develop that software - typically copyright is assigned in exchange for the money - and another common case of copyright assignment is when an individual developer submits a modification to be included in an official open source project. Such open source projects (like Apache) have a foundation that exists to hold the copyright so that the stewardship of the software can be maintained without a messy consultation with hundreds of dispersed pseudonymous bedroom hackers who happened to have made a small contribution.

28 October 2013

Show Me Numbers! Automated Browser Performance Testing

Here's my talk from AtlasCamp 2013, Atlassian's annual plugin developer conference held in Amsterdam.

My topic is Automated Browser Performance Testing.

JavaScript developers often talk about performance, always tweaking to make the user experience faster. The good developers carefully and accurately measure this performance in multiple browsers. The great developers automate this measurement.

In this talk I tell the full-stack story of the design and implementation of User Experience (UX) performance tests in JIRA. Learn techniques critical to improve not only actual performance, but also perceived performance - and when they are not the same thing!

23 April 2013

Nice Syntax for Ignoring Exceptions in Groovy

Save your breath. I don't need to hear how we shouldn't ignore exceptions. I agree with you. We should use exceptions as a part of an API and we should accept exception handling as an important, fundamental design element in our code. Happy?

OK now, sometimes you know you need to ignore an exception. Often it's a narrow exception type, for example, the following exceptions are frequently ignored with good reason (usually the exception cannot happen in practice):


Groovy inherits Java's exception handling syntax, the try, catch, finally blocks. While writing some code that ignored the exceptions I wondered what a Groovier way to do things would be. The following Config class is a kind of bean containing validated values read from a properties file. The defaults remain if the config file cannot be parsed for the given value.

Notice how the try {...} catch {...} looks like Groovy's closure passing syntax, as if there was a try method which is being passed a block. In fact I think Scala's equivalent is literally this (also, Scala is awesome). 

But while it looks that way, it doesn't quite act that way. 

But could it?

What I really wanted to have was something like this:

ignore (NumberFormatException) {
    threads = Integer.parseInt(config?.threads as String)

This would ignore only a NumberFormatException if it was thrown, if something else was thrown, it would be thrown out of the present block.

As I expected this is pretty easy to implement in Groovy and after a brief search I couldn't really find anything in the standard libraries or the blogosphere at large.

Here's my implemenation with some example code:

Run this on the live GroovyConsole yourself.

What do you think? 

31 January 2013

Activate Remote Disc with Command Line on MAC OSX

On Macs without a DVD drive (i.e. everything these days) OSX is configured to easily allow use of a "Remote Disc" wherever another machine has enabled DVD/CD sharing.

The trouble is the "Remote Disc" device was not showing up on my old Mac Mini's (broken DVD drive) and although I can usually avoid it I recently wanted to install something from a DVD. This machine has a DVD drive which is presumably why the option did not show up in the Finder preferences under Sidebar. 

Fortunately these preferences can be turned on with a command line as follows:

defaults write com.apple.NetworkBrowser ODSSupported -bool true
defaults write com.apple.NetworkBrowser EnableODiskBrowsing -bool true

Enter that in the terminal and then Restart Finder using the Force Quit menu so the settings take effect.

Remote Disc icon now shows up in the Finder Sidebar under Devices

Then you should see the remote disc icon and can mount remotely shared DVDs and CDs. Note that you must also enable DVD/CD sharing on the machine with the disc.

19 October 2011

New Stuxnet Virus Beta Version Discovered

Stuxnet, the most advanced computer virus ever discovered (by a long way), generally believed to be the first sophisticated example of a virus being used as an international military weapon, just got an upgrade. Most likely from its original authors.

At least, according to Symantec's analysis lab where a recently found version has been dissected on the operating table, it looks like the versions found in the wild "in Europe" were targeted test runs of a new version that was partly recompiled from the original source code which has so far remained secret.

The new version carries a different payload to the original Stuxnet which, it is generally believed, was intended to shut down or disrupt Iranian nuclear facilities. In fact the new version's actual payload is pluggable. Modules can be loaded dynamically from a command and control server located in India.

Duqu, as the new version is called, can read and write files, execute programs, capture keyboard input (to collect passwords) and generally open back doors for later use by the unknown attackers.

Check out the great article on this by Read Write Web and for a quick recap on the original Stuxnet, check this nicely produced - if creepy - video by Hungry Beast in Australia:

09 October 2011

Why Programming is Hard

Programming is not hard because computers are fussy. Programming is hard, but not for that reason.

There are plenty of pesky surface details in managing programming projects and programmers regularly find themselves burdened with managing tools and configuration details. More so when using Maven (Zing!).

But this is just work. Annoying repetitive work. It's not what makes programming hard.

In fact, programming is hard because there are so few limits to what you can do.

Certainly there are limits to computability, and there are famous problems and algorithms that define the state of the art in computer science, but for many programmers, knowing about these limits is enough to prevent them from pushing up against the hardness of these limits on a regular basis.

Programming is hard because the only limit is our own mental capability. It's always going to appear to be hard. It doesn't matter how smart you are.

If programming is not hard, then you're doing it wrong. Your brain is not being fully utilised? You are being limited by your tools and you need to replace them.

Any time spent dealing with repetitive details, remembering lots of things or even managing huge amounts of code, this time waste generally can be eliminated with more programming! This assumes you're in a postion to change the system enough and perhaps this is where practical project tradeoffs impose. And perhaps "the business" isn't interested in funding improvements to the open source build tool, or switching from Java to Scala. (Double Zing!) But those are not programming problems.

For me, because software is so soft, the goals can be altered subtly, continuously, destructively. Especially when ill-defined. The hardest thing generally comes down to dealing with the confusion of some form of not being clear what I'm trying to do. Unravelling the "is it this or is it that" trail of mental plans that led to the curent state. When this happens in the large, a software project is almost certain to fail.

But maybe that's just me. Do you think programming is not hard? Do you think it's hard for a different reason?

30 September 2011

32 Bit 64 Bit - All up in your Bitness

Here's one for the logophiles. English usage nerds. I'm coming out as a bit of a language nerd officially, today, so consider yourself warned.

In film they talk of footage. In transport, haulage. Then there are words like passage, patronage and of course cleavage. At school my friends and I would regularly put "-age" after words. "Foodage?" would mean "are you hungry?". "Skateage" was time spent skateboarding.

So what is it when you want to refer to how many bits something is?

You know your latest laptop is 64 bits and your last one was 32 bits. You know about the 16 bit retro video games like Sonic the Hedgehog and maybe even the "8 bit era" of the Commodore 64. So what do call that ... bitness?

I've heard it called bitness, and tweeted about it a while ago and got some interesting alternatives:

  • bitness
  • bittage
  • bittitude
  • bitality
  • bitosity
  • bitch
I guess the last one is a little challenging, linguistically. Maybe German?

Of course I had lots of replies to my tweet saying well you're talking about "word length" or "machine architecture" or other term specific to the usage. Well what if I'm talking about sound samples? "Is that a 16 bit sample or a 32 bit one?". Or colour space in digital imagery? Is that 24 bit colour? Ignoring bittitude is just like saying you can't call it footage, you have to call it film length.

We need to agree on a word that covers bitness and I think I'm coming down on the side of bittage.

Of course this is a question of crucial importance to the world (not - but I did warn you).

What do you think?

image credit jeffrey onMr Breakfast

24 August 2011

Last Minute Usability

Usability, like scalability and security cannot be added to a finished product any more than flavour can be added to a finished meal.

Delightfully Delicious
delicious photo by Shutter Ferret

19 April 2011

This is NOT the 80:20 Rule You Are Looking For

The 80:20 rule is based on the assumption that there is (often) an 80% (ish) subset of the realised value for a project that only costs 20% of the resources (e.g time) to complete. The rule, actually the Pareto Principle popularised by Richard Koch and others, like all rules of thumb, can be applied incorrectly.

It's not a license for doing things half-arsed.

06 April 2011

Automated Testing in Games

So I clearly upset a few game developers with my last blog post. Sorry about that! I do find it a bit surprising how strong some of the emotions were but I probably deserved the beating I got on reddit.

Second my post was too long and based on my stats, it looks like less that 1.0% of visitors read the article. Unless they can read a couple of thousand words in 20 seconds. Again, that was my fault.

But what was interesting was the responses to my article were pretty clearly divided into two main groups (disagreeing and flaming) and one small group (agreeing):

  1. You are on crack, we do this stuff (automated testing, etc) better than anyone.
  2. Game developers are moar awesome than normal developers and unit testing, agile and whatever else you are pushing doesn't apply to the game industry because of [one of]:
    1. How can you unit test games anyway that doesn't even make sense!
    2. Deadlines. We don't have time.
    3. Games have hundreds of gigs of assets.
    4. Console contracts / monetisation models etc.
  3. I agree that the game industry suffers from some poor tool and process adoption habits. 
Those who use these practices I've been talking about - and to keep things specific I think automated testing is an excellent candidate to focus the debate - these people generally didn't seem too upset with me. In fact I know plenty of game developers in this category. 

Those who believe that their game studios are backward generally cannot speak publicly about ANYTHING with the only few exceptions being people who have left the industry, like munificent who says:

I worked at EA for eight years. My running joke-that-was-not-a-joke was that we were twenty years behind the times, but I was probably exaggerating. Fifteen is a pretty good estimate.
At EA, I think the problem was that the company culture was built like this:
  1. A bunch of bedroom hackers in their teens started studios and shipped games ten to twenty years ago.
  2. Those games got successful, and the studios scaled up. The founders hired young people under them.
  3. More staff churn. Fresh grads come in, burned out experienced coders go out.

Munificent went on to qualify and contain his own statements as generalisations (as I did with mine).

Tool and process adoption in development teams is of particular interest to me and I have an ongoing conversation with several developers about the barriers to solving problems they have in this area.

As for the middle group, the question boils down to this:

If automated testing is to some degree not suited to the game industry, then why?

If the answers are things like "How could you test Starcraft 1?" or "You can't test for things like game balance" then I'm sorry that's not a convincing reason! How is it obviously impossible to automatically test Starcraft? I'm sure game developers heavily using automated testing (perhaps even Blizzard) will call bullshit and say "yes you can automatically test it".

You Can't Write a Test for X

I've seen so many of these debates and had them myself since the 90s and I've come to believe slowly by being convinced on a case by case basis that you can test the vast majority of what you think you can't test. I've learned this working in a number of different software projects in different industries.

If the answer is the cost benefit trade-offs on this project lead us to conclude it's not worth doing here and there then that's interesting. I'm looking for examples because it may well be a tooling problem.

I'm not saying that everything can be tested automatically, and perhaps the up-front cost of writing the tests is never outweighed by the benefits of having the tests because of game industry specific business features or whatever, but I think many of those things are currently subject to rapid change.

Game Revenue

WOW is a good case study. This is the sort of game that couldn't be funded 15 years ago. It plays similarly to the MUDs of the early 90s with a lot better graphics etc. but the market was too small for such a grand vision then. We were on 14.4k modems (or worse) and the number of internet users was tiny. WOW has recurring revenue. This is a feature unlike traditional games but not unlike traditional and modern non-game software.

With ongoing revenue you can fund ongoing development in a more continuous way. Updates can be done more frequently and codebases are encouraged to have longer lifespans. Longer lifespan codebases mean legacy code. There's nothing to encourage you to invest more in automated testing like having to maintain legacy code!

I think these are important factors, driven by the modern consumer internet and the market forces that come from it. These forces push game code towards a scenario where automated testing has a bigger benefit.

That doesn't address the cost side. You might think that the costs of automated testing, particularly unit testing, are fixed. Well in my experience of unit testing for the past 12+ years, the biggest barrier to automated testing is bad design. Here's what munificent (who has since moved on to Google) had to say about that:

Unit testing would be awesome, but the thing is... you need units for that. A lot of the game codebases I've seen are closer to "giant hairball" than "collection of components", which isn't amenable to unit testing.

When I was writing software in the 90s and 80s I never wrote automated tests. I thought testing had to be manual and had to be done when the system was "finished". I don't think that any more. When someone tells me that they have to test that way I ask why and I think I'm right to be suspicious of weak, vague answers.