Wolfram Alpha is amazing, but it has a fundamental flaw.

Wolfram Alpha is amazing. Go play with it now.

Some people really, really don't get it. They are frustrated that it isn't google, which of course it was never meant to be. If you're not sure what it is then I recommend the (overlong) intro video.

But basically, It's a data engine. A fact machine. It lets you pull and push and manipulate data.

For instance you could compare the mean wage of chiropractors vs journalists in seconds (topical!). Or do some maths. Or do lots of things.

It's some topical data about two professions.

And this is all the info your are given for your 'source'

It's finicky and has plenty of holes in it, but it's only just gone live and it is brilliant. I can't stress that enough.

Now on to why it sucks.

The cultural effects of a 'computational knowledge engine' that can both quantify and manipulate data is huge. But the ability for it to be misused is also huge. We all love wikipeida, but when journalists use it for their primary source of info it gets a little worrying. To offset that Wikipedia has a big list of references at the bottom of every page.

Rather than establish itself as an authority figure (despite the wishes of some of the more 'enthusiastic' admins) Wikipedia works hard to tell you how it knows something to be true, otherwise [citation needed] rears its head.

I don't get that same feeling from Wolfram Alpha. Firstly, the references are tucked away from view in a little pop up window rather than being an inherent part of the page. Secondly, the Primary source for everything I've tried so far is:

Wolfram|Alpha curated data, 2009

Which just isn't good enough. It is setting Alpha up to be an authority figure for data. But this isn't information that Wolfram et al. have discovered and published. This is information that Wolfram has collated.

Now, to be fair, 'background' sources are listed. And whilst they are most likely the true source of the data presented, no single study or report or finding is indicated to be where any specific fact came from. For example Alpha has decided that the UK produces 1.8 billion barrels of oil a day. Has it taken an average of the studies it looked at? Or has it weighted them? It doesn't tell me.

Every piece of data when fed into Alpha should have been tagged with details of where it came from. But it isn't. It's a mind boggling strange omission.

So much so that I can't help but feel this, may possibly, add weight to widespread opinion that Stephen Wolfram is a little bit, erm, arrogant. This letter that Richard Feynman (apparently) wrote to Stephen Wolfram is rather illuminating on the subject. The cynic in me whispers that Alpha wants to be your reference, not your intermediary.

Wolfram|Alpha is an amazing achievement, and one I will use an awful lot. (even if it is just to plot nonsense) Stephen Wolfram should be immensely proud of it.

But I can't help thinking how much more important Alpha would be as a proper research tool if it was a little bit more willing to explain where it learnt to be so clever.