Wolfram Alpha is amazing, but it has a fundamental flaw.

Wolfram Alpha is amazing. Go play with it now.

Some people really, really don't get it. They are frustrated that it isn't google, which of course it was never meant to be. If you're not sure what it is then I recommend the (overlong) intro video.

But basically, It's a data engine. A fact machine. It lets you pull and push and manipulate data.

For instance you could compare the mean wage of chiropractors vs journalists in seconds (topical!). Or do some maths. Or do lots of things.

It's some topical data about two professions.

And this is all the info your are given for your 'source'

It's finicky and has plenty of holes in it, but it's only just gone live and it is brilliant. I can't stress that enough.

Now on to why it sucks.

The cultural effects of a 'computational knowledge engine' that can both quantify and manipulate data is huge. But the ability for it to be misused is also huge. We all love wikipeida, but when journalists use it for their primary source of info it gets a little worrying. To offset that Wikipedia has a big list of references at the bottom of every page.

Rather than establish itself as an authority figure (despite the wishes of some of the more 'enthusiastic' admins) Wikipedia works hard to tell you how it knows something to be true, otherwise [citation needed] rears its head.

I don't get that same feeling from Wolfram Alpha. Firstly, the references are tucked away from view in a little pop up window rather than being an inherent part of the page. Secondly, the Primary source for everything I've tried so far is:

Wolfram|Alpha curated data, 2009

Which just isn't good enough. It is setting Alpha up to be an authority figure for data. But this isn't information that Wolfram et al. have discovered and published. This is information that Wolfram has collated.

Now, to be fair, 'background' sources are listed. And whilst they are most likely the true source of the data presented, no single study or report or finding is indicated to be where any specific fact came from. For example Alpha has decided that the UK produces 1.8 billion barrels of oil a day. Has it taken an average of the studies it looked at? Or has it weighted them? It doesn't tell me.

Every piece of data when fed into Alpha should have been tagged with details of where it came from. But it isn't. It's a mind boggling strange omission.

So much so that I can't help but feel this, may possibly, add weight to widespread opinion that Stephen Wolfram is a little bit, erm, arrogant. This letter that Richard Feynman (apparently) wrote to Stephen Wolfram is rather illuminating on the subject. The cynic in me whispers that Alpha wants to be your reference, not your intermediary.

Wolfram|Alpha is an amazing achievement, and one I will use an awful lot. (even if it is just to plot nonsense) Stephen Wolfram should be immensely proud of it.

But I can't help thinking how much more important Alpha would be as a proper research tool if it was a little bit more willing to explain where it learnt to be so clever.


Anonymous said...

Where you've written "admission" (4th paragraph from bottom), should that read "omission"?

Bill said...

Cheers! Fixed the typo.

I'm sure there are plenty more.

Upa said...
This comment has been removed by the author.
Upa said...

Lacking the references is of course a flaw.

Another one is that WolframAlpha will always have out-of-date information by design. It collects data from some external sources and tries to turn these data into something more complicated that just inverted index (as in Google case). This takes time and needs always be adjusted to changing formats of sources. In two words - will never work real-time.

HJ said...

That's 5 words.

Lave said...

I don't think anyone cares about it not working real time. Collating data in a way that allows it to be manipulated takes time, but allows you to do far more with it than you can otherwise.

Jhon said...

This takes time and needs always be adjusted to changing formats of sources.

You cannot go wrong on the best security systems