Yes, I agree completely. And I think you’re misunderstanding my point. I’m not saying the tech can’t do it, we all know that it can. The issue here is that most of the large organisations which need to be managing their data don’t do it, either because they do not have the tech, or are not prepared to invest in the processes and procedures to get their data into a state where tech can manage it, or because they do not even recognise that there is a problem.

(Report abuse)

Do you and I use the same technology??? The revolution has been in DATA! the storage, management and mining of DATA.

I only become aware of the strides as I look back over 35 years of playing with computers. This last week, I’ve been playing with a microcontroller. I’ve added a 256Kb serial Eeprom. 256Kbit or 32KByte is the same amount of memory as the huge IBM 370 I first used in 1972. The IBM took up 2 floors while my PCB fits easily into the palm of my hand. Other than the reduction in scale, there is considerable similarity between these two. One addresses memory locations using a HEX address. You have to construct your own FAT (File Allocation Table) and there is no easy way to see what you’ve stored. None of this applies to even the smallest modern computer.

With computers today, data mining is common and intelligent software writers allow for variations in spelling. This means one can find data, even if it isn’t an exact match.

Try this facinating internet mining tool:
http://cloud.clusty.com/

For more than 25 years programmers have been using field rules in databases to ensure the congruence of data. Modern software is particularly strong here.

Heuristic programming techniques continue to get stronger and more flexible. This means software often finds things that I assume are permanently lost in the sea of data.

Yes – lots of data is disgustingly inaccurate and flawed (a human fault). Programmers have developed software to handle this

Yes – data often doesn’t match across systems. Software can recognise similarities in the data to perform a best fit (usually a much better fit than humans can do).

Compatibility??? You modern IT people don’t know what that word means! Anything can be converted these days. I often think “incompatability” is an IT excuse for doing nothing. When I hear this, I ask “have you converted it to a delimited ASCII file” and I am sometimes amazed that the IT people don’t know what the heck this is. The more recent technologies such as XML and SPC make data compatibility much easier.

In my opinion, the IT problem is BCAK. It isn’t in the software or the data but what sits between the chair and the keyboard, us humans. Peter Drucker, the great management guru of the 1970’s used to say it takes 20 years for the population to grasp and adopt new technology and, to be honest, data processing has been moving much faster than that.

I believe strongly that one should avoid blaming ones tools when the real problem is just a lack of ones own skill or ability.

(Report abuse)

I’m busy shopping for a new bank, if you can recommend yours that would be lovely.

(Report abuse)

BTW - we can swap banks - mine knows all about my various accounts in one call to one person…

(Report abuse)