On Feb 29, 2012, at 6:05 PM, Rich Cooper wrote:
Moving data from one RDBMS to another is just not that hard. The problem is in understanding the data that people have actually typed into the database over the last thirty years. Different people put in different text descriptions all the time.
And what if you're working with non RDBMS systems? What about Model 204, System 2000, IDMS, Cincom, Adabas....? Out of fashion? Yes. Out of use? No.
IBM's IMS, a 40+ year old database is still generating $700M in revenue for IBM. I asked a friend if IBM was pressuring them to move to DB2. The answer was absolutely not. They were very content with what they had & in IBM's direction with IMS.
The applications are in place, they work within predictable boundaries. Who's going to pay to replace them?
I've had the pleasure of moving several PC databases. Ugly, slow process. Seems that the user—me—is, as you point out, inconsistent. I embed "knowledge" in the data fields. When I move from old to new, all sorts of stuff falls on the floor unless I'm paying close attention.
For applications in large organizations the challenge of "dirty data" is called "data profiling." You basically have to look at every single field, determine what the data value distributions are, uncover anomalies & on & on & on.
Apply rule#1 of management... "If it ain't broke, don't fix it."
Can ontologies, or just a slice of ontology wisdom be quickly applied to legacy systems to uncover necessary semantic understanding that walked out the door with the experts?