A new age for meta
Tech to gather, crunch and machine-learn the hell out of data has been racing ahead for the past decade, but one facet of the data infrastructure has been lagging, argues Prukalpa Sankar: our data about the data. It’s time to get our metadata to the level of the rest of our tools, with what she calls “Data Catalog 3.0”. The next stage of data cataloguing tools will be collaborative, will be able to catalogue more than just tables and columns, and offer an end-to-end description of an organisation’s data. (That’s quite a big promise right there.) Interestingly, the metadata catalogue will itself become a form of “big data”, and that offers exciting opportunities for the machine learning techniques that got us into this mess in the first place.