Use data and human intelligence in the right combination..

I just listened to Sebastian Wernicke’s Ted Talk: “How to use data to make a hit TV show”. The phrase Sebastian used which was “the data alone will not give you the future answer you need you use the intelligence from the brain as well”.

Often in discussions I have around the data savvy culture and implication of this evolution, people become fearful of their own role – thinking that these “data driven robots will replace everything they do and they will no longer be valuable”.  This is far from the reality. The human ability to ask the right questions, sense the weak signals and understand the implications are two unique skills that are seemingly impossible to replace at this stage.

What I took away from this video was how important it is for all us to develop and hone our question asking skills and start to learn how to best use the wide range to data analysis tools at our disposal. I would recommend reading “A more beautiful question” as this is the best book I found to develop your question asking skills. In addition, stopping yourself from doing the same thing in the same way is another approach I found valuable.

What are your thoughts on the balance between data driven and human intelligence/instinct driven?




Is there a set of Fundamental Data Types?

Is there a set of Fundamental Data Types?
Numerals and number systems have served us well as aids to quantify and represent the information around us, but today they are just one small part of the vast and complex digital shadow that is cast over our daily lives. Statistical methods help us explain how this shadow has evolved and can amazingly help predict its near-term future, its adjacent possibles. But why does that materially matter to us? So what?
Every piece of data that has ever been captured or created, had, has or will have a purpose, even if that purpose is not always immediately obvious. For instance it might help guide or trigger an action that will add value in the real world. That value might improve someone’s life, advance a product or process, or just turn a traffic light from red to green. The data associated with these acts is a catalyst to that value creation or is a characteristic of its outcome.
The term ‘catalyst’ is chosen to draw an analogy to chemistry and demonstrate how effective the idea of reacting data can be, when considered within the context of a fundamental data framework and is akin to the several branches of mathematics based on the same principle, using it to constrain how primitive number types work. A foundation of this framework, like with chemistry, has to be the elemental nature of data. That is its propensity to form in a fixed number of ways, or elemental types. These can also be thought of as exhibiting elemental properties. For instance, some data types must be highly volatile, while others must tend towards being inert, some must be heavy, while others lightweight and so on. Thinking like this not only simplifies the way that we perceive data, but also leads us to consider the ways in which these elemental data types might interact, or react, to make the world a better place. 
As for the fundamental data types themselves, we know of a few already, but there are many more out there to be labelled properly. Certainly we can add data associated with location and time to the list and we commonly refer to these as being geospatial and event data. But what about other types, such as personal, pricing, indexing data, and so on? Or perhaps more excitingly, the more exotic types of data, like metadata, analytical, aggregated and inferred data? The answers are nowhere near being clear yet, but certainly playing with data in this way is leading us to understand that our journey is only just beginning. For all we think we know about data, the real lessons are still to come. But wouldn’t it be fascinating if we had something like a periodic table of data, a touchstone we could turn to to help us experiment with data in safe, but as yet unimagined, ways…!
Co-authored with PhilTetlow: @DocPhilT