Technology Trend Series: Machine Learning

In the simplistic term, Machine Learning (ML) is all about making machine that can learn from surrounding and apply that learning in action. The more formal definition is from Tom Mitchell: “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.” Machine Learning is now has reached to the point where we actually don’t even realize its presence unless explicitly called out. Google, Amazon, IBM, Apple, Microsoft, Netflix, Uber, Tesla, etc. have adopted Machine Learning successfully. Google uses ML for spam filtering, image recognition, translation, self-driving car. Amazon uses ML for Recommendation system, Drone delivery, etc. Apple’s Siri, Microsoft’s Cortana and Amazon’s Alexa and Google’s Google Home use ML to provide Virtual Private Assistance (VPA). IBM’s Watson provides assistance to doctors to diagnose patients. Uber and Tesla created Machine Learning based autonomous vehicles. Machine Learning is used in the stock market for High Frequency Trading and in the postal service for handwriting recognition.

The Machine that Learn is essentially a computer algorithm to best model the input to its output. Carefully prepared large training dataset is fed into the algorithm to find out the optimal model. Machine Learning can be Supervised Learning, Unsupervised Learning, Reinforcement learning, Artificial Neural Network, Deep Learning, and so on. Let’s take an example of how a self-driving car uses machine learning: car is driven by a human driver and through various sensors: steering wheel, accelerator, brake, camera, GPS, radar, Laser, etc. it captures the data and the behavior of the human driver and used as training data set and find the optimal model to map the observed data from all the sensors to driver’s action.Machine Learning is progressing faster and superseding its own legacy. The drivers for continued massive growth and adoption of Machine Learning are the growing surge in data volume and complexities that conventional engineering approaches are increasingly unable to handle. For example, the amount of data created in the last 2 years exceeds the data created in the entire human history. So the traditional brute force computation is no longer a viable option to process this ocean of data. Meanwhile, as predicted in the “Moore’s Law”, the exponential growth of computing power is unleashing the power of Deep Learning. So, the businesses have to be fully prepared to utilize the most out of it to stay competitive in this disruptive and emerging technology landscape.

Machine Learning is ubiquitous in commercial usage and has reached to “Adolescent”, according to Gartner’s Maturity Level. Enterprises that have not yet started adopting the technology should start training and hiring talents in Machine Learning and Data Science and building infrastructure. The precursor of successful Machine Learning adaptation is to be able to manage “Big Data” for training the machines with the right training dataset. Enterprises need to invest more in dedicated Center of Excellence (CoE) for the “Emerging and disruptive Technology” like Machine Learning and embrace the transformation into their core business. For individual computer professionals, this is the time when the focus on learning the concept of Machine Learning and Artificial Intelligence, be familiarize with development tools and technologies (Matlab, R, Apache Mahout, Python ML libraries, etc.), specially, around data and algorithms, and finally dust off the the math and statistics skills long sitting unused. Those days aren't far away when the Java and .NET programmers would be considered as legacy programmers like the way the COBOL programmers were being considered.

Applying Agile in the snow cleaning job

We have just survived through the Nor'easter "Juno", and to my biggest relief, without any power disruption. As forcasted, the snow accumulation was more than 2 feet. I had cleaned 3 feet of snow in past but this time I thought to apply one of the Agile principle for the snow cleaning job which is: Deliver increments that add value to the customer (i.e. myself) in a small iteration. 

Let's go into detail of my Agile project. I have defined "Cleaning snow of my house" as the Epic. And here are the stories: 

1. Clean the entrance stair so that we can come out of our house through main entrance
2. Clean the snow of the driveway in front of the garage so that I can take out my van, which I use to drive on the highway
3. Clean the snow of the other driveway so that I can take the sedan out so that I can drive inside the city
4. Clean the sidewalk, which I have to do as a city law and also my kids would use that to go to school
5. Clean the bus stop at the end of the sidewalk, so that people can use that stop
6. Clean the patio snow, so that we can use the patio door to go out for fresh air
7. Clean the extra two spot of my drive way so that the guests can park the car

 I had started first prioritizing the stories based on the business value, to be said in simplistic term, how fast can I take my car out of my garage and driveway and then make a way out from the house. Also, I have re-prioritized the stories while progressing through the backlog of stories. I had also dropped the story#5: cleaning the bus stop due to the fact that the cost is too high for the benefit. Also, the snow plower guy from the city always dump the snow on the stop, so there's no point of cleaning this hardest part without getting sustaining benefit.

I have tried the Agile snow cleaning just for fun but after doing that I found it's actually fun and beneficial. It was "beneficial" because I was seeing the value of my work quicker and it was "fun" because I didn't have time to be tired by thinking too much down the road as I was taking a small bite at a time

Disclaimer: I have written this short post in March, 2015 after the massive snow apocalypse but kept it in Draft status as I didn't think this as valuable but while reviewing it today, I changed my mind and decided to publish.

Book Review of “Blink: the power of thinking without thinking”, Gladwell, Malcolm (2007)

“Blink” defies the long standing common sense and conventional wisdom within our society and, particularly, within our professional workspace, by putting an intriguing and thought provoking statement at the very beginning of the book as the cover page subtitle, i.e. “The Power of Thinking without Thinking”. The journey begins from there and ends through the six chapters of the book where the author, Malcolm Gladwell, explores various aspects of human psychology while making the case for his core propositions, i.e. the ingenuity of human subconscious, through his powerful skills of storytelling.

 

Gladwell started the book with a real life story that took place in a museum where a complex expert decision making, to figure out the authenticity of a Kouros’, fell apart. The professionals at the Getty museum meticulously investigated the Kouros for months. They used all the state-of-the-art techniques and skills they could ever use to verify its authenticity and finally declared that statue is an original work of art. However, Thomas Hoving, the former director of Metropolitan Museum of Art, New York, in his first glance, found the Kouros, suspiciously “fresh” and then proved to be unauthentic, contrary to the thorough and systematic investigation of all the experts at the Getty museum. This is what the author wants us to realize the power of intuition and snap judgment that often supersedes the experts’ long systematic analytical process.

 

The author claims that the “Thin Slicing”, the ability of our subconscious processing of the brain, called as the “Internal Computer”, to find patterns in situations based on very narrow slices of experience, is a central part of what it means to be human. The “Coup d’oeil” i.e. Power of Glance as well as snap decision make huge difference in the situations that demands to be really quick. To show the power of snap decision, the author brings the story of Vic Braden, one of the world’s top tennis coaches, who was able to call out a “double fault”, which is a rare event for a tennis player in a professional game, almost without any miss. There were no explainable clues that he could come up with on how he could make that snap decision just before the player’s going for double fault. In the book, it describes as: “Snap judgments are, first of all, enormously quickly: they rely on the thinnest slices of experience but they are unconscious. They take place behind a locked door and we are not very good at dealing with the fact of that locked door as they happen in our subconscious part of the brain processing”. Now the question may come to the mind that is there any formal way to create a process or structure that is conducive to make effective snap judgment? It turns out that, yes, there can be structures created to support spontaneity to foster snap judgment. Malcom Gladwell has shown that the spontaneity of decision can be better made through “less is more” i.e. by not overwhelming with voluminous data and processing. The proves in the book are: the mind blowing victory of Van Riper in the Millennium Challenge war game and simplifying the triage process of heart attack patients in Cook county hospital. Though spontaneity may sound like a random and chaotic structure but it’s far from that. The ability to become effectively spontaneous demands years of training in the relevant field.

 

If the Thin Slicing and Snap Judgments are all sounds like the panacea, let’s be clear that, it’s not the only color Gladwell wanted to paint the entire book with. There are dark sides that he wants the readers to be aware of. He reminds the shocking abject judgment in the American political history that he called as “The Warren Harding Error”. The dark side of Thin-slicing is that the snap judgments are made without getting below the surface of the situation that creates a condition where the “Blink” is often the root of prejudice and discrimination. The author has explained the famous Implicit Association Test (IAT) and showed how it unearths the bitter reality of our unconscious attitudes that may be utterly incompatible with our stated conscious values.

Gladwell then told us the story of Kenna who is a victim of snap decision not because that snap judgment was wrong but because, the snap judgment was inappropriately used to assess an unconventional musician. In the same line, Pepsi Challenge through a blind test was a famous failure for Coca Cola Company that entirely missed that what people says about a snap judgment doesn’t necessary holds true when the real consumption happens. In those instances, the experts’ skills of meticulous evaluation is much more effective than that of the thin slicing of experience as they might not be able to grasp the totality of an experience. So, it’s also important to use the right way to ask people for their snap judgments.

Gladwell then explored the science behind a fascinating side of human psychology i.e. the Art of Mind reading. He explains the research of Ekman and Friesen who have identified close to three thousand facial expressions and encoded them into “action-unit (AU)” to identify the facial displays of human emotion. Though human mind can be read through those “AU” but in a fast moving and stressed situation, people sometime lose the ability to read those clues, called as “temporary autism”, the drives people to bad judgment and inaccurate decision making. The case study of Diallo of Bronx, shot and killed by police officers, was such an example. He suggested that the gaffe of mind reading caused by “temporary autism” can be prevented by creating a white space i.e. giving enough time and space between the observation and decision making.

At the end of reading the six chapters of the book, readers may feel an apparent contradiction where the author blames the blind testing in one chapter and then advocates for the same in the next. But this would be clearer by his suggestion in the conclusion where he strongly recommends utilizing the moment of blink carefully. He puts forth a guiding principle on when to blink and when to think: “On straightforward choices, deliberate analysis is best. When questions of analysis and personal choice start to get complicated - when we have to juggle many different variables - then our subconscious thought processes may be superior, …contrary to conventional wisdom”. One thing the author could do differently in the book was to keep out the too much detail from the stories to retain the brevity of the book, but nonetheless, this is a fascinating book that took us through the complex ally of unconscious brain to show us the power of spontaneity and snap judgment i.e. “blink”.

Reference:

Book Review: Gladwell, Malcolm (2007). “Blink: the power of thinking without thinking”.

ISBN: 978-0-316-01066-5.