I spoke at yesterday’s Social Data Revolution class on Trust & Identity at Berkeley University on behalf of Airbnb. The class also had speakers from Uber and Reddit. You can see the live recording on youtube.
A world super power in electronics, Japaenese company Fujitsu claims that it no longer needs Data Scientists, and has automated their job! The company claims that
Data scientists use their skill to select a combination of algorithm and configuration to get the most accurate predictive model from the starting data
and that they have found a way to automate this searching over different configurations and models for the optimum. The diagram depicts a meta machine learning pipeline that tunes the hyper-parameters of a model in Spark or another language.
While it certainly makes sense to automate this potentially tedious optimisation, this will by no means deprecate the role of a Data Scientist. It is of course true that a Data Scientist has to intelligently choose an algorithm and its configuration, but this is a small part of the full life cycle of a data product that a Data Scientist is responsible for.
The processes of defining a metric to optimise, then obtaining and cleaning data, transforming data to informative feature, maybe also obtaining and cleaning labels (in the case of supervised learning) are all part of a Data Scientist’s responsibilities and need to be completed before an algorithm can be optimised.
Moreover, these processes constitute 90% of the blood, sweat, and tears of a Data Scientist that go into making a successful data product. Algorithm and configuration optimisation can give you a few percentage points boost in performance at most, but it is the accuracy of the data and intelligent feature sculpting which make the real difference.
Let’s hope Fujitsu does not sack all their Data Scientists just yet, or they may have a machine learning tuner with no data for the machine to learn from!
It’s finally happened. The Epicentre co-working space in Stockholm, Sweden has installed a near-field communication (NFC) chip into 50 volunteers. Described as like ‘having a vaccination’, the chip is inserted into the wrists of employees under their skin. This enables them to automatically clock in and and out of work, register loyalty points, and automate many other transactions.
Personally, the costs seem to far outweigh the benefits in this scenario. Other employers, especially military organisations and security companies are watching this new device keenly. However, for saving a few seconds each day and the perceived security threat of not having these, the intrusion into one’s private life cannot be worth it.
Wearable devices such as the Fitbit and Apple Watch are becoming ever more popular, but a non-removable device is a new frontier. The argument has always been that they allow us to collect more Big Data and process it more immediately to improve our lives, but its not clear why faster is always better.
Being able to track every movement and heart beat is the stuff of Orwell and Huxley while the benefits are cursory at best. Caution brave new world.
Yesterday I was invited by David Webster to talk to the team at innovative design company IDEO. IDEO is a cutting edge digital and physical design studio in Palo Alto that has been leading creativity for over 30 years. I was lucky enough to have a tour by David through their workshop, engineering office, and toy lab.
After the tour we had a joint Q&A with the whole team about how big data is used at Airbnb and how it might be used more in the design process at IDEO. Some key thoughts emerged:
The future for Data Science in Design is exciting and, when they start to overlap more, we will see changes in the world around around us accelerate even faster.
A Data Scientist needs to be able to extract insights from datasets that are orders of magnitude larger than what they were 5 years ago. And they need to extract this insight carefully, with statistical significance and integrity. Moreover, the insight is only as useful as the business need it solves.
As a regular interviewer at Airbnb for junior and senior Data Scientists, attention to data cleaning and diligence in statistical analysis are fundamental for successful candidates. Moreover, we look for people that understand the ‘why’ of a problem and the business impact of a solution. This is what differentiates a really smart candidate from a hired candidate.
Read Riley Newman, head of Data Science at Airbnb, describe his experience during the past 5 years of Airbnb’s hypergrowth in today’s VentureBeat article. Learn about how he scaled the team and brought data to the top of mind in every corner of the company. There’s also a picture of my team featured!