Apache Spark AI Summit

Last week I was lucky enough for Quantiful to send me to attend the 2018 Apache Spark AI Summit in San Francisco. Apache Spark is an open-source large scale computing framework. Essentially, it provides an easy way for data scientists to crunch and process large amounts of data. Apache Spark allows programmers to scale the processing of big data as well as perform machine learning at scale.

The summit was an extravagant affair hosted in the heart of the technology capital of the world, Silicon Valley. The speakers included big players in the analytics industry such as Andrej Karpathy (director of AI at Tesla), Chief Data Scientist of AllianceBernstein and the creators of Apache Spark from Berkley among others. The talks ranged from "Forecasting Bitcoin prices" to NASA’s "Using Deep Learning to find water on the moon".

Attending the summit was an enjoyable experience where I had the opportunity to learn about the latest technologies and developments in big data and AI. In particular, there are a few developments which I am excited to implement back home. The one I am most excited about is "mlflow".

mlflow is an API designed to help data scientists build, test and productionise machine learning models in a reproducible, managed workflow. This will allow Quantiful to quickly release improvements to our own Machine Learning API and ensure our clients get the best solution delivered in the fastest way.

Overall the Spark AI summit reassured me that even though we are a small-start up based in NZ our technical sophistication and thinking ranks well enough to impress some of the best minds in the industry. I am excited to bring my learnings home and extend our products sophistication in order to deliver the best results for our clients.