Deep Learning for Biotechnology on Qubole
Matt Der, Chief Technology Officer, Notch
Watch Now
In the biological sciences, hypothesis-driven experiments and bottom-up design experiments rely on predicting what will happen with new cells and molecules. Machine learning excels at prediction and has become more democratized, making it an important component in the biotech toolkit. We use Merck's Kaggle competition as a representative task in this domain that involves predicting molecular activity from numeric descriptors of chemical structure. Our approach utilizes deep neural networks using the Keras library in a Qubole notebook, which is conveniently attached to an autoscaled Spark cluster. We use Spark to distribute the hyperparameter search for optimizing the neural net.