03-09-2019, 03:13 PM
Microsoft and Intel partner to speed deep learning workloads on Azure
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/03/microsoft-and-intel-partner-to-speed-deep-learning-workloads-on-azure.png" width="587" height="183" title="" alt="" /></div><div><p><em>This post is co-authored with Ravi Panchumarthy and Mattson Thieme from Intel.</em></p>
<p>We are happy to announce that Microsoft and Intel are partnering to bring optimized deep learning frameworks to Azure. These optimizations are available in a new offering on the Azure marketplace called the <a href="http://aka.ms/dsvm/intel" target="_blank" rel="noopener noreferrer">Intel Optimized Data Science VM for Linux (Ubuntu).</a></p>
<p><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/03/microsoft-and-intel-partner-to-speed-deep-learning-workloads-on-azure.png"></p>
<p>Over the last few years, deep learning has become the state of the art for several machine learning and cognitive applications. Deep learning is a machine learning technique that leverages neural networks with multiple layers of non-linear transformations, so that the system can learn from data and build accurate models for a wide range of machine learning problems. Computer vision, language understanding, and speech recognition are all examples of deep learning at play today. Innovations in deep neural networks in these domains have enabled these algorithms to reach <a href="https://blogs.microsoft.com/ai/historic-achievement-microsoft-researchers-reach-human-parity-conversational-speech-recognition/">human level performance in vision, speech recognition</a> and <a href="https://blogs.microsoft.com/ai/machine-translation-news-test-set-human-parity/">machine translation</a>. Advances in this field continually excite data scientists, organizations and media outlets alike. To many organizations and data scientists, doing deep learning well at scale poses challenges due to technical limitations.</p>
<p>Often, default builds of popular deep learning frameworks like TensorFlow are not fully optimized for training and inference on CPU. In response, Intel has open-sourced framework optimizations for Intel® Xeon processors. Now, through partnering with Microsoft, Intel is helping you accelerate your own deep learning workloads on Microsoft Azure with this new marketplace offering.</p>
<blockquote>
<p><em>“Microsoft is always looking at ways in which our customers can get the best performance for a wide range of machine learning scenarios on Azure. We are happy to partner with Intel to combine the toolsets from both the companies and offer them in a convenient pre-integrated package on the Azure marketplace for our users”</em> </p>
</blockquote>
<blockquote>
<p>– Venky Veeraraghavan, Partner Group Program manager, ML platform team, Microsoft.</p>
</blockquote>
<h2>Accelerating Deep Learning Workloads on Azure</h2>
<p>Built on the top of the popular <a href="http://aka.ms/dsvm">Data Science Virtual Machine (DSVM)</a>, this offer adds on new Python environments that contain Intel’s optimized versions of TensorFlow and MXNet. These optimizations leverage the <a href="https://www.intel.com/content/www/us/en/architecture-and-technology/avx-512-overview.html">Intel® Advanced Vector Extensions 512 (Intel® AVX-512)</a> and <a href="https://01.org/mkl-dnn">Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN)</a> to accelerate training and inference on Intel® Xeon® Processors. When running on an Azure F72s_v2 VM instance, <a href="https://www.intel.ai/intel-optimized-data-science-virtual-machine-azure">these optimizations yielded an average of 7.7X speedup in training</a> throughput across all standard CNN topologies. You can <a href="https://www.intel.ai/intel-optimized-data-science-virtual-machine-azure">find more details on the optimization practice here</a>.</p>
<p>As a data scientist or AI developer, this change is quite transparent. You still code with the standard TensorFlow or MXNet frameworks. You can also use the new set of Python (conda) environments (intel_tensorflow_p36, intel_mxnet_p36) on the DSVM to run your code to take full advantage of all the optimizations on an Intel® Xeon Processor based F-Series or H-Series VM instance on Azure. Since this product is built using the DSVM as the base image, all the <a href="https://docs.microsoft.com/azure/machine-learning/data-science-virtual-machine/overview#whats-included-in-the-data-science-vm">rich tools for data science and machine learning</a> are still available to you. Once you develop your code and train your models, you can deploy them for inferencing on either the cloud or edge.</p>
<blockquote>
<p><em>“Intel and Microsoft are committed to democratizing artificial intelligence by making it easy for developers and data scientists to take advantage of Intel hardware and software optimizations on Azure for machine learning applications. The Intel Optimized Data Science Virtual Machine (DSVM) provides up to a 7.7X speedup on existing frameworks without code modifications, benefiting Microsoft and Intel customers worldwide”</em></p>
</blockquote>
<blockquote>
<p><i>– </i>Binay Ackalloor, Director Business Development, AI Products Group, Intel.</p>
</blockquote>
<h2>Performance</h2>
<p>In Intel’s benchmark tests run on Azure F72s_v2 instance, here are the results comparing the optimized version of TensorFlow with the standard TensorFlow builds.</p>
<p><img alt="Bar graph of Default TensorFlow vs. Intel Optimization for TensorFlow" height="447" src="http://www.sickgaming.net/blog/wp-content/uploads/2019/03/microsoft-and-intel-partner-to-speed-deep-learning-workloads-on-azure-1.png" title width="859"></p>
<p><strong>Figure 1:</strong> Intel® Optimization for TensorFlow provides an <strong>average of 7.7X increase</strong> (average indicated by the red line) in training throughput on major CNN topologies. <a href="https://github.com/IntelAI/azure-applications/tree/master/scripts/benchmark">Run your own benchmarks using tf_cnn_benchmarks</a>. Performance results are based on Intel testing as of 01/15/2019. <a href="https://www.intel.ai/intel-optimized-data-science-virtual-machine-azure">Find the complete testing configuration here</a>.</p>
<h2>Getting Started</h2>
<p>To get started with the Intel Optimized DSVM, click on the <a href="http://aka.ms/dsvm/intel">offer in the Azure Marketplace</a>, then click “GET IT NOW”. Once you answer a few simple questions on the Azure Portal, your VM is created with all the DSVM tool sets and the Intel optimized deep learning frameworks pre-configured and ready to use.</p>
<p><img alt="Screenshot of Intel Optimized Data Science VM for Linux in Azure marketplace" height="629" src="http://www.sickgaming.net/blog/wp-content/uploads/2019/03/microsoft-and-intel-partner-to-speed-deep-learning-workloads-on-azure-2.png" title width="886"></p>
<p>The Intel Optimized Data Science VM is an ideal environment to develop and train deep learning models on Intel Xeon processor-based VM instances on Azure. Microsoft and Intel will continue their long partnership to explore additional AI solutions and framework optimizations to other services on Azure like the <a href="http://aka.ms/azureml">Azure Machine Learning service</a> and <a href="https://azure.microsoft.com/services/iot-edge/">Azure IoT Edge</a>.</p>
<h2>Next steps</h2>
</div>
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/03/microsoft-and-intel-partner-to-speed-deep-learning-workloads-on-azure.png" width="587" height="183" title="" alt="" /></div><div><p><em>This post is co-authored with Ravi Panchumarthy and Mattson Thieme from Intel.</em></p>
<p>We are happy to announce that Microsoft and Intel are partnering to bring optimized deep learning frameworks to Azure. These optimizations are available in a new offering on the Azure marketplace called the <a href="http://aka.ms/dsvm/intel" target="_blank" rel="noopener noreferrer">Intel Optimized Data Science VM for Linux (Ubuntu).</a></p>
<p><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/03/microsoft-and-intel-partner-to-speed-deep-learning-workloads-on-azure.png"></p>
<p>Over the last few years, deep learning has become the state of the art for several machine learning and cognitive applications. Deep learning is a machine learning technique that leverages neural networks with multiple layers of non-linear transformations, so that the system can learn from data and build accurate models for a wide range of machine learning problems. Computer vision, language understanding, and speech recognition are all examples of deep learning at play today. Innovations in deep neural networks in these domains have enabled these algorithms to reach <a href="https://blogs.microsoft.com/ai/historic-achievement-microsoft-researchers-reach-human-parity-conversational-speech-recognition/">human level performance in vision, speech recognition</a> and <a href="https://blogs.microsoft.com/ai/machine-translation-news-test-set-human-parity/">machine translation</a>. Advances in this field continually excite data scientists, organizations and media outlets alike. To many organizations and data scientists, doing deep learning well at scale poses challenges due to technical limitations.</p>
<p>Often, default builds of popular deep learning frameworks like TensorFlow are not fully optimized for training and inference on CPU. In response, Intel has open-sourced framework optimizations for Intel® Xeon processors. Now, through partnering with Microsoft, Intel is helping you accelerate your own deep learning workloads on Microsoft Azure with this new marketplace offering.</p>
<blockquote>
<p><em>“Microsoft is always looking at ways in which our customers can get the best performance for a wide range of machine learning scenarios on Azure. We are happy to partner with Intel to combine the toolsets from both the companies and offer them in a convenient pre-integrated package on the Azure marketplace for our users”</em> </p>
</blockquote>
<blockquote>
<p>– Venky Veeraraghavan, Partner Group Program manager, ML platform team, Microsoft.</p>
</blockquote>
<h2>Accelerating Deep Learning Workloads on Azure</h2>
<p>Built on the top of the popular <a href="http://aka.ms/dsvm">Data Science Virtual Machine (DSVM)</a>, this offer adds on new Python environments that contain Intel’s optimized versions of TensorFlow and MXNet. These optimizations leverage the <a href="https://www.intel.com/content/www/us/en/architecture-and-technology/avx-512-overview.html">Intel® Advanced Vector Extensions 512 (Intel® AVX-512)</a> and <a href="https://01.org/mkl-dnn">Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN)</a> to accelerate training and inference on Intel® Xeon® Processors. When running on an Azure F72s_v2 VM instance, <a href="https://www.intel.ai/intel-optimized-data-science-virtual-machine-azure">these optimizations yielded an average of 7.7X speedup in training</a> throughput across all standard CNN topologies. You can <a href="https://www.intel.ai/intel-optimized-data-science-virtual-machine-azure">find more details on the optimization practice here</a>.</p>
<p>As a data scientist or AI developer, this change is quite transparent. You still code with the standard TensorFlow or MXNet frameworks. You can also use the new set of Python (conda) environments (intel_tensorflow_p36, intel_mxnet_p36) on the DSVM to run your code to take full advantage of all the optimizations on an Intel® Xeon Processor based F-Series or H-Series VM instance on Azure. Since this product is built using the DSVM as the base image, all the <a href="https://docs.microsoft.com/azure/machine-learning/data-science-virtual-machine/overview#whats-included-in-the-data-science-vm">rich tools for data science and machine learning</a> are still available to you. Once you develop your code and train your models, you can deploy them for inferencing on either the cloud or edge.</p>
<blockquote>
<p><em>“Intel and Microsoft are committed to democratizing artificial intelligence by making it easy for developers and data scientists to take advantage of Intel hardware and software optimizations on Azure for machine learning applications. The Intel Optimized Data Science Virtual Machine (DSVM) provides up to a 7.7X speedup on existing frameworks without code modifications, benefiting Microsoft and Intel customers worldwide”</em></p>
</blockquote>
<blockquote>
<p><i>– </i>Binay Ackalloor, Director Business Development, AI Products Group, Intel.</p>
</blockquote>
<h2>Performance</h2>
<p>In Intel’s benchmark tests run on Azure F72s_v2 instance, here are the results comparing the optimized version of TensorFlow with the standard TensorFlow builds.</p>
<p><img alt="Bar graph of Default TensorFlow vs. Intel Optimization for TensorFlow" height="447" src="http://www.sickgaming.net/blog/wp-content/uploads/2019/03/microsoft-and-intel-partner-to-speed-deep-learning-workloads-on-azure-1.png" title width="859"></p>
<p><strong>Figure 1:</strong> Intel® Optimization for TensorFlow provides an <strong>average of 7.7X increase</strong> (average indicated by the red line) in training throughput on major CNN topologies. <a href="https://github.com/IntelAI/azure-applications/tree/master/scripts/benchmark">Run your own benchmarks using tf_cnn_benchmarks</a>. Performance results are based on Intel testing as of 01/15/2019. <a href="https://www.intel.ai/intel-optimized-data-science-virtual-machine-azure">Find the complete testing configuration here</a>.</p>
<h2>Getting Started</h2>
<p>To get started with the Intel Optimized DSVM, click on the <a href="http://aka.ms/dsvm/intel">offer in the Azure Marketplace</a>, then click “GET IT NOW”. Once you answer a few simple questions on the Azure Portal, your VM is created with all the DSVM tool sets and the Intel optimized deep learning frameworks pre-configured and ready to use.</p>
<p><img alt="Screenshot of Intel Optimized Data Science VM for Linux in Azure marketplace" height="629" src="http://www.sickgaming.net/blog/wp-content/uploads/2019/03/microsoft-and-intel-partner-to-speed-deep-learning-workloads-on-azure-2.png" title width="886"></p>
<p>The Intel Optimized Data Science VM is an ideal environment to develop and train deep learning models on Intel Xeon processor-based VM instances on Azure. Microsoft and Intel will continue their long partnership to explore additional AI solutions and framework optimizations to other services on Azure like the <a href="http://aka.ms/azureml">Azure Machine Learning service</a> and <a href="https://azure.microsoft.com/services/iot-edge/">Azure IoT Edge</a>.</p>
<h2>Next steps</h2>
</div>