Quobyte Inc., a developer of modern storage system software, will be offering a TensorFlow plug-in for its distributed file system, providing increased throughput performance and linear scalability for ML-powered applications.
Using Quobyte storage with TensorFlow helps to simplify and streamline the operation of machine learning.
Quobyte’s TensorFlow File System Plug-in allows TensorFlow applications to talk directly to Quobyte, bypassing the operating system kernel to significantly reduce kernel mode context switches and lower CPU usage.
While Quobyte storage can be used with all stages of ML, the resulting increased GPU utilization from the TensorFlow plug-in speeds up model training of ML workflows.
The TensorFlow plug-in can be used to train models locally on sample data sets and use the Google Cloud Platform for training at scale because Quobyte runs on-prem and in the cloud.
Additionally, because it bypasses the kernel entirely, Quobyte’s TensorFlow plug-in works with both current and older versions of Linux, providing a full range of flexible deployment options for use in ML.
TensorFlow is an open source library for numerical computation and large-scale machine learning used across industries such as autonomous vehicles, robotics, financial services, healthcare, government, aerospace, defense, and many others.
Quobyte provides users the flexibility to train anywhere and seamlessly move models into production to better support ML workloads from the data center to the cloud to the edge.
Additional benefits of Quobyte’s TensorFlow File System Plug-in include:
- The ability to leverage HDD and SSD to get the best price-performance ratio without cumbersome tiering Prefetching of training data can deliver substantial performance improvement. Much machine-generated data uses a sequential naming convention that makes it ideal for prefetching.
- Infinite scalability that allows users to grow storage in terms of throughput and capacity when they need it. As ML project requirements change – oftentimes more quickly than anticipated – the Quobyte installation will adapt. Disks or servers can be quickly and easily added when needed to provide more capacity or performance without any interruption to applications or services.
- Multi-tenancy that provides additional security by allowing users to define isolated namespaces and physical separation of data/workloads inside the same cluster. Administrators can further isolate tenants by controlling to which physical hardware they have access in order to ensure performance and that data is not accessible to any unauthorized users on the network.
“With the higher accuracy of results, scalability to handle bigger data sets, and flexibility to run on-prem to the cloud, and edge, we believe we are providing an optimal experience that allows customers to fully leverage the value of their Machine Learning infrastructure investments,” said Bjorn Kolbeck, Quobyte CEO.
For more information about this news, visit www.quobyte.com.