Google Cloud has developed a new software service that helps organizations accomplish large-scale, online data transfers. The goal is to help take the complexity out of data transfers and move data faster than existing online tools like gsutil.
"In migrating to the cloud, enterprises can realize the full value of their data—building new apps and experiences faster and accessing analytics and machine learning solutions to generate new insights," Ash Ahluwalia, product manager at Google Cloud, told 5 Minute Briefing. "But getting data to the cloud is often challenging. The current transfer options for on-premises-to-cloud can be time-consuming and complex, with offline transfers requiring additional logistical support and online transfers through gsutil lacking the scale for petabyte-level, over-network transfers."
Ahluwalia said the new service, which is called "Transfer Service for on-premises data," now in beta, will enable enterprises to complete large-scale, over-network batch transfers from their on-premise sources to Google Cloud Storage. Transfer Service for on-premises data is a managed solution that lets organizations move their data without needing to engineer their own custom software or invest in an off-the-shelf solution.
"It’s fast enough to process transfers at tens of gigabytes per second, capable of processing petabytes and billions of files, and it provides built-in, fail-safe and reliability measures," he said.
According to Google Cloud it is also designed to be reliable and secure, so that if agent failures occur, in-progress transfers will not be impacted. And with performance optimizations included from the application to the transport layer, the service can use an organization's available bandwidth to minimize transfer times, and service requires no code or maintenance.
To get started, customers will need to first install and start the on-premise software (the agent), then go to the Cloud Console and submit directories to transfer to Cloud Storage. When transferring data, the service will parallelize the transfer across many agents, and then coordinate these agents to transfer the data over a secure internet connection to Cloud Storage. The service also features a fully self-service GUI with detailed transfer logs so that organizations can create, monitor, and manage transfer jobs with confidence.
Google Cloud's Transfer Service for on-premises data is now available in beta. Learn more about how the service works and here.