Skip to main content
 

We need open, accelerated file transfer (and Aspera is not the answer)

TechCrunch reports that IBM bought Aspera yesterday:

Aspera’s software is built on “fasp,” its patented file transfer technology. Fasp is designed to leverage a company’s wide area network (WAN) and commodity hardware to achieve speeds that are faster than FTP and HTTP over a secure network. A WAN is essentially a company’s network across a large geographic region. Aspera’s technology optimizes the WAN through its software that allows for granularity in the way the technology is used. Through the process, Aspera optimizes the bandwidth, latency, bottlenecks and a host of other factors.

Essentially, Aspera is an optimized, proprietary protocol built on UDP. The sender and receiver needs to have Aspera software (or software that licenses Aspera's technology) installed for the transfer to take place.

IBM's thesis that we need faster file transfers is obviously correct. Uploading large data or media files is a pain, and it's not a problem that's going away. While downstream bandwidth is getting better all the time, upstream bandwidth often suffers in comparison - and between big data, more sophisticated applications and our insatiable appetite for video, the size of the files we're moving across the Internet is going up.

[Disclosure: right now I'm CTO at latakoo, a startup that helps enterprises (including TV news networks) move large amounts of media data around. We use a toolbox of compression techniques, as well as a global upload network, to make managing media files much faster. Journalists use us to, for example, upload video from Air Force One while the plane is taxiing down the runway.]

Don't think there's a problem? 5 minutes of recorded video on an iPhone 5 can be 1gb or more in size. Try uploading that from anything other than a super-fast broadband connection.

The trouble is that a protocol like Aspera's naturally leads to lock-in, and can only solve the problem for certain kinds of software. Unless fasp is open-sourced now that it's owned by IBM, every piece of software you own will need to have a license for their patents in order to take advantage of it. Not only does that lead to more expensive software, but it also limits the innovation that can happen around the protocol. If your business relies on fast file transfers, that's a lose-lose situation for you: you're locked into an expensive ecosystem, and there's little chance of a disruptive incomer to tweak and play with the protocol. Because of that, folks will just invent new protocols. When an improvement is made, it's likely you'll have to buy into a whole new ecosystem, as opposed to just upgrade to the next version. You'll hop from lock-in to lock-in.

Knowing that we need to speed up uploads, and use all available bandwidth, we need to think about other, more open approaches. UDP makes a lot of sense as a bedrock, but an open protocol designed for resilient file transfer is needed - and then, with a first version prototyped, we need to seed libraries all over the place, in every available language. We need to design it for backwards-compatibility. And we need to make sure it remains unencumbered with patents (possibly, in today's climate, by defensively registering a patent and then widely granting a license).

Our approach with Latakoo has always been to use standard protocols and optimize the files that are being transferred over the top. That works really well, and ensures wide compatibility with all kinds of networks, from corporate infrastructure through satellite modems connections. Even better would be to be able to rely on a well-supported open protocol that optimizes the upload stream in ways that TCP does not (and then, continue to optimize content with this as a starting point). Bittorrent exists, of course, and is wonderful - but is often blocked because of unfortunate connotations and liabilities relating to media piracy.

Developing such a protocol is in the interests of Netflix, Google, and, yes, Latakoo. Faster file transfers are empowering for every user, and save large service providers money. They're easier to integrate with, and allow for an ecosystem where the customer is in control. Everybody wins.

· Posts · Share this post