You’re likely familiar with the most common file transfer protocols: FTP/S, SFTP and HTTP/S. However, a number of other communication protocols for file transfer exist. This blog aims to help you understand more about the larger world of file transfer protocols so you can choose the right file transfer tool for the job.
For each communications protocol, a brief description of how it operates, typical use cases and a link to the published standard will be included.
Accelerated File Transfer Protocol (AFTP)
Overview: JSCAPE by Redwood developed this proprietary file transfer protocol to improve file transfer speed over high latency networks by running its control channel over TCP and its data channel over UDP. This file transfer configuration can increase transfer speeds up to 100x compared to TCP-based file transfer protocols such as FTP/S, SFTP or HTTP/S.
Use cases: AFTP is ideal for organizations transferring higher data volumes across long distances via high-bandwidth networks due to its ability to sidestep latency slowdowns experienced in TCP transfers. Industry applications include banking and finance, media and entertainment, industrial manufacturing, data architecture and more.
Published standard and references: JSCAPE AFTP documentation
Applicability Statement 2 (AS2)
Overview: The AS2 file transfer protocol began as an inventory control and ordering system developed to deliver business structured data. Similar to email, AS2 encodes files as individual attachments within an AS2 “message.” The message is sent via HTTP/S to a trading partner and can be signed and encrypted. AS2 also allows receipt confirmation messages to be sent.
Two “higher” variations of Applicability Statement transfer exist:
- AS3, which focuses on message formatting for Electronic Data Interchange (EDI) that can be delivered over any file transfer protocol.
- AS4, a newer version of AS2 gaining adoption, which operates as an XML-based web service and provides improved partner interaction and delivery meta-data support.
Use cases: The retail and consumer goods industries commonly use AS2 to exchange purchase orders, invoices and other business documents in EDI format.
Published standard and references: The IETF’s RFC 4130 details how AS2 transfers operate
You may also be interested in how to set up an AS2 server in JSCAPE.
Background Intelligent Transfer Service (BITS)
Overview: The BITS file transfer protocol is designed to transfer non-time-sensitive files with minimal performance and operational impact on the transferring machine and network. It does so by performing transfers using idle bandwidth remaining after the transferring machine’s other active tasks.
Use cases: BITS is a Windows application, and is an ideal file transfer protocol for scenarios such as non-critical updates, constrained bandwidth environments or very large or long transfers from individual machines.
Published standard and references: Microsoft’s BITS standard
BitTorrent
Overview: Probably the most well-known peer-to-peer file sharing protocol, BitTorrent allows a decentralized group of clients to pool together and share data between them. This coordination is often made possible by tracker servers that keep records of which machines have copies of which files, but it is not a requirement. The protocol makes many small data requests, secures data via SHA-256 encryption, makes extensive use of hashing to protect and confirm files and includes its own file transfer protocol, the BitTorrent File System (BTFS).
Use cases: BitTorrent is ideal for distributing large files to multiple users, such as streaming or game servers. It also improves transfer speeds and bandwidth usage in environments with slower networks or hardware by spreading the data loads across multiple machines via many small data requests (instead of a single client/server connection).
Published standard and references: BitTorrent protocol specification
Cross File Transfer
Overview: This proprietary file transfer protocol was originally developed for the French banking system prior to the widespread adoption of the Internet, and now supports TCP/IP connections as well. It is a common machine file transfer method used by larger European organizations. It requires the installation of the transfer application on all machines involved in a transfer due to its coordination of approved transfer sites in a centrally managed CFT directory.
Use cases: Cross File Transfer is ideally suited for high-security transfers that may involve data transformation or file manipulation as part of the transfer, such as banking. It offers a number of MFT automation features similar to JSCAPE.
Published standard and references: See Axway’s documentation
Fast and Secure Protocol (FASP)
Overview: IBM’s proprietary FASP file transfer protocol was designed to solve issues with file transfer throughput and speed over TCP. FASP relies on connectionless data transmission over UDP to try and achieve the highest possible packet velocity. To ensure minimal data loss, the protocol relies on a proprietary packet loss reduction algorithm that measures the time of transfer and response from the sender to the data endpoint, corrects for network latency and estimates the packets that should have been received during that time.
Use cases: FASP was developed to transfer large data volumes as quickly as possible. It is a good choice for organizations that conduct significant transfers across a number of geographically dispersed, wide-area networks that operate at varying levels of speed and reliability.
Published standard and references: IBM operates FASP under the Aspera brand name, with documentation at this link
File Transfer Protocol (FTP) and File Transfer Protocol Secure (FTPS)
Overview: One of the most common file transfer protocols in use today, FTP began as a simple, user-friendly way to transfer files from one machine to another. However, it did not require data encryption. As a result, it has largely been superceded by FTPS, which adds encryption support via TLS, or SFTP, which uses SSH to protect its data.
Use cases: FTP/S is widely used for file transfers over the internet that need to balance speed, security and backward compatibility with other systems that might be using a variety of connection security methods. It also offers a strong level of authentication security due to its use of public certificates.
Published standard and references: FTP’s most current documentation can be found at this link
Hypertext Transfer Protocol (HTTP) and Hypertext Transfer Protocol Secure (HTTPS)
Overview: The core application layer protocol for today’s internet, HTTP/S is one of the most ubiquitous file transfer protocols in the world. It was designed to facilitate data transfer between multiple clients and servers transmitting a variety of data formats and allowing a number of intermediate connections between two end-points for optimal traffic and data routing. HTTPS encrypts communications via TLS in order to ensure that server authentication, data privacy and integrity are all preserved.
Use cases: HTTP/S is the ideal protocol to serve data requests to a large number of applications or clients used by humans. It is capable of delivering a number of media formats with specific human-readable styling, and is the standard for providing human interactive experiences online.
Published standard and references: RFC 7231
Odette File Transfer Protocol (OFTP)
Overview: This file transfer protocol was designed in the 1980s by the Organization for data exchange by teletransmission in Europe (Odette), with the goal of replacing typical paper-based business correspondence with digital communications, known as EDI. As such, OFTP prioritizes standardized document exchange, confirmed acknowledgment of file receipts and the ability to transfer in a peer-to-peer environment (as opposed to a server-client approach). OFTP can also be secured and authenticated using TLS.
Use cases: OFTP is ideally suited for human-readable document exchange in a “paperwork” heavy environment between two peer organizations. Typical use cases involve stock or purchase orders, inventory confirmations, shipping confirmations, invoices and more in industries like logistics, billing, accounting or insurance. OFTP has been the core file transfer protocol for the European automotive industry since its inception.
Published standard and references: RFC 5024 contains an overview of the protocol, while the Odette organization maintains an OFTP implementation guide at this link.
Note: Cerberus’s MFT server by Redwood partner JSCAPE supports OFTP transfer.
rsync
Overview: A standard open-source file transfer utility on most Linux distributions, the rsync protocol was developed to transfer and synchronize files between two different systems. rsync offers a number of command line and configuration tools and graphical interface applications as well.
Use Cases: Typically used to connect to and update local or remote storage, rsync is an ideal tool for file mirroring, backup preservation, logging and other data duplication tasks.
Published Standard and References: The rsync website, and the rsync project on Github
Secure Copy Protocol (SCP)
Overview: SCP is a relatively simple file transfer protocol designed to copy a single file at a time from one machine to another via SSH. It does not offer any file manipulation commands but can provide some transfer compression and access reporting. As such, it typically transfers data at higher speeds than protocols such as SFTP or FTPS.
Use cases: SCP is best for smaller-size or one-off data transfers that don’t require any file manipulation, such as distributing firmware updates. It’s also a good choice for automated scripting of repetitive transfers.
Published standard and references: SCP is part of the OpenSSH program, which maintains SCP documentation at this link.
SSH File Transfer Protocol (SFTP)
Overview: In many ways the successor to FTP, SFTP is probably the most common file transfer protocol after HTTP/S due to its blend of encrypted transfer and broad cross-platform compatibility. SFTP transfers are typically handled via a standalone SFTP or SSH server, with all data traffic being encrypted via SSH. SFTP can transmit a full suite of file and directory manipulation commands, as well as commands governing the transfer itself. SFTP can also transmit metadata such as time stamps for ease of logging and data integrity purposes.
Use cases: SFTP is an ideal protocol for many situations where you need to transmit data in a user-friendly, encrypted manner. Organizations that transfer significant amounts of data externally from their LAN will likely find that SFTP is more compatible with network security devices, and it is also ideal for organizations operating in more secure compliance-centric environments where access levels and data security must be regularly audited.
Published standard and references: SSH Working Group’s most recent SFTP documentation
Trivial File Transfer Protocol (TFTP)
Overview: One of the earliest file transfer protocols, TFTP is a simple, low-resource protocol that can only read or write files from or to a specified remote server. It was primarily created to support network equipment booting, rather than high-volume transfers, and has no user authentication support or security provisions (although it can run under secured layers).
Use cases: TFTP remains in use today, primarily to provide network boot or configuration files, as well as firmware updates.
Published standard and references: RFC 1350
Tsunami UDP Protocol
Overview: An open-source file transfer protocol designed to address TCP-based file transfer slowdowns created by bandwidth delay, Tsunami limits TCP-based communication to the control connection while handling all bulk data transfer via UDP. The protocol was designed to facilitate downloads and does not allow client uploads back to servers.
Use cases: Tsunami could be considered for use cases where a downloading client operates on a slower network or at a significant geographic distance from the server.
Published standard and references: Tsunami UDP Protocol on Sourceforge and Github
UDP-Based Data Transfer Protocol (UDT)
Overview: A combined bandwidth optimization and file transfer protocol, UDT began as an academic bulk data transfer tool and has evolved to specialize in high-speed WAN transfers. UDT’s control and data channels both operate via UDP, and the protocol offers a customizable congestion control functionality to ensure other network traffic does not get overwhelmed by large transfers (even TCP traffic). UDT can operate all its transfers from a single port, making it a good tool for cautious network managers, and providing an API for custom scripting. Note that encryption and security of UDT transfers are currently evolving, but are not inherently part of the protocol.
Use cases: UDT is a popular choice for organizations seeking to transmit large data sets over longer distances at high speeds, such as international researchers, multinational financial institutions, data centers and more. The protocol’s flexibility and customization options also make it a good choice for high-load automation activities.
Published standard and references: UDT open-source code and documentation on Sourceforge
UDP-Based File Transfer Protocol (UFTP)
Overview: UFTP focuses on one-to-many file transfers over large geographic distances, such as satellite and broadcast transfers. The protocol uses TLS-based encryption to secure its data and allows a common key to be shared by multiple receivers. Because of its broadcast nature, UFTP has been designed to include multicast tunneling that can occur by UFTP proxy servers.
Use cases: UFTP lends itself well to data transfers that must traverse multiple disparate networks, such as streams of live broadcasts or satellite data transfers.
Published standard and references: UFTP’s open-source code and documentation resides on Sourceforge
We hope the above information has helped you decide which file transfer protocol to use. If you have any questions, our experts at Cerberus or JSCAPE will be happy to discuss your needs.
Bonus protocol: File transfer in space!
If you’ve ever wondered how to transfer files off-planet, look no further than the Space Packet Protocol. This standard for interstellar file transfer was developed by NASA’s Consultative Committee for Space Data Systems and is the best way to send data out to the stars.