BTP Client Library
The Blob Transfer Protocol (BTP) is a lightweight file transfer and remote shell protocol based on CSP. BTP supports upload and download of files, as well as basic file operations such as file listing, delete, copy and move. This document explains the protocol and the client implementation in btp-client
and the btp
command.
On the Satlab products, BTP is mainly used for downloading payload data and to upload new software images. The remote shell interface can be used for advanced debugging on Linux systems if necessary.
BTP is built around a server task listening on a single port on the subsystem. Please consult the device client library for the BTP port of a specific device.
File Tranfer
The main feature of BTP is tranfer of files to and from the device. BTP works by dividing the transferred file into a number of blocks, and maintaining a bitmap of received blocks on the receiving end. The transfer progress is stored in a map-file that is named after the file being transferred. This allows the transfer to be aborted and resumed, e.g. to download a file over multiple passes of the satellite.
The figure below shows the message sequence for a download. A download starts with the client transmitting a download request to the server. This packet contains the filename to download and the size of the transfer blocks. This allows the block size to be adjusted to match the MTU of the connection to the server. A connection timeout value is also specified on the download request. The download request packet also contains fields to specify the backend to download from and fields to download only a part of a file from a specific offset. Currently, the “file” backend is implemented and the size
and offset
fields are not honored.
The server will reply with a download reply packet, which contains an error field that is used to report problems in the request, plus fields to return the size and CRC32 checksum of the file to be transferred.
Next follows the actual transfer of the file blocks. The client first sends a download status request, that contains sequence number of the last received block and of the block with the largest sequence number received. The status (received or not received) of the blocks between these two sequence numbers are marked in a bitfield. The download status request is acknowledged with a download status reply with a error field. The client should now transmit a block request, that can either request a number of blocks from a specific sequence number or request the next missing blocks as specified by the status packet. The server will then send the requested blocks and go back and wait for a new status request or another block request. This sequence is looped until the file is complete at which point the client should send a complete request to gracefully close the connection.
The file upload sequence is similar to the download sequence, except that some of the packet directions are obviously reversed. Instead of an download request, an upload request should be sent which contains the same fields as the download request plus additional fields to specify the size and CRC32 checksum of the file to be uploaded. An upload request also includes a force
boolean field to command the server to ignore any previous progress of the file, and a paranoid
boolean field which can be used to request all blocks to be flushed to disk as soon as they are received. In order to detect a file that has already been uploaded, the upload reply packet contains the checksum and size of the current file on the server.
The upload request/reply sequence is followed by a upload status request, where the server will reply with a similar sequence number and bitfield packet as in the download case. The client can now transmit block reply packets to the server, while periodically requesting upload status until the file transfer is complete. Again, the transfer is finished with a complete request to close the connection gracefully.
File Operations
In addition to file transfers, BTP also supports basic file operations. The delete, copy and move commands are the simplest and only consists of a single request packet with the backend name, and the filename to be deleted, copied or moved. The server replies with a reply packet that only contains an error code to signal if the file operation was successfully applied or not.
File listing is performed by sending a list request with the backend and path to list, plus the maximum number of file entries to reply with. The server will reply with a list reply packet that includes an error code and the number of file entries found in the path. Next, the server will then send file entry reply packets up to the number requested, or until entry packets have been sent for all files in the path. The client can now request subsequent file entries or retransmission of lost file entry packets by sending a file entry request for a specific sequence number.
Remote Shell
For Linux systems, BTP supports a remote shell over CSP similar to the well-known telnet used on TCP/IP networks. The remote shell can either be used to execute a single command or to run in an interactive session.
A remote shell session is initiated by sending a shell request to the server, containing the command to execute or an empty string to start interactive mode. A shell request contains a 16-bit count
field that is used to limit the maximum number of reply packets the server should send, to prevent accidentally blocking the radio link. A single flag byte in the request packet can be used to request a character-delimited session where a packet is sent for every character entered on the client, instead of waiting for a complete line. This can be used to run remote applications that require the terminal to be in cbreak or raw mode.
The server will return output of the executed output in one or more shell reply packets, each containing an error code and up to 80 characters of output. The replies are sequence numbered and an included flag
byte is used to mark when no more packets are to be expected. The client can then transmit another shell request with a new command. Alternatively, the client can send an empty shell request with the done
bit set in the flag field to end the session.
BTP library and tools
The client BTP protocol as described in the previous sections is implemented in the btp-client
library and used with the satctl
tool. The packet types are all defined in include/btp/types.h
and the client interface is defined in include/btp/client.h
and implemented in src/client.c
. The btp
command, implemented in src/cmd_btp.c
, is a convenient command line tool that wraps the client library.
The following listing shows the output from the command when run without arguments:
[satctl] help btp
usage: btp <command> <remote file> [local file]
Block Transfer Protocol client
<command> is one of push, pull, list, rm, mv, cp or shell.
Options:
-n NODE, Connect to NODE
-p PORT, Connect to PORT
-b BACKEND, Use BACKEND as backend
-s BLOCKSIZE,Transfer data in BLOCKSIZE sized blocks
-c COUNT, Request COUNT blocks at a time
-o OFFSET, Start OFFSET bytes into file
-z SIZE, Download only SIZE bytes
-t TIMEOUT, Set timeout to TIMEOUT ms
-a ATTEMPTS, Retry connection up to ATTEMPTS tries
-l LIMIT, Stop after LIMIT bytes downloaded
-q, Quiet mode. Do not print progress bar
-r, Enable RDP for transfer
-f, Delete data and map file before starting transfer
-d, Paranoid mode. Always sync blocks to disk
-j, Single character terminal mode
The commands btp push
and btp pull
are used to upload and download files. To e.g. download a file called 1M.bin
from the root directory of the system on CSP address 22, use:
[satctl] btp -n22 pull /1M.bin 1M.bin
100% [########################################] 1048576
182.3 kbit/s (1048576 bytes in 44.940s)
The push
command works in the same way, except that the local and remote path are swapped. Similarly, to show the contents of the file /testfile
on address 22 using the remote shell interface, use:
[satctl] btp -n22 shell cat /testfile
This is the contents of testfile
API Reference
Defines
-
BTP_OFFSET_NEXT
Magic offset value to indicate next available block in btp_client_get_blocks/btp_client_send_blocks offset argument.
Typedefs
-
typedef void (*progress_cb)(struct btp_context *btp, void *arg)
Progress callback function.
This function is repeatedly called by the BTP stack when transferring data, and can be used to show the transfer progress.
- Param btp:
BTP context
- Param arg:
Pointer value passed through btp_client_get_blocks/send_blocks.
Functions
-
struct btp_context *btp_client_connect(uint8_t host, uint8_t port, uint32_t timeout, uint32_t opts, unsigned int attempts)
Open new BTP connection to host.
- Parameters:
host – CSP host to connect to
port – CSP port on host with BTP server
timeout – Timeout in milliseconds
opts – Extra CSP connection options
attempts – Number of times to try each BTP operation
- Returns:
A btp_context object on success, NULL on failure
-
int btp_client_disconnect(struct btp_context *btp)
Close an active BTP connection.
- Parameters:
btp – An open btp_context
- Returns:
0 on success, error code on failure
-
int btp_client_upload(struct btp_context *btp, char *backend, char *remotepath, char *localpath, uint8_t block_size, uint32_t timeout_csum, uint32_t timeout_server, bool force)
Begin upload of a file.
- Parameters:
btp – An open btp_context from btp_client_connect
backend – Backend to upload to
remotepath – Absolute path where file should be uploaded
localpath – Absolute path to the file to upload
block_size – Number of bytes to upload per block
force – If true, force upload even if transfer is already complete
- Returns:
0 on success, error code on failure
-
int btp_client_download(struct btp_context *btp, char *backend, char *remotepath, char *localpath, uint8_t block_size, uint32_t timeout_csum, uint32_t timeout_server, bool force)
Begin download of a file.
- Parameters:
btp – An open btp_context from btp_client_connect
backend – Backend to download from
remotepath – Absolute path to remote file to download
localpath – Absolute path where downloaded file should be stored
block_size – Number of bytes to download per block
force – If true, force download even if transfer is already complete
- Returns:
0 on success, error code on failure
-
int btp_client_request_status(struct btp_context *btp)
Update upload status.
- Parameters:
btp – An open btp_context from btp_client_connect
- Returns:
0 on success, error code on failure
-
int btp_client_send_status(struct btp_context *btp)
Send download status.
- Parameters:
btp – An open btp_context from btp_client_connect
- Returns:
0 on success, error code on failure
-
int btp_client_get_blocks(struct btp_context *btp, unsigned int offset, unsigned int count, progress_cb cb, void *cbarg)
Request blocks in download.
- Parameters:
btp – An open btp_context from btp_client_connect
offset – Start from block offset
count – Number of blocks to download
cb – Progress callback function
cbarg – Argument passed to callback
- Returns:
0 on success, error code on failure
-
int btp_client_send_blocks(struct btp_context *btp, unsigned int offset, unsigned int count, progress_cb cb, void *cbarg)
Send blocks in upload.
- Parameters:
btp – An open btp_context from btp_client_connect
offset – Start from block offset
count – Number of blocks to upload
cb – Progress callback function
cbarg – Argument passed to callback
- Returns:
0 on success, error code on failure
-
int btp_client_complete(struct btp_context *btp)
Finish active transfer.
- Parameters:
btp – An open btp_context from btp_client_connect
- Returns:
0 on success, error code on failure
-
bool btp_client_finished(struct btp_context *btp)
Test is tranfer is finished.
- Parameters:
btp – An open btp_context from btp_client_connect
- Returns:
true if transfer is complete, false otherwise
-
int btp_client_remove(struct btp_context *btp, char *backend, char *path)
Remove file.
- Parameters:
btp – An open btp_context from btp_client_connect
backend – Backend to remove file on
path – Absolute path to remote file to remove
- Returns:
0 on success, error code on failure
-
int btp_client_move(struct btp_context *btp, char *backend, char *frompath, char *topath)
Move file.
- Parameters:
btp – An open btp_context from btp_client_connect
backend – Backend to move file on
frompath – Absolute path to file to move
topath – Absolute path where file should be moved
- Returns:
0 on success, error code on failure
-
int btp_client_copy(struct btp_context *btp, char *backend, char *frompath, char *topath)
Copy file.
- Parameters:
btp – An open btp_context from btp_client_connect
backend – Backend to copy file on
frompath – Absolute path to file to copy
topath – Absolute path where file should be copied
- Returns:
0 on success, error code on failure
-
int btp_client_list(struct btp_context *btp, uint16_t count, char *backend, char *name)
List files.
- Parameters:
btp – An open btp_context from btp_client_connect
count – Maximum number of files to list
backend – Backend to list files on
name – Absolute path to list
- Returns:
0 on success, error code on failure
-
int btp_client_shell(struct btp_context *btp, uint16_t count, char *cmd, bool raw)
Create shell connection.
- Parameters:
btp – An open btp_context from btp_client_connect
count – Maximum number of reply packets
cmd – Command to execute. Use cmd[0] = ‘\0’ for interactive mode
raw – If true, transmit one packet per character
- Returns:
0 on success, error code on failure
-
struct btp_context
- #include <client.h>
BTP context struct.
Public Members
-
uint8_t host
-
uint8_t port
-
uint32_t timeout
-
unsigned int attempts
-
bool complete
-
bool raw
-
enum btp_xfer_state state
-
csp_conn_t *conn
-
int fd
-
char filename[BLOB_NAME_SIZE]
-
char filename_partial[BLOB_NAME_SIZE + 5]
-
uint32_t blocks
-
uint32_t block_size
-
uint32_t size
-
uint32_t progress
-
uint32_t checksum
-
uint32_t next
-
uint32_t bits
-
uint8_t *block_status
-
uint8_t host
Enums
-
enum btp_errcode
BTP error codes.
Values:
-
enumerator BTP_EOK
No error.
-
enumerator BTP_EINVAL
Invalid argument.
-
enumerator BTP_ENOENT
No such file or directory.
-
enumerator BTP_ENOSYS
Function not implemented.
-
enumerator BTP_ETIMEDOUT
Operation timed out.
-
enumerator BTP_EPROTO
Protocol error.
-
enumerator BTP_ENXIO
No such device or address.
-
enumerator BTP_EIO
Input/output error.
-
enumerator BTP_ENOMEM
Not enough space.
-
enumerator BTP_ESTALE
Stale file handle.
-
enumerator BTP_EUNKNOWN
Unknown error.
-
enumerator BTP_EEXIST
File exists.
-
enumerator BTP_EDISCON
Disconnect.
-
enumerator BTP_EOK
Functions
-
const char *btp_error(enum btp_errcode err)
Convert BTP error to string.
- Parameters:
err – BTP error code
- Returns:
Pointer to string representation of error, or “Unknown error”
-
enum btp_errcode btp_errno_to_error(int erno)
Convert errno value to BTP error.
- Parameters:
erno – errno value
- Returns:
BTP error value or BTP_EUNKNOWN
Protocol Reference
Defines
-
BACKEND_NAME_SIZE
-
BLOB_NAME_SIZE
-
BTP_BITFIELD_LENGTH
-
SHELL_INP_SIZE
-
SHELL_OUTP_SIZE
-
BTP_DEFAULT_PORT
-
SHELL_FLAG_CHAR
-
SHELL_FLAG_DONE
-
shell_request_size(_chars)
-
shell_reply_size(_chars)
Enums
-
enum btp_xfer_state
Values:
-
enumerator STATE_IDLE
-
enumerator STATE_CONNECTED
-
enumerator STATE_UPLOAD
-
enumerator STATE_DOWNLOAD
-
enumerator STATE_LIST
-
enumerator STATE_IDLE
-
enum btp_type
Values:
-
enumerator BTP_BLOCK_REQUEST
Block request.
-
enumerator BTP_BLOCK_REPLY
Block reply.
-
enumerator BTP_DW_REQUEST
Download blob request.
-
enumerator BTP_DW_REPLY
Download blob reply.
-
enumerator BTP_UP_REQUEST
Upload blob request.
-
enumerator BTP_UP_REPLY
Upload blob reply.
-
enumerator BTP_STAT_PULL_REQUEST
Blob pull status request.
-
enumerator BTP_STAT_PULL_REPLY
Blob pull status reply.
-
enumerator BTP_STAT_PUSH_REQUEST
Blob push status request.
-
enumerator BTP_STAT_PUSH_REPLY
Blob push status reply.
-
enumerator BTP_COMPLETE_REQUEST
Completion request.
-
enumerator BTP_COMPLETE_REPLY
Completion reply.
-
enumerator BTP_LIST_REQUEST
List blobs request.
-
enumerator BTP_LIST_REPLY
List blobs reply.
-
enumerator BTP_ENTRY_REQUEST
Entry request.
-
enumerator BTP_ENTRY_REPLY
Entry reply.
-
enumerator BTP_REMOVE_REQUEST
Remove blob request.
-
enumerator BTP_REMOVE_REPLY
Remove blob reply.
-
enumerator BTP_MOVE_REQUEST
Move blob request.
-
enumerator BTP_MOVE_REPLY
Move blob reply.
-
enumerator BTP_COPY_REQUEST
Copy blob request.
-
enumerator BTP_COPY_REPLY
Copy blob reply.
-
enumerator __BTP_INFO_REQUEST
-
enumerator __BTP_INFO_REPLY
-
enumerator __BTP_LOCK_REQUEST
-
enumerator __BTP_LOCK_REPLY
-
enumerator __BTP_UNLOCK_REQUEST
-
enumerator __BTP_UNLOCK_REPLY
-
enumerator __BTP_GETATTRIB_REQUEST
-
enumerator __BTP_GETATTRIB_REPLY
-
enumerator __BTP_SETATTRIB_REQUEST
-
enumerator __BTP_SETATTRIB_REPLY
-
enumerator BTP_SHELL_REQUEST
Shell request.
-
enumerator BTP_SHELL_REPLY
Shell reply.
-
enumerator BTP_BLOCK_REQUEST
-
struct btp_packet
- #include <types.h>
Generic packet type.
-
struct btp_blockreq
- #include <types.h>
Block pull request.
-
struct btp_blockrep
- #include <types.h>
Block pull reply.
-
struct btp_dwreq
- #include <types.h>
Download request.
-
struct btp_dwrep
- #include <types.h>
Download reply.
-
struct btp_upreq
- #include <types.h>
Upload request.
-
struct btp_uprep
- #include <types.h>
Upload reply.
-
struct btp_stat_pullrequest
- #include <types.h>
Status pull request.
-
struct btp_stat_pullreply
- #include <types.h>
Status pull reply.
-
struct btp_stat_pushrequest
- #include <types.h>
Status push request.
Public Members
-
uint8_t type
Must be BTP_STAT_PUSH_REQUEST.
-
uint8_t type
-
struct btp_stat_pushreply
- #include <types.h>
Status push reply.
-
struct btp_completerequest
- #include <types.h>
Complete request.
Public Members
-
uint8_t type
Must be BTP_COMPLETE_REQUEST.
-
uint8_t type
-
struct btp_completereply
- #include <types.h>
Complete reply.
-
struct btp_listreq
- #include <types.h>
List request.
-
struct btp_listrep
- #include <types.h>
List reply.
-
struct btp_entryreq
- #include <types.h>
Entry request.
-
struct btp_entryrep
- #include <types.h>
Entry reply.
-
struct btp_removereq
- #include <types.h>
Remove request.
-
struct btp_removerep
- #include <types.h>
Remove reply.
-
struct btp_movereq
- #include <types.h>
Move request.
-
struct btp_moverep
- #include <types.h>
Move reply.
-
struct btp_copyreq
- #include <types.h>
Copy request.
-
struct btp_copyrep
- #include <types.h>
Copy reply.
-
struct btp_shellreq
- #include <types.h>
Shell request.
-
struct btp_shellrep
- #include <types.h>
Shell reply.