BTP Client Library

The Blob Transfer Protocol (BTP) is a lightweight file transfer and remote shell protocol based on CSP. BTP supports upload and download of files, as well as basic file operations such as file listing, delete, copy and move. This document explains the protocol and the client implementation in btp-client and the btp command.

On the Satlab products, BTP is mainly used for downloading payload data and to upload new software images. The remote shell interface can be used for advanced debugging on Linux systems if necessary.

BTP is built around a server task listening on a single port on the subsystem. Please consult the device client library for the BTP port of a specific device.

File Tranfer

The main feature of BTP is tranfer of files to and from the device. BTP works by dividing the transferred file into a number of blocks, and maintaining a bitmap of received blocks on the receiving end. The transfer progress is stored in a map-file that is named after the file being transferred. This allows the transfer to be aborted and resumed, e.g. to download a file over multiple passes of the satellite.

The figure below shows the message sequence for a download. A download starts with the client transmitting a download request to the server. This packet contains the filename to download and the size of the transfer blocks. This allows the block size to be adjusted to match the MTU of the connection to the server. A connection timeout value is also specified on the download request. The download request packet also contains fields to specify the backend to download from and fields to download only a part of a file from a specific offset. Currently, the “file” backend is implemented and the size and offset fields are not honored.

The server will reply with a download reply packet, which contains an error field that is used to report problems in the request, plus fields to return the size and CRC32 checksum of the file to be transferred.

../../../_images/btp.svg

Next follows the actual transfer of the file blocks. The client first sends a download status request, that contains sequence number of the last received block and of the block with the largest sequence number received. The status (received or not received) of the blocks between these two sequence numbers are marked in a bitfield. The download status request is acknowledged with a download status reply with a error field. The client should now transmit a block request, that can either request a number of blocks from a specific sequence number or request the next missing blocks as specified by the status packet. The server will then send the requested blocks and go back and wait for a new status request or another block request. This sequence is looped until the file is complete at which point the client should send a complete request to gracefully close the connection.

The file upload sequence is similar to the download sequence, except that some of the packet directions are obviously reversed. Instead of an download request, an upload request should be sent which contains the same fields as the download request plus additional fields to specify the size and CRC32 checksum of the file to be uploaded. An upload request also includes a force boolean field to command the server to ignore any previous progress of the file, and a paranoid boolean field which can be used to request all blocks to be flushed to disk as soon as they are received. In order to detect a file that has already been uploaded, the upload reply packet contains the checksum and size of the current file on the server.

The upload request/reply sequence is followed by a upload status request, where the server will reply with a similar sequence number and bitfield packet as in the download case. The client can now transmit block reply packets to the server, while periodically requesting upload status until the file transfer is complete. Again, the transfer is finished with a complete request to close the connection gracefully.

File Operations

In addition to file transfers, BTP also supports basic file operations. The delete, copy and move commands are the simplest and only consists of a single request packet with the backend name, and the filename to be deleted, copied or moved. The server replies with a reply packet that only contains an error code to signal if the file operation was successfully applied or not.

File listing is performed by sending a list request with the backend and path to list, plus the maximum number of file entries to reply with. The server will reply with a list reply packet that includes an error code and the number of file entries found in the path. Next, the server will then send file entry reply packets up to the number requested, or until entry packets have been sent for all files in the path. The client can now request subsequent file entries or retransmission of lost file entry packets by sending a file entry request for a specific sequence number.

Remote Shell

For Linux systems, BTP supports a remote shell over CSP similar to the well-known telnet used on TCP/IP networks. The remote shell can either be used to execute a single command or to run in an interactive session.

A remote shell session is initiated by sending a shell request to the server, containing the command to execute or an empty string to start interactive mode. A shell request contains a 16-bit count field that is used to limit the maximum number of reply packets the server should send, to prevent accidentally blocking the radio link. A single flag byte in the request packet can be used to request a character-delimited session where a packet is sent for every character entered on the client, instead of waiting for a complete line. This can be used to run remote applications that require the terminal to be in cbreak or raw mode.

The server will return output of the executed output in one or more shell reply packets, each containing an error code and up to 80 characters of output. The replies are sequence numbered and an included flag byte is used to mark when no more packets are to be expected. The client can then transmit another shell request with a new command. Alternatively, the client can send an empty shell request with the done bit set in the flag field to end the session.

BTP library and tools

The client BTP protocol as described in the previous sections is implemented in the btp-client library and used with the satctl tool. The packet types are all defined in include/btp/types.h and the client interface is defined in include/btp/client.h and implemented in src/client.c. The btp command, implemented in src/cmd_btp.c, is a convenient command line tool that wraps the client library.

The following listing shows the output from the command when run without arguments:

[satctl] help btp
usage: btp <command> <remote file> [local file]
Block Transfer Protocol client

<command> is one of push, pull, list, rm, mv, cp or shell.

Options:
 -n NODE,     Connect to NODE
 -p PORT,     Connect to PORT
 -b BACKEND,  Use BACKEND as backend
 -s BLOCKSIZE,Transfer data in BLOCKSIZE sized blocks
 -c COUNT,    Request COUNT blocks at a time
 -o OFFSET,   Start OFFSET bytes into file
 -z SIZE,     Download only SIZE bytes
 -t TIMEOUT,  Set timeout to TIMEOUT ms
 -a ATTEMPTS, Retry connection up to ATTEMPTS tries
 -l LIMIT,    Stop after LIMIT bytes downloaded
 -q,          Quiet mode. Do not print progress bar
 -r,          Enable RDP for transfer
 -f,          Delete data and map file before starting transfer
 -d,          Paranoid mode. Always sync blocks to disk
 -j,          Single character terminal mode

The commands btp push and btp pull are used to upload and download files. To e.g. download a file called 1M.bin from the root directory of the system on CSP address 22, use:

[satctl] btp -n22 pull /1M.bin 1M.bin
100% [########################################] 1048576
182.3 kbit/s (1048576 bytes in 44.940s)

The push command works in the same way, except that the local and remote path are swapped. Similarly, to show the contents of the file /testfile on address 22 using the remote shell interface, use:

[satctl] btp -n22 shell cat /testfile
This is the contents of testfile

API Reference

Defines

BTP_OFFSET_NEXT

Magic offset value to indicate next available block in btp_client_get_blocks/btp_client_send_blocks offset argument.

Typedefs

typedef void (*progress_cb)(struct btp_context *ctx, void *arg)

Progress callback function.

This function is repeatedly called by the BTP stack when transferring data, and can be used to show the transfer progress.

Parameters
  • ctx: BTP context

  • arg: Pointer value passed through btp_client_get_blocks/send_blocks.

Functions

struct btp_context *btp_client_connect(uint8_t host, uint8_t port, uint32_t timeout, uint32_t opts)

Open new BTP connection to host.

Return

A btp_context object on success, NULL on failure

Parameters
  • host: CSP host to connect to

  • port: CSP port on host with BTP server

  • timeout: Timeout in milliseconds

  • opts: Extra CSP connection options

int btp_client_disconnect(struct btp_context *ctx)

Close an active BTP connection.

Return

0 on success, error code on failure

Parameters

int btp_client_upload(struct btp_context *ctx, char *backend, char *filename, char *localfile, uint8_t block_size, uint32_t offset, uint32_t size, uint32_t timeout, bool force, bool paranoid)

Begin upload of a file.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • backend: Backend to upload to

  • filename: Absolute path where file should be uploaded

  • localfile: Absolute path to the file to upload

  • block_size: Number of bytes to upload per block

  • offset: If nonzero, skip the first offset bytes in localfile

  • size: If nonzero, only upload size bytes

  • timeout: Timeout in milliseconds

  • force: If true, force upload even if transfer is already complete

  • paranoid: If true, sync blocks to disk after each block transfer

int btp_client_download(struct btp_context *ctx, char *backend, char *filename, char *localfile, uint8_t block_size, uint32_t offset, uint32_t size, uint32_t timeout, bool force, bool paranoid)

Begin download of a file.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • backend: Backend to download from

  • filename: Absolute path to remote file to download

  • localfile: Absolute path where downloaded file should be stored

  • block_size: Number of bytes to download per block

  • offset: If nonzero, skip the first download bytes in filename

  • size: If nonzero, only download size bytes

  • timeout: Timeout in milliseconds

  • force: If true, force download even if transfer is already complete

  • paranoid: If true, sync blocks to disk after each block transfer

int btp_client_request_status(struct btp_context *ctx, uint32_t timeout)

Update upload status.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • timeout: Timeout in milliseconds

int btp_client_send_status(struct btp_context *ctx, uint32_t timeout)

Send download status.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • timeout: Timeout in milliseconds

int btp_client_get_blocks(struct btp_context *ctx, unsigned int offset, unsigned int count, uint32_t timeout, progress_cb cb, void *cbarg)

Request blocks in download.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • offset: Start from block offset

  • count: Number of blocks to download

  • timeout: Timeout in milliseconds

  • cb: Progress callback function

  • cbarg: Argument passed to callback

int btp_client_send_blocks(struct btp_context *ctx, unsigned int offset, unsigned int count, uint32_t timeout, progress_cb cb, void *cbarg)

Send blocks in upload.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • offset: Start from block offset

  • count: Number of blocks to upload

  • timeout: Timeout in milliseconds

  • cb: Progress callback function

  • cbarg: Argument passed to callback

int btp_client_complete(struct btp_context *ctx, uint32_t timeout)

Finish active transfer.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • timeout: Timeout in milliseconds

bool btp_client_finished(struct btp_context *ctx)

Test is tranfer is finished.

Return

true if transfer is complete, false otherwise

Parameters

int btp_client_remove(struct btp_context *ctx, char *backend, char *path, uint32_t timeout)

Remove file.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • backend: Backend to remove file on

  • path: Absolute path to remote file to remove

  • timeout: Timeout in milliseconds

int btp_client_move(struct btp_context *ctx, char *backend, char *frompath, char *topath, uint32_t timeout)

Move file.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • backend: Backend to move file on

  • frompath: Absolute path to file to move

  • topath: Absolute path where file should be moved

  • timeout: Timeout in milliseconds

int btp_client_copy(struct btp_context *ctx, char *backend, char *frompath, char *topath, uint32_t timeout)

Copy file.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • backend: Backend to copy file on

  • frompath: Absolute path to file to copy

  • topath: Absolute path where file should be copied

  • timeout: Timeout in milliseconds

int btp_client_list(struct btp_context *ctx, uint16_t count, char *backend, char *name, uint32_t timeout)

List files.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • count: Maximum number of files to list

  • backend: Backend to list files on

  • name: Absolute path to list

  • timeout: Timeout in milliseconds

int btp_client_shell(struct btp_context *ctx, uint16_t count, char *cmd, bool raw, uint32_t timeout)

Create shell connection.

Return

0 on success, error code on failure

Parameters
  • ctx: An open btp_context from btp_client_connect

  • count: Maximum number of reply packets

  • cmd: Command to execute. Use cmd[0] = ‘\0’ for interactive mode

  • raw: If true, transmit one packet per character

  • timeout: Timeout in milliseconds

struct btp_context
#include <client.h>

BTP context struct.

Public Members

uint8_t host
uint8_t port
bool connected
bool complete
bool paranoid
bool raw
enum btp_xfer_state state
csp_conn_t *conn
int fd
int fd_map
char filename[BLOB_NAME_SIZE]
char mapname[BLOB_NAME_SIZE + 4]
uint32_t blocks
uint32_t block_size
uint32_t size
uint32_t progress
uint32_t checksum
uint32_t next
uint32_t bits
uint8_t *block_status

Enums

enum btp_errcode

BTP error codes.

Values:

enumerator BTP_EOK

No error.

enumerator BTP_EINVAL

Invalid argument.

enumerator BTP_ENOENT

No such file or directory.

enumerator BTP_ENOSYS

Function not implemented.

enumerator BTP_ETIMEDOUT

Operation timed out.

enumerator BTP_EPROTO

Protocol error.

enumerator BTP_ENXIO

No such device or address.

enumerator BTP_EIO

Input/output error.

enumerator BTP_ENOMEM

Not enough space.

enumerator BTP_ESTALE

Stale file handle.

enumerator BTP_EUNKNOWN

Unknown error.

enumerator BTP_EDISCON

Disconnect.

Functions

const char *btp_error(enum btp_errcode err)

Convert BTP error to string.

Return

Pointer to string representation of error, or “Unknown error”

Parameters
  • err: BTP error code

enum btp_errcode btp_errno_to_error(int erno)

Convert errno value to BTP error.

Return

BTP error value or BTP_EUNKNOWN

Parameters
  • erno: errno value

Protocol Reference

Defines

BACKEND_NAME_SIZE
BLOB_NAME_SIZE
MAP_FORMAT
MAP_SYMBOL_OK
MAP_SYMBOL_MISS
BTP_BITFIELD_LENGTH
SHELL_INP_SIZE
SHELL_OUTP_SIZE
BTP_DEFAULT_PORT
SHELL_FLAG_CHAR
SHELL_FLAG_DONE
shell_request_size(_chars)
shell_reply_size(_chars)

Enums

enum btp_xfer_state

Values:

enumerator STATE_IDLE
enumerator STATE_CONNECTED
enumerator STATE_UPLOAD
enumerator STATE_DOWNLOAD
enumerator STATE_LIST
enum btp_type

Values:

enumerator BTP_BLOCK_REQUEST

Block request.

enumerator BTP_BLOCK_REPLY

Block reply.

enumerator BTP_DW_REQUEST

Download blob request.

enumerator BTP_DW_REPLY

Download blob reply.

enumerator BTP_UP_REQUEST

Upload blob request.

enumerator BTP_UP_REPLY

Upload blob reply.

enumerator BTP_STAT_PULL_REQUEST

Blob pull status request.

enumerator BTP_STAT_PULL_REPLY

Blob pull status reply.

enumerator BTP_STAT_PUSH_REQUEST

Blob push status request.

enumerator BTP_STAT_PUSH_REPLY

Blob push status reply.

enumerator BTP_COMPLETE_REQUEST

Completion request.

enumerator BTP_COMPLETE_REPLY

Completion reply.

enumerator BTP_LIST_REQUEST

List blobs request.

enumerator BTP_LIST_REPLY

List blobs reply.

enumerator BTP_ENTRY_REQUEST

Entry request.

enumerator BTP_ENTRY_REPLY

Entry reply.

enumerator BTP_REMOVE_REQUEST

Remove blob request.

enumerator BTP_REMOVE_REPLY

Remove blob reply.

enumerator BTP_MOVE_REQUEST

Move blob request.

enumerator BTP_MOVE_REPLY

Move blob reply.

enumerator BTP_COPY_REQUEST

Copy blob request.

enumerator BTP_COPY_REPLY

Copy blob reply.

enumerator __BTP_INFO_REQUEST
enumerator __BTP_INFO_REPLY
enumerator __BTP_LOCK_REQUEST
enumerator __BTP_LOCK_REPLY
enumerator __BTP_UNLOCK_REQUEST
enumerator __BTP_UNLOCK_REPLY
enumerator __BTP_GETATTRIB_REQUEST
enumerator __BTP_GETATTRIB_REPLY
enumerator __BTP_SETATTRIB_REQUEST
enumerator __BTP_SETATTRIB_REPLY
enumerator BTP_SHELL_REQUEST

Shell request.

enumerator BTP_SHELL_REPLY

Shell reply.

enum btp_completion

Values:

enumerator BTP_CPLT_OK

File received, close connection.

enumerator BTP_CPLT_CRC32

Reply with CRC32, keep connection open.

enumerator BTP_CPLT_DISCON

Disconnect and keep map.

enumerator BTP_CPLT_ABORT

Remove file and map.

struct btp_packet
#include <types.h>

Generic packet type.

Public Members

uint8_t type
uint8_t err
struct btp_blockreq
#include <types.h>

Block pull request.

Public Members

uint8_t type

Must be BTP_BLOCK_REQUEST.

uint32_t offset
uint32_t count
struct btp_blockrep
#include <types.h>

Block pull reply.

Public Members

uint8_t type

Must be BTP_BLOCK_REPLY.

uint8_t err
uint32_t seq
uint32_t total
uint32_t block
uint8_t data[0]
struct btp_dwreq
#include <types.h>

Download request.

Public Members

uint8_t type

Must be BTP_DW_REQUEST.

char backend[5]
char name[38]
uint8_t block_size
uint32_t offset
uint32_t size
uint32_t timeout
struct btp_dwrep
#include <types.h>

Download reply.

Public Members

uint8_t type

Must be BTP_DW_REPLY.

uint8_t err
uint32_t checksum
uint32_t size
struct btp_upreq
#include <types.h>

Upload request.

Public Members

uint8_t type

Must be BTP_UP_REQUEST.

char backend[5]
char name[38]
uint8_t block_size
uint32_t checksum
uint32_t size
uint32_t timeout
uint8_t force
uint8_t paranoid
struct btp_uprep
#include <types.h>

Upload reply.

Public Members

uint8_t type

Must be BTP_UP_REPLY.

uint8_t err
uint32_t checksum
uint32_t size
struct btp_stat_pullrequest
#include <types.h>

Status pull request.

Public Members

uint8_t type

Must be BTP_STAT_PULL_REQUEST.

uint32_t next
uint32_t bits
uint8_t bitfield[10]
struct btp_stat_pullreply
#include <types.h>

Status pull reply.

Public Members

uint8_t type

Must be BTP_STAT_PULL_REPLY.

uint8_t err
struct btp_stat_pushrequest
#include <types.h>

Status push request.

Public Members

uint8_t type

Must be BTP_STAT_PUSH_REQUEST.

struct btp_stat_pushreply
#include <types.h>

Status push reply.

Public Members

uint8_t type

Must be BTP_STAT_PUSH_REPLY.

uint8_t err
uint32_t next
uint32_t bits
uint8_t bitfield[10]
struct btp_completerequest
#include <types.h>

Complete request.

Public Members

uint8_t type

Must be BTP_COMPLETE_REQUEST.

struct btp_completereply
#include <types.h>

Complete reply.

Public Members

uint8_t type

Must be BTP_COMPLETE_REPLY.

uint8_t err
struct btp_listreq
#include <types.h>

List request.

Public Members

uint8_t type

Must be BTP_LIST_REQUEST.

uint16_t count
char backend[5]
char name[38]
struct btp_listrep
#include <types.h>

List reply.

Public Members

uint8_t type

Must be BTP_LIST_REPLY.

uint8_t err
uint8_t entries
struct btp_entryreq
#include <types.h>

Entry request.

Public Members

uint8_t type

Must be BTP_ENTRY_REQUEST.

uint16_t count
struct btp_entryrep
#include <types.h>

Entry reply.

Public Members

uint8_t type

Must be BTP_ENTRY_REPLY.

uint8_t err
uint8_t seq
char name[38]
uint32_t size
uint32_t mode
uint32_t etype
uint16_t uid
uint16_t gid
uint16_t nlink
uint32_t mtime
struct btp_removereq
#include <types.h>

Remove request.

Public Members

uint8_t type

Must be BTP_REMOVE_REQUEST.

char backend[5]
char name[38]
struct btp_removerep
#include <types.h>

Remove reply.

Public Members

uint8_t type

Must be BTP_REMOVE_REPLY.

uint8_t err
struct btp_movereq
#include <types.h>

Move request.

Public Members

uint8_t type

Must be BTP_MOVE_REQUEST.

char backend[5]
char from[38]
char to[38]
struct btp_moverep
#include <types.h>

Move reply.

Public Members

uint8_t type

Must be BTP_MOVE_REPLY.

uint8_t err
struct btp_copyreq
#include <types.h>

Copy request.

Public Members

uint8_t type

Must be BTP_COPY_REQUEST.

char backend[5]
char from[38]
char to[38]
struct btp_copyrep
#include <types.h>

Copy reply.

Public Members

uint8_t type

Must be BTP_COPY_REPLY.

uint8_t err
struct btp_shellreq
#include <types.h>

Shell request.

Public Members

uint8_t type

Must be BTP_SHELL_REQUEST.

uint8_t flags
uint16_t count
char input[80]
struct btp_shellrep
#include <types.h>

Shell reply.

Public Members

uint8_t type

Must be BTP_SHELL_REPLY.

uint8_t err
uint8_t seq
uint8_t flags
char output[80]