# BTP Client Library

The Blob Transfer Protocol (BTP) is a lightweight file transfer and remote shell protocol based on CSP. BTP supports upload and download of files, as well as basic file operations such as file listing, delete, copy, mkdir, rmdir and move. This document explains the protocol and the client implementation in `btp-client` and the `btp` command.

On the Satlab products, BTP is mainly used for downloading payload data and to upload new software images. The remote shell interface can be used for advanced debugging on Linux systems if necessary.

BTP is built around a server task listening on a single port on the subsystem. Please consult the device client library for the BTP port of a specific device.

## File Transfer

The main feature of BTP is transfer of files to and from the device. BTP works by dividing the transferred file into a number of blocks, and maintaining a bitmap of received blocks on the receiving end. The transfer progress is stored in a map-file that is named after the file being transferred. This allows the transfer to be aborted and resumed, e.g. to download a file over multiple passes of the satellite.

The figure below shows the message sequence for a download. A download starts with the client transmitting a download request to the server. This packet contains the filename to download and the size of the transfer blocks. This allows the block size to be adjusted to match the MTU of the connection to the server. A connection timeout value is also specified on the download request. The download request packet also contains fields to specify the backend to download from and fields to download only a part of a file from a specific offset. Currently, the “file” backend is implemented and the `size` and `offset` fields are not honored.

The server will reply with a download reply packet, which contains an error field that is used to report problems in the request, plus fields to return the size and CRC32 checksum of the file to be transferred.

Next follows the actual transfer of the file blocks. The client first sends a download status request, that contains sequence number of the last received block and of the block with the largest sequence number received. The status (received or not received) of the blocks between these two sequence numbers are marked in a bitfield. The download status request is acknowledged with a download status reply with an error field. The client should now transmit a block request, that can either request a number of blocks from a specific sequence number or request the next missing blocks as specified by the status packet. The server will then send the requested blocks and go back and wait for a new status request or another block request. This sequence is looped until the file is complete at which point the client should send a complete request to gracefully close the connection.

The file upload sequence is similar to the download sequence, except that some of the packet directions are obviously reversed. Instead of a download request, an upload request should be sent which contains the same fields as the download request plus additional fields to specify the size and CRC32 checksum of the file to be uploaded. An upload request also includes a `force` boolean field to command the server to ignore any previous progress of the file. In order to detect a file that has already been uploaded, the upload reply packet contains the checksum and size of the current file on the server.

The upload request/reply sequence is followed by an upload status request, where the server will reply with a similar sequence number and bitfield packet as in the download case. The client can now transmit block reply packets to the server, while periodically requesting upload status until the file transfer is complete. Again, the transfer is finished with a complete request to close the connection gracefully.

## File Operations

In addition to file transfers, BTP also supports basic file operations. Delete, copy and move commands are the simplest and only consists of a single request packet with the backend name, and the filename to be deleted, copied or moved. The server replies with a reply packet that only contains an error code to signal if the file operation was successfully applied or not.

File listing is performed by sending a list request with the backend and path to list, plus the maximum number of file entries to reply with. The server will reply with a list reply packet that includes an error code and the number of file entries found in the path. Next, the server will then send file entry reply packets up to the number requested, or until entry packets have been sent for all files in the path. The client can now request subsequent file entries or retransmission of lost file entry packets by sending a file entry request for a specific sequence number.

## Remote Shell

For Linux systems, BTP supports a remote shell over CSP similar to the well-known telnet used on TCP/IP networks. The remote shell can either be used to execute a single command or to run in an interactive session.

A remote shell session is initiated by sending a shell request to the server, containing the command to execute or an empty string to start interactive mode. A shell request contains a 16-bit `count` field that is used to limit the maximum number of reply packets the server should send, to prevent accidentally blocking the radio link. A single flag byte in the request packet can be used to request a character-delimited session where a packet is sent for every character entered on the client, instead of waiting for a complete line. This can be used to run remote applications that require the terminal to be in cbreak or raw mode.

The server will return output of the executed output in one or more shell reply packets, each containing an error code and up to 80 characters of output. The replies are sequence numbered and an included `flag` byte is used to mark when no more packets are to be expected. The client can then transmit another shell request with a new command. Alternatively, the client can send an empty shell request with the `done` bit set in the flag field to end the session.

## BTP library and tools

The client BTP protocol as described in the previous sections is implemented in the `btp-client` library and used with the `satctl` tool. The packet types are all defined in `include/btp/types.h` and the client interface is defined in `include/btp/client.h` and implemented in `src/client.c`. The `btp` command, implemented in `src/cmd_btp.c`, is a convenient command line tool that wraps the client library.

The following listing shows the output from the command when run without arguments:

    [satctl] help btp
    usage: btp <command> <remote file> [local file]
    Block Transfer Protocol client

    <command> is one of push, pull, list, remove, move, copy, mkdir, rmdir or shell.

    Options:
     -p PORT,     Connect to PORT
     -b BACKEND,  Use BACKEND as backend
     -s BLOCKSIZE,Transfer data in BLOCKSIZE sized blocks
     -c COUNT,    Request COUNT blocks at a time
     -o OFFSET,   Start OFFSET bytes into file
     -z SIZE,     Download only SIZE bytes
     -t TIMEOUT,  Set timeout to TIMEOUT ms
     -a ATTEMPTS, Retry connection up to ATTEMPTS tries
     -q,          Quiet mode. Do not print progress bar
     -r,          Enable RDP for transfer
     -f,          Delete data and map file before starting transfer
     -j,          Single character terminal mode

The commands `btp push` and `btp pull` are used to upload and download files. To e.g. download a file called `1M.bin` from the root directory of the system on CSP address 22, use:

    [satctl] btp pull 22 /1M.bin 1M.bin
    100% [########################################] 1048576
    275.2 kbit/s (1048576 bytes in 29.771s)

The `push` command works in the same way, except that the local and remote path are swapped. Similarly, to show the contents of the file `/testfile` on address 22 using the remote shell interface, use:

    [satctl] btp shell 22 cat /testfile
    This is the contents of testfile

Note that setting the count option to a value above 25 is not recommended when transferring BTP traffic via a UDP link as this will degrade performance.

## API Reference

Defines

BTP_OFFSET_NEXT

  Magic offset value to indicate next available block in btp_client_get_blocks/btp_client_send_blocks offset argument.

BITS_PER_BYTE

Typedefs

typedef void (*btp_progress_cb)(struct btp_context *btp, void *arg)

  Progress callback function.

  This function is repeatedly called by the BTP stack when transferring data, and can be used to show the transfer progress.

  Param btp:

    BTP context

  Param arg:

    Pointer value passed through btp_client_get_blocks/send_blocks.

typedef int (*btp_list_cb)(struct btp_entryrep *erep, void *arg)

Functions

struct btp_context *btp_client_connect(uint8_t host, uint8_t port, uint32_t timeout, uint32_t opts, unsigned int attempts)

  Open new BTP connection to host.

  Parameters:

    - host – CSP host to connect to

    - port – CSP port on host with BTP server

    - timeout – Timeout in milliseconds

    - opts – Extra CSP connection options

    - attempts – Number of times to try each BTP operation

  Returns:

    A btp_context object on success, NULL on failure

int btp_client_disconnect(struct btp_context *btp)

  Close an active BTP connection.

  Parameters:

    btp – An open btp_context

  Returns:

    0 on success, error code on failure

int btp_client_upload(struct btp_context *btp, const char *backend, const char *remotepath, const char *localpath, uint8_t block_size, uint32_t timeout_csum, uint32_t timeout_server, bool force)

  Begin upload of a file.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - backend – Backend to upload to

    - remotepath – Absolute path where file should be uploaded

    - localpath – Absolute path to the file to upload

    - block_size – Number of bytes to upload per block

    - timeout_csum – Timeout for calculating checksum

    - timeout_server – Server-side timeout

    - force – If true, force upload even if transfer is already complete

  Returns:

    0 on success, error code on failure

int btp_client_download(struct btp_context *btp, const char *backend, const char *remotepath, const char *localpath, uint8_t block_size, uint32_t timeout_csum, uint32_t timeout_server, bool force)

  Begin download of a file.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - backend – Backend to download from

    - remotepath – Absolute path to remote file to download

    - localpath – Absolute path where downloaded file should be stored

    - block_size – Number of bytes to download per block

    - timeout_csum – Timeout for calculating checksum

    - timeout_server – Server-side timeout

    - force – If true, force download even if transfer is already complete

  Returns:

    0 on success, error code on failure

int btp_client_request_status(struct btp_context *btp)

  Update upload status.

  Parameters:

    btp – An open btp_context from btp_client_connect

  Returns:

    0 on success, error code on failure

int btp_client_send_status(struct btp_context *btp)

  Send download status.

  Parameters:

    btp – An open btp_context from btp_client_connect

  Returns:

    0 on success, error code on failure

int btp_client_get_blocks(struct btp_context *btp, unsigned int offset, unsigned int count, btp_progress_cb cb, void *cbarg)

  Request blocks in download.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - offset – Start from block offset

    - count – Number of blocks to download

    - cb – Progress callback function

    - cbarg – Argument passed to callback

  Returns:

    0 on success, error code on failure

int btp_client_send_blocks(struct btp_context *btp, unsigned int offset, unsigned int count, btp_progress_cb cb, void *cbarg)

  Send blocks in upload.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - offset – Start from block offset

    - count – Number of blocks to upload

    - cb – Progress callback function

    - cbarg – Argument passed to callback

  Returns:

    0 on success, error code on failure

int btp_client_complete(struct btp_context *btp, uint32_t timeout_csum)

  Finish active transfer.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - timeout_csum – Timeout for calculating checksum

  Returns:

    0 on success, error code on failure

bool btp_client_finished(struct btp_context *btp)

  Test is tranfer is finished.

  Parameters:

    btp – An open btp_context from btp_client_connect

  Returns:

    true if transfer is complete, false otherwise

int btp_client_remove(struct btp_context *btp, const char *backend, const char *path)

  Remove file.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - backend – Backend to remove file on

    - path – Absolute path to remote file to remove

  Returns:

    0 on success, error code on failure

int btp_client_move(struct btp_context *btp, const char *backend, const char *frompath, const char *topath)

  Move file.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - backend – Backend to move file on

    - frompath – Absolute path to file to move

    - topath – Absolute path where file should be moved

  Returns:

    0 on success, error code on failure

int btp_client_copy(struct btp_context *btp, const char *backend, const char *frompath, const char *topath)

  Copy file.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - backend – Backend to copy file on

    - frompath – Absolute path to file to copy

    - topath – Absolute path where file should be copied

  Returns:

    0 on success, error code on failure

int btp_client_list(struct btp_context *btp, uint16_t count, const char *backend, const char *name, btp_list_cb cb, void *cbarg)

  List files.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - count – Maximum number of files to list

    - backend – Backend to list files on

    - name – Absolute path to list

    - cb – Progress callback function

    - cbarg – Argument passed to callback

  Returns:

    0 on success, error code on failure

int btp_client_shell(struct btp_context *btp, uint16_t count, const char *cmd, bool raw)

  Create shell connection.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - count – Maximum number of reply packets

    - cmd – Command to execute. Use cmd[0] = ‘\0’ for interactive mode

    - raw – If true, transmit one packet per character

  Returns:

    0 on success, error code on failure

int btp_client_mkdir(struct btp_context *btp, const char *backend, const char *path)

  Make directory.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - backend – Backend to make directory on

    - path – Absolute path to the directory to make

  Returns:

    0 on success, error code on failure

int btp_client_rmdir(struct btp_context *btp, const char *backend, const char *path)

  Remove directory.

  Parameters:

    - btp – An open btp_context from btp_client_connect

    - backend – Backend to remove directory on

    - path – Absolute path to the directory to remove

  Returns:

    0 on success, error code on failure

struct block_status

  Public Members

  bool complete
  uint32_t blocks
  uint32_t block_size
  uint32_t next
  uint32_t bits
  uint8_t block_status[BTP_BITFIELD_LENGTH]

struct btp_context

  Public Members

  uint8_t host
  uint8_t port
  uint32_t timeout
  unsigned int attempts
  bool raw
  enum btp_xfer_state state
  csp_conn_t *conn
  int fd
  char filename[BLOB_NAME_SIZE]
  char filename_partial[BLOB_NAME_SIZE + 5]
  struct block_status block_status
  uint32_t size
  uint32_t progress
  uint32_t checksum

Enums

enum btp_errcode

  BTP error codes.

  Values:

  enumerator BTP_EOK

    No error.

  enumerator BTP_EINVAL

    Invalid argument.

  enumerator BTP_ENOENT

    No such file or directory.

  enumerator BTP_ENOSYS

    Function not implemented.

  enumerator BTP_ETIMEDOUT

    Operation timed out.

  enumerator BTP_EPROTO

    Protocol error.

  enumerator BTP_ENXIO

    No such device or address.

  enumerator BTP_EIO

    Input/output error.

  enumerator BTP_ENOMEM

    Not enough space.

  enumerator BTP_ESTALE

    Stale file handle.

  enumerator BTP_EUNKNOWN

    Unknown error.

  enumerator BTP_EEXIST

    File exists.

  enumerator BTP_EINTR

    Keyboard interrupt.

  enumerator BTP_EDISCON

    Disconnect.

Functions

const char *btp_error(enum btp_errcode err)

  Convert BTP error to string.

  Parameters:

    err – BTP error code

  Returns:

    Pointer to string representation of error, or “Unknown error”

enum btp_errcode btp_errno_to_error(int erno)

  Convert errno value to BTP error.

  Parameters:

    erno – errno value

  Returns:

    BTP error value or BTP_EUNKNOWN

## Protocol Reference

Defines

BACKEND_NAME_SIZE

BLOB_NAME_SIZE

BTP_BITFIELD_LENGTH

SHELL_INP_SIZE

SHELL_OUTP_SIZE

BTP_DEFAULT_PORT

SHELL_FLAG_CHAR

SHELL_FLAG_DONE

shell_request_size(_chars)

shell_reply_size(_chars)

Enums

enum btp_xfer_state

  Values:

  enumerator STATE_IDLE
  enumerator STATE_CONNECTED
  enumerator STATE_UPLOAD
  enumerator STATE_DOWNLOAD
  enumerator STATE_LIST

enum btp_type

  Values:

  enumerator BTP_BLOCK_REQUEST

    Block request.

  enumerator BTP_BLOCK_REPLY

    Block reply.

  enumerator BTP_DW_REQUEST

    Download blob request.

  enumerator BTP_DW_REPLY

    Download blob reply.

  enumerator BTP_UP_REQUEST

    Upload blob request.

  enumerator BTP_UP_REPLY

    Upload blob reply.

  enumerator BTP_STAT_PULL_REQUEST

    Blob pull status request.

  enumerator BTP_STAT_PULL_REPLY

    Blob pull status reply.

  enumerator BTP_STAT_PUSH_REQUEST

    Blob push status request.

  enumerator BTP_STAT_PUSH_REPLY

    Blob push status reply.

  enumerator BTP_COMPLETE_REQUEST

    Completion request.

  enumerator BTP_COMPLETE_REPLY

    Completion reply.

  enumerator BTP_LIST_REQUEST

    List blobs request.

  enumerator BTP_LIST_REPLY

    List blobs reply.

  enumerator BTP_ENTRY_REQUEST

    Entry request.

  enumerator BTP_ENTRY_REPLY

    Entry reply.

  enumerator BTP_REMOVE_REQUEST

    Remove blob request.

  enumerator BTP_REMOVE_REPLY

    Remove blob reply.

  enumerator BTP_MOVE_REQUEST

    Move blob request.

  enumerator BTP_MOVE_REPLY

    Move blob reply.

  enumerator BTP_COPY_REQUEST

    Copy blob request.

  enumerator BTP_COPY_REPLY

    Copy blob reply.

  enumerator __BTP_INFO_REQUEST
  enumerator __BTP_INFO_REPLY
  enumerator __BTP_LOCK_REQUEST
  enumerator __BTP_LOCK_REPLY
  enumerator __BTP_UNLOCK_REQUEST
  enumerator __BTP_UNLOCK_REPLY
  enumerator __BTP_GETATTRIB_REQUEST
  enumerator __BTP_GETATTRIB_REPLY
  enumerator __BTP_SETATTRIB_REQUEST
  enumerator __BTP_SETATTRIB_REPLY
  enumerator BTP_SHELL_REQUEST

    Shell request.

  enumerator BTP_SHELL_REPLY

    Shell reply.

  enumerator BTP_MKDIR_REQUEST

    Mkdir blob request.

  enumerator BTP_MKDIR_REPLY

    Mkdir blob reply.

  enumerator BTP_RMDIR_REQUEST

    Rmdir blob request.

  enumerator BTP_RMDIR_REPLY

    Rmdir blob reply.

enum btp_completion

  Values:

  enumerator BTP_CPLT_OK

    File received, close connection.

  enumerator BTP_CPLT_CRC32

    Reply with CRC32, keep connection open.

  enumerator BTP_CPLT_DISCON

    Disconnect and keep map.

  enumerator BTP_CPLT_ABORT

    Remove file and map.

struct btp_packet
  #include <types.h>

  Generic packet type.

  Public Members

  uint8_t type
  uint8_t err

struct btp_blockreq
  #include <types.h>

  Block pull request.

  Public Members

  uint8_t type

    Must be BTP_BLOCK_REQUEST.

  uint32_t offset
  uint32_t count

struct btp_blockrep
  #include <types.h>

  Block pull reply.

  Public Members

  uint8_t type

    Must be BTP_BLOCK_REPLY.

  uint8_t err
  uint32_t seq
  uint32_t total
  uint32_t block
  uint8_t data[0]

struct btp_dwreq
  #include <types.h>

  Download request.

  Public Members

  uint8_t type

    Must be BTP_DW_REQUEST.

  char backend[5]
  char name[38]
  uint8_t block_size
  uint32_t deprecated1

    Deprecated, set to 0.

  uint32_t deprecated2

    Deprecated, set to 0.

  uint32_t timeout

struct btp_dwrep
  #include <types.h>

  Download reply.

  Public Members

  uint8_t type

    Must be BTP_DW_REPLY.

  uint8_t err
  uint32_t checksum
  uint32_t size

struct btp_upreq
  #include <types.h>

  Upload request.

  Public Members

  uint8_t type

    Must be BTP_UP_REQUEST.

  char backend[5]
  char name[38]
  uint8_t block_size
  uint32_t checksum
  uint32_t size
  uint32_t timeout
  uint8_t force
  uint8_t paranoid

struct btp_uprep
  #include <types.h>

  Upload reply.

  Public Members

  uint8_t type

    Must be BTP_UP_REPLY.

  uint8_t err
  uint32_t checksum
  uint32_t size

struct btp_stat_pullrequest
  #include <types.h>

  Status pull request.

  Public Members

  uint8_t type

    Must be BTP_STAT_PULL_REQUEST.

  uint32_t next
  uint32_t bits
  uint8_t bitfield[10]

struct btp_stat_pullreply
  #include <types.h>

  Status pull reply.

  Public Members

  uint8_t type

    Must be BTP_STAT_PULL_REPLY.

  uint8_t err

struct btp_stat_pushrequest
  #include <types.h>

  Status push request.

  Public Members

  uint8_t type

    Must be BTP_STAT_PUSH_REQUEST.

struct btp_stat_pushreply
  #include <types.h>

  Status push reply.

  Public Members

  uint8_t type

    Must be BTP_STAT_PUSH_REPLY.

  uint8_t err
  uint32_t next
  uint32_t bits
  uint8_t bitfield[10]

struct btp_completerequest
  #include <types.h>

  Complete request.

  Public Members

  uint8_t type

    Must be BTP_COMPLETE_REQUEST.

struct btp_completereply
  #include <types.h>

  Complete reply.

  Public Members

  uint8_t type

    Must be BTP_COMPLETE_REPLY.

  uint8_t err

struct btp_listreq
  #include <types.h>

  List request.

  Public Members

  uint8_t type

    Must be BTP_LIST_REQUEST.

  uint16_t count
  char backend[5]
  char name[38]

struct btp_listrep
  #include <types.h>

  List reply.

  Public Members

  uint8_t type

    Must be BTP_LIST_REPLY.

  uint8_t err
  uint8_t entries

struct btp_entryreq
  #include <types.h>

  Entry request.

  Public Members

  uint8_t type

    Must be BTP_ENTRY_REQUEST.

  uint16_t count

struct btp_entryrep
  #include <types.h>

  Entry reply.

  Public Members

  uint8_t type

    Must be BTP_ENTRY_REPLY.

  uint8_t err
  uint8_t seq
  char name[38]
  uint32_t size
  uint32_t mode
  uint32_t etype
  uint16_t uid
  uint16_t gid
  uint16_t nlink
  uint32_t mtime

struct btp_removereq
  #include <types.h>

  Remove request.

  Public Members

  uint8_t type

    Must be BTP_REMOVE_REQUEST.

  char backend[5]
  char name[38]

struct btp_removerep
  #include <types.h>

  Remove reply.

  Public Members

  uint8_t type

    Must be BTP_REMOVE_REPLY.

  uint8_t err

struct btp_movereq
  #include <types.h>

  Move request.

  Public Members

  uint8_t type

    Must be BTP_MOVE_REQUEST.

  char backend[5]
  char from[38]
  char to[38]

struct btp_moverep
  #include <types.h>

  Move reply.

  Public Members

  uint8_t type

    Must be BTP_MOVE_REPLY.

  uint8_t err

struct btp_copyreq
  #include <types.h>

  Copy request.

  Public Members

  uint8_t type

    Must be BTP_COPY_REQUEST.

  char backend[5]
  char from[38]
  char to[38]

struct btp_copyrep
  #include <types.h>

  Copy reply.

  Public Members

  uint8_t type

    Must be BTP_COPY_REPLY.

  uint8_t err

struct btp_shellreq
  #include <types.h>

  Shell request.

  Public Members

  uint8_t type

    Must be BTP_SHELL_REQUEST.

  uint8_t flags
  uint16_t count
  char input[80]

struct btp_shellrep
  #include <types.h>

  Shell reply.

  Public Members

  uint8_t type

    Must be BTP_SHELL_REPLY.

  uint8_t err
  uint8_t seq
  uint8_t flags
  char output[80]

struct btp_mkdirreq
  #include <types.h>

  Mkdir request.

  Public Members

  uint8_t type

    Must be BTP_MKDIR_REQUEST.

  char backend[5]
  char name[38]

struct btp_mkdirrep
  #include <types.h>

  Mkdir reply.

  Public Members

  uint8_t type

    Must be BTP_MKDIR_REPLY.

  uint8_t err

struct btp_rmdirreq
  #include <types.h>

  Rmdir request.

  Public Members

  uint8_t type

    Must be BTP_RMDIR_REQUEST.

  char backend[5]
  char name[38]

struct btp_rmdirrep
  #include <types.h>

  Rmdir reply.

  Public Members

  uint8_t type

    Must be BTP_RMDIR_REPLY.

  uint8_t err