blackbox_logger.c File Reference

Implementation of the blackbox logger. More...

#include "blackbox_logger_internal.h"
#include "esp_log.h"
#include "utils.h"

Include dependency graph for blackbox_logger.c:

Functions
static blackbox_logger_err_t	is_data_entry_internal (const uint8_t *data_ptr, const size_t data_size)
	Check if valid data entry.
static blackbox_logger_err_t	validate_entry_internal (const uint8_t *data_ptr, const size_t data_size)
	Validate entry.
size_t	get_entries_chunk_size_with_ceiling_helper (const uint8_t *buf_ptr, const size_t buf_size, const size_t ceiling_size, const size_t align_to)
	Get maximum chunk size of entries with ceiling.
static void	timer_flush_flash_internal (void *logger_ptr)
	Flush flash callback for timer.
static blackbox_logger_err_t	update_parameters_internal (const blackbox_logger_t *logger_ptr)
	Update Logger configuration parameters structure.
static blackbox_logger_err_t	esp_to_blackbox_error_map (const esp_err_t err)
	Maps esp errors to logger errors.
static blackbox_logger_err_t	binary_search_recover_helper (const blackbox_logger_t *logger_ptr)
	Binary search partition sectors beginnings to find maximum ID.
static blackbox_logger_err_t	linear_recover_helper (const blackbox_logger_t *logger_ptr)
	Search linearly from given logger parameters until maximum ID is found.
static blackbox_logger_err_t	recover_logger_internal (blackbox_logger_t *logger_ptr)
	Recover Logger parameters.
blackbox_logger_err_t	blackbox_logger_init (blackbox_logger_t *logger_ptr, blackbox_logger_config_t *config_ptr)
	Initialize BlackBox Logger.
blackbox_logger_err_t	blackbox_logger_write (blackbox_logger_t *logger_ptr, const blackbox_logger_entry_view_t *entry_ptr)
	Write an entry to flash.
blackbox_logger_err_t	blackbox_logger_read (const blackbox_logger_t *logger_ptr, const uint32_t read_offset, blackbox_logger_entry_view_t *entry_ptr, const size_t entry_buf_max_capacity)
	Read an entry from flash.
blackbox_logger_err_t	blackbox_logger_flush (blackbox_logger_t *logger_ptr)
	Flush buffered data to flash.
blackbox_logger_err_t	blackbox_logger_next_replay (blackbox_logger_t *logger_ptr, blackbox_logger_iter_cb_t is_entry_processed_cb_func, void *cb_arg_void_ptr, blackbox_logger_entry_view_t *entry_ptr, const size_t entry_buf_max_capacity)
	Replay next available log entry.
blackbox_logger_err_t	blackbox_logger_iterate_replay (blackbox_logger_t *logger_ptr, blackbox_logger_iter_cb_t is_entry_processed_cb_func, void *cb_arg_void_ptr, blackbox_logger_entry_view_t *entry_ptr, const size_t entry_buf_max_capacity)
	Iterate over available log entries.
void	blackbox_logger_get_parameters_defaults (blackbox_logger_parameters_t *parameter_ptr)
	Get Logger parameters defaults.
bool	blackbox_logger_has_replay_data (const blackbox_logger_t *logger_ptr)
	Is log data available for replay.
void	blackbox_logger_deinit (blackbox_logger_t *logger_ptr)
	De-init BlackBox Logger.

Variables
static const char *	TAG = "BB_LOG"
static blackbox_logger_parameters_t	blackbox_logger_default_parameters

Detailed Description

Implementation of the blackbox logger.

Author

Abanoub Salah

• This module uses a circular buffer approach to ensure we only perform block-erases when necessary, extending flash life
• Recovery done with linear search starting from beginning if provided hint prove invalid
• Write done using local memory buffer 'batch buffer' before flushing. flushing is forced under
  • Buffer is full or
  • Preset-Time passed since last buffer write
• Read done using a user-provided memory buffer

Function Documentation

binary_search_recover_helper()

static blackbox_logger_err_t binary_search_recover_helper ( const blackbox_logger_t *  logger_ptr )

static

Binary search partition sectors beginnings to find maximum ID.

Parameters

logger_ptr

Pointer to BlackBox Logger instance

Returns

blackbox_logger_err_t Search result

Return values

BLACKBOX_LOGGER_OK	Search success
BLACKBOX_LOGGER_ERR_FAIL	Search fail
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)

Note

Logger's parameters instance updated on success

Here is the call graph for this function:

Here is the caller graph for this function:

blackbox_logger_deinit()

void blackbox_logger_deinit

(

blackbox_logger_t *

logger_ptr

)

De-init BlackBox Logger.

De-init BlackBox Logger by flushing then stopping and deleting the timer and free-up used resources

Parameters

logger_ptr

Pointer to BlackBox Logger instance

Here is the call graph for this function:

Here is the caller graph for this function:

blackbox_logger_flush()

blackbox_logger_err_t blackbox_logger_flush

(

blackbox_logger_t *

logger_ptr

)

Flush buffered data to flash.

• Flush buffered data to flash taking into account wrap-around at flash-end
• Calls notify callback if set and replay available
• Log entry format is as follows
  • [HEADER][PAYLOAD][TAIL][ALIGNMENT]...[HEADER][PAYLOAD][TAIL][ALIGNMENT]
• Where:
  • [HEADER]: Is a prefix to the payload with
    • Header magic number
    • Monotonically increasing ID
    • Payload length
  • [PAYLOAD]: Entry data
  • [TAIL]: Is a postfix to the payload with
    • CRC-16-Modbus for [HEADER][PAYLOAD]
    • Tail magic number
  • [ALIGNMENT]: Stuffing to ensure next entry is aligned with system imposed alignment

Parameters

logger_ptr

Pointer to BlackBox Logger instance

Returns

blackbox_logger_err_t Flush result

Return values

BLACKBOX_LOGGER_OK	Flush success
BLACKBOX_LOGGER_ERR_FAIL	Failed to write/erase partition
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)
BLACKBOX_LOGGER_ERR_NO_MEM	Flash is full
BLACKBOX_LOGGER_ERR_TIMEOUT	Timed out waiting for resource

Note

Writes won't extend to next sector to prevent entry fragmentation that means every sector beginning may have a start of an entry and will fail if entry is bigger than sector length

Function does not write buffer as a whole chunk it tries to fill available sector space first if possible to save on space

- Replay/Write offsets interaction behavior is configurable in kconfig

• Keep old entries: Returns with 'no memory' error code
• Replace old entries: Advance replay offset to the beginning of next write sector jumping-over any replay leftover in sector since they will get erased with the whole sector for coming fresh writes

Here is the call graph for this function:

Here is the caller graph for this function:

blackbox_logger_get_parameters_defaults()

void blackbox_logger_get_parameters_defaults

(

blackbox_logger_parameters_t *

parameter_ptr

)

Get Logger parameters defaults.

Parameters

parameter_ptr

Pointer to parameters structure

Here is the caller graph for this function:

blackbox_logger_has_replay_data()

bool blackbox_logger_has_replay_data

(

const blackbox_logger_t *

logger_ptr

)

Is log data available for replay.

Parameters

logger_ptr

Pointer to BlackBox Logger instance

Returns

true if data available, false otherwise

Here is the caller graph for this function:

blackbox_logger_init()

blackbox_logger_err_t blackbox_logger_init	(	blackbox_logger_t *	logger_ptr,
		blackbox_logger_config_t *	config_ptr
	)

Initialize BlackBox Logger.

Initialize logger with provided configurations using parameters as hints for recover also setup a one-shot timer with a preset-time for flushing buffered data to flash

Parameters

logger_ptr	Pointer to BlackBox Logger instance
config_ptr	Pointer to BlackBox Logger configuration instance

Returns

blackbox_logger_err_t Initialize result

Return values

BLACKBOX_LOGGER_OK	Initialize success
BLACKBOX_FAIL	Initialize Fail
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)
BLACKBOX_LOGGER_ERR_NO_MEM	No available memory for resource allocation

Here is the call graph for this function:

Here is the caller graph for this function:

blackbox_logger_iterate_replay()

blackbox_logger_err_t blackbox_logger_iterate_replay	(	blackbox_logger_t *	logger_ptr,
		blackbox_logger_iter_cb_t	is_entry_processed_cb_func,
		void *	cb_arg_void_ptr,
		blackbox_logger_entry_view_t *	entry_ptr,
		const size_t	entry_buf_max_capacity
	)

Iterate over available log entries.

Iterate over available log entries calling a callback function on success stopping-on empty log or when callback returns false

Parameters

logger_ptr	Pointer to BlackBox Logger instance
is_entry_processed_cb_func	Callback for when entry is available
cb_arg_void_ptr	Void pointer to callback argument
entry_ptr	Pointer to entry to read to
entry_buf_max_capacity	Provided buffer maximum capacity

Returns

blackbox_logger_err_t Replay result

Return values

BLACKBOX_LOGGER_OK	Replay success
BLACKBOX_LOGGER_ERR_FAIL	Failed to read from partition
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)
BLACKBOX_LOGGER_ERR_NO_MEM	No available memory for entry data
BLACKBOX_LOGGER_ERR_TIMEOUT	Timed out waiting for resource
BLACKBOX_LOGGER_ERR_INVALID_CRC	Invalid CRC for the entry
BLACKBOX_LOGGER_ERR_INVALID_ENTRY	Invalid entry found
BLACKBOX_LOGGER_ERR_PROCESS_FAIL	Callback refused/failed to process entry

Note

See blackbox_logger_next_replay notes since this function loop call it

Here is the call graph for this function:

blackbox_logger_next_replay()

blackbox_logger_err_t blackbox_logger_next_replay	(	blackbox_logger_t *	logger_ptr,
		blackbox_logger_iter_cb_t	is_entry_processed_cb_func,
		void *	cb_arg_void_ptr,
		blackbox_logger_entry_view_t *	entry_ptr,
		const size_t	entry_buf_max_capacity
	)

Replay next available log entry.

calling a callback function on next available log entry then advance replay offset on callback returns true otherwise returns with error code

Parameters

logger_ptr	Pointer to BlackBox Logger instance
is_entry_processed_cb_func	Callback for when entry is available
cb_arg_void_ptr	Void pointer to callback argument
entry_ptr	Pointer to entry to read to
entry_buf_max_capacity	Provided buffer maximum capacity

Returns

blackbox_logger_err_t Replay result

Return values

BLACKBOX_LOGGER_OK	Replay success
BLACKBOX_LOGGER_ERR_FAIL	Failed to read from partition
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)
BLACKBOX_LOGGER_ERR_NO_MEM	No available memory for entry data
BLACKBOX_LOGGER_ERR_TIMEOUT	Timed out waiting for resource
BLACKBOX_LOGGER_ERR_INVALID_CRC	Invalid CRC for the entry
BLACKBOX_LOGGER_ERR_INVALID_ENTRY	Invalid entry found
BLACKBOX_LOGGER_ERR_PROCESS_FAIL	Callback refused/failed to process entry

Note

Function assumes every entry header starts at an aligned memory address

replay_offset gets incremented if callback function return true which means entry was processed

- On-Corrupt entry recovery strategy

• If entry validation fails, assume flash corruption or power-loss mid-write
• Synchronize replay_offset to write_offset to prevent undefined behavior
• This guarantees forward progress at the cost of losing corrupted entries

Here is the call graph for this function:

Here is the caller graph for this function:

blackbox_logger_read()

blackbox_logger_err_t blackbox_logger_read	(	const blackbox_logger_t *	logger_ptr,
		uint32_t	read_offset,
		blackbox_logger_entry_view_t *	entry_ptr,
		size_t	entry_buf_max_capacity
	)

Read an entry from flash.

Read to a buffer with a capacity provided by caller from a given flash offset

Parameters

logger_ptr	Pointer to BlackBox Logger instance
read_offset	Offset to read from
entry_ptr	Pointer to entry to be logged
entry_buf_max_capacity	Provided buffer maximum capacity

Returns

blackbox_logger_err_t read result

Return values

BLACKBOX_LOGGER_OK	Read success
BLACKBOX_LOGGER_ERR_FAIL	Failed to read from partition
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)
BLACKBOX_LOGGER_ERR_NO_MEM	No available memory for entry data
BLACKBOX_LOGGER_ERR_TIMEOUT	Timed out waiting for resource
BLACKBOX_LOGGER_ERR_INVALID_CRC	Invalid CRC for the entry
BLACKBOX_LOGGER_ERR_INVALID_ENTRY	Invalid entry found

Here is the call graph for this function:

Here is the caller graph for this function:

blackbox_logger_write()

blackbox_logger_err_t blackbox_logger_write	(	blackbox_logger_t *	logger_ptr,
		const blackbox_logger_entry_view_t *	entry_ptr
	)

Write an entry to flash.

Write to buffer first with a pre-defined maximum length if exceeded, it gets flushed. Entries are wrapped between header and tail with crc16 to ensures data integrity after power loss Look at blackbox_logger_flush for more details

Parameters

logger_ptr	Pointer to BlackBox Logger instance
entry_ptr	Pointer to entry to be logged

Returns

blackbox_logger_err_t Write result

Return values

BLACKBOX_LOGGER_OK	Write success
BLACKBOX_LOGGER_ERR_FAIL	Failed to flush writing buffer
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)
BLACKBOX_LOGGER_ERR_NO_MEM	No available memory for resource allocation
BLACKBOX_LOGGER_ERR_TIMEOUT	Timed out waiting for resource

Note

Maximum entry length is set by 'batch size' in kconfig and must be less than or equal to 'sector size' and sized must be aligned to system imposed alignments

Here is the call graph for this function:

Here is the caller graph for this function:

esp_to_blackbox_error_map()

static blackbox_logger_err_t esp_to_blackbox_error_map ( const esp_err_t  err )

static

Maps esp errors to logger errors.

Parameters

err

ESP error

Returns

blackbox_logger_err_t Blackbox Logger error

Here is the caller graph for this function:

get_entries_chunk_size_with_ceiling_helper()

size_t get_entries_chunk_size_with_ceiling_helper	(	const uint8_t *	buf_ptr,
		const size_t	buf_size,
		const size_t	ceiling_size,
		const size_t	align_to
	)

Get maximum chunk size of entries with ceiling.

Get maximum bytes to take from buffer that contains entries and doesn't exceed ceiling size

Parameters

buf_ptr	Pointer to entries buffer
buf_size	Buffer size
ceiling_size	Size of maximum bytes to get as ceiling
align_to	Align to nearest

Returns

size_t Maximum bytes to take from buffer

Note

Function doesn't validate entries, it just reads headers

Here is the call graph for this function:

Here is the caller graph for this function:

is_data_entry_internal()

static blackbox_logger_err_t is_data_entry_internal ( const uint8_t *  data_ptr, const size_t  data_size  )

static

Check if valid data entry.

Parameters

data_ptr	Pointer to data
data_size	Size of data buffer

Returns

blackbox_logger_err_t Is data entry result BLACKBOX_LOGGER_OK Valid data entry

Return values

BLACKBOX_LOGGER_ERR_NO_MEM	No available memory for entry data
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)
BLACKBOX_LOGGER_ERR_INVALID_CRC	Invalid CRC for the entry
BLACKBOX_LOGGER_ERR_INVALID_ENTRY	Invalid entry

Here is the call graph for this function:

Here is the caller graph for this function:

linear_recover_helper()

static blackbox_logger_err_t linear_recover_helper ( const blackbox_logger_t *  logger_ptr )

static

Search linearly from given logger parameters until maximum ID is found.

Parameters

logger_ptr

Pointer to BlackBox Logger instance

Returns

blackbox_logger_err_t Search result

Return values

BLACKBOX_LOGGER_OK	Search success
BLACKBOX_LOGGER_ERR_FAIL	Search fail
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)

Note

Logger's parameters instance updated on success

Here is the call graph for this function:

Here is the caller graph for this function:

recover_logger_internal()

static blackbox_logger_err_t recover_logger_internal ( blackbox_logger_t *  logger_ptr )

static

Recover Logger parameters.

Recover Logger parameters first using provided parameters if not valid it binary searches for 'head sector' a sector which has the highest entry ID at it's beginning then linearly search that 'head sector' for the highest ID

Parameters

logger_ptr

Pointer to BlackBox Logger instance

Returns

blackbox_logger_err_t Recover result

Return values

BLACKBOX_LOGGER_OK	Recover success
BLACKBOX_LOGGER_ERR_FAIL	Recover fail
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)
BLACKBOX_LOGGER_ERR_TIMEOUT	Timed out waiting for resource

Here is the call graph for this function:

Here is the caller graph for this function:

timer_flush_flash_internal()

static void timer_flush_flash_internal ( void *  logger_ptr )

static

Flush flash callback for timer.

Flush buffer to flash when triggered by the timer

Parameters

logger_ptr

Pointer to BlackBox Logger instance

Here is the call graph for this function:

Here is the caller graph for this function:

update_parameters_internal()

static blackbox_logger_err_t update_parameters_internal ( const blackbox_logger_t *  logger_ptr )

static

Update Logger configuration parameters structure.

Update configuration parameters if logger parameters changed at any point

Parameters

logger_ptr

Pointer to BlackBox Logger instance

Returns

blackbox_logger_err_t

Return values

BLACKBOX_LOGGER_OK	Update success
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)

Here is the caller graph for this function:

validate_entry_internal()

static blackbox_logger_err_t validate_entry_internal ( const uint8_t *  data_ptr, const size_t  data_size  )

static

Validate entry.

Checks header, tail and CRC validity

Parameters

data_ptr	Pointer to data
data_size	Size of data

Returns

blackbox_logger_err_t Validate entry result

Return values

BLACKBOX_LOGGER_OK	Valid entry
BLACKBOX_LOGGER_ERR_INVALID_ARG	Provided invalid argument(s)
BLACKBOX_LOGGER_ERR_INVALID_CRC	Invalid CRC for the entry
BLACKBOX_LOGGER_ERR_INVALID_ENTRY	Invalid entry

Here is the call graph for this function:

Here is the caller graph for this function:

Variable Documentation

blackbox_logger_default_parameters

blackbox_logger_parameters_t blackbox_logger_default_parameters

static

Initial value:

= {
    .write_offset = 0, .replay_offset = 0, .last_id = 0, .is_dirty = false}

Blackbox Logger default parameters structure

TAG

const char* TAG = "BB_LOG"

static

Blackbox Logger TAG name