The XCORE-VOICE Solution consists of example designs and a C-based SDK for the development of audio front-end applications to support far-field voice use cases on the xcore.ai family of chips (XU316). The XCORE-VOICE examples are currently based on FreeRTOS or bare-metal, leveraging the flexibility of the xcore.ai platform and providing designers with a familiar environment to customize and develop products.
XCORE-VOICE example designs include turn-key solutions to enable easier product development for smart home applications such as light switches, thermostats, and home appliances. xcore.ai’s unique architecture providing powerful signal processing and accelerated AI capabilities combined with the XCORE-VOICE framework allows designers to incorporate keyword, event detection, or advanced local dictionary support to create a complete voice interface solution. Bridging designs including PDM microphone to host aggregation are also included showcasing the use of xcore.ai as an interfacing and bridging solution for deployment in existing systems.
The C SDK is composed of the following components:
Peripheral IO libraries including; UART, I2C, I2S, SPI, QSPI, PDM microphones, and USB. These libraries support bare-metal and RTOS application development.
Libraries core to DSP applications, including vectorized math and voice processing DSP. These libraries support bare-metal and RTOS application development.
Libraries for speech recognition applications. These libraries support bare-metal and RTOS application development.
Libraries that enable multi-core FreeRTOS development on xcore including a wide array of RTOS drivers and middleware.
Pre-build and validated audio processing pipelines.
Code Examples - Examples showing a variety of xcore features based on bare-metal and FreeRTOS programming.
Documentation - Tutorials, references and API guides.
The XCORE-VOICE Solution takes advantage of the flexible software-defined xcore-ai architecture to support numerous far-field voice use cases through the available example designs and the ability to construct user-defined audio pipeline from the SW components and libraries in the C-based SDK.
These include:
Voice Processing components
Two PDM microphone interfaces
Digital signal processing pipeline
Full duplex, stereo, Acoustic Echo Cancellation (AEC)
Reference audio via I2S with automatic bulk delay insertion
Point noise suppression via interference canceller
Switchable stationary noise suppressor
Programmable Automatic Gain Control (AGC)
Flexible audio output routing and filtering
Support for Sensory, Cyberon or other 3rd party Automatic Speech Recognition (ASR) software
Device Interface components
Full speed USB2.0 compliant device supporting USB Audio Class (UAC) 2.0
Flexible Peripheral Interfaces
Programmable digital general-purpose inputs and outputs
Example Designs utilizing above components
Far-Field Voice Local Command
Low Power Far-Field Voice Local Command
Far-Field Voice Assistance
Firmware Management
Boot from QSPI Flash
Default firmware image for power-on operation
Option to boot from a local host processor via SPI
Device Firmware Update (DFU) via USB or I2C
Power Consumption
FFD/FFVA: 300-350mW (Typical)
Low Power FFD: 110mW (Full-Power), 54mW (Low-Power), <50mW possible with Sensory’s LPSD under certain conditions.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Quick Start Guide$$$Obtaining the Hardware£££doc/shared/introduction.html#obtaining-the-hardware
The XK-VOICE-L71 DevKit and Hardware Manual can be obtained from the XK-VOICE-L71 product information page.
The XCORE-AI-EXPLORER DevKit and Hardware Manual used in the Microphone Aggregation example can be obtained from the XK-VOICE-L71 product information page.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Quick Start Guide$$$Obtaining the Software£££doc/shared/introduction.html#obtaining-the-software
XCORE ® -VOICE Solutions$$$XCORE-VOICE Quick Start Guide$$$Obtaining the Software$$$Development Tools£££doc/shared/introduction.html#development-tools
It is recommended that you download and install the latest release of the XTC Tools. XTC Tools 15.3.0 or newer are required. If you already have the XTC Toolchain installed, you can check the version with the following command:
xcc --version
XCORE ® -VOICE Solutions$$$XCORE-VOICE Quick Start Guide$$$Obtaining the Software$$$Application Demonstrations£££doc/shared/introduction.html#application-demonstrations
If you only want to run the example designs, pre-built firmware and other software can be downloaded from the XCORE-VOICE product information page.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Quick Start Guide$$$Obtaining the Software$$$Source Code£££doc/shared/introduction.html#source-code
If you wish to modify the example designs, a zip archive of all source code can be downloaded from the XCORE-VOICE product information page.
If you have previously cloned the repository or downloaded a zip file of source code, the following commands can be used to update and fetch the submodules:
XCORE ® -VOICE Solutions$$$XCORE-VOICE Quick Start Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Example designs$$$Demonstration£££sln-voice-low-power-ffd-quick-start.html#demonstration
This is the far-field voice local command (FFD) example design. Two examples are provided: both examples include speech recognition and a local dictionary. One example uses the Sensory TrulyHandsfree™ (THF) libraries, and the other one uses the Cyberon DSPotter™ libraries.
When a wakeword phrase is detected followed by a command phrase, the application will output an audio response and a discrete message over I2C and UART.
Sensory’s THF and Cyberon’s DSpotter™ libraries ship with an expiring development license. The Sensory one will suspend recognition after 11.4 hours or 107 recognition events, and the Cyberon one will suspend recognition after 100 recognition events. After the maximum number of recognitions is reached, a device reset is required to resume normal operation. To perform a reset, either power cycle the device or press the SW2 button.
Production software runs on a special device. Contact Cyberon, Sensory or XMOS sales for information about production use of the device.
Requirements
XK-VOICE-L71 board
Powered speaker(s) with 3.5mm jack connection (OPTIONAL)
Being returned to the prompt means flashing has completed, and the XTAG4 may be disconnected.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Quick Start Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Example designs$$$Speech Recognition£££sln-voice-low-power-ffd-quick-start.html#speech-recognition
Speak one of the wakewords followed by one of the commands from the lists below.
There are three LED states:
Flashing Green = Waiting for Wake Word
Solid Red & Green = Waiting for or Processing Command
Fast Flashing Red = Evaluation period has expired
The application resets waiting for the wakeword (flashing green). Upon recognizing ‘Hello XMOS’ or ‘Hello Cyberon’ (DSpotter™ model only), waiting begins for a command (solid red & green).
After a period of inactivity, or successful command processing the application returns to waiting for wakeword (flashing green).
Sensory TrulyHandsfree™ and Cyberon DSpotter™ models detect the same commands, as listed below.
Wakewords
Hello XMOS
Hello Cyberon (DSpotter™ model only)
Dictionary Commands
Switch on the TV
Switch off the TV
Channel up
Channel down
Volume up
Volume down
Switch on the lights
Switch off the lights
Brightness up
Brightness down
Switch on the fan
Switch off the fan
Speed up the fan
Slow down the fan
Set higher temperature
Set lower temperature
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command£££sln-voice-low-power-ffd-programming-guide.html#low-power-far-field-voice-local-command
XCORE ® -VOICE Solutions$$$XCORE-VOICE Quick Start Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Example designs$$$Demonstration£££sln-voice-low-power-ffd-quick-start.html#demonstration
The low power far-field voice local command (Low Power FFD) example design targets low power
speech recognition using Sensory’s TrulyHandsfree™ (THF) speech recognition and local dictionary.
When the small wake word model running on tile 1 recognizes a wake word utterance, the device
transitions to full power mode where tile 0’s command model begins receiving audio samples,
continuing the command recognition process. On command recognition, the application outputs a
discrete message over I2C and UART.
Sensory’s THF software ships with an expiring development license. It will suspend recognition
after 11.4 hours or 107 recognition events; after which, a device reset is required to resume
normal operation. To perform a reset, either power cycle the device or press the SW2 button.
Note that SW2 is only functional while in full power mode (this application is configured to hold
the device in full-power mode on such license expiration events).
Being returned to the prompt means flashing has completed, and the XTAG4 may be disconnected.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Quick Start Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Example designs$$$Speech Recognition£££sln-voice-low-power-ffd-quick-start.html#speech-recognition
Speak one of the wake words followed by one of the commands from the lists below.
There are four LED states:
Solid Red = Low Power. Waiting for wake word.
Blinking Green = Full power. Waiting for command.
Solid Red & Green = Full power. Processing command.
Flickering Red = Full power. End of evaluation (device reset required).
On startup, the application enters low power mode and waits for the wake word. Upon wake word
recognition, the device enters full power mode and waits for a command. Upon command recognition,
the device will queue the command for processing. On each wake word or command recognition, a timer
is reset (per tile). On expiration of the intent engine’s timer, the device will request a transition
to low power. The other tile may reject the request in cases where its timer has not expired or other
application-specific reasons.
These are the XCORE-VOICE far-field voice assistant example designs demonstrating:
2-microphone far-field voice assistant front-end
Audio pipeline including echo cancelation, interference cancelling and noise suppression
Stereo reference input and voice assistant output each supported as I2S or USB (UAC2.0)
This application can be used out of the box as a voice processor solution, or extended to run local wakeword engines.
These applications features a full duplex acoustic echo cancellation stage, which can be provided reference audio via I2S or USB audio. An audio output ASR stream is also available via I2S or USB audio.
Connect either end of the ribbon cable to the XTAG4, and the other end to the XK-VOICE-L71 board as shown (Image shows piggybacked connection to RPi. Standalone operation is also supported):
Open a music player on host PC, and play a stereo file.
Check music is playing through powered speakers.
Adjust volume using music player or speakers.
Open Audacity and configure to communicate with kit. Input Device: XCORE-VOICE Voice Processor and Output Device: XCORE-VOICE Voice Processor
Set recording channels to 2 (Stereo) in Device
Set Project Rate to 48000Hz in Selection Toolbar.
Click Record (press ‘r’) to start capturing audio streamed from the XCORE-VOICE device.
Talk over music; move around the room while talking.
Stop music player.
Click Stop (press space) to stop recording. Audacity records single audio channel streamed from the XCORE-VOICE kit including extracted voice signal.
Click dropdown menu next to Audio Track, and select Split Stereo To Mono.
Click Solo on left channel of split processed audio. Increase Gain slider if necessary.
Click Play (press space) to playback processed audio.
Only your voice is audible. Playback music is removed by acoustic echo cancellation; voice is isolated by interference canceller; background noise is removed by noise suppression algorithms.
Copyright (c) 2017 Amazon.com, Inc., licensed under the MIT License
Sensory TrulyHandsfree™
The Sensory TrulyHandsfree™ speech recognition library is Copyright (C) 1995-2022 Sensory Inc. and is provided as an expiring development license. Commercial licensing is granted by Sensory Inc.
Cyberon DSpotter™
For any licensing questions about Cyberon DSpotter™ speech recognition library please contact Cyberon Corporation.
It is recommended that you download and install the latest release of the XTC Tools. XTC Tools 15.3.0 or newer are required for building, running, flashing and debugging the example applications.
CMake 3.21 or newer and Git are also required for building the example applications.
A standard C/C++ compiler is required to build applications for the host PC. Windows users may use Build Tools for Visual Studio command-line interface.
It is recommended to use Ninja as the build system for native Windows firmware builds.
To install Ninja follow install instructions at https://ninja-build.org/ or on Windows
install with winget by running the following commands in PowerShell:
# InstallwingetinstallNinja-build.ninja# Reload user Path$env:Path=[System.Environment]::GetEnvironmentVariable("Path","User")
XCORE-VOICE host builds should also work using other Windows GNU development environments like GNU Make, MinGW or Cygwin.
This is the far-field voice local command (FFD) example design. Three examples are provided: all examples include speech recognition and a local dictionary. One example uses the Sensory TrulyHandsfree™ (THF) libraries, and the other ones use the Cyberon DSPotter™ libraries. The two examples with the Cyberon DSPotter™ libraries differ in the audio source fed into the intent engine. One example uses the audio source from the microphone array, and the other uses the audio source from the I2S interface.
The examples using the microphone array as the audio source include an audio pipeline with the following stages:
Interference Canceler (IC) + Voice To Noise Ratio Estimator (VNR)
Noise Suppressor (NS)
Adaptive Gain Control (AGC)
The FFD examples provide several options to inform the host of a possible intent detected by the intent engine. The device can notify the host by:
sending the intent ID over a UART interface upon detecting the intent
sending the intent ID over an I2C master interface upon detecting the intent
allowing the host to poll the last detected intent ID over the I2C slave interface
listening to an audio message over an I2S interface
When a wakeword phrase is detected followed by a command phrase, the application will output an audio response and a discrete message over I2C and UART.
Sensory’s THF and Cyberon’s DSpotter™ libraries ship with an expiring development license. The Sensory one will suspend recognition after 11.4 hours or 107 recognition events, and the Cyberon one will suspend recognition after 100 recognition events. After the maximum number of recognitions is reached, a device reset is required to resume normal operation. To perform a reset, either power cycle the device or press the SW2 button.
This example application features audio playback responses. Speakers can be connected to the LINE OUT on the XK-VOICE-L71.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Configuring the Firmware£££sln-voice-low-power-ffd-configuration.html#configuring-the-firmware
The default application performs as described in the Overview. There are numerous compile time options that can be added to change the example design without requiring code changes. To change the options explained in the table below, add the desired configuration variables to the APP_COMPILE_DEFINITIONS cmake variable in the .cmake file located in the examples/ffd/ folder.
If options are changed, the application firmware must be rebuilt.
FFD Compile Options
Compile Option
Description
Default Value
appconfINTENT_ENABLED
Enables/disables the intent engine, primarily for debug.
1
appconfINTENT_RESET_DELAY_MS
Sets the period after the wake up phrase has been heard for a valid command phrase
5000
appconfINTENT_RAW_OUTPUT
Set to 1 to output all keywords found, skipping the internal wake up and command state machine
0
appconfAUDIO_PLAYBACK_ENABLED
Enables/disables the audio playback command response
1
appconfINTENT_UART_OUTPUT_ENABLED
Enables/disables the UART intent message
1
appconfINTENT_UART_DEBUG_INFO_ENABLED
Enables/disables the UART intent debug information
0
appconfI2C_MASTER_DAC_ENABLED
Enables/disables configuring the DAC over I2C master
1
appconfINTENT_I2C_MASTER_OUTPUT_ENABLED
Enables/disables sending the intent message over I2C master
1
appconfINTENT_I2C_MASTER_DEVICE_ADDR
Sets the address of the I2C device receiving the intent via the I2C master interface
0x01
appconfINTENT_I2C_SLAVE_POLLED_ENABLED
Enables/disables allowing another device to poll the intent message via I2C slave
0
appconfI2C_SLAVE_DEVICE_ADDR
Sets the address of the I2C device receiving the intent via the I2C slave interface
0x42
appconfINTENT_I2C_REG_ADDRESS
Sets the address of the I2C register to store the intent message, this value can be read via the I2C slave interface
0x01
appconfUART_BAUD_RATE
Sets the baud rate for the UART tx intent interface
9600
appconfUSE_I2S_INPUT
Replace I2S audio source instead of the microphone array audio source.
0
appconfI2S_MODE
Select I2S mode, supported values are appconfI2S_MODE_MASTER and appconfI2S_MODE_SLAVE
master
appconfI2S_AUDIO_SAMPLE_RATE
Select the sample rate of the I2S interface, supported values are 16000 and 48000
16000
appconfRECOVER_MCLK_I2S_APP_PLL
Enables/disables the recovery of the MCLK from the Software PLL application; this removes the need to use an external MCLK.
0
appconfINTENT_TRANSPORT_DELAY_MS
Sets the delay between host wake up requested and I2C and UART keyword code transmission
50
appconfINTENT_QUEUE_LEN
Sets the maximum number of detected intents to hold while waiting for the host to wake up
10
appconfINTENT_WAKEUP_EDGE_TYPE
Sets the host wake up pin GPIO edge type. 0 for rising edge, 1 for falling edge
0
appconfAUDIO_PIPELINE_SKIP_IC_AND_VNR
Enables/disables the IC and VNR
0
appconfAUDIO_PIPELINE_SKIP_NS
Enables/disables the NS
0
appconfAUDIO_PIPELINE_SKIP_AGC
Enables/disables the AGC
0
Note
The example_ffd_i2s_input_cyberon has different default values from the ones in the table above.
The list of updated values can be found in the APP_COMPILE_DEFINITIONS list in examples\ffd\ffd_i2s_input_cyberon.cmake.
Configuring the I2C interfaces
The I2C interfaces are used to configure the DAC and to communicate with the host. The I2C interface can be configured as a master or a slave.
The DAC must be configured at bootup via the I2C master interface.
The I2C master is used when the FFD example asynchronously sends intent messages to the host. The I2C slave is used when the host wants to read intent messages from the FFD example through polling.
Note
The I2C interface cannot operate as both master and slave simultaneously. The FFD example design uses the I2C master interface to configure the DAC at device initialisation.
However, if the host reads intent messages from the FFD example using the I2C slave interface, the I2C master interface will be disabled after the DAC configuration is complete.
To send the intent ID via the I2C master interface when a command is detected, set the following variables:
appconfINTENT_I2C_MASTER_OUTPUT_ENABLED to 1.
appconfINTENT_I2C_MASTER_DEVICE_ADDR to the desired address used by the I2C slave device.
appconfINTENT_I2C_SLAVE_POLLED_ENABLED to 0, this will disable the I2C slave interface.
To configure the FFD example so that the host can poll for the intent via the I2C slave interface, set the following variables:
appconfINTENT_I2C_SLAVE_POLLED_ENABLED to 1.
appconfI2C_SLAVE_DEVICE_ADDR to the desired address used by the I2C master device.
appconfINTENT_I2C_REG_ADDRESS to the desired register read by the I2C master device.
appconfINTENT_I2C_MASTER_OUTPUT_ENABLED to 0, this will disable the I2C master interface after initialization.
The handling of the I2C slave registers is done in the examples\ffd\src\i2c_reg_handling.c file. The variable appconfINTENT_I2C_REG_ADDRESS is used in the callback function read_device_reg().
Configuring the I2S interface
The I2S interface is used to play the audio command response to the DAC, and/or to receive the audio samples from the host. The I2S interface can be configured as either a master or a slave.
To configure the I2S interface, set the following variables:
appconfI2S_ENABLED to 1.
appconfI2S_MODE to the desired mode, either appconfI2S_MODE_MASTER or appconfI2S_MODE_SLAVE.
appconfI2S_AUDIO_SAMPLE_RATE to the desired sample rate, either 16000 or 48000.
appconfRECOVER_MCLK_I2S_APP_PLL to 1 if an external MCLK is not available, otherwise set it to 0.
appconfAUDIO_PLAYBACK_ENABLED to 1, if the intent audio is to be played back.
appconfUSE_I2S_INPUT to 1, if the I2S audio source is to be used instead of the microphone array audio source.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Linux or macOS£££sln-voice-ffva-deploying-linux-macos-programming-guide.html#deploying-the-firmware-with-linux-or-macos
This document explains how to deploy the software using CMake and Make.
Note
In the commands below <speech_engine> can be either sensory or cyberon, depending on the choice of the speech recognition engine and model.
Note
The Cyberon speech recognition engine is integrated in two examples. The example_ffd_cyberon use the microphone array as the audio source, and the example_ffd_i2s_input_cyberon uses the I2S interface as the audio source.
In the rest of this section, we use only the example_ffd_<speech_engine> as an example.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Host Applications£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-host-applications
This application requires a host application to create the flash data partition. Run the following commands in the root folder to build the host application using your native Toolchain:
Note
Permissions may be required to install the host applications.
cmake -B build_hostcd build_hostmake install
The host applications will be installed at /opt/xmos/bin, and may be moved if desired. You may wish to add this directory to your PATH variable.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-firmware
After having your python environment activated, run the following commands in the root folder to build the firmware:
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Running the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#running-the-firmware
Before running the firmware, the filesystem and model must be flashed to the
data partition.
Within the root of the build folder, run:
make flash_app_example_ffd_<speech_engine>
After this command completes, the application will be running.
After flashing the data partition, the application can be run without
reflashing. If changes are made to the data partition components, the
application must be reflashed.
From the build folder run:
xrun --xscope example_ffd_<speech_engine>.xe
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Debugging the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#debugging-the-firmware
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows£££doc/programming_guide/ffva/deploying/native_windows.html#deploying-the-firmware-with-native-windows
This document explains how to deploy the software using CMake and Ninja. If you are not using native Windows MSVC build tools and instead using a Linux emulation tool such as WSL, refer to Deploying the Firmware with Linux or macOS.
To install Ninja follow install instructions at https://ninja-build.org/ or on Windows
install with winget by running the following commands in PowerShell:
# InstallwingetinstallNinja-build.ninja# Reload user Path$env:Path=[System.Environment]::GetEnvironmentVariable("Path","User")
Note
In the commands below <speech_engine> can be either sensory or cyberon, depending on the choice of the speech recognition engine and model.
Note
The Cyberon speech recognition engine is integrated in two examples. The example_ffd_cyberon use the microphone array as the audio source, and the example_ffd_i2s_input_cyberon uses the I2S interface as the audio source.
In the rest of this section, we use only the example_ffd_<speech_engine> as an example.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Host Applications£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-host-applications
This application requires a host application to create the flash data partition. Run the following commands in the root folder to build the host application using your native Toolchain:
Note
Permissions may be required to install the host applications.
Note
A C/C++ compiler, such as Visual Studio or MinGW, must be included in the path.
Before building the host application, you will need to add the path to the XTC Tools to your environment.
The host applications will be installed at %USERPROFILE%\.xmos\bin, and may be moved if desired. You may wish to add this directory to your PATH variable.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-firmware
After having your python environment activated, run the following commands in the root folder to build the firmware:
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Running the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#running-the-firmware
Before running the firmware, the filesystem and model must be flashed to the data partition.
Within the root of the build folder, run:
ninja flash_app_example_ffd_<speech_engine>
After this command completes, the application will be running.
After flashing the data partition, the application can be run without reflashing. If changes are made to the data partition components, the application must be reflashed.
From the build folder run:
xrun --xscope example_ffd_<speech_engine>.xe
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Debugging the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#debugging-the-firmware
This section describes the connections that would need to be made to an external host for plug and play integration with existing devices.
When an intent is found, the XCORE device will check if the host is awake, by checking the Host Status GPIO pin. If the host is awake the intent code will be transmitted over I2C and/or UART.
If the host is not awake, the XCORE device will trigger a transition of the Wakeup GPIO pin. This can be configured to be a rising or falling edge. The XCORE device will then wait for a fixed period of time, set at compile time, before transmitting the intent over the I2C and/or UART interface. This behavior can be changed as desired by modifying the intent handling code.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$UART£££sln-voice-low-power-ffd-host-integration.html#uart
UART Connections
FFD Connection
Host Connection
J4:24
UART RX
J4:20
GND
I2C
I2C Connections
FFD Connection
Host Connection
J4:3
SDA
J4:5
SCL
J4:9
GND
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$GPIO£££sln-voice-low-power-ffd-host-integration.html#gpio
The estimated power usage of the example application varies from 100-141 mW. This will vary based on component tolerances and any user added code and/or user added compile options.
FFD Resources
Resource
Tile 0
Tile 1
Total Memory Free
145k
208k
Runtime Heap Memory Free
38k
42k
FFD CPU Usage
Core ID
Typical Mean CPU Usage (%)
Standard Deviation CPU Usage (%)
Typical Min CPU usage (%, 10ms rolling)
Typical Max CPU usage (%, 10ms rolling)
tile[0], core[0]
0.006
0.345
0.000
21.030
tile[0], core[1]
0.072
2.031
0.000
80.690
tile[0], core[2]
0.082
2.287
0.000
100.000
tile[0], core[3]
1.666
2.906
0.000
54.560
tile[0], core[4]
65.925
27.828
0.000
91.220
tile[1], core[0]
0.014
0.540
0.000
27.440
tile[1], core[1]
99.990
0.505
74.000
100.000
tile[1], core[2]
99.990
0.507
73.870
100.000
tile[1], core[3]
18.272
13.259
0.000
98.220
tile[1], core[4]
17.231
11.048
0.000
37.260
Note that these are typical usage statistics for a representative run of the application on hardware. Core allocations may shift run-to-run in a scheduled RTOS.
These statistics are generated by slicing the representative run into 10 ms chunks and calculating % time per chunk not spent in the FreeRTOS IDLE tasks.
Therefore, the underlying distribution of these 10 ms bins should not be assumed to be Normal; this has implications on e.g. the interpretation of the Standard Deviation given here.
FFD Power Usage
Power State
Power (mW)
Always
114
The description of the software is split up by folder:
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Local Command$$$Modifying the Software$$$examples/ffd/bsp_config£££sln-voice-ffd-bsp-config.html#examples-ffd-bsp-config
This folder contains bsp_configs for the FFD application. More information on bsp_configs can be found in the RTOS Framework documentation.
FFD bsp_config
Filename/Directory
Description
dac directory
DAC ports for supported bsp_configs
XCORE-AI-EXPLORER directory
experimental bsp_config, not recommended for general use
XCORE-AI-EXPLORER_EXT directory
experimental bsp_config, not recommended for general use
XK_VOICE_L71 directory
default FFD application bsp_config
XK_VOICE_L71_EXT directory
USB debug extension FFD application bsp_config
bsp_config.cmake
cmake for adding FFD bsp_configs
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Local Command$$$Modifying the Software$$$examples/ffd/filesystem_support£££sln-voice-ffd-filesystem-support.html#examples-ffd-filesystem-support
This folder contains filesystem contents for the FFD application.
FFD filesystem_support
Filename/Directory
Description
50.wav
Playback for intent ID 50
1.wav
Playback for intent ID 1
3.wav
Playback for intent ID 3
4.wav
Playback for intent ID 4
5.wav
Playback for intent ID 5
6.wav
Playback for intent ID 6
7.wav
Playback for intent ID 7
8.wav
Playback for intent ID 8
9.wav
Playback for intent ID 9
10.wav
Playback for intent ID 10
11.wav
Playback for intent ID 11
12.wav
Playback for intent ID 12
13.wav
Playback for intent ID 13
14.wav
Playback for intent ID 14
15.wav
Playback for intent ID 15
16.wav
Playback for intent ID 16
17.wav
Playback for intent ID 17
18.wav
Playback for intent ID 18
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Local Command$$$Modifying the Software$$$examples/ffd/src£££sln-voice-ffd-src.html#examples-ffd-src
This folder contains the core application source.
FFD src
Filename/Directory
Description
gpio_ctrl directory
contains general purpose input handling and LED handling tasks
intent_engine directory
contains intent engine code
intent_handler directory
contains intent handling code
rtos_conf directory
contains default FreeRTOS configuration headers
app_conf_check.h
header to validate app_conf.h
app_conf.h
header to describe app configuration
config.xscope
xscope configuration file
ff_appconf.h
default fatfs configuration header
main.c
main application source file
xcore_device_memory.c
model loading from filesystem source file
xcore_device_memory.h
model loading from filesystem header file
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Audio Pipeline£££sln-voice-low-power-ffd-src.html#audio-pipeline
The audio pipeline module provides the application with three API functions:
This function has the role of creating the audio pipeline, with two optional application pointers which are provided to the application in the audio_pipeline_input() and audio_pipeline_output() callbacks.
In FFD, the audio pipeline is initialized with no additional arguments, and instantiates a 3 stage pipeline on tile 1, as described in:
Audio Pipeline
This function has the role of launching tasks on each tile. For those familiar with XCORE, it is comparable to the main par loop in an XC main.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$vApplicationMinimalIdleHook£££sln-voice-low-power-ffd-src.html#vapplicationminimalidlehook
This is a FreeRTOS callback. By calling “waiteu” without events configured, this has the effect of both MIPs and power savings on XCORE.
If replacing the existing model, these are the only two functions that are required to be populated.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$intent_engine_create£££sln-voice-low-power-ffd-intent-engine.html#intent-engine-create
This function has the role of creating the model running task and providing a pointer, which can be used by the application to handle the output intent result. In the case of the default configuration, the application provides a FreeRTOS Queue object.
The ASR engine is on tile 0 in both FFD and FFVA, but the audio pipeline output is on tile 1 for FFD and on tile 0 for FFVA.
The call to intent_engine_intertile_task_create() will create two threads on tile 0. One thread is the ASR engine thread. The other thread is an intertile rx thread, which will interface with the audio pipeline output.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$intent_engine_ready_sync£££sln-voice-low-power-ffd-intent-engine.html#intent-engine-ready-sync
This function is called by both tiles and serves to ensure that tile 0 is ready to receive
audio samples before starting the audio pipeline. This is a preventative measure to avoid dropping
samples at startup.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$intent_engine_sample_push£££sln-voice-low-power-ffd-intent-engine.html#intent-engine-sample-push
This function has the role of sending the ASR output channel from the audio pipeline to the intent engine.
The ASR engine is on tile 0 in both FFD and FFVA, but the audio pipeline output is on tile 1 for FFD and on tile 0 for FFVA.
The call to intent_engine_samples_send_remote() will send the audio samples to the previously configured intertile rx thread.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$intent_engine_process_asr_result£££sln-voice-low-power-ffd-intent-engine.html#intent-engine-process-asr-result
This function can be replaced by the application to handle the intent in a completely different manner.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Local Command$$$Modifying the Software$$$Miscellaneous Functions£££sln-voice-intent-engine.html#miscellaneous-functions
The following helper functions are provided for supporting the command processing features that are unique to the default FFD application:
intent_engine_keyword_queue_count
intent_engine_keyword_queue_complete
intent_engine_stream_buf_reset
intent_engine_play_response
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Local Command$$$Modifying the Software$$$modules/asr/intent_handler£££sln-voice-intent-handler.html#modules-asr-intent-handler
This folder contains ASR output handling modules for the FFD and FFVA applications.
ASR Intent handler
Filename/Directory
Description
audio_response directory
include folder for handling audio responses to keywords
intent_handler.c
contains the implementation of default intent handling code
If replacing the existing handler code, this is the only function that is required to be populated.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$intent_handler_create£££sln-voice-low-power-ffd-intent-handler.html#intent-handler-create
This function has the role of creating the keyword handling task for the ASR engine. In the case of the Sensory and Cyberon models, the application provides a FreeRTOS Queue object. This handler is on the same tile as the speech recognition engine, tile 0.
The call to intent_handler_create() will create one thread on tile 0. This thread will receive ID packets from the ASR engine over a FreeRTOS Queue object and output over various IO interfaces based on configuration.
The FFD example design consists of three major software blocks, the audio pipeline, keyword spotter, and keyword handler. This section will go into detail on how to replace each/all of these subsystems.
It is highly recommended to be familiar with the application as a whole before attempting replacing these functional units. This information can be found here:
Software Description
See Software Description for more details on the memory footprint and CPU usage of the major software components.
The audio pipeline can be replaced by making changes to the audio_pipeline.c file.
It is up to the user to ensure that the input and output frames of the audio pipeline remain the same, or the remainder of the application will not function properly.
This section will walk through an example of replacing the XMOS NS stage, with a custom stage foo.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Modifying the Software$$$Declaration and Definition of DSP Context£££doc/programming_guide/ffva/software_modifications.html#declaration-and-definition-of-dsp-context
To add or remove a peripheral IO, modify the bsp_config accordingly. Refer to documentation inside the RTOS Framework on how to instantiate different RTOS peripheral drivers.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Direct Control£££sln-voice-low-power-ffd-replacing-keyword-engine-block.html#direct-control
In a single controller system, the XCORE can be used to control peripherals directly.
The proc_keyword_res task can be modified as follows:
Intent Handler (intent_handler.c)
staticvoidproc_keyword_res(void*args){QueueHandle_tq_intent=(QueueHandle_t)args;int32_tid=0;while(1){xQueueReceive(q_intent,&id,portMAX_DELAY);/* User logic here */}}
This code example will receive the ID of each intent, and can be populated by any user application logic. User logic can use other RTOS drivers to control various peripherals, such as screens, motors, lights, etc, based on the intent engine outputs.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Local Command$$$Modifying the Software$$$Speech Recognition - Sensory£££sln-voice-ffd-speech-recognition-sensory.html#speech-recognition-sensory
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$License£££sln-voice-low-power-ffd-speech-recognition.html#license
The Sensory TrulyHandsFree™ (THF) speech recognition library is Copyright (C) 1995-2022 Sensory Inc., All Rights Reserved.
Sensory THF software requires a commercial license granted by Sensory Inc.
This software ships with an expiring development license. It will suspend recognition after 11.4 hours
or 107 recognition events.
The Sensory THF speech recognition engine runs proprietary models to identify keywords in an audio stream. Models can be generated using VoiceHub.
Two models are provided - one in US English and one in Mainland Mandarin. The US English model is used by default. To modify the software to use the Mandarin model, see the comment at the top of the ffd_sensory.cmake file. Make sure run the following commands to rebuild and re-flash the data partition:
make cleanmake flash_app_example_ffd_sensory -j
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Local Command$$$Modifying the Software$$$Dictionary command table£££sln-voice-ffd-speech-recognition-cyberon.html#dictionary-command-table
English Language Demo
Utterances
Type
Return code (decimal)
Hello XMOS
keyword
1
Switch on the TV
command
3
Switch off the TV
command
4
Channel up
command
5
Channel down
command
6
Volume up
command
7
Volume down
command
8
Switch on the lights
command
9
Switch off the lights
command
10
Brightness up
command
11
Brightness down
command
12
Switch on the fan
command
13
Switch off the fan
command
14
Speed up the fan
command
15
Slow down the fan
command
16
Set higher temperature
command
17
Set lower temperature
command
18
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Application Integration£££sln-voice-low-power-ffd-speech-recognition.html#application-integration
In depth information on out of the box integration can be found here: Host Integration
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Local Command$$$Modifying the Software$$$Speech Recognition - Cyberon£££sln-voice-ffd-speech-recognition-cyberon.html#speech-recognition-cyberon
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$License£££sln-voice-low-power-ffd-speech-recognition.html#license
Cyberon DSpotter™ software requires a commercial license granted by Cyberon Corporation.
This software ships with an expiring development license. It will suspend recognition after 100 recognition events.
Production versions of the DSpotter™ library are unrestricted when running on a specially licensed XMOS device. Please contact Cyberon or XMOS sales for further information.
The Cyberon DSpotter™ speech recognition engine runs proprietary models to identify keywords in an audio stream.
One model for US English is provided. For any technical questions or additional models please contact Cyberon.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Local Command$$$Modifying the Software$$$Dictionary command table£££sln-voice-ffd-speech-recognition-cyberon.html#dictionary-command-table
English Language Demo
Utterances
Type
Return code (decimal)
Hello XMOS
keyword
1
Hello Cyberon
keyword
1
Switch on the TV
command
2
Switch off the TV
command
3
Channel up
command
4
Channel down
command
5
Volume up
command
6
Volume down
command
7
Switch on the lights
command
8
Switch off the lights
command
9
Brightness up
command
10
Brightness down
command
11
Switch on the fan
command
12
Switch off the fan
command
13
Speed up the fan
command
14
Slow down the fan
command
15
Set higher temperature
command
16
Set lower temperature
command
17
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Application Integration£££sln-voice-low-power-ffd-speech-recognition.html#application-integration
In depth information on out of the box integration can be found here: Host Integration
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command£££sln-voice-low-power-ffd-programming-guide.html#low-power-far-field-voice-local-command
The low power far-field voice local command (Low Power FFD) example design targets low power
speech recognition using Sensory’s TrulyHandsfree™ (THF) speech recognition and local dictionary.
When the small wake word model running on tile 1 recognizes a wake word utterance, the device
transitions to full power mode where tile 0’s command model begins receiving audio samples,
continuing the command recognition process. On command recognition, the application outputs a
discrete message over I2C and UART.
Tile 0’s command model, in combination with a timer, determines when to request a transition to low
power. Tile 1 may accept or reject this request based on its own timer that is reset on wake word
recognitions and potentially other application-specific events. The figure below illustrates the
general behavior.
When in low power mode, tile 0 is effectively disabled along with any peripheral/IO associated with
that tile.
Sensory’s THF software ships with an expiring development license. It will suspend recognition
after 11.4 hours or 107 recognition events; after which, a device reset is required to resume
normal operation. To perform a reset, either power cycle the device or press the SW2 button.
Note that SW2 is only functional while in full power mode (this application is configured to hold
the device in full-power mode on such license expiration events).
More information on the Sensory speech recognition library can be found here: Speech Recognition
Connect the xTAG to the debug header, as shown below.
Connect the micro USB XTAG4 and micro USB XK-VOICE-L71 to the programming host.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Configuring the Firmware£££sln-voice-low-power-ffd-configuration.html#configuring-the-firmware
The default application performs as described in the Overview. There
are numerous compile time options that can be added to change the example design without requiring
code changes. To change the options explained in the table below, add the desired configuration
variables to the APP_COMPILE_DEFINITIONS CMake variable located in the example’s CMake file
here.
If options are changed, the application firmware must be rebuilt.
Low Power FFD Compile Options
Compile Option
Description
Default Value
appconfINTENT_RESET_DELAY_MS
Sets the period after the wake word phrase or subsequent command/wake word phrase has been heard for a valid command phrase
4000
appconfINTENT_UART_OUTPUT_ENABLED
Enables/disables the UART intent message
1
appconfINTENT_I2C_MASTER_OUTPUT_ENABLED
Enables/disables sending the intent message over I2C master
1
appconfUART_BAUD_RATE
Sets the baud rate for the UART tx intent interface
9600
appconfINTENT_I2C_MASTER_DEVICE_ADDR
Sets the I2C slave address to transmit the intent to
0x01
appconfINTENT_TRANSPORT_DELAY_MS
Sets the delay between host wake up requested and I2C and UART keyword code transmission
50
appconfINTENT_QUEUE_LEN
Sets the maximum number of detected intents to hold while waiting for the host to wake up
10
appconfINTENT_WAKEUP_EDGE_TYPE
Sets the host wake up pin GPIO edge type. 0 for rising edge, 1 for falling edge
0
appconfAUDIO_PIPELINE_SKIP_IC_AND_VNR
Enables/disables the IC and VNR
0
appconfAUDIO_PIPELINE_SKIP_NS
Enables/disables the NS
0
appconfAUDIO_PIPELINE_SKIP_AGC
Enables/disables the AGC
0
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Linux or macOS£££sln-voice-ffva-deploying-linux-macos-programming-guide.html#deploying-the-firmware-with-linux-or-macos
This document explains how to deploy the software using CMake and Make.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Host Applications£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-host-applications
This application requires a host application to create the flash data partition. Run the following
commands in the root folder to build the host application using your native toolchain:
Note
Permissions may be required to install the host applications.
cmake -B build_hostcd build_hostmake install
The host applications will be installed at /opt/xmos/bin, and may be moved if desired. You may
wish to add this directory to your PATH variable.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-firmware
After having your python environment activated, run the following commands in the root folder to build the firmware:
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Running the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#running-the-firmware
Before running the firmware, the filesystem and command model must be flashed to the data partition.
Within the root of the build folder, run:
make flash_app_example_low_power_ffd_sensory
After this command completes, the application will be running.
After flashing the data partition, the application can be run without reflashing. If changes are
made to the data partition components, the application must be reflashed.
From the build folder run:
xrun --xscope example_low_power_ffd_sensory.xe
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Debugging the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#debugging-the-firmware
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows£££doc/programming_guide/ffva/deploying/native_windows.html#deploying-the-firmware-with-native-windows
This document explains how to deploy the software using CMake and Ninja. If you are not using
native Windows MSVC build tools and instead using a Linux emulation tool such as WSL, refer to
Deploying the Firmware with Linux or macOS.
To install Ninja follow install instructions at https://ninja-build.org/ or on Windows
install with winget by running the following commands in PowerShell:
# InstallwingetinstallNinja-build.ninja# Reload user Path$env:Path=[System.Environment]::GetEnvironmentVariable("Path","User")
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Host Applications£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-host-applications
This application requires a host application to create the flash data partition. Run the following
commands in the root folder to build the host application using your native toolchain:
Note
Permissions may be required to install the host applications.
Note
A C/C++ compiler, such as Visual Studio or MinGW, must be included in the path.
Before building the host application, you will need to add the path to the XTC Tools to your environment.
The host applications will be installed at %USERPROFILE%\.xmos\bin, and may be moved if desired.
You may wish to add this directory to your PATH variable.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-firmware
After having your python environment activated, run the following commands in the root folder to build the firmware:
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Running the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#running-the-firmware
Before running the firmware, the filesystem and command model must be flashed to the data partition.
Within the root of the build folder, run:
ninja flash_app_example_low_power_ffd_sensory
After this command completes, the application will be running.
After flashing the data partition, the application can be run without reflashing. If changes are
made to the data partition components, the application must be reflashed.
From the build folder run:
xrun --xscope example_low_power_ffd_sensory.xe
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Debugging the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#debugging-the-firmware
This section describes the connections that would need to be made to an external host for plug and
play integration with existing devices.
When an intent is found, the XCORE device will check if the host is awake, by checking the Host
Status GPIO pin. If the host is awake the intent code will be transmitted over I2C and/or UART.
If the host is not awake, the XCORE device will trigger a transition of the Wakeup GPIO pin. This
can be configured to be a rising or falling edge. The XCORE device will then wait for a fixed
period of time, set at compile time, before transmitting the intent over the I2C and/or UART
interface. This behavior can be changed as desired by modifying the intent handling code.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$UART£££sln-voice-low-power-ffd-host-integration.html#uart
UART Connections
Low Power FFD Connection
Host Connection
J4:24
UART RX
J4:20
GND
I2C
I2C Connections
Low Power FFD Connection
Host Connection
J4:3
SDA
J4:5
SCL
J4:9
GND
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$GPIO£££sln-voice-low-power-ffd-host-integration.html#gpio
The approximate resource utilizations for Low Power FFD are shown in the table below.
Low Power FFD Resources
Resource
Tile 0
Tile 1
Unused CPU Time (600MHz | 200MHz)
50%
10%
Total Memory Free
19.1k
5.3k
Runtime Heap Memory Free
219k
12.4k
The estimated (core) power usage for Low Power FFD are shown in the table below. Additional power
savings may be possible using Sensory’s Low Power Sound Detect (LPSD) option which approaches sub-50mW
operation in Low Power mode. These measurements will vary based on component tolerances and any user
added code and/or user added compile options.
Low Power FFD Power Usage
Power State
Core Power (mW)
Low Power
54
Full Power
110
The description of the software is split up by folder:
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$bsp_config£££sln-voice-low-power-ffd-bsp-config.html#bsp-config
This folder contains bsp_configs for the Low Power FFD application. More information on bsp_configs
can be found in the RTOS Framework documentation.
Low Power FFD bsp_config
Filename/Directory
Description
dac directory
DAC ports for supported bsp_configs (not used in example, disabled)
XK_VOICE_L71 directory
default Low Power FFD application bsp_config
bsp_config.cmake
cmake for adding Low Power FFD bsp_configs
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$filesystem_support£££sln-voice-low-power-ffd-filesystem-support.html#filesystem-support
This folder contains filesystem contents for the Low Power FFD application.
Low Power FFD filesystem_support
Filename/Directory
Description
demo.txt
A file for demonstrative purposes containing the text “Hello World!”. This file is not used or interacted with in this application.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$model£££sln-voice-low-power-ffd-model.html#model
This folder contains the Sensory wake word and command model files the Low Power FFD application.
Note
Only a subset of the files below are used. See low_power_ffd.cmake for the files used by the
application. Also note the nibble-swapped net-file is manually generated, via the nibble_swap
tool found in lib_qspi_fast_read.
The command model’s net-file, in binary-form (nibble swapped, for supporting fast flash reads)
command-pc62w-6.1.0-op10-prod-net.c
The command model’s net-file, in source form
command-pc62w-6.1.0-op10-prod-search.bin
The command model’s search-file, in binary form
command-pc62w-6.1.0-op10-prod-search.c
The command model’s search-file, in source form
command-pc62w-6.1.0-op10-prod-search.h
The command model’s search header-file
command.snsr
The command model’s Sensory THF/TNL SDK “snsr” file
wakeword-pc60w-6.1.0-op10-prod-net.bin
The wake word model’s net-file, in binary-form
wakeword-pc60w-6.1.0-op10-prod-net.c
The wake word model’s net-file, in source form
wakeword-pc60w-6.1.0-op10-prod-search.bin
The wake word model’s search-file, in binary form
wakeword-pc60w-6.1.0-op10-prod-search.c
The wake word model’s search-file, in source form
wakeword-pc60w-6.1.0-op10-prod-search.h
The wake word model’s search header-file
wakeword.snsr
The wake word model’s Sensory THF/TNL SDK “snsr” file
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$src£££sln-voice-low-power-ffd-src.html#src
This folder contains the core application source.
FFD src
Filename/Directory
Description
gpio_ctrl directory
contains general purpose input handling and LED handling tasks
intent_engine directory
contains intent engine code
intent_handler directory
contains intent handling code
power directory
contains low power control logic and related audio buffer
rtos_conf directory
contains default FreeRTOS configuration headers
wakeword directory
contains wake word detection code
app_conf_check.h
header to validate app_conf.h
app_conf.h
header to describe app configuration
config.xscope
xscope configuration file
ff_appconf.h
default fatfs configuration header
main.c
main application source file
device_memory_impl.c
contains XCORE device memory functions for supporting ASR functionality
device_memory_impl.h
header for the device memory implementation
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Audio Pipeline£££sln-voice-low-power-ffd-src.html#audio-pipeline
The audio pipeline module provides the application with three API functions:
This function has the role of creating the audio pipeline, with two optional application pointers
which are provided to the application in the audio_pipeline_input() and audio_pipeline_output() callbacks.
In Low Power FFD, the audio pipeline is initialized with no additional arguments, and instantiates a
3 stage pipeline on tile 1, as described in:
Audio Pipeline
This function has the role of receiving the processed audio pipeline output.
In Low Power FFD, the output is sent to both the wake word handler and the intent engine. Because
the intent engine will be suspended in low power mode and that there is a finite time that it takes
to resume full power operation, there is a ring buffer placed between the audio output received
from this routine and the intent engine’s stream buffer.
This function has the role of launching tasks on each tile. For those familiar with XCORE, it is comparable to the main par loop in an XC main.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$vApplicationMinimalIdleHook£££sln-voice-low-power-ffd-src.html#vapplicationminimalidlehook
This is a FreeRTOS callback. By calling “waiteu” without events configured, this has the effect of both MIPs and power savings on XCORE.
This function is the application C entry point on tile 1, provided by the SDK.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$src/gpio_ctrl£££sln-voice-low-power-ffd-gpio-ctrl.html#src-gpio-ctrl
This folder contains the GPIO and LED related functionality for the Low Power FFD application.
Low Power FFD gpio_ctrl
Filename/Directory
Description
gpi_ctrl.c
The general purpose input control source file. Implements SW2 reset logic.
gpi_ctrl.h
The general purpose input control header file.
leds.c
The LED task source file. Handles the applications LED indications.
leds.h
The LED task header file.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$src/intent_engine£££sln-voice-low-power-ffd-intent-engine.html#src-intent-engine
This folder contains the intent engine module for the low power FFD application.
Low Power FFD Intent Engine
Filename/Directory
Description
intent_engine_io.c
contains additional io intent engine code
intent_engine_support.c
contains general intent engine support code
intent_engine.c
contains the implementation of default intent engine code
These APIs provide the functionality needed to feed audio pipeline samples into the ASR engine.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$intent_engine_create£££sln-voice-low-power-ffd-intent-engine.html#intent-engine-create
This function has the role of creating the model running task and providing a pointer, which can be
used by the application to handle the output intent result. In the case of the default configuration,
the application provides a FreeRTOS Queue object.
In Low Power FFD, the audio pipeline output is on tile 1 and the ASR engine on tile 0.
intent_engine_create snippet (intent_engine_io.c)
intent_engine_intertile_task_create(priority);
The call to intent_engine_intertile_task_create() will create two threads on tile 0. One thread is
the ASR engine thread. The other thread is an intertile RX thread, which will interface with the
audio pipeline output.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$intent_engine_ready_sync£££sln-voice-low-power-ffd-intent-engine.html#intent-engine-ready-sync
This function is called by both tiles and serves to ensure that tile 0 is ready to receive
audio samples before starting the audio pipeline. This is a preventative measure to avoid dropping
samples at startup.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$intent_engine_sample_push£££sln-voice-low-power-ffd-intent-engine.html#intent-engine-sample-push
This function has the role of sending the ASR output channel from the audio pipeline to the intent engine.
In Low Power FFD, the audio pipeline output is on tile 1 and the ASR engine on tile 0.
The call to intent_engine_samples_send_remote() will send the audio samples to the previously
configured intertile RX thread.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$intent_engine_process_asr_result£££sln-voice-low-power-ffd-intent-engine.html#intent-engine-process-asr-result
This function can be replaced by the application to handle the intent in a completely different manner.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Low Power Components£££sln-voice-low-power-ffd-intent-engine.html#low-power-components
The following APIs are the intent engine mechanisms needed by the power control task.
In this implementation, it is the responsibility of tile 0 (intent engine tile) to determine when
to request a transition into low power mode; however, tile 1 may reject the request. When tile 1
accepts the request (via LOW_POWER_ACK), the power control task calls intent_engine_low_power_accept.
When tile 1 rejects the request (via LOW_POWER_NAK), the power control task calls
intent_engine_full_power_request.
Note
There is an additional LOW_POWER_HALT response where the power control task calls
intent_engine_halt. This is primarily for end-of-evaluation handling logic for the underlying
ASR engine and is not needed for a normal application.
After tile 1 accepts the low power request, tile 0 begins preparations for entering low power by
locking various resources and waiting for any enqueued commands to finish up. The helper functions
below are provided for this purpose.
Before tile 1 sends LOW_POWER_ACK it also stops pushing audio samples via intent_engine_sample_push.
After receiving the low power response, the application may clear the stream buffer and keyword
queue to avoid processing stale samples/commands when returning to full power mode. The functions
below provide this functionality.
Since it is possible that a command is spoken/recognized between the time when tile 0 requests
low power and when tile 1 responds to the request, the application should not reset these
buffer entities until it has received LOW_POWER_ACK; otherwise, recognized commands may be lost.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Evaluation Specific Components£££sln-voice-low-power-ffd-intent-engine.html#evaluation-specific-components
The following functions are provided for the primary purpose of facilitating the evaluation of the
ASR model. The provided ASR models have evaluation periods which will end due to various factors.
When the evaluation period ends, the application logic halts the intent engine via intent_engine_halt.
This is primarily to ensure the device remains in full-power mode to allow functionality that may be
exclusive to tile 0 to function.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$src/intent_handler£££sln-voice-low-power-ffd-intent-handler.html#src-intent-handler
This folder contains ASR output handling modules for the Low Power FFD application.
FFD Intent handler
Filename/Directory
Description
intent_handler.c
contains the implementation of default intent handling code
If replacing the existing handler code, this is the only function that is required to be populated.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$intent_handler_create£££sln-voice-low-power-ffd-intent-handler.html#intent-handler-create
This function has the role of creating the keyword handling task for the ASR engine. In the case of
the Sensory model, the application provides a FreeRTOS Queue object. This handler is on the same
tile as the Sensory engine, tile 0.
The call to intent_handler_create() will create one thread on tile 0. This thread will receive ID
packets from the ASR engine over a FreeRTOS Queue object and output over various IO interfaces based
on configuration.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$src/power£££sln-voice-low-power-ffd-power.html#src-power
This folder contains the low power control logic and supporting logic.
Low Power FFD power
Filename/Directory
Description
low_power_audio_buffer.c
Implementation of an audio sample ring buffer. Aids in responsiveness to commands during a transition to full power mode.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$power_control_task_create£££sln-voice-low-power-ffd-power.html#power-control-task-create
Creates and starts the power control task. To be called by each tile.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$power_control_exit_low_power£££sln-voice-low-power-ffd-power.html#power-control-exit-low-power
Applicable only for Tile 1. Begins a transition to full power mode and is intended to be called by
the power_state_set() routine.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$power_control_state_get£££sln-voice-low-power-ffd-power.html#power-control-state-get
Applicable only for Tile 1. Gets the current power state.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$power_control_halt£££sln-voice-low-power-ffd-power.html#power-control-halt
Applicable only for Tile 1. Halts the power control task. This is provided primarily for
end-of-evaluation logic, but severs to terminate the low power logic. When halted, the system
remains in full power mode.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$power_control_req_low_power£££sln-voice-low-power-ffd-power.html#power-control-req-low-power
Applicable only for Tile 0. Requests a transition to low power mode.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$power_control_ind_complete£££sln-voice-low-power-ffd-power.html#power-control-ind-complete
Applicable only for Tile 0. Indication that the last step for preparing for a low power transition
has completed and allows the power control task to continue with final steps. This is primarily to
ensure the LED indications are up-to-date before driver locks are taken (which include GPIO/LED control).
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Power State Components£££sln-voice-low-power-ffd-power.html#power-state-components
The power state module provides the application with the following primary API functions:
This module is also responsible for providing the base power state datatype (power_state_t) used by
other low power logic.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$power_state_init£££sln-voice-low-power-ffd-power.html#power-state-init
Initializes the power state module. Responsible to initializing the underlying timer that effectively
determines whether a low power request by Tile 0 is accepted or rejected.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$power_state_set£££sln-voice-low-power-ffd-power.html#power-state-set
Used by Tile 1’s application to signal full power events (such as wake word detection or other
application-specific events). Used by Tile 1’s power control logic to signal low power only after
Tile 0 has requested low power mode and the local timer has expired.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$power_state_timer_expired_get£££sln-voice-low-power-ffd-power.html#power-state-timer-expired-get
Used by the Tile 1’s power control logic to determine whether to accept or reject a low power request by Tile 0.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$src/wakeword£££sln-voice-low-power-ffd-wakeword.html#src-wakeword
This folder contains the wake word recognition functionality for the Low Power FFD application.
Low Power FFD wakeword
Filename/Directory
Description
wakeword.c
The wake word engine source file. Responsible for the transfer of audio samples into the ASR and handling of wake word detection events.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$wakeword_init£££sln-voice-low-power-ffd-wakeword.html#wakeword-init
This function performs the required initialization for the wakeword_handler() function to
operate. This involves initializing an instance of devmem_manager_t for use by the ASR abstraction
layer and initialization of the ASR unit itself. It is to be called once during startup before any
call to wakeword_handler() occurs.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$wakeword_handler£££sln-voice-low-power-ffd-wakeword.html#wakeword-handler
This function performs wake word detection logic and reports back to the caller a result, indicating
whether a wake word was recognized. Note: this routine is called by audio_pipeline_output(), meaning
this routine’s logic should be kept to a minimum to ensure timing requirements are met.
In this implementation a single wake word ID of 1 is defined. Minimal adaptation is needed to support
other models supporting other IDs or more than one valid wake word.
The Low Power FFD example design consists of four major software blocks: the audio pipeline,
ASR engine (wake word and intent engines), intent handler, and power control. This section will go
into detail on how to replace each subsystem.
It is highly recommended to be familiar with the application as a whole before attempting replacing
these functional units. This information can be found here:
Software Description
See Software Description for more details on the memory footprint and
CPU usage of the major software components.
The audio pipeline can be replaced by making changes to the audio_pipeline.c file.
It is up to the user to ensure that the input and output frames of the audio pipeline remain the
same, or the remainder of the application will not function properly.
This section will walk through an example of replacing the XMOS NS stage, with a custom stage foo.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Modifying the Software$$$Declaration and Definition of DSP Context£££doc/programming_guide/ffva/software_modifications.html#declaration-and-definition-of-dsp-context
It is also possible to add or remove stages. Refer to the RTOS Framework documentation on the
generic pipeline sw_service.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Replacing ASR Engine Block£££sln-voice-low-power-ffd-replacing-keyword-engine-block.html#replacing-asr-engine-block
Replacing the keyword spotter engine has the potential to require significant changes due to various
feature extraction input requirements and varied output logic.
The generic intent engine API only requires two functions be declared:
Intent API (intent_engine.h)
/* Generic interface for intent engines */int32_tintent_engine_create(uint32_tpriority,void*args);int32_tintent_engine_sample_push(asr_sample_t*buf,size_tframes);
Refer to the existing Sensory model implementation for details on how the output handler is set up,
how the audio is conditioned to the expected model format, and how it receives frames from the audio
pipeline.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Modifying the Software$$$Replacing Example Design Interfaces£££doc/programming_guide/ffva/software_modifications.html#replacing-example-design-interfaces
It may be desired to have a different output interface to talk to a host, or not have a host at all
and handle the intent local to the XCORE device.
To add or remove a peripheral IO, modify the bsp_config accordingly. Refer to documentation inside
the RTOS Framework on how to instantiate different RTOS peripheral drivers.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Direct Control£££sln-voice-low-power-ffd-replacing-keyword-engine-block.html#direct-control
In a single controller system, the XCORE can be used to control peripherals directly.
The proc_keyword_res task can be modified as follows:
Intent Handler (intent_handler.c)
staticvoidproc_keyword_res(void*args){QueueHandle_tq_intent=(QueueHandle_t)args;int32_tid=0;while(1){xQueueReceive(q_intent,&id,portMAX_DELAY);/* User logic here */}}
This code example will receive the ID of each intent, and can be populated by any user application
logic. User logic can use other RTOS drivers to control various peripherals, such as screens,
motors, lights, etc, based on the intent engine outputs.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Replacing Example Power Control Logic£££sln-voice-low-power-ffd-replacing-keyword-engine-block.html#replacing-example-power-control-logic
Depending on the peripherals used in the end application, the requirements and handling of the
power control/state logic may need adaptation. The power control logic operates in a task where a
state machine that is common to both tiles is used. During steady state, each tile is expected to
remain is the same state. During transitions each tile executes its own state transition logic.
Below outlines the various functions that may need adaptation for a given application.
Locking drivers (power_control.c)
staticvoiddriver_control_lock(void){#if ON_TILE(POWER_CONTROL_TILE_NO)rtos_osal_mutex_get(&gpio_ctx_t0->lock,RTOS_OSAL_WAIT_FOREVER);#elsertos_osal_mutex_get(&qspi_flash_ctx->mutex,RTOS_OSAL_WAIT_FOREVER);/* User logic here */#endif}
Unlocking drivers (power_control.c)
staticvoiddriver_control_unlock(void){#if ON_TILE(POWER_CONTROL_TILE_NO)rtos_osal_mutex_put(&gpio_ctx_t0->lock);#else/* User logic here */rtos_osal_mutex_put(&qspi_flash_ctx->mutex);#endif}
This implementation also includes function calls that are for evaluation/diagnosis purposes and may
be removed for end applications. This includes calls to:
led_indicate_awake
led_indicate_asleep
When removing these calls, the associated call to power_control_ind_complete must either be moved
to another location in the application (this is currently handled in led.c’s led_task) or logic
associated with TASK_NOTIF_MASK_LP_IND_COMPLETE should be removed/disabled. The power_control_ind_complete
routine provides a basic means for the power control task to wait for another asynchronous process
to complete before proceeding with the state transition logic.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Speech Recognition£££sln-voice-low-power-ffd-speech-recognition.html#speech-recognition
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$License£££sln-voice-low-power-ffd-speech-recognition.html#license
The Sensory TrulyHandsFree™ (THF) speech recognition library is Copyright (C) 1995-2022 Sensory Inc., All Rights Reserved.
Sensory THF software requires a commercial license granted by Sensory Inc.
This software ships with an expiring development license. It will suspend recognition after 11.4 hours
or 107 recognition events.
The Sensory THF speech recognition engine runs proprietary models to identify keywords in an audio stream. Models can be generated using VoiceHub.
Two models are provided for the purpose of Low Power FFD. The small wake word model running on tile 1
is approximately 67KB. The command model running on tile 0 is approximately 289KB. On tile 1, the
Sensory runtime and application supporting code consumes approximately 239KB of SRAM. On tile 0, the
Sensory runtime and application supporting code consumes approximately 210KB of SRAM.
With the command model in flash, the Sensory engine requires a core frequency of at least 450 MHz to
keep up with real time. Additionally, the intent engine that is responsible for processing the
commands must be on the same tile as the flash.
To run with a different model, see the SetSensorymodelvariables section of the low_power_ffd.cmake file. There several variables are set pointing to files that are part of the VoiceHub generated model download. Change these variables to point to the files you downloaded. This can be done for both the wakeword and command models. The command model “net.bin” file, because it is placed in flash memory, must first be nibble swapped. A utility is provided that is part of the host applications built during install. Run that application with the following command:
Make sure run the following commands to rebuild and re-flash the data partition:
make cleanmake flash_app_example_low_power_ffd -j
You may also wish to modify the command ID-to-string lookup table which is located in the src/intent_engine/intent_engine_io.c source file.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Wake Word Dictionary£££sln-voice-low-power-ffd-speech-recognition.html#wake-word-dictionary
English Language Wake Words
Return code (decimal)
Utterance
1
Hello XMOS
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Command Dictionary£££sln-voice-low-power-ffd-speech-recognition.html#command-dictionary
English Language Commands
Return code (decimal)
Utterance
1
Switch on the TV
2
Channel up
3
Channel down
4
Volume up
5
Volume down
6
Switch off the TV
7
Switch on the lights
8
Brightness up
9
Brightness down
10
Switch off the lights
11
Switch on the fan
12
Speed up the fan
13
Slow down the fan
14
Set higher temperature
15
Set lower temperature
16
Switch off the fan
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Low Power Far-field Voice Local Command$$$Modifying the Software$$$Application Integration£££sln-voice-low-power-ffd-speech-recognition.html#application-integration
In depth information on out of the box integration can be found here: Host Integration
This is the XCORE-VOICE far-field voice assistant example design.
This application can be used out of the box as a voice processor solution, or expanded to run local wakeword engines.
This application features a full duplex acoustic echo cancellation stage, which can be provided reference audio via I2S or USB audio. An audio output ASR stream is also available via I2S or USB audio.
By default, there are two audio integration options. The INT (Integrated) configuration uses I2S for reference and output audio streams. The UA (USB Accessory) configuration uses USB UAC 2.0 for reference and output audio streams.
Connect the xTAG to the debug header, as shown below.
Connect the micro USB XTAG4 and micro USB XK-VOICE-L71 to the programming host.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Linux or macOS£££sln-voice-ffva-deploying-linux-macos-programming-guide.html#deploying-the-firmware-with-linux-or-macos
This document explains how to deploy the software using CMake and Make.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Host Applications£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-host-applications
This application requires a host application to create the flash data partition. Run the following commands in the root folder to build the host application using your native Toolchain:
Note
Permissions may be required to install the host applications.
cmake -B build_hostcd build_hostmake install
The host applications will be installed at /opt/xmos/bin, and may be moved if desired. You may wish to add this directory to your PATH variable.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-firmware
After having your python environment activated, run the following commands in the root folder to build the I2S firmware:
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Running the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#running-the-firmware
Before the firmware is run, the filesystem must be loaded.
Inside of the build folder root, after building the firmware, run one of:
make flash_app_example_ffva_int_fixed_delaymake flash_app_example_ffva_int_cyberon_fixed_delaymake flash_app_example_ffva_ua_adec_altarch
Once flashed, the application will run.
After the filesystem has been flashed once, the application can be run without flashing. If changes are made to the filesystem image, the application must be reflashed.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Upgrading the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#upgrading-the-firmware
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Linux or macOS$$$UA variant£££sln-voice-ffva-deploying-linux-macos-programming-guide.html#ua-variant
The UA variants of this application contain DFU over the USB DFU Class V1.1 transport method.
To create an upgrade image from the build folder run:
make create_upgrade_img_example_ffva_ua_adec_altarch
Once the application is running, a USB DFU v1.1 tool can be used to perform various actions. This example will demonstrate with dfu-util commands. Installation instructions for the respective operating systems can be found here.
The DFU interprets the flash as 3 separate partitions, the read only factory image, the read/write upgrade image, and the read/write data partition containing the filesystem.
The factory image can be read back by running:
dfu-util -e -d ,20b1:4001 -a 0 -U readback_factory_img.bin
The factory image can not be written to.
From the build folder, the upgrade image can be written by running:
dfu-util -e -d ,20b1:4001 -a 1 -D example_ffva_ua_adec_altarch_upgrade.bin
The upgrade image can be read back by running:
dfu-util -e -d ,20b1:4001 -a 1 -U readback_upgrade_img.bin
On system reboot, the upgrade image will always be loaded if valid. If the upgrade image is invalid, the factory image will be loaded. To revert back to the factory image, you can upload a file containing the word 0xFFFFFFFF.
The data partition image can be read back by running:
dfu-util -e -d ,20b1:4001 -a 2 -U readback_data_partition_img.bin
The data partition image can be written by running:
dfu-util -e -d ,20b1:4001 -a 2 -D readback_data_partition_img.bin
Note that the data partition will always be at the address specified in the initial flashing call.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Linux or macOS$$$INT variant£££sln-voice-ffva-deploying-linux-macos-programming-guide.html#int-variant
The INT variants of this application contain DFU over I2C.
To create an upgrade image from the build folder run:
make create_upgrade_img_example_ffva_int_fixed_delay
Once the application is running, the xvf_dfu tool can be used to perform various actions. Installation instructions for Raspbian OS can be found here.
Before running the xvf_dfu host application, the I2C_ADDRESS value in the file transport_config.yaml located in the same folder as the binary file xvf_dfu must be updated. This value must match the one set for appconf_CONTROL_I2C_DEVICE_ADDR in the platform_conf.h file.
The DFU interprets the flash as 3 separate partitions, the read only factory image, the read/write upgrade image, and the read/write data partition containing the filesystem.
The factory image can be read back by running:
xvf_dfu --upload-factory readback_factory_img.bin
The factory image can not be written to.
From the build folder, the upgrade image can be written by running:
On system reboot, the upgrade image will always be loaded if valid. If the upgrade image is invalid, the factory image will be loaded. To revert back to the factory image, you can upload a file containing the word 0xFFFFFFFF.
The FFVA-INT variants include some version numbers:
APP_VERSION_MAJOR
APP_VERSION_MINOR
APP_VERSION_PATCH
These values are defined in the app_conf.h file, and they can read by running:
xvf_dfu --version
The data partition image cannot be read or write using the xvf_dfu host application.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Debugging the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#debugging-the-firmware
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows£££doc/programming_guide/ffva/deploying/native_windows.html#deploying-the-firmware-with-native-windows
This document explains how to deploy the software using CMake and Ninja. If you are not using native Windows MSVC build tools and instead using a Linux emulation tool, refer to Deploying the Firmware with Linux or macOS.
To install Ninja follow install instructions at https://ninja-build.org/ or on Windows
install with winget by running the following commands in PowerShell:
# InstallwingetinstallNinja-build.ninja# Reload user Path$env:Path=[System.Environment]::GetEnvironmentVariable("Path","User")
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Host Applications£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-host-applications
This application requires a host application to create the flash data partition. Run the following commands in the root folder to build the host application using your native Toolchain:
Note
Permissions may be required to install the host applications.
Note
A C/C++ compiler, such as Visual Studio or MinGW, must be included in the path.
Before building the host application, you will need to add the path to the XTC Tools to your environment.
The host applications will be installed at %USERPROFILE%\.xmos\bin, and may be moved if desired. You may wish to add this directory to your PATH variable.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Building the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#building-the-firmware
After having your python environment activated, run the following commands in the root folder to build the I2S firmware:
After the filesystem has been flashed once, the application can be run without flashing. If changes are made to the filesystem image, the application must be reflashed.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Upgrading the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#upgrading-the-firmware
The UA variants of this application contain DFU over the USB DFU Class V1.1 transport method.
In this section DFU over I2C for the INT variants is not covered. The INT variants require an I2C connection to the host, and Windows doesn’t support this feature.
To create an upgrade image from the build folder run:
Once the application is running, a USB DFU v1.1 tool can be used to perform various actions. This example will demonstrate with dfu-util commands. Installation instructions for respective operating system can be found here
The DFU interprets the flash as 3 separate partitions, the read only factory image, the read/write upgrade image, and the read/write data partition containing the filesystem.
The factory image can be read back by running:
dfu-util -e -d ,20b1:4001 -a 0 -U readback_factory_img.bin
The factory image can not be written to.
From the build folder, the upgrade image can be written by running:
dfu-util -e -d ,20b1:4001 -a 1 -D example_ffva_ua_adec_altarch_upgrade.bin
The upgrade image can be read back by running:
dfu-util -e -d ,20b1:4001 -a 1 -U readback_upgrade_img.bin
On system reboot, the upgrade image will always be loaded if valid. If the upgrade image is invalid, the factory image will be loaded. To revert back to the factory image, you can upload an file containing the word 0xFFFFFFFF.
The data partition image can be read back by running:
dfu-util -e -d ,20b1:4001 -a 2 -U readback_data_partition_img.bin
The data partition image can be written by running:
dfu-util -e -d ,20b1:4001 -a 2 -D readback_data_partition_img.bin
Note that the data partition will always be at the address specified in the initial flashing call.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Deploying the Firmware with Native Windows$$$Debugging the Firmware£££doc/programming_guide/ffva/deploying/native_windows.html#debugging-the-firmware
This example design can be integrated with existing solutions or modified to be a single controller solution.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Modifying the Software$$$Out of the Box Integration£££doc/programming_guide/ffva/design.html#out-of-the-box-integration
Out of the box integration varies based on configuration.
INT requires I2S connections to the host. Refer to the schematic, connecting the host reference audio playback to the ADC I2S and the host input audio to the DAC I2S. Out of the box, the INT configuration requires an externally generated MCLK of 12.288 MHz. 24.576 MHz is also supported and can be changed via the compile option MIC_ARRAY_CONFIG_MCLK_FREQ, found in ffva_int.cmake.
UA requires a USB connection to the host.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Modifying the Software$$$Support for ASR engine£££doc/programming_guide/ffva/design.html#support-for-asr-engine
The example_ffva_int_cyberon_fixed_delay provides an example about how to include an ASR engine, the Cyberon DSPotter™.
Most of the considerations made in the section about the FFD devices are still valid for the FFVA example. The only notable difference is that the pipeline output in the FFVA example
is on the same tile as the ASR engine, i.e. tile 0.
Note
Both the audio pipeline and the ASR engine process use the same sample block length. appconfINTENT_SAMPLE_BLOCK_LENGTH and appconfAUDIO_PIPELINE_FRAME_ADVANCE are both 240.
The application consists of a PDM microphone input which is fed through the XMOS-VOICE DSP blocks. The output ASR channel is then output over I2S or USB.
The DFU process is internally managed by the DFU controller module within the firmware.
This module is tasked with overseeing the DFU state machine and executing DFU operations.
The list of states and transactions are represented in the diagram in Fig. 1.
the appIDLE and appDETACH states are not implemented, and the device is started in the dfuIDLE state
the device goes into the dfuIDLE state when a SET_ALTERNATE message is received
the device is rebooted when a DFU_DETACH command is received.
The DFU allows the following operations:
download of an upgrade image to the device
upload of factory and upgrade images from the device
reboot of the device.
The rest of this section describes the message sequence charts of the supported operations.
A message sequence chart of the download operation is below:
Message sequence chart of the download operation
Note
The end of the image transfer is indicated by a DFU_DNLOAD message of size 0.
Note
The DFU_DETACH message is used to trigger the reboot.
Note
For the I2C implementation, specification of the block number in download is not supported; all downloads must start with block number 0 and must be run to completion. The device will track this progress internally.
A message sequence chart of the reboot operation is below:
Message sequence chart of the reboot operation
Note
The DFU_DETACH message is used to trigger the reboot.
A message sequence chart of the upload operation is below:
Message sequence chart of the upload operation
Note
The end of the image transfer is indicated by a DFU_UPLOAD message of size less than the transport medium maximum; this is 4096 bytes in UA and 128 bytes in INT.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Modifying the Software$$$DFU over USB implementation£££dfu-usb-interface-design.html#dfu-over-usb-implementation
The UA variant of the device makes use of a USB connection for handling DFU operations.
This interface is a relatively standard, specification-compliant implementation.
The implementation is encapsulated within the tinyUSB library, which provides a USB stack for the sln_voice.
DFU over I2C implementation
The INT variant of the device presents a DFU interface that may be controlled
over I2C.
Fig. 5 shows the modules involved in
processing the DFU commands. The I2C£££sln-voice-low-power-ffd-host-integration.html#i2c task has a dedicated logical core so that it is always ready
to receive and send control messages. The DFU state machine is driven by the control commands. The DFU state
machine interacts with a separate RTOS task in
order to asynchronously perform flash read/write operations.
sln_voice Control Plane Components Diagram
Fig. 6 shows the interaction
between the Device Control module and the DFU Servicer.
In this diagram, boxes with the same colour reside in the same RTOS task.
sln_voice Device Control – Servicer Flow Chart
This diagram shows a critical aspect of the DFU control operation.
The Device Control module, having placed a command on a Servicer’s command
queue, waits on the Gateway queue for a response.
As a result, it ensures processing of a single control command at a time.
Limiting DFU control operation to a single command in-flight reduces the
complexity of the control protocol and eliminates several potential error
cases.
The FFVA-INT uses a packet protocol to receive control commands and send each
corresponding response.
Because packet transmission occurs over a very short-haul transport, as in
I2C, the protocol does not include fields for error detection or correction such as start-of-frame and
end-of-frame symbols, a cyclical redundancy check or an error correcting code.
Fig. 7 depicts the structure of each packet.
sln_voice Control Plane Packet Diagram
Packets containing a response from the FFVA-INT to the host application place
a status value in the first byte of the payload.
Mirroring the USB DFU specification, the INT DFU implementation supports a set of 9
control commands intended to drive the state machine, along with an additional 2
utility commands:
DFU commands
Name
ID
Length
Payload Structure
Purpose
DFU_DETACH
0
1
Payload unused
Write-only command. Restarts the device. Payload is required for protocol, but is discarded within the device. This command has a defined purpose in the USB DFU specification, but in a deviation to that specification it is used with I2C simply to reboot the device. Future versions of the XMOS DFU-by-device-control protocol (but not future versions of this product) may choose to alter the function of this command to more closely align with the USB DFU specification.
DFU_DNLOAD
1
130
2 bytes length marker, followed by 128 bytes of data buffer
Write-only command. The first two bytes indicate how many bytes of data are being transmitted in this packet. These bytes are little-endian, so byte 0 represents the low byte and byte 1 represents the high byte of an unsigned 16b integer. The remaining 128 bytes are a data buffer for transfer to the device. All control command packets are a fixed length, and therefore all 128 bytes must be included in the command, even if unused. For example, a payload with length of 100 should have the first 100 bytes of data set, but must send an additional 28 bytes of arbitrary data.
DFU_UPLOAD
2
130
2 bytes length marker, followed by 128 bytes of data buffer
Read-only command. The first two bytes indicate how many bytes of data are being transmitted in this packet. These bytes are little-endian, so byte 0 represents the low byte and byte 1 represents the high byte of an unsigned 16b integer. The remaining 128 bytes are a data buffer of data received from the device. All control command packets are a fixed length, and therefore this buffer will be padded to length 128 by the device before transmission. The device will, as per the USB DFU specification, mark the end of the upload process by sending a “short frame” - a packet with a length marker less than 128 bytes.
DFU_GETSTATUS
3
5
1 byte representing device status, 3 bytes representing the requested timeout, 1 byte representing the next device state.
Read-only command. The first byte returns the device status code, as described in the USB DFU specification in the table in section 6.1.2. The next 3 bytes represent the amount of time the host should wait, in ms, before issuing any other commands. This timeout is used in the DNLOAD process to allow the device time to write to flash. This value is little-endian, so bytes 1, 2, and 3 represent the low, middle, and high bytes respectively of an unsigned 24b integer. The final byte returns the number of the state that the device will move into immediately following the return of this request, as described in the USB DFU specification in the table in section 6.1.2.
DFU_CLRSTATUS
4
1
Payload unused
Write-only command. Moves the device out of state 10, dfuERROR. Payload is required for protocol, but is discarded within the device.
DFU_GETSTATE
5
1
1 byte representing current device state.
Read-only command. The first (and only) byte represents the number of the state that the device is currently in, as described in the USB DFU specification in the table in section 6.1.2.
DFU_ABORT
6
1
Payload unused
Write-only command. Aborts an ongoing upload or download process. Payload is required for protocol, but is discarded within the device.
DFU_SETALTERNATE
64
1
1 byte representing either factory (0) or upgrade (1) DFU target images
Write-only command. Sets which of the factory or upgrade images should be targeted by any subsequent upload or download commands. Use of this command entirely resets the DFU state machine to initial conditions: the device will move to dfuIDLE, clear all error conditions, wipe all internal DFU data buffers, and reset all other DFU state apart from the DFU_TRANSFERBLOCK value. This command is included to emulate the SET_ALTERNATE request available in USB.
DFU_TRANSFERBLOCK
65
2
2 bytes, representing the target transfer block for an upload process.
Read/write command. Sets/gets a 2 byte value specifying the transfer block number to use for a subsequent upload operation. A complete image may be conceptually divided into 128-byte blocks. These blocks may then be numbered from 0 upwards. Setting this value sets which block will be returned by a subsequent DFU_UPLOAD request. This value is initialised to 0, and autoincrements after each successful DFU_UPLOAD request has been serviced. Therefore, to read a whole image from the start, there is no need to issue this command - this command need only be used to select a specific section to read. Because this value is automatically incremented after a DFU_UPLOAD command is successfully serviced, reading it will give the value of the next block to be read (and this will be one greater than the previous block read, if it has not been altered in the interim). This value is reset to 0 at the successful completion of a DFU_UPLOAD process. It is not reset after a DFU_ABORT, nor after a DFU_SETALTERNATE call. This command is included to emulate the ability in a USB request to send values in the header of the request - the device control protocol used here does not allow sending any data with a read request such as DFU_UPLOAD.
DFU_GETVERSION
88
3
3 bytes, representing major.minor.patch version of device
Read-only command. Bytes 0, 1, and 2 represent the major, minor, and patch versions respectively of the device. This is a utility command intended to provide an easy mechanism by which to verify that a firmware download has been successful.
DFU_REBOOT
89
1
Payload unused
Write-only command. Restarts the device. Payload is required for protocol, but is discarded within the device. This is a utility command intended to provide a clear and unambiguous interface for restarting the device. Use of this command should be preferred over DFU_DETACH for this purpose.
When writing a custom compliant host application, the use of XMOS’ fwk_rtos
library is advised; the device_control library provided there gives a host
API that can communicate effectively with the FFVA-INT. A description of the I2C bus activity
during the execution of the above DFU commands is provided below, in the
instance that usage of the device_control library is inconvenient or
impossible.
The FFVA-INT I2C address is set by default as 0x42. This may be
confirmed by examination of the appconf_CONTROL_I2C_DEVICE_ADDR define in the
platform_conf.h file. The I2C address may also be altered by editing this file.
The DFU resource has an internal “resource ID” of 0xF0. This maps to the
register that read/write operations on the DFU resource should target -
therefore, the register to write to will always be 0xF0.
To issue a write command (e.g. DFU_SETALTERNATE):
First, set up a write to the device address. For a default device
configuration, a write operation will always start by a write token to 0x42
(START, 7 bits of address [0x42], R/W bit [0 to specify write]), wait for ACK,
followed by specifying the register to write [Resource ID 0xF0]
(and again wait for ACK).
Then, write the command ID (in this example, 64 [0x40]) from the above table.
Then, write the total transfer size, including the register byte. In this
example, that will be 4 bytes (register byte, command ID, length byte, and 1
byte of payload), so write 0x04.
Finally, send the payload - e.g. 1 to set the alternate setting to “upgrade”.
The full sequence for this write command will therefore be START, 7 bits of
address [0x42], 0 (to specify write), hold for ACK, 0xF0, hold for ACK, 0x40,
hold for ACK, 0x04, hold for ACK, 0x01, hold for ACK, STOP.
To complete the transaction, the device must then be queried; set up a read to
0x42 (START, 7 bits of address [0x42], R/W bit [1 to specify read], wait for
ACK). The device will clock-stretch until it is ready, at which point it will
release the clock and transmit one byte of status information. This will be a
value from the enum control_ret_t from device_control_shared.h,
found in modules\rtos\modules\sw_services\device_control\api.
To issue a read command (e.g. DFU_GETSTATUS):
Set up a write to the device; as above, this will mean sending START,
7 bits of device address [0x42], 0 (to specify write), hold for ACK. Send the
DFU resource ID [0xF0], hold for ACK.
Then, write the command ID (in this example, 3), bitwise ANDed with 0x80 (to
specify this as a read command) - in this example therefore 0x83 should be
sent, and hold for ACK.
Then, write the total length of the expected reply. In this example, the
command has a payload of 5 bytes. The device will also prepend the payload
with a status byte. Therefore, the expected reply length will be 6 bytes
[0x06]. Hold for ACK.
Then, issue a repeated START. Follow this with a read from the device:
the repeated START, 7 bits of device address [0x42], 1 (to specify read), hold
for ACK. The device will clock-stretch until it is ready. It will then send
a status byte (from the enum control_ret_t as described above), followed
by a payload of requested data - in this example, the device will send 5
bytes. ACK each received byte. After the last expected byte, issue a STOP.
This application features 16kHz and 48kHz audio input and output. The XMOS DPS blocks operate on 16kHz audio. Input streams are downsampled when needed. Output streams are upsampled when needed. When in I2S modes This function is called by the bsp_config to enable the I2S sample rate conversion.
The FFVA example design consists of three major software blocks, the audio interface, audio pipeline, and placeholder for a keyword handler. This section will go into detail on how to modify each/all of these subsystems.
It is highly recommended to be familiar with the application as a whole before attempting replacing these functional units.
See Memory and CPU Requirements for more details on the memory footprint and CPU usage of the major software components.
The audio pipeline can be replaced by making changes to the audio_pipeline.c file.
It is up to the user to ensure that the input and output frames of the audio pipeline remain the same, or the remainder of the application will not function properly.
This section will walk through an example of replacing the XMOS NS stage, with a custom stage foo.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Modifying the Software$$$Declaration and Definition of DSP Context£££doc/programming_guide/ffva/software_modifications.html#declaration-and-definition-of-dsp-context
It is also possible to add or remove stages. Refer to the RTOS Framework documentation on the generic pipeline sw_service.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Modifying the Software$$$Changing the ASR engine£££doc/programming_guide/ffva/software_modifications.html#changing-the-asr-engine
THE FFVA provides an example with a specific ASR engine. A different ASR engine can be used by updating and adding the necessary files in modules\asr.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$Far-field Voice Assistant$$$Modifying the Software$$$Replacing Example Design Interfaces£££doc/programming_guide/ffva/software_modifications.html#replacing-example-design-interfaces
It may be desired to have a different input or output interfaces to talk to a host.
One example use case may be to create a hybrid audio solution where reference frames or output audio streams are used over an interface other than I2S or USB.
Audio Pipeline Input (main.c)
voidaudio_pipeline_input(void*input_app_data,int32_t**input_audio_frames,size_tch_count,size_tframe_count){(void)input_app_data;int32_t**mic_ptr=(int32_t**)(input_audio_frames+(2*frame_count));staticintflushed;while(!flushed){size_treceived;received=rtos_mic_array_rx(mic_array_ctx,mic_ptr,frame_count,0);if(received==0){rtos_mic_array_rx(mic_array_ctx,mic_ptr,frame_count,portMAX_DELAY);flushed=1;}}rtos_mic_array_rx(mic_array_ctx,mic_ptr,frame_count,portMAX_DELAY);/* Your ref input source here */}
Refer to documentation inside the RTOS Framework on how to instantiate different RTOS peripheral drivers. Populate the above code snippet with your input frame source. Refer to the default application for an example of populating reference via I2S or USB.
Audio Pipeline Output (main.c)
intaudio_pipeline_output(void*output_app_data,int32_t**output_audio_frames,size_tch_count,size_tframe_count){(void)output_app_data;/* Your output sink here */#if appconfWW_ENABLEDww_audio_send(intertile_ctx,frame_count,(int32_t(*)[2])output_audio_frames);#endifreturnAUDIO_PIPELINE_FREE_FRAME;}
Refer to documentation inside the RTOS Framework on how to instantiate different RTOS peripheral drivers. Populate the above code snippet with your output frame sink. Refer to the default application for an example of outputting the ASR channel via I2S or USB.
To add or remove a peripheral IO, modify the bsp_config accordingly. Refer to documentation inside the RTOS Framework on how to instantiate different RTOS peripheral drivers.
This example is deprecated and will be moved into a separate
Application Note and may be removed in the next major release.
This example provides a bridge between 16 PDM microphones to either
TDM16 slave or USB Audio and targets the xcore-ai explorer board.
This application is to support cases where many microphone inputs need
to be sent to a host where signal processing will be performed. Please
see the other examples in sln_voice where signal processing is performed
within the xcore in firmware.
This example uses a modified mic_array with multiple decimator threads to
support 16 DDR microphones on a single 8 bit input port. The example is written as
‘bare-metal’ and runs directly on the XCORE device without an RTOS.
It is recommended to use Ninja or xmake as the make system under Windows.
Ninja has been observed to be faster than xmake, however xmake comes natively with XTC tools.
This firmware has been tested with Ninja version v1.11.1.
Ensure Ninja is on the command line path. You can add to the path
permanently by following these steps
https://www.computerhope.com/issues/ch000549.htm. Alternatively you
may set the path in the current command line session using something
like setPATH=%PATH%;C:\Users\xmos\utils\ninja
After having your python environment activated, run the following commands in the root folder to build the firmware:
The design consists of a number of tasks connected via the xcore-ai silicon communication channels.
The decimators in the microphone array are configured to produce a 48 kHz PCM output.
The 16 output channels are loaded into a 16 slot TDM slave peripheral running at 24.576 MHz bit
clock or a USB Audio Class 2 asynchronous interface and are optionally
amplified. The TDM build also provides a simple I2C slave interface to allow
gains to be controlled at run-time. The USB build supports USB Audio Class 2 compliant volume controls.
For the TDM build, a simple TDM16 master peripheral is included as well as a local
24.576 MHz clock source so that mic_array and TDM16 slave operation may be tested
standalone through the use of jumper cables. These may be removed when
integrating into a system with TDM16 master supplied.
The applications are written on bare metal and use logical cores (hardware threads)
to implement the functional blocks. Each of the tasks are connected using channels provided in the
xcore-ai architecture. The thread diagrams are shown in Fig. 8 and Fig. 9.
Both the TDM and USB aggregator examples share a common PDM front end. This consists of an 8 bit port
with each data line connected to two PDM microphones each configured to provide data
on a different clock edge. The 3.072 MHz clock for the PDM microphones is provided by the xcore-ai
device on a 1 bit port and clocks all PDM microphones. The PDM clock is divided down from the 24.576 MHz
local MCLK.
The data collected by the 8 bit port is sent to the lib_mic_array block which de-interleaves
the PDM data streams and performs decimation of the PDM data down to 48 kHz 32 bit PCM samples.
Due to the large number of microphones the PDM capture stage uses four hardware threads on tile[0]; one for the microphone
capture and three for decimation. This is needed to divide the processing workload and meet timing comfortably.
Samples are forwarded to the next stage at a rate of 48 kHz resulting in a packet of 16
PCM samples per exchange.
The 16 channels of 48 kHz PCM streams are collected by Hub and are amplified using a
saturated gain stage. The initial gain is set to 100, since a gain of 1 sounds very
quiet due to the mic_array output being scaled to allow acoustic
overload of the microphones without clipping within the decimators. This value can be
overridden using the MIC_GAIN_INIT define in app_conf.h.
Additionally for the TDM configuration, the Hub task also checks for control packets
from I2C which may be used to dynamically update the individual gains at runtime.
A single hardware thread contains the task and a triple buffer scheme is used to ensure there is always
a free buffer available to write into regardless of the relative phase between the production
and consumption of microphone samples.
The Hub task has plenty of timing slack and is a suitable place for adding signal processing
if needed.
The TDM build supports a 16-slot TDM slave Tx peripheral from the fwk_io sub-module. In this application
it runs at 24.576 MHz bit clock which supports 16 channels of 32 bit, 48 kHz samples per frame.
The TDM component uses a single hardware thread.
For the purpose of debugging a simple TDM 16 Master Rx component is provided. This allows the transmitted
TDM frames from the application to be received and checked without having to connect an external
TDM Master. It may be deleted / disconnected without affecting the core application.
Note
The simple TDM 16 Master Rx component is not regression tested and is for evaluation of TDM 16 Slave Tx in this application only.
The xcore-ai device has a total resource count of 2 x 524288 Bytes of memory and 2 x 8 hardware threads across two tiles.
This application uses around half of the processing resources and a tiny fraction of the available memory
meaning there is plenty of space inside the chip for additional functionality if needed.
For the TDM build, there are 32 registers which control the gain of each of the 16 output
channels. The 8 bit registers contain the upper 8 bit and lower 8 bit of the
microphone gain respectively. The initial gain is set to 100, since 1 is
quiet due to the mic_array output being scaled to allow acoustic
overload of the microphones without clipping. Typically a gain of a few
hundred works for normal conditions. The gain is only applied after the
lower byte is written.
The gain applied is saturating so no overflow will occur, only clipping.
Register
Value
0
Channel 0 upper gain byte
1
Channel 0 lower gain byte
2
Channel 1 upper gain byte
3
Channel 1 lower gain byte
4
Channel 2 upper gain byte
5
Channel 2 lower gain byte
6
Channel 3 upper gain byte
7
Channel 3 lower gain byte
8
Channel 4 upper gain byte
9
Channel 4 lower gain byte
10
Channel 5 upper gain byte
11
Channel 5 lower gain byte
12
Channel 6 upper gain byte
13
Channel 6 lower gain byte
14
Channel 7 upper gain byte
15
Channel 7 lower gain byte
16
Channel 8 upper gain byte
17
Channel 8 lower gain byte
18
Channel 9 upper gain byte
19
Channel 9 lower gain byte
20
Channel 10 upper gain byte
21
Channel 10 lower gain byte
22
Channel 11 upper gain byte
23
Channel 11 lower gain byte
24
Channel 12 upper gain byte
25
Channel 12 lower gain byte
26
Channel 13 upper gain byte
27
Channel 13 lower gain byte
28
Channel 14 upper gain byte
29
Channel 14 lower gain byte
30
Channel 15 upper gain byte
31
Channel 15 lower gain byte
If using a raspberry Pi as the I2C host you may use the following
commands:
$ i2cset -y 1 0x3c 0 0 #Set the gain on mic channel 0 to 50
$ i2cset -y 1 0x3c 1 50 #Set the gain on mic channel 0 to 50
$ i2cget -y 1 0x3c 0 #Get the upper byte of gain on mic channel 0
$ i2cget -y 1 0x3c 1 #Get the lower byte of gain on mic channel 0
$ i2cset -y 1 0x3c 16 1 #Set the gain on mic channel 8 to 256
$ i2cset -y 1 0x3c 15 0 #Set the gain on mic channel 8 to 256
This example is based on the RTOS framework and drivers. This choice simplifies the example design, but it leads to high latency in the system.
The main sources of latency are:
Large block size used for ASRC processing: this is necessary to minimise latency associated with the intertile context and thread switching overhead.
Large size of the buffer to which the ASRC output samples are written: a stable level (half full) must be reached before the start of streaming out over USB.
RTOS task scheduling overhead between the tasks.
bInterval of USB in the RTOS drivers is set to 4, i.e. one frame every 1 ms.
Block based implementation of the USB and I2S RTOS drivers.
The expected latencies for USB at 48 kHz are as follows:
USB -> ASRC -> I2S: from 8 ms at I2S at 192 kHz to 22 ms at 44.1 kHz
I2S -> ASRC -> USB: from 13 ms at I2S at 192 kHz to 19 ms at 44.1 kHz
For a proposed implementation with lower latency, please refer to the bare-metal examples below:
This is the XCORE-VOICE Asynchronous Sampling Rate Converter (ASRC) example design.
The example system implements a stereo I2S Slave and a stereo Adaptive UAC2.0 interface and exchanges data between the two interfaces.
Since the two interfaces are operating in different clock domains, there is an ASRC block between them that converts from the input to the output sampling rate.
There are two ASRC blocks, one each in the I2S -> ASRC -> USB and USB -> ASRC -> I2S path, as illustrated in the ASRC example top level system diagram.
The diagram also shows the rate calculation path, which monitors and computes the instantaneous ratio between the ASRC input and output sampling rate.
The rate ratio is used by the ASRC task to dynamically adapt filter coefficients using spline interpolation in its filtering stage.
ASRC example top level system diagram
The I2S Slave interface is a stereo 32 bit interface supporting sampling rates between 44.1 kHz - 192 kHz.
The USB interface is a stereo, 32 bit, 48 kHz, High-Speed, USB Audio Class 2, Adaptive interface.
The ASRC algorithm implemented in the lib_src library is used for the ASRC processing.
The ASRC processing is block based and works on a block size of 244 samples per channel in the I2S -> ASRC -> USB path and 96 samples per channel in the USB -> ASRC -> I2S path.
This example application is supported on the XK-VOICE-L71 board.
In addition to the XK-VOICE-L71 board, it requires an XTAG4 to program and debug the device.
To demonstrate the audio exchange between the I2S and USB interface, the XK-VOICE-L71 device needs to be connected to an I2S Master device.
To do this, connect the BCLK, MCLK, DOUT, DIN pins of the RASPBERRY PI HOST INTERFACE header (J4) on the XK-VOICE-L71 to the I2S Master.
The table XK-VOICE-L71 RPI host interface header (J4) connections lists the pins on the XK-VOICE-L71 RPI header and the signals on the I2S Master that they need to be connected to.
It is recommended to use Ninja or xmake as the make system under Windows.
Ninja has been observed to be faster than xmake, however xmake comes natively with XTC tools.
This firmware has been tested with Ninja version v1.11.1.
To install Ninja, follow these steps:
Download ninja.exe from here.
This firmware has been tested with Ninja version v1.11.1.
Ensure Ninja is on the command line path. It can be added to the path
permanently by following the steps listed
here. Alternatively,
set the path in the current command line session using something
like setPATH=%PATH%;C:\Users\xmos\utils\ninja
To build for the first time, activate your python environment, run cmake to create the
make files:
Following initial cmake build, for subsequent builds, as long as new source files are not added, just type:
$ ninja example_asrc_demo.xe
cmake needs to be rerun to discover any new source files added.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Example Designs$$$ASRC Application$$$Overview$$$Running the app£££table-pin-connections-label.html#running-the-app
To run the app, either xrun or xflash can be used. Connect the XK-VOICE-L71 board to the host and type the following
to run with real-time debug output enabled:
$ xrun --xscope example_asrc_demo.xe
or to flash the application so that it always boots after a power cycle:
When the example runs, the audio received by the device on the I2S Slave interface at the I2S interface sampling rate is
sample rate converted using the ASRC to the USB sampling rate and streamed out from the device over the USB interface. Similarly,
the audio streamed out by the USB host into the USB interface of the device is sample rate converted to the I2S interface sampling
rate and streamed out from the device over the I2S Slave interface.
This example supports dynamic changes of the I2S interface sampling frequency at runtime. It detects the I2S sampling rate change and reconfigures
the system for the new rate.
The ASRC demo application is a two tile application developed to run on the XK-VOICE-L71 board running at a core frequency of 600 MHz.
It is a FreeRTOS based application where all the application blocks are implemented as FreeRTOS tasks.
Each tile has 5 bare metal cores dedicated to running RTOS tasks and since all processing is done within RTOS tasks, each core has 120 MHz of bandwidth
available.
The tasks can roughly be categorised as belonging to the USB driver, I2S driver or the application code categories.
The actual ASRC processing happens in four tasks across the two tiles; the usb_audio_out_asrc task, i2s_audio_recv_asrc task, and two instances of asrc_one_channel task, one on each tile.
This is described in more detail in the Application components section below.
Most of the tasks are involved in the ASRC processing data path, while a few are involved in monitoring the input and output data rates
and computing the rate ratio, which is the ratio between the frequencies at the input and output of the ASRC tasks.
The rate ratio is provided to the ASRC tasks every asrc_process_frame() call. Details about the rate ratio calculation are described in the rate_server section below.
This application presents a stereo, 48 kHz, 32 bit, high-speed, Adaptive UAC2.0 USB interface.
It has two endpoints, Endpoint 0 for control and Endpoint 1 for bidirectional isochronous USB audio.
The USB application level driver is TinyUSB based.
The usb_xud_thread, usb_isr, usb_task and usb_adaptive_clk_manager implement the USB driver.
Together, these tasks handle the USB communication with the host and also monitor the average USB rate seen by the device.
The average USB rate is used for calculating the rate ratios that are
sent to the asrc_process_frame() function. This is described more in the rate_server section.
The usb_xud_thread runs XUD_Main which implements the USB HIL driver. It runs on a dedicated bare metal core so cannot be preempted by other RTOS tasks.
It interfaces with the USB app level thread (usb_task) via shared memory and dedicated channels between the XUD_Main and each endpoint.
XUD_Main notifies the connected endpoint of a USB transfer completion through an interrupt on the respective channel. This interrupt is serviced by the usb_isr routine.
usb_task implements the app level USB driver functionality. The app level USB driver is based on TinyUSB which hooks into the application by means of callback functions.
The usb_isr task is triggered by the interrupt and parses the data transferred from XUD and places it on a queue that the usb_task blocks on for further processing.
For example, on completion of an EP1 OUT transfer, the transfer completion gets notified on the usb_xud_thread -> usb_isr -> usb_task path,
and the usb_task calls the tud_audio_rx_done_post_read_cb() function to have the application process the data received from the host.
On completion of an EP1 IN transfer, the transfer completion again follows the usb_xud_thread -> usb_isr -> usb_task path, and usb_task calls the tud_audio_tx_done_pre_load_cb()
callback function to have the application load the EP1 IN data for the next transfer.
samples_to_host_stream_buf and samples_from_host_stream_buf are circular buffers shared between the application and the USB driver and allow for decoupling one from the other.
The data frame received over USB from the host is written to the samples_from_host_stream_buf by the TinyUSB callback function tud_audio_rx_done_post_read_cb(),
while the application reads USB_TO_I2S_ASRC_BLOCK_LENGTH samples of data out of it.
Similarly, the application writes the ASRC output block of data to the samples_to_host_stream_buf while the TinyUSB callback function tud_audio_tx_done_pre_load_cb()
reads from it to send one frame of data to the USB host.
usb_adaptive_clk_manager task is responsible for calculating the average USB rate as seen by the device. The average rate is calculated over a 16-second moving window.
The averaging smooths out any jitter seen in the USB SOF timestamps that are used for calculating the rate.
I2S Driver components
This application presents a stereo 32 bit, I2S Slave interface that supports I2S sampling rates of 44.1, 48, 88.2, 96, 176.4 and 192 kHz.
The I2S driver supports tracking dynamic sampling rate (SR) changes and recalculates the nominal sampling rate after detecting a SR change event.
It also continuously monitors the timespan over which a fixed number of samples are received. This information is then used by the application for
calculating the average I2S rate seen by the device.
i2s_slave_thread, I2S send_buffer and receive_buffer and rtos_i2s_isr make up the I2S driver components.
i2s_slave_thread implements the I2S HIL driver. The HIL level driver calls into the application callback functions for i2s_init(), i2s_restart_check(), i2s_receive() and i2s_send().
These functions, in addition to handling I2S send and receive data, also detect sampling rate changes and gather information for tracking the average sampling rate.
I2S send_buffer and receive_buffer are circular buffers shared between the driver and the application and contain data received over I2S (receive_buffer) and data the application wants to send over I2S (send_buffer).
These buffers allow for decoupling the I2S HIL driver from the ASRC application. The driver reads from and writes to these buffers at the I2S sample rate while the application can read and write blocks of data to these buffers equal to the ASRC input or output block size.
The application calls rtos_i2s_rx() to read I2S_TO_USB_ASRC_BLOCK_LENGTH samples of data from the receive_buffer. The i2s_slave_thread independently calls i2s_receive() callback function to write a sample of data as it gets received over I2S.
Similarly, the application calls rtos_i2s_tx() to write ASRC output size block of data into the send_buffer. Meanwhile, the driver independently calls the callback function i2s_send() to read a sample of data to send over the I2S.
rtos_i2s_isr interrupt is used to ensure that the application calls to rtos_i2s_rx() and rtos_i2s_tx() block only on RTOS primitives when waiting for read data to be available or buffer space to be available when writing data.
usb_audio_out_asrc, i2s_audio_recv_asrc, asrc_one_channel_task, usb_to_i2s_intertile, i2s_to_usb_intertile and the rate_server tasks make up the non-driver components of the application.
usb_audio_out_asrc performs ASRC on data received from the USB host to the device. It waits to get notified by the TinyUSB callback function tud_audio_rx_done_post_read_cb() when there are one or more ASRC input blocks (96 USB samples) of data in the samples_from_host_stream_buf.
It does ASRC processing of the first channel while coordinating with the asrc_one_channel_task for processing the second channel in parallel and sends the processed output to the other tile on the inter-tile context.
i2s_audio_recv_asrc performs ASRC on data received over the I2S interface by the device. It blocks on the rtos_i2s_rx() function to receive one ASRC input block (244 I2S samples) of data from I2S and performs ASRC on one channel
while coordinating with the asrc_one_channel_task for processing the second channel in parallel. It then sends the processed output to the other tile on the inter-tile context.
asrc_one_channel_task performs ASRC on a single channel of data. There is one of these on each tile. It waits on an RTOS message queue for an ASRC input block to be available, does ASRC processing on the block and posts the completion notification on another message queue.
usb_to_i2s_intertile task receives the ASRC output data generated by usb_audio_out_asrc over the inter-tile context onto the I2S tile and writes it to the I2S send_buffer.
It has other rate-monitoring related responsibilities that are described in the rate_server section.
i2s_to_usb_intertile task receives the ASRC output data generated by i2s_audio_recv_asrc over the inter-tile context onto the USB tile and writes it to the USB samples_to_host_stream_buf.
It has other rate-monitoring related responsibilities that are described in the rate_server section.
The I2S -> ASRC -> USB data path diagram shows the application tasks involved in the I2S -> ASRC -> USB path processing and their interaction with each other.
I2S -> ASRC -> USB data path
The USB -> ASRC -> I2S data path diagram shows the application tasks involved in the USB -> ASRC -> I2S path processing and their interaction with each other.
The ASRC process_frame API requires the caller to calculate and send the instantaneous ratio between the ASRC input and output rate. The rate_server is responsible for calculating these rate ratios for both USB -> ASRC -> I2S and I2S -> ASRC -> USB directions.
Additionally, the application also monitors the average buffer fill levels of the buffers holding ASRC output to prevent any overflows or underflows of the respective buffer. A gradual drift in the buffer fill level indicates that the rate ratio is being under or over calculated by the rate_server.
This could happen either due to jitter in the actual rates or precision limitations when calculating the rates.
The average fill level of the buffer is monitored and a closed-loop error correction factor is calculated to keep the buffer level at an expected stable level.
The error estimated based on the buffer fill level is used to compute the estimated rate ratio from the initial rate ratio. This estimated rate ratio is then sent to the ASRC process_frame() API.
The rate_server runs on the I2S tile (tile 1) and is periodically triggered from the USB tile (tile 0) by the usb_to_i2s_intertile task. The rate_server is triggered once after every 16 frames are written to the samples_to_host_stream_buf.
The following information is needed for calculating the rate ratios:
The average I2S rate
The average USB rate
An error factor computed based on the USB samples_to_host_stream_buf fill level
An error factor computed based on the I2S sendbuffer fill level
A USB mic_interface_open flag indicating if the USB host is streaming out from the device,
since the rate ratio in the I2S -> ASRC -> USB direction is calculated only when the host is reading data from the device
A USB spkr_interface_open flag indicating if the USB host is streaming into the device,
since the rate ratio in the USB -> ASRC -> I2S direction is calculated only when the host is sending data to the device
Of the above, the USB related information (2, 3, 5 and 6 above) is available on the USB tile. When triggering the rate_server, the i2s_to_usb_intertile task gets this information,
either calculating it or getting it through shared memory from other USB tasks on the same tile, and sends it to the rate_server over the inter-tile context using the structure below.
The I2S related information (1 and 4 above) is calculated in the rate_server itself with information available for calculating these available through shared memory from other tasks on this tile.
After calculating the rates, the rate_server sends the rate ratio for the USB -> ASRC -> I2S side to the usb_to_i2s_intertile task over the inter-tile context and it is made available to the
usb_audio_out_asrc task through shared memory. The I2S -> ASRC -> USB side rate ratio is also made available to the i2s_audio_recv_asrc task through shared memory since it runs on the same tile as the rate server.
The Rate calculation code flow diagram shows the code flow during the rate ratio calculation process, focussing on the usb_to_intertile task that triggers the rate_server and the rate_server task where the rate ratios are calculated.
Rate calculation code flow
Handling I2S sampling rate change events
The I2S driver monitors the I2S nominal rate and provides this information to the application. When an I2S sampling rate change happens:
The ASRC instances on both tiles are re-initialised with the new sampling rate.
The buffers that are used for buffer-fill-level based correction are reset. Streaming out of them is paused while zeroes are sent out over both USB and I2S.
Once the buffers fill to a stable level, streaming out from them resumes.
The average buffer level calculation state is reset and the average buffer level calculation starts afresh.
New stable buffer levels are also calculated and the buffer levels are now corrected against these new stable averages.
Note that the device starts with the nominal I2S sampling rate set to zero. Device startup therefore follows the same path as an I2S sampling rate change where the sampling rate goes from zero to first detected nominal sampling rate.
Everything described above therefore also applies to the device startup behaviour.
Handling USB speaker interface close -> open events
When the USB host stops streaming to the device and then starts again, this event is detected through calls to the tud_audio_set_itf_close_EP_cb and tud_audio_set_itf_cb functions.
The ASRC output buffer in the USB -> ASRC -> I2S path (I2S send_buffer) is reset.
Zeroes are then sent over I2S until the buffer fills to a stable level, when we resume streaming out of this buffer to send samples over I2S.
The average buffer calculation state for the I2S send_buffer is also reset and a new stable average is calculated against which the average buffer levels are corrected.
Handling USB mic interface close -> open events
If the USB host stops streaming from the device and then starts again, this event is detected through calls to the tud_audio_set_itf_close_EP_cb and tud_audio_set_itf_cb functions.
The ASRC output buffer in the I2S -> ASRC -> USB is reset (USB samples_to_host_stream_buf).
Zeroes are streamed to the host until the buffer fills to a stable level, when we resume streaming out of this buffer to send samples over USB.
The average buffer calculation state for the USB samples_to_host_stream_buf is also reset and a new stable average is calculated against which the average buffer levels are corrected.
Out of the 524288 bytes of memory available per tile, this application uses approximately 262000 bytes of memory on Tile 0
and 208000 bytes of memory on Tile 1.
Profiling the CPU usage for this application using an RTOS friendly profiling tool is still TBD.
However, profiling some application tasks has taken place. These numbers along with some already existing profiling numbers for the drivers are listed in the Tile 0 tasks MIPS and Tile 1 tasks MIPS tables.
Each tile has 5 bare-metal cores being used for running RTOS tasks so each core has a fixed bandwidth of 120 MHz available.
Ports of the Sensory and Cyberon speech recognition libraries are provided.
Speech Recognition Ports
Filename/Directory
Description
modules/asr directory
include folder for ASR modules and ports
module/asr/sensory directory
contains the Sensory library and associated port code
module/asr/Cyberon directory
contains the Cyberon library and associated port code
modules/asr/CmakeLists.txt
CMakeLists file for adding ASR port targets
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Memory and CPU Requirements£££sln-voice-memory-cpu.html#memory-and-cpu-requirements
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Memory and CPU Requirements$$$Memory£££sln-voice-memory-cpu.html#memory
The table below lists the approximate memory requirements for the larger software components. All memory use estimates in the table below are based on the default configuration for the feature. Alternate configurations will require more or less memory. The estimates are provided as guideline to assist application developers judge the memory cost of extending the application or benefit of removing an existing feature. It can be assumed that the memory requirement of components not listed in the table below are under 5 kB.
Memory Requirements
Component
Memory Use (kB)
Stereo Adaptive Echo Canceler (AEC)
275
Sensory Speech Recognition Engine
180
Cyberon Speech Recognition Engine
125
Interference Canceler (IC) + Voice To Noise Ratio Estimator (VNR)
130
USB
20
Noise Suppressor (NS)
15
Adaptive Gain Control (AGC)
11
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Memory and CPU Requirements$$$CPU£££sln-voice-memory-cpu.html#cpu
The table below lists the approximate CPU requirements in MIPS for the larger software components. All CPU use estimates in the table below are based on the default configuration for the feature. Alternate configurations will require more or less MIPS. The estimates are provided as guideline to assist application developers judge the MIP cost of extending the application or benefits of removing an existing feature. It can be assumed that the memory requirement of components not listed in the table below are under 1%.
The following formula was used to convert CPU% to MIPS:
MIPS = (CPU% / 100%) * (600 MHz / 5 cores)
CPU Requirements (@ 600 MHz)
Component
CPU Use (%)
MIPS Use
USB XUD
100
120
I2S (slave mode)
80
96
Stereo Adaptive Echo Canceler (AEC)
80
96
Sensory Speech Recognition Engine
80
96
Cyberon Speech Recognition Engine
72
87
Interference Canceler (IC) + Voice To Noise Ratio Estimator (VNR)
This section includes instructions on anticipated or common software modifications.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$How-Tos$$$Changing the input and output sample rate£££doc/programming_guide/howto.html#changing-the-input-and-output-sample-rate
In the example design app_conf.h file, change appconfAUDIO_PIPELINE_SAMPLE_RATE to either 16000 or 48000.
I2S AEC reference input audio & USB processed audio output
The FFVA example design includes 2 basic configurations; INT and UA. The INT configuration is setup with I2S for input and output audio. The UA configuration is setup with USB for input and output audio. This HOWTO explains how to modify the FFVA example design for I2S input audio and USB output audio.
In the ffva_ua.cmake file, changing the appconfAEC_REF_DEFAULT to appconfAEC_REF_I2S will result in the expected input frames.
For integrating with I2S there are a few other differences from the default UA configuration. When integrating with an external Raspberry Pi BCLK and LRCLK, you will want the following FFVA_UA_COMPILE_DEFINITIONS:
appconfI2S_AUDIO_SAMPLE_RATE can also be 16000. Only 48k and 16k conversions is supported in FFVA.
The default FFVA INT device doesn’t require an external MCLK, but this setting can be changed by setting appconfEXTERNAL_MCLK=1. In this case the FFVA example application will sit at initialization until it can lock on to that clock source, so it MUST be active during boot.
Since the FFVA example application is not receiving reference audio through USB in this configuration, USB adaptive mode will not adapt to the input. By default, FFVA will output the configured nominal rate.
If you enable appconfAEC_REF_DEFAULT=appconfAEC_REF_I2S and appconfI2S_MODE=appconfI2S_MODE_MASTER. You need to invert I2S_DATA_IN and I2S_MIC_DATA in the bsp_config/XK_VOICE_L71/XK_VOICE_L71.xn file to have the reference audio play properly.
Lastly, with I2S enabled the DAC is always initialized by the FFVA example application. If FFVA cannot be the I2C host then it is up to the host to initialize the DAC, like in the AVS demo.
If you want to customize the XTC Tools commands like xflash and xrun, you can see what commands CMake is running by adding VERBOSE=1 to your build command line. For example:
make run_my_target VERBOSE=1
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Frequently Asked Questions$$$fatfs_mkimage: not found£££sln-voice-faq.html#fatfs-mkimage-not-found
This issue occurs when the fatfs_mkimage host utility cannot be found. The most common cause for these issues are an incomplete installation of XCORE-VOICE.
Ensure that the host applications build and install has been completed. Verify that the fatfs_mkimage binary is installed to a location on PATH, or that the default application installation folder is added to PATH.
One potential issue with the low power FFD application is a crash after adding new code:
xrun: Program received signal ET_ECALL, Application exception. [Switching to tile[1] core[1]] 0x0008a182 in pdm_rx_isr ()
This generally occurs when there is not enough processing time available on tile 1, or when interrupts were disabled for too long, causing the mic array driver to fail to meet timing. To resolve reduce the processing time, minimize context switching and other actions that require kernel locks, and/or increase the tile 1 core clock frequency.
The clock dividers are set high to minimize core power consumption. This can make debugging a challenge or impossible. Even adding a simple printf can cause critical timing to be missed. In order to debug with the low-power features enabled, temporarily modify the clock dividers in app_conf.h.
XCORE ® -VOICE Solutions$$$XCORE-VOICE Programming Guide$$$Frequently Asked Questions$$$xcc2clang.exe: error: no such file or directory£££sln-voice-faq.html#xcc2clang-exe-error-no-such-file-or-directory
Those strange characters at the beginning of the path are known as a byte-order mark (BOM). CMake adds them to the beginning of the response files it generates during the configure step. Why does it add them? Because the MSVC compiler toolchain requires them. However, some compiler toolchains, like gcc and xcc, do not ignore the BOM. Why did CMake think the compiler toolchain was MSVC and not the XTC toolchain? Because of a bug in which certain versions of CMake and certain versions of Visual Studio do not play nice together. The good news is that this appears to have been addressed in CMake version 3.22.3. Update to CMake version 3.22.2 or newer.
At the core of the Voice Framework are high-performance audio processing algorithms. The algorithms are connected in a pipeline that takes its input from a pair of the microphone and executes a series of signal processing algorithms to extract a voice signal from a complex soundscape. The audio pipeline can accept a reference signal from a host system which is used to perform Acoustic Echo Cancellation (AEC) to remove audio being played by the host. The audio pipeline provides two different output channels - one that is optimized for Automatic Speech Recognition systems and the other for voice communications.
A flexible audio signal routing infrastructure and a range of digital inputs and outputs enables the Voice Framework to be integrated into a wide range of system configurations, that can be configured at start up and during operation through a set of control registers. In addition, all source code is provided to allow for full customization or the addition of other audio processing algorithms.
lib_aec is a library which provides functions that can be put together to perform Acoustic Echo Cancellation (AEC)
on input mic data using the input reference data to model the room echo characteristics. lib_aec library functions
make use of functionality provided in lib_xcore_math to perform DSP operations. For more details refer to
AEC Overview.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Repository Structure£££getting-started.html#repository-structure
modules/lib_aec - The actual lib_aec library directory within https://github.com/xmos/fwk_voice/. Within lib_aec
api/ - Headers containing the public API for lib_aec.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Requirements£££getting-started.html#requirements
lib_aec is included as part of the fwk_voice github repository
and all requirements for cloning and building fwk_voice apply. lib_aec is compiled as a static library as part of
overall fwk_voice build. It depends on lib_xcore_math.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Structure£££getting-started.html#api-structure
The API can be categorised into high level and low level functions.
High level API has fewer input arguments and is simpler. However, it provides limited options for calling functions in parallel
across multiple threads. Keeping API simplicity in mind, most of the high level API functions accept a pointer to the AEC state
structure as an input and modify the relevant part of the AEC state. API and example documentation provides more
details about the fields within the state modified when calling a given function. High level API functions allow
2 levels of parallelism:
Single level of parallelism where for a given function, main and shadow filter processing can happen in parallel.
Two levels of parallelism where a for a given function, processing across multiple channels as well as main and shadow filter can be done in parallel.
Low level API has more input arguments but allows more freedom for running in parallel across multiple threads. Low
level API function names begin with a aec_l2_ prefix.
Depending on the low level API used, functions can be run in parallel to work over a range of bins or a range of phases.
This API is still a work in progress and will be fully supported in the future.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Getting and Building£££getting-started.html#getting-and-building
This repo is got as part of the parent fwk_voice repo clone. It is compiled as a static library as part of fwk_voice
compilation process.
To include lib_aec in an application as a static library, the generated libfwk_voice_module_lib_aec.a can then be linked into the
application. Be sure to also add lib_aec/api as an include directory for the application.
The lib_aec library provides functions that can be put together to
perform Automatic Echo Cancellation on input microphone data by using
input reference data to model the echo characteristics of the room.
The echo canceller takes in one or more channels of microphone (mic)
input and one or more channels of reference input data. The mic input is
the input captured by the device microphones. Reference input is the
audio that is played out of the device speakers. The echo canceller uses
the reference input to model the room echo characteristics for each
mic-loudspeaker pair and outputs an echo cancelled version of the mic
input. AEC uses adaptive filters, one per mic-speaker pair to constantly
remove echo from the the mic input. The filters continually adapt to the
acoustic environment to accommodate changes in the room created by
events such as doors opening or closing and people moving about.
Echo cancellation is performed on a frame by frame basis. Each frame is
made of 15msec chunks of data, which is 240 samples at 16kHz input
sampling frequency, per input channel. For example, for a 2 mic channel
and 2 reference channel input configuration, an input frame is made of
2x240 samples of mic data and 2x240 samples of reference data. Input
data is expected to be in fixed point 32bit 1.31 format. Further, in
this example, there will be a total of 4 adaptive filters;
\(\hat{H}_{y0x0}\), \(\hat{H}_{y0x1}\), \(\hat{H}_{y1x0}\)
and \(\hat{H}_{y1x1}\), monitoring the echo seen in mic channel 0
from reference channel 0 and 1 and echo seen in mic channel 1 from
reference channel 0 and 1.
Microphone data is referred to as \(y\) when in time domain and
\(Y\) when in frequency domain. In general throughout the code,
names starting with lower case represent time domain and those beginning
with upper case represent frequency domain. For example \(error\) is
the filter error and \(Error\) is the spectrum of the filter error.
Reference input is referred to as \(x\) in time domain and \(X\)
when in frequency domain. Filter is referred to as \(\hat{h}\) in
time domain and \(\hat{H}\) in frequency domain.
A filter has multiple phases. The term phases refers to the tail length
of the filter. A filter with more phases or a longer tail length will be
able to model a more reverberant room response leading to better echo
cancellation.
There are 2 types of adaptive filters used in the AEC. These are
referred to as main filter and shadow filter. The main filter as the
name suggests is the main filter that is used to generate the echo
cancelled output of the AEC. Shadow filter is a filter that used to
quickly detect and respond to changes in the room transfer function.
There is one main filter and one shadow filter per \(x\)-\(y\)
pair. Typically the main filter has more phases than the shadow filter.
Fewer phases in the shadow filter enable it to rapidly detect and
respond to changes while more phases in main filter lead to deeper
convergence and hence better echo cancellation at the AEC output.
Before starting AEC processing or every time there’s a configuration
change, the user needs to call aec_init() to initialise the echo
canceller for a desired configuration. Once the AEC is initialised, the
library functions can be called in a logical order to perform echo
cancellation on a frame by frame basis. Refer to the aec_1_thread and
aec_2_threads examples to see how the functions are called to perform
echo cancellation using one thread or 2 threads.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Reference£££modules/voice/modules/lib_vnr/doc/src/reference/index.html#api-reference
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Acoustic Echo Canceller Library$$$API Reference$$$AEC Data Structure and Enum Definitions£££aec-types.html#aec-data-structure-and-enum-definitions
groupaec_types
Enums
enumaec_adaption_e
Values:
enumeratorAEC_ADAPTION_AUTO
Compute filter adaption config every frame.
enumeratorAEC_ADAPTION_FORCE_ON
Filter adaption always ON.
enumeratorAEC_ADAPTION_FORCE_OFF
Filter adaption always OFF.
enumshadow_state_e
Values:
enumeratorLOW_REF
Not much reference so no point in acting on AEC filter logic.
enumeratorERROR
something has gone wrong, zero shadow filter
enumeratorZERO
shadow filter has been reset multiple times, zero shadow filter
enumeratorRESET
copy main filter to shadow filter
enumeratorEQUAL
main filter and shadow filter are similar
enumeratorSIGMA
shadow filter bit better than main, reset sigma_xx for faster convergence
counter for tracking shadow filter copy to main filter
structaec_shared_state_t
#include <aec_state.h>
AEC shared state structure.
Data structures holding AEC persistent state that is common between main filter and shadow filter. aec_state_t::shared_state for both main and shadow filter point to the common aec_shared_t structure. [aec_shared_state_t]
BFP array pointing to the reference input spectrum phases. The term phase refers to the spectrum data for a frame. Multiple phases means multiple frames of data.
For example, 10 phases would mean the 10 most recent frames of data. Each phase spectrum, pointed to by X_fifo[i][j]->data is stored as a length AEC_FD_FRAME_LENGTH, complex 32bit array.
The phases are ordered from most recent to least recent in the X_fifo. For example, for an AEC configuration of 2 x-channels and 10 phases per x channel, 10 frames of X data spectrum is stored in the X_fifo. For a given x channel, say x channel 0, X_fifo[0][0] points to the most recent frame’s X spectrum and X_fifo[0][9] points to the last phase, i.e the least recent frame’s X spectrum.
BFP array pointing to time domain mic input processing block. The y data values are stored as length AEC_PROC_FRAME_LENGTH, 32bit integer array per y channel.
BFP array pointing to time domain reference input processing block. The x data values are stored as length AEC_PROC_FRAME_LENGTH, 32bit integer array per x channel.
BFP array pointing to time domain mic input values from the previous frame. These are put together with the new samples received in the current frame to make a AEC_PROC_FRAME_LENGTH processing block. The prev_y data values are stored as length (AEC_PROC_FRAME_LENGTH - AEC_FRAME_ADVANCE), 32bit integer array per y channel.
BFP array pointing to time domain reference input values from the previous frame. These are put together with the new samples received in the current frame to make a AEC_PROC_FRAME_LENGTH processing block. The prev_x data values are stored as length (AEC_PROC_FRAME_LENGTH - AEC_FRAME_ADVANCE), 32bit integer array per x channel.
BFP array pointing to sigma_XX values which are the weighted average of the X_energy signal. The sigma_XX data is stored as 32bit integer array of length AEC_FD_FRAME_LENGTH
Exponential moving average of the time domain mic signal energy. This is calculated by calculating energy per sample and summing across all samples. Stored in a y channels array with every value stored as a 32bit integer mantissa and exponent.
Exponential moving average of the time domain reference signal energy. This is calculated by calculating energy per sample and summing across all samples. Stored in a x channels array with every value stored as a 32bit integer mantissa and exponent.
Energy of the mic input spectrum. This is calculated by calculating the energy per bin and summing across all bins. Stored in a y channels array with every value stored as a 32bit integer mantissa and exponent.
Sum of the X_energy across all bins for a given x channel. Stored in a x channels array with every value stored as a 32bit integer mantissa and exponent.
Data structures holding AEC persistent state. There are 2 instances of aec_state_t maintained within AEC; one for main filter and one for shadow filter specific state. [aec_state_t]
BFP array pointing to estimated mic signal spectrum. The Y_data data values are stored as length AEC_FD_FRAME_LENGTH, complex 32bit array per y channel.
BFP array pointing to adaptive filter error signal spectrum. The Error data is stored as length AEC_FD_FRAME_LENGTH, complex 32bit array per y channel.
BFP array pointing to the adaptive filter spectrum. The filter spectrum is stored as a num_y_channels x total_phases_across_all_x_channels array where each H_hat[i][j] entry points to the spectrum of a single phase.
Number of phases in the filter refers to its tail length. A filter with more phases would be able to model a longer echo thereby causing better echo cancellation.
For example, for a 2 y-channels, 3 x-channels, 10 phases per x channel configuration, the filter spectrum phases are stored in a 2x30 array. For a given y channel, say y channel 0, H_hat[0][0] to H_hat[0][9] points to 10 phases of H_haty0x0, H_hat[0][10] to H_hat[0][19] points to 10 phases of H_haty0x1 and H_hat[0][20] to H_hat[0][29] points to 10 phases of H_haty0x2.
Each filter phase data which is pointed to by H_hat[i][j].data is stored as AEC_FD_FRAME_LENGTH complex 32bit array.
BFP array pointing to all phases of reference input spectrum across all x channels. Here, the reference input spectrum is saved in a 1 dimensional array of phases, with x channel 0 phases followed by x channel 1 phases and so on. For example, for a 2 x-channels, 10 phases per x channel configuration, X_fifo_1d[0] to X_fifo_1d[9] points to the 10 phases for channel 0 and X_fifo[10] to X_fifo[19] points to the 10 phases for channel 1.
Each X data spectrum phase pointed to by X_fifo_1d[i][j].data is stored as length AEC_FD_FRAME_LENGTH complex 32bit array.
BFP array pointing to the X_energy data which is the energy per bin of the X spectrum summed over all phases of the X data. X_energy data is stored as a length AEC_FD_FRAME_LENGTH, integer 32bit array per x channel.
BFP array pointing to time domain overlap data values which are used in the overlap add operation done while calculating the echo canceller time domain output. Stored as a length 32, 32 bit integer array per y channel.
Exponential moving average of the time domain adaptive filter error signal energy. Stored in an x channels array with every value stored as a 32bit integer mantissa and exponent.
Maximum X energy across all values of X_energy for a given x channel. Stored in an x channels array with every value stored as a 32bit integer mantissa and exponent.
pointer to the state data shared between main and shadow filter.
unsignednum_phases
Number of filter phases per x-y pair that AEC filter is configured for. This is the input argument num_main_filter_phases or num_shadow_filter_phases, depending on which filter the aec_state_t is instantiated for, passed in aec_init() call.
Maximum number of microphone input channels supported in the library. Microphone input to the AEC refers to the input from the device’s microphones from which AEC removes the echo created in the room by the device’s loudspeakers.
AEC functions follow the convention of using \(y\) and \(Y\) for referring to time domain and frequency domain representation of microphone input.
The num_y_channels passed into aec_init() call should be less than or equal to AEC_LIB_MAX_Y_CHANNELS. This define is only used for defining data structures in the aec_state. The library code implementation uses only the num_y_channels aec is initialised for in the aec_init() call.
AEC_LIB_MAX_X_CHANNELS
Maximum number of reference input channels supported in the library. Reference input to the AEC refers to a copy of the device’s speaker output audio that is also sent as an input to the AEC. It is used to model the echo characteristics between a mic-loudspeaker pair.
AEC functions follow the convention of using \(x\) and \(X\) for referring to time domain and frequency domain representation of reference input.
The num_x_channels passed into aec_init() call should be less than or equal to AEC_LIB_MAX_X_CHANNELS. This define is only used for defining data structures in the aec_state. The library code implementation uses only the num_x_channels aec is initialised for in the aec_init() call.
AEC_FRAME_ADVANCE
AEC frame size This is the number of samples of new data that the AEC works on every frame. 240 samples at 16kHz is 15msec. Every frame, the echo canceller takes in 15msec of mic and reference data and generates 15msec of echo cancelled output.
AEC_PROC_FRAME_LENGTH
Time domain samples block length used internally in AEC’s block LMS algorithm
AEC_FD_FRAME_LENGTH
Number of bins of spectrum data computed when doing a DFT of a AEC_PROC_FRAME_LENGTH length time domain vector. The AEC_FD_FRAME_LENGTH spectrum values represent the bins from DC to Nyquist.
AEC_LIB_MAX_PHASES
Maximum total number of phases supported in the AEC library This is the maximum number of total phases supported in the AEC library. Total phases are calculated by summing phases across adaptive filters for all x-y pairs.
For example. for a 2 y-channels, 2 x-channels, 10 phases per x channel configuration, there are 4 adaptive filters, H_haty0x0, H_haty0x1, H_haty1x0 and H_haty1x1, each filter having 10 phases, so the total number of phases is 40. When aec_init() is called to initialise the AEC, the num_y_channels, num_x_channels and num_main_filter_phases parameters passed in should be such that num_y_channels * num_x_channels * num_main_filter_phases is less than equal to AEC_LIB_MAX_PHASES.
This define is only used when defining data structures within the AEC state structure. The AEC algorithm implementation uses the num_main_filter_phases and num_shadow_filter_phases values that are passed into aec_init().
AEC_UNUSED_TAPS_PER_PHASE
Overlap data length
AEC_FFT_PADDING
Extra 2 samples you need to allocate in time domain so that the full spectrum (DC to nyquist) can be stored after the in-place FFT. NOT USER MODIFIABLE.
This function initializes AEC data structures for a given configuration. The configuration parameters num_y_channels, num_x_channels, num_main_filter_phases and num_shadow_filter_phases are passed in as input arguments.
This function needs to be called at startup to first initialise the AEC and subsequently whenever the AEC configuration changes.
main_state, shadow_state and shared_state structures must start at double word aligned addresses.
main_mem_pool and shadow_mem_pool must point to memory buffers big enough to support main and shadow filter processing. AEC state aec_state_t and shared state aec_shared_state_t structures contain only the BFP data structures used in the AEC. The memory these BFP structures will point to needs to be provided by the user in the memory pool main and shadow filters memory pool. An example memory pool structure is present in aec_memory_pool_t and aec_shadow_filt_memory_pool_t.
main_mem_pool and shadow_mem_pool must also start at double word aligned addresses.
Example
#include"aec_memory_pool.h"aec_state_tDWORD_ALIGNEDmain_state;aec_state_tDWORD_ALIGNEDshadow_state;aec_shared_state_tDWORD_ALIGNEDaec_shared_state;uint8_tDWORD_ALIGNEDaec_mem[sizeof(aec_memory_pool_t)];uint8_tDWORD_ALIGNEDaec_shadow_mem[sizeof(aec_shadow_filt_memory_pool_t)];unsignedy_chans=2,x_chans=2;unsignedmain_phases=10,shadow_phases=5;// There is one main and one shadow filter per x-y channel pair, so for this example there will be 4 main and 4// shadow filters. Each main filter will have 10 phases and each shadow filter will have 5 phases.aec_init(&main_state,&shadow_state,&shared_state,aec_mem,aec_shadow_mem,y_chans,x_chans,main_phases,shadow_phases);
Parameters:
main_state – [inout] AEC state structure for holding main filter specific state
shadow_state – [inout] AEC state structure for holding shadow filter specific state
shared_state – [inout] Shared state structure for holding state that is common to main and shadow filter
main_mem_pool – [inout] Memory pool containing main filter memory buffers
shadow_mem_pool – [inout] Memory pool containing shadow filter memory buffers
num_y_channels – [in] Number of mic input channels
num_x_channels – [in] Number of reference input channels
num_main_filter_phases – [in] Number of phases in the main filter
num_shadow_filter_phases – [in] Number of phases in the shadow filter
Initialise AEC data structures for processing a new frame.
This is the first function that is called when a new frame is available for processing. It takes the new samples as input and combines the new samples and previous frame’s history to create a processing block on which further processing happens. It also initialises some data structures that need to be initialised at the beginning of a frame.
Note
y_data and x_data buffers memory is free to be reused after this function call.
This function calculates the energy of frequency domain data used in the AEC. Frequency domain data in AEC is in the form of complex 32bit vectors and energy is calculated as the squared magnitude of the input vector.
Calculate Discrete Fourier Transform (DFT) spectrum of an input time domain vector.
This function calculates the spectrum of a real 32bit time domain vector. It calculates an N point real DFT where N is the length of the input vector to output a complex N/2+1 length complex 32bit vector. The N/2+1 complex output values represent spectrum samples from DC up to the Nyquist frequency.
The DFT calculation is done in place. After this function call the input and output BFP structures data fields point to the same memory. Since DFT is calculated in place, use of the input BFP struct is undefined after this function.
To allow for inplace transform from N real 32bit values to N/2+1 complex 32bit values, the input vector should have 2 extra real 32bit samples worth of memory. This means that input->data should point to a buffer of length input->length+2
After this function input->data and output->data point to the same memory address.
Calculate inverse Discrete Fourier Transform (DFT) of an input spectrum.
This function calculates a N point inverse real DFT of a complex 32bit where N is 2*(length-1) where length is the length of the input vector. The output is a real 32bit vector of length N.
The inverse DFT calculation is done in place. After this operation the input and the output BFP structures data fields point to the same memory. Since the calculation is done in place, use of input BFP struct after this function is undefined.
After this function input->data and output->data point to the same memory address.
XFIFO is a FIFO of the most recent X frames, where X is spectrum of one frame of reference input. There’s a common X FIFO that is shared between main and shadow filters. It holds num_main_filter_phases most recent X frames and the shadow filter uses num_shadow_filter_phases most recent frames out of it.
This function calculates the energy per X sample index summed across the X FIFO phases. This function also calculates the maximum energy across all samples indices of the output energy vector
Note
This function implements some speed optimisations which introduce quantisation error. To stop quantisation error build up, in every call of this function, energy for one sample index, which is specified in the recalc_bin argument, is recalculated without the optimisations. There are a total of AEC_FD_FRAME_LENGTH samples in the energy vector, so recalc_bin keeps cycling through indexes 0 to AEC_PROC_FRAME_LENGTH/2.
Parameters:
state – [inout] AEC state. state->X_energy[ch] and state->max_X_energy[ch] are updated
ch – [in] channel index for which energy calculations are done
recalc_bin – [in] The sample index for which energy is recalculated to eliminate quantisation errors
This function updates the X FIFO by removing the oldest X frame from it and adding the current X frame to it. This function also calculates sigmaXX which is the exponential moving average of the current X frame energy
Parameters:
state – [inout] AEC state structure. state->shared_state->X_fifo[ch] and state->shared_state->sigma_XX[ch] are updated.
ch – [in] X channel index for which to update X FIFO
Calculate error spectrum and estimated mic signal spectrum.
This function calculates the error spectrum (Error) and estimated mic input spectrum (Y_hat) Y_hat is calculated as the sum of all phases of the adaptive filter multiplied by the respective phases of the reference input spectrum. Error is calculated by subtracting Y_hat from the mic input spectrum Y
Parameters:
state – [inout] AEC state structure. state->Error[ch] and state->Y_hat[ch] are updated
ch – [in] mic channel index for which to compute Error and Y_hat
This function calculates the average coherence between mic input signal (y) and estimated mic signal (y_hat). A metric is calculated using y and y_hat and the moving average (coh) and a slow moving average (coh_slow) of that metric is calculated. The coherence values are used to distinguish between situations when filter adaption should continue or freeze and update mu accordingly.
Parameters:
state – [inout] AEC state structure. state->shared_state->coh_mu_state[ch].coh and state->shared_state->coh_mu_state[ch].coh_slow are updated
ch – [in] mic channel index for which to calculate average coherence
This function is responsible for windowing the filter error signal and creating AEC filter output that can be propagated to downstream stages. output is calculated by overlapping and adding current frame’s windowed error signal with the previous frame windowed error. This is done to smooth discontinuities in the output as the filter adapts.
Parameters:
state – [inout] AEC state structure. state->error[ch]
output – [out] pointer to the output buffer
ch – [in] mic channel index for which to calculate output
This function calculates the normalisation spectrum of the reference input signal. This normalised spectrum is later used during filter adaption to scale the adaption to the size of the input signal. The normalisation spectrum is calculated as a time and frequency smoothed energy of the reference input spectrum.
The normalisation spectrum is calculated differently for main and shadow filter, so a flag indicating whether this calculation is being done for the main or shadow filter is passed as an input to the function
Parameters:
state – [inout] AEC state structure. state->inv_X_energy[ch] is updated
ch – [in] reference channel index for which to calculate normalisation spectrum
is_shadow – [in] flag indicating filter type. 0: Main filter, 1: Shadow filter
Compare and update filters. Calculate the adaption step size mu.
This function has 2 responsibilities. First, it compares the energies in the error spectrums of the main and shadow filter with each other and with the mic input spectrum energy, and makes an estimate of how well the filters are performing. Based on this, it optionally modifies the filters by either resetting the filter coefficients or copying one filter into another. Second, it uses the coherence values calculated in aec_calc_coherence as well as information from filter comparison done in step 1 to calculate the adaption step size mu.
Parameters:
main_state – [inout] AEC state structure for the main filter
shadow_state – [inout] AEC state structure for the shadow filter
This function calculates a parameter referred to as T that is later used to scale the reference input spectrum in the filter update step. T is a function of the adaption step size mu, normalisation spectrum inv_X_energy and the filter error spectrum Error.
Parameters:
state – [inout] AEC state structure. state->T[x_ch] is updated
This function updates the adaptive filter spectrum (H_hat). It calculates the delta update that is applied to the filter by scaling the X FIFO with the T values computed in aec_compute_T() and applies the delta update to H_hat. A gradient constraint FFT is then applied to constrain the length of each phase of the filter to avoid wrapping when calculating y_hat
Parameters:
state – [inout] AEC state structure. state->H_hat[y_ch] is updated
The X FIFO BFP structure is maintained in 2 forms - as a 2 dimensional [x_channels][num_phases] and as a [x_channels * num_phases] 1 dimensional array. This is done in order to optimally access the X FIFO as needed in different functions. After the X FIFO is updated with the current X frame, this function is called in order to copy the 2 dimensional BFP structure into it’s 1 dimensional counterpart.
Parameters:
state – [inout] AEC state structure. state->X_fifo_1d is updated
Calculate a correlation metric between the microphone input and estimated microphone signal.
This function calculates a metric of resemblance between the mic input and the estimated mic signal. The correlation metric, along with reference signal energy is used to infer presence of near and far end signals in the AEC mic input.
Parameters:
state – [in] AEC state structure. state->y and state->y_hat are used to calculate the correlation metric
ch – [in] mic channel index for which to calculate the metric
This function implements a quick check for detecting activity on the input channels. It detects signal presence by checking if the maximum sample in the time domain input frame is above a given threshold.
Parameters:
input_data – [in] Pointer to input data frame. Input is assumed to be in Q1.31 fixed point format.
active_threshold – [in] Threshold for detecting signal activity
num_channels – [in] Number of input data channels
Returns:
0 if no signal activity on the input channels, 1 if activity detected on the input channels
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$On GitHub£££page_vnr_features_state_h.html#on-github
lib_aec is present as part of fwk_voice. Get the latest version of fwk_voice from
https://github.com/xmos/fwk_voice. lib_aec is present within the modules/lib_aec directory in fwk_voice
lib_ns is a library which performs Noise Suppression (NS), by estimating the noise and
subtracting it from frame. lib_ns library functions make use of functionality
provided in lib_xcore_math to perform DSP operations. For more details, refer to NS Overview.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Repository Structure£££getting-started.html#repository-structure
modules/lib_ns - The actual lib_ns library directory within https://github.com/xmos/fwk_voice/.
Within lib_ns
api/ - Headers containing the public API for lib_ns.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Requirements£££getting-started.html#requirements
lib_ns is included as part of the fwk_voice github repository and all requirements for cloning
and building fwk_voice apply. lib_ns is compiled as a static library as part of the overall
fwk_voice build. It depends on lib_xcore_math.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Getting and Building£££getting-started.html#getting-and-building
This module is part of the parent fwk_voice repo clone. It is compiled as a static library as part of
fwk_voice compilation process.
To include lib_ns in an application as a static library, the generated libfwk_voice_module_lib_ns.a can then be linked
into the application. Add lib_ns/api to the include directories when building the application.
The lib_ns library provides an API to implement Noise
Suppression within an application.
The noise suppressor estimates the probability of speech presence and dynamically
adapts its coefficients to estimate the noise levels to subtract from the input.
The filter will automatically reset its noise estimations every 10 frames.
The NS takes as input a frame of data from an audio channel. This could be the
microphone input or the output of another module in the application.
Noise Suppression is performed on a frame-by-frame basis. Each frame consists of
15ms of data, which is 240 samples at 16kHz input sampling frequency. Input data is
expected to be in a fixed-point 32-bit 1.31 format.
Before processing any frames, the application must configure and initialise the
NS instance by calling ns_init(). Then for each frame,
ns_process_frame() will update the NS instance’s internal state and produce
the output frame by applying the NS algorithm to the input frame.
If multiple channels need to be processed by the application, or multiple outputs
are required, an independent instance of the NS must be run for each channel.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Reference£££modules/voice/modules/lib_vnr/doc/src/reference/index.html#api-reference
This function initialises the NS state with the provided configuration. It must be called at startup to initialise the NS before processing any frames, and can be called at any time after that to reset the NS instance, returning the internal NS state to its defaults.
This function updates the NS’s internal state based on the input 1.31 frame, and returns an output 1.31 frame containing the result of the NS algorithm applied to the input.
The input and output pointers can be equal to perform the processing in-place.
Length of the frame of data on which the NS will operate.
NS_PROC_FRAME_LENGTH
Time domain samples block length used internally.
NS_PROC_FRAME_BINS
Number of bins of spectrum data computed when doing a DFT of a NS_PROC_FRAME_LENGTH length time domain vector. The NS_PROC_FRAME_BINS spectrum values represent the bins from DC to Nyquist.
NS_INT_EXP
The exponent used internally to keep q1.31 format.
NS_WINDOW_LENGTH
The length of the window applied in time domain
structns_state_t
#include <ns_state.h>
NS state structure.
This structure holds the current state of the NS instance and members are updated each time that ns_process_frame() runs. Many of these members are exponentially-weighted moving averages (EWMA) which influence the behaviour of the NS filter. The user should not directly modify any of these members.
This header contains definitions for data structure and defines.
This header is automatically included by ns_api.h
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$On GitHub£££page_vnr_features_state_h.html#on-github
lib_ns is present as part of fwk_voice. Get the latest version of fwk_voice from
https://github.com/xmos/fwk_voice. lib_ns is present within the modules/lib_ns directory in fwk_voice.
To use the functions in this library in an application, include ns_api.h in the application source file.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Automatic Gain Control Library£££modules/voice/modules/lib_agc/doc/index.html#automatic-gain-control-library
lib_agc is a library which performs Automatic Gain Control (AGC), with support for Loss Control.
For more details, refer to AGC Overview.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Repository Structure£££getting-started.html#repository-structure
modules/lib_agc - The actual lib_agc library directory within https://github.com/xmos/fwk_voice/.
Within lib_agc
api/ - Headers containing the public API for lib_agc.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Requirements£££getting-started.html#requirements
lib_agc is included as part of the fwk_voice github repository and all requirements for cloning
and building fwk_voice apply. lib_agc is compiled as a static library as part of the overall
fwk_voice build. It depends on lib_xcore_math.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Getting and Building£££getting-started.html#getting-and-building
This module is part of the parent fwk_voice repo clone. It is compiled as a static library as part of
fwk_voice compilation process.
To include lib_agc in an application as a static library, the generated libfwk_voice_module_lib_agc.a can then be linked
into the application. Add lib_agc/api to the include directories when building the application.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Automatic Gain Control Library$$$AGC Overview£££id1.html#agc-overview
The lib_agc library provides an API to implement Automatic Gain Control within
an application. The goal of the AGC algorithm is to provide consistent output
levels for voice audio.
The gain control can adapt to maintain the amplitude of the peak of the frame
within an upper and lower bound configured for the AGC instance. When used in an
application with a Voice to Noise Ratio estimator (VNR), the AGC will adapt only when
voice activity is detected, so that speech in the input signal is amplified
above other sounds.
The AGC also has a Loss Control feature which can be used when the application
has an Acoustic Echo Canceller (AEC). This feature uses data from the AEC to
adjust the gain applied to reduce residual echoes by attenuating the audio when
near-end speech is not present.
The AGC takes as input a frame of data from an audio channel. This could be the
microphone input or the output of another module in the application.
Gain control is performed on a frame-by-frame basis. Each frame consists of 15ms
of data, which is 240 samples at 16kHz input sampling frequency. Input data is
expected to be in a fixed-point 32-bit 1.31 format.
Before processing any frames, the application must configure and initialise the
AGC instance by calling agc_init(). Then for each frame,
agc_process_frame() will update the AGC instance’s internal state and produce
the output frame by applying the AGC algorithm to the input frame.
The gain values in this module for AGC gain and Loss Control gain are
multiplicative factors that are applied to scale the input frame. Therefore, a
fixed gain value of 1.0 (without loss control) will create no change to the input.
If multiple channels need to be processed by the application, or multiple outputs
are required, an independent instance of the AGC must be run for each channel.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Reference£££modules/voice/modules/lib_vnr/doc/src/reference/index.html#api-reference
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Automatic Gain Control Library$$$API Reference$$$AGC API Functions£££agc-func.html#agc-api-functions
This function initialises the AGC state with the provided configuration. It must be called at startup to initialise the AGC before processing any frames, and can be called at any time after that to reset the AGC instance, returning the internal AGC state to its defaults.
This function updates the AGC’s internal state based on the input frame and meta-data, and returns an output containing the result of the AGC algorithm applied to the input.
The input and output pointers can be equal to perform the processing in-place.
output – [out] Array to return the resulting frame of data
input – [in] Array of frame data on which to perform the AGC
meta_data – [in] Meta-data structure with VNR/AEC data
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Automatic Gain Control Library$$$API Reference$$$AGC Pre-Defined Profiles£££agc-profiles.html#agc-pre-defined-profiles
groupagc_profiles
Defines
AGC_PROFILE_ASR
AGC profile tuned for Automatic Speech Recognition (ASR).
AGC_PROFILE_FIXED_GAIN
AGC profile tuned to apply a fixed gain.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Automatic Gain Control Library$$$API Reference$$$AGC API Structure Definitions£££agc-defines.html#agc-api-structure-definitions
groupagc_defs
Defines
AGC_FRAME_ADVANCE
Length of the frame of data on which the AGC will operate.
AGC_META_DATA_NO_VNR
If the application has no VNR, adapt_on_vnr must be disabled in the configuration. This pre-processor definition can be assigned to the vnr_flag in agc_meta_data_t in that situation to make it clear in the code that there is no VNR.
AGC_META_DATA_NO_AEC
If the application has no AEC, lc_enabled must be disabled in the configuration. This pre-processor definition can be assigned to the aec_ref_power and aec_corr_factor in agc_meta_data_t in that situation to make it clear in the code that there is no AEC.
structagc_config_t
#include <agc_api.h>
AGC configuration structure.
This structure contains configuration settings that can be changed to alter the behaviour of the AGC instance.
Members with the “lc_” prefix are parameters for the Loss Control feature.
Public Members
intadapt
Boolean to enable AGC adaption; if enabled, the gain to apply will adapt based on the peak of the input frame and the upper/lower threshold parameters.
intadapt_on_vnr
Boolean to enable adaption based on the VNR meta-data; if enabled, adaption will always be performed when voice activity is detected. This must be disabled if the application doesn’t have a VNR.
intsoft_clipping
Boolean to enable soft-clipping of the output frame.
Loss control gain to apply when far-end activity only is detected.
structagc_state_t
#include <agc_api.h>
AGC state structure.
This structure holds the current state of the AGC instance and members are updated each time that agc_process_frame() runs. Many of these members are exponentially-weighted moving averages (EWMA) which influence the adaption of the AGC gain or the loss control feature. The user should not directly modify any of these members, except the config.
The current configuration of the AGC. Any member of this configuration structure can be modified and that change will take effect on the next run of agc_process_frame().
EWMA of the far-end correlation for detecting double-talk.
structagc_meta_data_t
#include <agc_api.h>
AGC meta data structure.
This structure holds meta-data about the current frame to be processed, and must be updated to reflect the current frame before calling agc_process_frame().
Public Members
intvnr_flag
Boolean to indicate the detection of voice activity in the current frame.
Correlation factor between the microphone input and the AEC’s estimated microphone signal.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Automatic Gain Control Library$$$API Reference$$$AGC Header Files£££modules/voice/modules/lib_agc/doc/src/reference/header_files.html#agc-header-files
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Automatic Gain Control Library$$$API Reference$$$agc_api.h£££id1.html#agc-api-h
pagepage_agc_api_h
This header should be included in application source code to gain access to the lib_agc public functions API.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Automatic Gain Control Library$$$API Reference$$$agc_profiles.h£££id2.html#agc-profiles-h
pagepage_agc_profiles_h
This header contains pre-defined profiles for AGC configurations. These profiles can be used to initialise the agc_config_t data for use with agc_init().
This header is automatically included by agc_api.h.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$On GitHub£££page_vnr_features_state_h.html#on-github
lib_agc is present as part of fwk_voice. Get the latest version of fwk_voice from
https://github.com/xmos/fwk_voice. lib_agc is present within the modules/lib_agc directory in fwk_voice.
lib_adec is a library which provides functions for measuring and correcting delay offsets between the reference
and loudspeaker signals.
lib_adec depends on lib_aec and lib_xcore_math libraries. For more details about the ADEC, refer to
ADEC Overview
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Repository Structure£££getting-started.html#repository-structure
modules/lib_adec - The actual lib_adec library directory within https://github.com/xmos/fwk_voice/. Within lib_adec
api/ - Headers containing the public API for lib_adec.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Getting and Building£££getting-started.html#getting-and-building
lib_adec is included as part of the fwk_voice github repository
and all requirements for cloning and building fwk_voice apply. lib_adec is compiled as a static library as part of
overall fwk_voice build. To include lib_adec in an application as a static library, the generated libfwk_voice_module_lib_adec.a can then be linked into the application. Be sure to also add lib_adec/api as an include directory for the application.
The ADEC module provides functions to estimate and automatically correct for delay offsets between the reference and the
loudspeakers.
Acoustic echo cancellation is an adaptive filtering process which compares the reference audio to that received from the
microphones. It models the reverberation time of a room, i.e. the time it takes for acoustic reflections to decay to
insignificance. The time window modelled by the AEC is finite, and to maximise its performance it is important to ensure
that the reference audio is presented to the AEC time aligned to the audio being reproduced by the loudspeakers. The
reference audio path delay and the audio reproduction path delay may be significantly different, requiring additional
delay to be inserted into one of the two paths, to correct this delay difference.
The ADEC module provides functionality for
Measuring the current delay
Using the measured delay along with AEC performance related metadata collected from the echo canceller to monitor AEC and make decisions about reconfiguring the AEC and correcting bulk delay offsets.
The metadata collected from AEC contains statistics such as the ERLE, the peak power seen in the adaptive filter and the
peak power to average power ratio of the adaptive filter.
The ADEC algorithm works in 2 modes - normal mode and delay estimation mode.
In its normal mode ADEC monitors the AEC performance and requests small delay corrections. Using the statistics from the AEC, the ADEC estimates a metric called the
AEC goodness which is an estimate of how well the echo canceller is performing. Based on the estimated AEC goodness and the current measured delay, the ADEC can
request for a delay correction to be applied at the input of the echo canceller.
If the AEC is seen as consistently bad, the ADEC transitions to a delay estimation mode and requests for
A special delay to be applied at AEC input that will enable measuring the actual delay in both delay scenarios; microphone input arriving at the AEC earlier in time than the reference input as well as microphone input arriving late in time wrt reference input.
A restart of AEC in a new configuration that has more adaptive filter phases, in order of have a longer filter tail length that is suitable for delay estimation.
Once the ADEC has a measure of the new delay, it requests a delay correction and a reconfiguration of the AEC back to its normal
mode and goes back to its normal mode of monitoring AEC performance and correcting for small delay offsets.
Before processing any frames, the application must configure and initialise the ADEC instance by calling adec_init(). Then for each frame, adec_estimate_delay() will estimate the current delay and adec_process_frame() will use the current frame’s AEC statistics and the estimated delay to monitor the AEC and request possible AEC and delay configuration changes.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Reference£££modules/voice/modules/lib_vnr/doc/src/reference/index.html#api-reference
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Automatic Delay Estimation and Correction Library$$$API Reference$$$ADEC API Functions£££adec-func.html#adec-api-functions
This function initialises ADEC state for a given configuration. It must be called at startup to initialise the ADEC data structures before processing any frames, and can be called at any time after that to reset the ADEC instance, returning the internal ADEC state to its defaults.
Example with ADEC configured for delay estimation only at startup
adec_state_tadec_state;adec_config_tadec_conf;adec_conf.bypass=1;// Bypass automatic DE correctionadec_conf.force_de_cycle_trigger=1;// Force a delay correction cycle, so that delay correction happens once after initialisationadec_init(&adec_state,&adec_conf);// Application needs to ensure that adec_state->adec_config.force_de_cycle_trigger is set to 0 after ADEC has requested a transition to delay estimation mode once in order to ensure that delay is corrected only at startup.
Example with ADEC configured for automatic delay estimation and correction
Perform ADEC processing on an input frame of data.
This function takes information about the latest AEC processed frame and the latest measured delay estimate as input, and decides if a delay correction between input microphone and reference signals is required. If a correction is needed, it outputs a new requested input delay, optionally accompanied with a request for AEC restart in a different configuration. It updates the internal ADEC state structure to reflect the current state of the ADEC process.
This function measures the microphone signal delay wrt the reference signal. It does so by looking for the phase with the peak energy among all AEC filter phases and uses the peak energy phase index as the estimate of the microphone delay. Along with the measured delay, it also outputs information about the peak phase energy that can then be used to gauge the AEC filter convergence and the reliability of the measured delay.
Number of frames far we look back to smooth the peak to average filter power ratio history.
ADEC_PEAK_LINREG_HISTORY_SIZE
Number of frames of peak power history we look at while computing AEC goodness metric. Not NOT USER MODIFIABLE.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Automatic Delay Estimation and Correction Library$$$API Reference$$$ADEC Data Structure and Enum definitions£££adec-types.html#adec-data-structure-and-enum-definitions
groupadec_types
Enums
enumadec_mode_t
Values:
enumeratorADEC_NORMAL_AEC_MODE
ADEC processing mode where it monitors AEC performance and requests small delay correction.
enumeratorADEC_DELAY_ESTIMATOR_MODE
ADEC processing mode for bulk delay correction in which it measures for a new delay offset.
structadec_config_t
#include <adec_state.h>
ADEC configuration structure.
This is used to provide configuration when initialising ADEC at startup. A copy of this structure is present in the ADEC state structure and available to be modified by the application for run time control of ADEC configuration.
Public Members
int32_tbypass
Bypass ADEC decision making process. When set to 1, ADEC evaluates the current input frame metrics but doesn’t make any delay correction or aec reset and reconfiguration requests
int32_tforce_de_cycle_trigger
Force trigger a delay estimation cycle. When set to 1, ADEC bypasses the ADEC monitoring process and transitions to delay estimation mode for measuring delay offset.
structde_output_t
#include <adec_state.h>
Delay estimator output structure.
Public Members
int32_tmeasured_delay_samples
Estimated microphone delay in time domain samples.
Flag indicating if ADEC is requesting an input delay correction
int32_trequested_mic_delay_samples
Mic delay in samples requested by ADEC. Relevant when delay_change_request_flag is 1. Note that this value is a signed integer. A positive requested_mic_delay_samples requires the microphone to be delayed so the application needs to delay the input mic signal by requested_mic_delay_samples samples. A negative requested_mic_delay_samples means ADEC is requesting the input mic signal to be moved earlier in time. This, the application should do my delaying the input reference signal by abs(requested_mic_delay_samples) samples.
int32_treset_aec_flag
flag indicating ADEC’s request for a reset of part of the AEC state to get AEC filter to start adapting from a 0 filter. ADEC requests this when a small delay correction needs to be applied that doesn’t require a full reset of the AEC.
int32_tdelay_estimator_enabled_flag
Flag indicating if AEC needs to be run configured in delay estimation mode.
int32_trequested_delay_samples_debug
Requested delay samples without clamping to +- MAX_DELAY_SAMPLES. Used only for debugging.
structaec_to_adec_t
#include <adec_state.h>
Input structure containing current frame’s information from AEC.
Flag indicating if there is activity on reference input channels.
structadec_state_t
#include <adec_state.h>
ADEC state structure.
This structure holds the current state of the ADEC instance and members are updated each time that adec_process_frame() runs. Many of these members are statistics from tracking the AEC performance. The user should not directly modify any of these members, except the config.
ADEC’s mode of operation. Can be operating in normal AEC or delay estimation mode.
int32_tgated_milliseconds_since_mode_change
milliseconds elapsed since a delay change was last requested. Used to ensure that delay corrections are not requested too early without allowing enough time for aec filter to converge.
int32_tlast_measured_delay
Last measured delay.
int32_tpeak_power_history_idx
index storing the head of the peak_power_history circular buffer
int32_tpeak_power_history_valid
Flag indicating whether the peak_power_history buffer has been filled at least once.
int32_tsf_copy_flag
Flag indicating if shadow to main filter copy has happened at least once in the AEC.
int32_tconvergence_counter
Counter indicating number of frames the AEC shadow filter has been attempting to converge.
int32_tshadow_flag_counter
Counter indicating number of frame the AEC shadow filter has been better than the main filter.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$On GitHub£££page_vnr_features_state_h.html#on-github
lib_adec is present as part of fwk_voice. Get the latest version of fwk_voice from
https://github.com/xmos/fwk_voice. lib_adec is present within the modules/lib_adec directory in fwk_voice
lib_ic is a library which provides functions that together perform Interference Cancellation (IC)
on two channel input mic data by adapting to and modelling the room transfer characteristics. lib_ic library functions
make use of functionality provided in lib_aec for the core normalised LMS blocks which in turn uses
lib_xcore_math to perform DSP low-level optimised operations. For more details refer to IC Overview.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Repository Structure£££getting-started.html#repository-structure
modules/lib_ic - The actual lib_ic library directory within https://github.com/xmos/fwk_voice/.
Within lib_ic:
api/ - Headers containing the public API for lib_ic.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Requirements£££getting-started.html#requirements
lib_ic is included as part of the fwk_voice github repository
and all requirements for cloning and building fwk_voice apply. lib_ic is compiled as a static library as part of
overall fwk_voice build. It depends on lib_aec and lib_xcore_math.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Structure£££getting-started.html#api-structure
The API is presented as three simple functions. These are initialisation, filtering and adaption. Initialisation is called once
at startup and filtering and adaption is called once per frame of samples. The performance requirement is relative low (around 12MIPS)
and as such is supplied as a single threaded implementation only.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Getting and Building£££getting-started.html#getting-and-building
This repo is obtained as part of the parent fwk_voice repo clone. It is
compiled as a static library as part of fwk_voice compilation process.
To include lib_ic in an application as a static library, the generated libfwk_voice_module_lib_ic.a can then be linked into the
application. Be sure to also add lib_ic/api as an include directory for the application.
The Interference Canceller (IC) suppresses static noise from point sources such as cooker hoods, washing machines,
or radios for which there is no reference audio signal available. When the Voice to Noise Ratio estimator (VNR) input
indicates the absence of voice, the IC adapts to remove noise from point sources in the environment. When the VNR
signal indicates the presence of voice, the IC suspends adaptation which allows the voice source to be passed but
maintains suppression of the interfering noise sources which have been previously adapted to.
It can offer much greater, and automatic, cancellation of broad-band noise sources when compared to beam forming
techniques.
It is designed to work at a sample rate of 16kHz and has a fixed configuration of two input microphones and a single
output channel.
The interference canceller is based on an AEC architecture and attempts to cancel one microphone signal from the other in
the absence of voice. In this way, it builds an estimate of the difference in transfer functions between the two
microphones for any present noise sources. Since the transfer function includes spatial information about the noise
sources, applying this filter to the mic input allows any signals originating from the noise source to be cancelled.
The IC uses an adaptive filter which continually adapts to the acoustic environment to accommodate changes in the room
created by events such as doors opening or closing and people moving about. However, it will hold the current transfer
function in the presence of voice meaning it does not adapt to desired audio sources, which can be a person speaking.
The cancellation is performed on a frame by frame basis. Each frame is made of 15msec chunks of data, which is 240
new samples at 16kHz input sampling frequency, per input channel. This is combined with previous audio data to form
a 512 sample frame which allows for sufficient overlap for effective operation of the filter.
The first channel of input microphone data is referred to as y when in time domain and Y when in frequency
domain. The second channel of input microphone data is referred to as x when in time domain and X when in frequency
domain. The y signal is effectively used as the signal containing noise that needs to be cancelled and the x signal
is the reference from which the transfer function is estimated and consequently the noise signal estimated before it
is subtracted from y.
In general throughout the code, names starting with lower case represent time domain and those beginning with
upper case represent frequency domain. For example error is the filter error and Error is the spectrum of
the filter error. The filter coefficient array referred to as h_hat in time domain and H_hat in frequency domain.
The filter has multiple phases each of 15ms. The term phases refers to the tail length of the filter. A filter with more phases or a
longer tail length will be able to model a more reverberant room response leading to better interference cancellation
but, as with all normalised LMS based architectures, will be slower to converge in the case of a transfer function change.
Before starting the IC processing the user must call ic_init() to initialise the IC. If the configuration parameters are
to be set to non-defaults please modify these after ic_init() or in the lib_ic API Definitions£££ic-defines.html#lib-ic-api-definitions file.
Once the IC is initialised, the library functions can be called in a order to perform interference cancellation on
a frame by frame basis.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Reference£££modules/voice/modules/lib_vnr/doc/src/reference/index.html#api-reference
Initialise IC and VNR data structures and set parameters according to ic_defines.h.
This is the first function that must called after creating an ic_state_t instance.
Parameters:
state – [inout] pointer to IC state structure
Returns:
Error status of the VNR inference engine initialisation that is done as part of ic_init. 0 if no error, one of TfLiteStatus error enum values in case of error.
This should be called once per new frame of IC_FRAME_ADVANCE samples. The y_data array contains the microphone data that is to have the noise subtracted from it and x_data is the noise reference source which is internally delayed before being fed into the adaptive filter. Note that the y_data input array is internally delayed by the call to ic_filter() and so contains the delayed y_data afterwards. Typically it does not matter which mic channel is connected to x or y_data as long as the separation is appropriate. The performance of this filter has been optimised for a 71mm mic separation distance.
Parameters:
state – [inout] pointer to IC state structure
y_data – [inout] array reference of mic 0 input buffer. Modified during call
x_data – [in] array reference of mic 1 input buffer
output – [out] array reference containing IC processed output buffer
Calculate voice to noise ratio estimation for the input and output of the IC.
This function can be called after each call to ic_filter. It will calculate voice to noise ratio which can be used to give information to ic_adapt and to the AGC.
Parameters:
state – [inout] pointer to IC state structure
input_vnr_pred – [inout] voice to noise estimate of the IC input
output_vnr_pred – [inout] voice to noise estimate of the IC output
Adapts the IC filter according to previous frame’s statistics and VNR input.
This function should be called after each call to ic_filter. Filter and adapt functions are separated so that the external VNR can operate on each frame.
Parameters:
state – [inout] pointer to IC state structure
vnr – [in] VNR Voice-to-Noise ratio estimation
lib_ic API State Structure
groupic_state
Enums
enumadaption_config_e
Values:
enumeratorIC_ADAPTION_AUTO
enumeratorIC_ADAPTION_FORCE_ON
enumeratorIC_ADAPTION_FORCE_OFF
enumcontrol_flag_e
Values:
enumeratorHOLD
enumeratorADAPT
enumeratorADAPT_SLOW
enumeratorUNSTABLE
enumeratorFORCE_ADAPT
enumeratorFORCE_HOLD
structic_config_params_t
#include <ic_state.h>
IC configuration structure.
This structure contains configuration settings that can be changed to alter the behaviour of the IC instance. An instance of this structure is is automatically included as part of the IC state.
It controls the behaviour of the main filter and normalisation thereof. The initial values for these configuration parameters are defined in ic_defines.h and are initialised by ic_init().
Public Members
uint8_tbypass
Boolean to control bypassing of filter stage and adaption stage. When set the delayed y audio samples are passed unprocessed to the output. It is recommended to perform an initialisation of the instance after bypass is set as the room transfer function may have changed during that time.
int32_tgamma_log2
Up scaling factor for X energy calculation used for normalisation.
uint32_tsigma_xx_shift
Down scaling factor for X energy for used for normalisation.
Delta value used in denominator to avoid large values when calculating inverse X energy.
structic_adaption_controller_config_t
#include <ic_state.h>
IC adaption controller configuration structure.
This structure contains configuration settings that can be changed to alter the behaviour of the adaption controller. This includes processing of the raw VNR probability input and optional stability controller logic. It is automatically included as part of the IC state and initialised by ic_init().
The initial values for these configuration parameters are defined in ic_defines.h.
Enum which controls the way mu and leakage_alpha are being adjusted.
structic_adaption_controller_state_t
#include <ic_state.h>
IC adaption controller state structure.
This structure contains state used for the instance of the adaption controller logic. It is automatically included as part of the IC state and initialised by ic_init().
Configuration parameters for the adaption controller.
structic_state_t
#include <ic_state.h>
IC state structure.
This is the main state structure for an instance of the Interference Canceller. Before use it must be initialised using the ic_init() function. It contains everything needed for the IC instance including configuration and internal state of both the filter, adaption logic and adaption controller.
BFP array pointing to the frequency domain T used for adapting the filter coefficients (H). Note there is no associated storage because we re-use the x input array as a memory optimisation.
State and configuration parameters for the IC adaption controller.
vnr_pred_state_tvnr_pred_state
Input and Output VNR Prediction related state
lib_ic API Definitions
groupic_defines
Defines
IC_INIT_MU
Initial MU value applied on startup. MU controls the adaption rate of the IC and is normally adjusted by the adaption rate controller during operation.
IC_INIT_EMA_ALPHA
Alpha used for calculating y_ema_energy, x_ema_energy and error_ema_energy.
IC_INIT_LEAKAGE_ALPHA
Alpha used for leaking away H_hat, allowing filter to slowly forget adaption. This value is adjusted by the adaption rate controller if instability is detected.
IC_FILTER_PHASES
The number of filter phases supported by the IC. Each filter phase represents 15ms of filter length. Hence a 10 phase filter will allow cancellation of noise sources with up to 150ms of echo tail length. There is a tradeoff between adaption speed and maximum cancellation of the filter; increasing the number of phases will increase the maximum cancellation at the cost of increased xCORE resource usage and slower adaption times.
IC_Y_CHANNEL_DELAY_SAMPS
This is the delay, in samples that one of the microphone signals is delayed in order for the filter to be effective. A larger number increases the delay through the filter but may improve cancellation. The group delay through the IC filter is 32 + this number of samples.
IC_INIT_SIGMA_XX_SHIFT
Down scaling factor for X energy calculation used for normalisation.
IC_INIT_GAMMA_LOG2
Up scaling factor for X energy calculation for used for LMS normalisation.
IC_INIT_DELTA
Delta value used in denominator to avoid large values when calculating inverse X energy.
IC_INIT_FAST_RATIO_THRESHOLD
Fast ratio threshold to detect instability.
IC_INIT_ENERGY_ALPHA
Alpha for EMA input/output energy calculation.
IC_INIT_HIGH_INPUT_VNR_HOLD_LEAKAGE_ALPHA
Leakage alpha used in case vnr detects high voice probability.
IC_INIT_INSTABILITY_RECOVERY_LEAKAGE_ALPHA
Leakage alpha used in the case where instability is detected. This allows the filter to stabilise without completely forgetting the adaption.
IC_INIT_ADAPT_COUNTER_LIMIT
Limits number of frames for which mu and leakage_alpha could be adapted.
IC_INIT_INPUT_VNR_THRESHOLD
VNR input threshold which decides whether to hold or adapt the filter.
IC_INIT_INPUT_VNR_THRESHOLD_HIGH
VNR high threshold to leak the filter is the speech level is high.
IC_INIT_INPUT_VNR_THRESHOLD_LOW
VNR low threshold to adapt faster when the speech level is low.
IC_INIT_VNR_PRED_ALPHA
Alpha for EMA VNR prediction calculation.
IC_INIT_INPUT_VNR_PRED
Initial value for the input VNR prediction.
IC_INIT_OUTPUT_VNR_PRED
Initial value for the output VNR prediction.
IC_Y_CHANNELS
Number of Y channels input. This is fixed at 1 for the IC. The Y channel is delayed and used to generate the estimated noise signal to subtract from X. In practical terms it does not matter which microphone is X and which is Y. NOT USER MODIFIABLE.
IC_X_CHANNELS
Number of X channels input. This is fixed at 1 for the IC. The X channel is the microphone from which the estimated noise signal is subtracted. In practical terms it does not matter which microphone is X and which is Y. NOT USER MODIFIABLE.
IC_FRAME_LENGTH
Time domain samples block length used internally in the IC’s block LMS algorithm. NOT USER MODIFIABLE.
IC_FRAME_ADVANCE
IC new samples frame size This is the number of samples of new data that the IC works on every frame. 240 samples at 16kHz is 15msec. Every frame, the IC takes in 15msec of mic data and generates 15msec of interference cancelled output. NOT USER MODIFIABLE.
IC_FD_FRAME_LENGTH
Number of bins of spectrum data computed when doing a DFT of a IC_FRAME_LENGTH length time domain vector. The IC_FD_FRAME_LENGTH spectrum values represent the bins from DC to Nyquist. NOT USER MODIFIABLE.
FFT_PADDING
Extra 2 samples you need to allocate in time domain so that the full spectrum (DC to nyquist) can be stored after the in-place FFT. NOT USER MODIFIABLE.
This header contains definitions for data structures used in lib_ic. It also contains the configuration sub-structures which control the operation of the interference canceller during run-time.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$On GitHub£££page_vnr_features_state_h.html#on-github
lib_ic is present as part of fwk_voice. Get the latest version of fwk_voice from
https://github.com/xmos/fwk_voice. The lib_ic module can be found in the modules/lib_ic directory in fwk_voice.
To use the functions in this library in an application, include ic_api.h in the application source file
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library£££modules/voice/modules/lib_vnr/doc/index.html#voice-to-noise-ratio-estimator-library
lib_vnr is a library which estimates the ratio of speech signal in noise for an input audio stream.
lib_vnr library functions uses lib_xcore_math to perform DSP using low-level optimised operations, and lib_tflite_micro and lib_nn to perform inference using an optimised TensorFlow Lite model.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Repository Structure£££getting-started.html#repository-structure
modules/lib_vnr - The lib_vnr library directory within https://github.com/xmos/fwk_voice/.
Within lib_vnr:
api/ - Header files containing the public API for lib_vnr.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Requirements£££getting-started.html#requirements
lib_vnr is included as part of the fwk_voice github repository and all requirements for cloning and building fwk_voice apply. It depends on lib_xcore_math
and the xmos-ai-tools python package.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Structure£££getting-started.html#api-structure
The API is split into 2 parts; feature extraction and inference. The feature extraction API processes an input audio frame to extract features that are input to the inference stage.
The inference API has functions for running inference using the VNR TensorFlow Lite model to predict the speech to noise ratio.
Both feature extraction and inference APIs have initialisation functions that are called only once at device initialisation and processing functions that are called every frame.
The performance requirement is relative low, around 5 MIPS for initialisation and 3 MIPS for processing, and as such is supplied as a single threaded implementation only.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$Getting and Building£££getting-started.html#getting-and-building
The VNR estimator module is obtained as part of the parent fwk_voice repo clone. It is present in fwk_voice/modules/lib_vnr
Both feature extraction and the inference parts of lib_vnr can be compiled as static libraries. The application can link against libfwk_voice_module_lib_vnr_features.a
and/or libfwk_voice_module_lib_vnr_inference.a and add lib_vnr/api/features and/or lib_vnr/api/inference and lib_vnr/api/common as include directories.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$VNR Inference Model£££getting-started.html#vnr-inference-model
The VNR estimator module uses a neural network model to predict the SNR of speech in noise for incoming data. The model used is a pre trained TensorFlow Lite model
that has been optimised for the XCORE architecture using the xmos-ai-tools xformer.
The optimised model is compiled as part of the VNR Inference Engine. Changing the model at runtime is not supported.
If changing to a different model, the application needs to generate the model related files and recompile.
This process is automated through the build system, as described below.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$VNR Inference Model$$$Integrating a TensorFlow Lite model into the VNR module£££getting-started.html#integrating-a-tensorflow-lite-model-into-the-vnr-module
To integrate the new TensorFlow Lite model into the VNR module:
Put an unoptimised model into fwk_voice/modules/lib_vnr/python/model/model_output/trained_model.tflite
Rerun the build tool of our choice (make or ninja, for example)
This will use xmos-ai-tools to optimise .tflite model for xcore and generate .cpp and .h files
into fwk_voice/modules/lib_vnr/src/inference/model/. Those generated files will be picked by the build system and compiled into the VNR module.
The process described above only generates an optimised model that would run on a single core.
Any new models replacing the existing one should have the same set of input features,
input and output size, and data types as the existing model.
If changes to the features are made, the feature extraction code must be updated.
Note that the VNR is used to control the IC behavior, and so its performance may also change.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$VNR Overview£££id1.html#vnr-overview
The VNR (Voice to Noise Ratio) estimator predicts the signal to noise ratio of a speech signal in noise, using a pre-trained neural network. The VNR neural network model outputs a value between 0 and 1, with 1 indicating the strongest speech, and 0, the weakest speech compared to noise in a frame of audio data.
The VNR module processes VNR_FRAME_ADVANCE new audio pcm samples every frame. The time domain input is transformed to frequency domain using a 512 point DFT. A MEL filterbank is then applied to compress the DFT output spectrum into fewer data points. The MEL filter outputs of VNR_PATCH_WIDTH most recent frames are normalised and fed as input features to the VNR prediction model which runs an inference over the features to output the VNR estimate value.
VNR estimations can be very helpful in voice processing pipelines. Applications for VNR include intelligent power management, control of adaptive
filters for reducing noise sources and improved performance of AGC (Automatic Gain Control) blocks that provide a more natural listening experience.
The VNR API is split into 2 parts; feature extraction and inference. This is done to allow multiple sets of features to use the same inference engine.
The VNR feature extraction is further split into 2 parts; a function to form the input frame that the feature extraction can run on, and a function to do the actual feature extraction. The function for forming the input frame starts from VNR_FRAME_ADVANCE new pcm samples and creates the DFT output that is used as input to the MEL filterbank. This has been separated from the rest of the feature extraction to support cases where the VNR might be using the DFT output computed in another module for extracting features.
The pre-trained, optimised for XCORE TensorFlow Lite model, that is used for VNR inference has been compiled as part of the VNR inference static library. There’s no support for providing a new model to the inference engine at run time.
Before starting the feature extraction, the user must call vnr_input_state_init() and vnr_feature_state_init() to initialise the form input frame and feature extraction state. Before starting inference, the user must call vnr_inference_init() to initialise the inference engine.
There are no user configurable parameters within the VNR and so no arguments are required and no configuration structures need be tuned.
Once the VNR is initialised, the vnr_form_input_frame(), vnr_extract_features() and vnr_inference() functions should be called on a frame by frame basis.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Reference£££modules/voice/modules/lib_vnr/doc/src/reference/index.html#api-reference
Create the input frame for processing through the VNR estimator.
This function takes in VNR_FRAME_ADVANCE new samples, combines them with previous frame’s samples to form a VNR_PROC_FRAME_LENGTH samples input frame of time domain data, and outputs the DFT spectrum of the input frame. The DFT spectrum is output in the BFP structure and data memory provided by the user.
The frequency spectrum output from this function is processed through the VNR feature extraction stage.
If sharing the DFT spectrum calculated in some other module, vnr_form_input_frame() is not needed.
input_state – [inout] pointer to the VNR input state structure
X – [out] pointer to a variable of type bfp_complex_s32_t that the user allocates. The user doesn’t need to initialise this bfp variable. After this function, X is updated to point to the DFT output spectrum and can be passed as input to the feature extraction stage.
X_data – [out] pointer to VNR_FD_FRAME_LENGTH values of type complex_s32_t that the user allocates. After this function, the DFT spectrum values are written to this array, and X->data points to X_data memory.
new_x_frame – [in] Pointer to VNR_FRAME_ADVANCE new time domain samples
This function takes in DFT spectrum of the VNR input frame and does the feature extraction. The features are written to the feature_patch BFP structure and feature_patch_data memory provided by the user. The feature output from this function are passed as input to the VNR inference engine.
Parameters:
vnr_feature_state – [inout] Pointer to the VNR feature extraction state structure
feature_patch – [out] Pointer to the bfp_s32_t structure allocated by the user. The user doesn’t need to initialise this BFP structure before passing it to this function. After this function call feature_patch will be updated and will point to the extracted features. It can then be passed to the inference stage.
feature_patch_data – [out] Pointer to the VNR_PATCH_WIDTH * VNR_MEL_FILTERS int32_t values allocated by the user. The extracted features will be written to the feature_patch_data array and the BFP structure’s feature_patch->data will point to this array.
lib_vnr inference engine API Functions
groupvnr_inference_api
Functions
int32_tvnr_inference_init()
Initialise the inference_engine object and load the VNR model into the inference engine.
This function calls lib_tflite_micro functions to initialise the inference engine and load the VNR model into it. It is called once at startup. The memory required for the inference engine object as well as the tensor arena size required for inference is statically allocated as global buffers in the VNR module. The VNR model is compiled as part of the VNR module.
This function invokes the inference engine. It takes in a set of features corresponding to an input frame of data and outputs the VNR prediction value. The VNR output is a single value ranging between 0 and 1 returned in float_s32_t format, with 0 being the lowest SNR and 1 being the strongest possible SNR in speech compared to noise.
Parameters:
vnr_output – [out] VNR prediction value.
features – [in] Input feature vector. Note that this is not passed as a const pointer and the feature memory is overwritten as part of the inference computation.
lib_vnr #defines common to feature extraction and inference
groupvnr_defines
Defines
VNR_MEL_FILTERS
Number of filters in the MEL filterbank used in the VNR feature extraction.
VNR_PATCH_WIDTH
Number of frames that make up a full set of features for the inference to run on.
lib_vnr feature extraction #defines and data structure definitions
groupvnr_features_state
Defines
VNR_PROC_FRAME_LENGTH
Time domain samples block length used internally in VNR DFT computation. NOT USER MODIFIABLE.
<>
VNR_FRAME_ADVANCE
VNR new samples frame size This is the number of samples of new data that the VNR processes every frame. 240 samples at 16kHz is 15msec. NOT USER MODIFIABLE.
VNR_FD_FRAME_LENGTH
Number of bins of spectrum data computed when doing a DFT of a VNR_PROC_FRAME_LENGTH length time domain vector. The VNR_FD_FRAME_LENGTH spectrum values represent the bins from DC to Nyquist. NOT USER MODIFIABLE.
Feature buffer containing the most recent VNR_MEL_FILTERS frames’ MEL frequency spectrum.
lib_vnr Header Files
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Reference$$$vnr_features_api.h£££vnr-api-h.html#vnr-features-api-h
pagepage_vnr_features_api_h
This header contains lib_vnr features extraction API functions.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Reference$$$vnr_inference_api.h£££page_vnr_features_api_h.html#vnr-inference-api-h
pagepage_vnr_inference_api_h
This header contains lib_vnr inference engine API functions.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Reference$$$vnr_defines.h£££vnr-common-defines-h.html#vnr-defines-h
pagepage_vnr_defines_h
This header contains the lib_vnr public #defines that are common to both feature extraction and inference.
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$API Reference$$$vnr_features_state.h£££id1.html#vnr-features-state-h
pagepage_vnr_features_state_h
This header contains lib_vnr feature extraction related public #defines and data structure definitions
XCORE ® -VOICE Solutions$$$Audio Processing$$$Audio Features$$$Voice To Noise Ratio Estimator Library$$$On GitHub£££page_vnr_features_state_h.html#on-github
lib_vnr is present as part of fwk_voice. Get the latest version of fwk_voice from
https://github.com/xmos/fwk_voice. The lib_vnr module can be found in the modules/lib_vnr directory in fwk_voice.
The xcore platform provides a range of powerful, flexible and economic crossover processors for the use in wide-ranging applications. The XCore platform provides:
Fast compute
Flexibility
Economy
Scalablity
Security
Fast time to market
Architecture & Hardware Guide
At the heart of the platform, the Architecture & Hardware Guide describes the multicore processors. Multiple xcore processors can themselves be “networked” together with seamless communications.
The Programming Guide describes how logical cores of an xcore processor can act independently to behave like highly responsive hardware peripherals, or can work as a team to apply all available CPU cycles onto a single compute task.
The xcore processors are accompanied by the XTC Tools. As well as providing a powerful toolchain for application development, the toolkit assists with application deployment and upgrade.
Traditionally, xcore multi-core processors have been programmed using the XC language. The XC language allows the programmer to statically place tasks on the available hardware cores and wire them together with channels to provide inter-process communication. The XC language also exposes “events,” which are unique to the xcore architecture and are a useful alternative to interrupts.
Using the XC language, it is possible to write dedicated application software with deterministic timing and very low latency between I/O and tasks.
While XC elegantly enables the intrinsic, unique capabilities of the xcore architecture, there often needs to be higher level application type software running alongside it. The programming model that makes the lower level deterministic software possible may not be best suited for many higher level parts of an application that do not require deterministic timing. Where strict real-time execution is not required, higher level abstractions can be used to manage finite hardware resources, and provide a more familiar programming environment.
A symmetric multiprocessing (SMP) real time operating system (RTOS) can be used to simplify xcore application designs, as well as to preserve the hard real-time benefits provided by the xcore architecture for the lower level software functions that require it.
This document assumes familiarity with real time operating systems in general. Familiarity with FreeRTOS specifically should not be required, but will be helpful. For current up to date documentation on FreeRTOS see the documentation section on the FreeRTOS website.
To support this new programming model for xcore, XMOS has extended the popular and free FreeRTOS kernel to support SMP. This allows for the kernel’s scheduler to be started on any number of available xcore logical cores per tile, leaving the remaining free to support other program elements that combine to create complete systems. Once the scheduler is started, FreeRTOS threads are placed on cores dynamically at runtime, rather than statically at compile time. All the usual FreeRTOS rules for thread scheduling are followed, except that rather than only running the single highest priority thread that is ready at any given time, multiple threads may run simultaneously. The threads chosen to run are always the highest priority threads that are ready. When there are more threads of a single priority that are ready to run than the number of cores available, they are scheduled in a round robin fashion. Dynamic scheduling allows FreeRTOS to optimize physical core usage based on priority and availability at runtime, opening up the potential for using tile wide MIPs more efficiently than what could be manually specified in a static compile time setting.
One of xcore’s primary strengths is its guarantee of deterministic behavior and timing. RTOS threads can also benefit from this determinism provided by the xcore architecture. An RTOS thread with interrupts disabled and a high enough priority behaves just as a bare-metal thread. An SMP RTOS kernel does not need to preempt a high priority thread because it has many other cores to utilize to schedule lower priority threads. Using an SMP RTOS allows developers to concentrate on specific requirements of their application without worrying about what affect they might have on non-preemptable thread response times. Furthermore, modification of the program in the future is much easier because the developer does not have to worry about affecting existing responsiveness with changes in unrelated areas. The non-preemptable threads will not be effected by adding lower-priority functionality.
Another xcore strength is it’s performance. xcore.ai provides lightning fast general purpose compute, AI acceleration, powerful DSP and instantaneous I/O control. RTOS threads can also benefit from the performance provided by the xcore architecture, allowing an application developer to dynamically shift performance usage from one application feature to another.
The standard FreeRTOS kernel supports dynamic task priorities, while the FreeRTOS-SMP kernel adds the following additional APIs:
vTaskCoreAffinitySet
vTaskCoreAffinityGet
vTaskPreemptionDisable
vTaskPreemptionEnable
Together, these API enable a developer to take full advantage of xcore’s performance.
Some additional configuration options are also available to the FreeRTOS-SMP Kernel:
To further leverage the xcore hardware and the FreeRTOS programming model, XMOS provides support for asymmetric multiprocessing (AMP) per tile. Each XMOS chip contains at least two tiles, which consist of their own set of logical xcore cores, IO, memory space, and more. XMOS provides a build method and variety of software drivers to allow an application to be created that is an AMP system containing, multiple SMP FreeRTOS kernels.
To help ease development of xcore applications using an SMP RTOS, XMOS provides several SMP RTOS compatible drivers. These include, but are not necessarily limited to:
Documentation on each of these drivers can be found under the RTOS Drivers section in the RTOS framework documentation pages.
It is worth noting that most of these drivers utilize a lightweight RTOS abstraction layer, meaning that they are not dependent on FreeRTOS. Conceivably they should work on any SMP RTOS, provided an abstraction layer for it is provided. This abstraction layer is found under the path modules/rtos/modules/osal. At the moment the only available SMP RTOS for xcore is the XMOS SMP FreeRTOS, but more may become available in the future.
The RTOS framework also includes some higher level RTOS compatible software services, some of which call the aforementioned drivers. These include, but are not necessarily limited to:
DHCP server
FAT filesystem
HTTP parser
JSON parser
MQTT client
SNTP client
TLS
USB stack
WiFi connection manager
Documentation on several software services can be found under the RTOS Services section in the RTOS framework documentation pages.
This document is intended to help you start your first FreeRTOS application on xcore. We assume you have read FreeRTOS Application Programming and that you are familiar with FreeRTOS.
A fully functional example application that can be found in the RTOS framework under the path examples/freertos/explorer_board. This application is a reference for how to use an RTOS drivers or software service, and serves as an example for how to structure an SMP RTOS application for xcore. Additional code to initialize the SoC platform for this example is provided by a board support configuration library modules/rtos/modules/board_support/XCORE-AI-EXPLORER_2V0/platform
This example application runs two instances of SMP FreeRTOS, one on each of the processor’s two tiles. Because each tile has its own memory which is not shared between them, this can be viewed as a single asymmetric multiprocessing (AMP) system that comprises two SMP systems. A FreeRTOS thread that is created on one tile will never be scheduled to run on the other tile. Similarly, an RTOS object that is created on a tile, such as a queue, can only be accessed by threads and ISRs that run on that tile and never by code running on the other tile.
That said, the example application is programmed and built as a single coherent application, which will be familiar to programmers who have previously programmed for the xcore in the XC programming language. Data that must be shared between threads running on different tiles is sent via a channel using the RTOS intertile driver, which under the hood uses a streaming channel between the tiles.
Most of the I/O interface drivers in fact provide a mechanism to share driver instances between tiles that utilizes this intertile driver. For those familiar with XC programming, this can be viewed as a C alternative to XC interfaces.
For example, a SPI interface might be available on tile 0. Normally, initialization code that runs on tile 0 sets this interface up and then starts the driver. Without any further initialization, code that runs on tile 1 will be unable to access this interface directly, due both to not having direct access to tile 0’s memory, as well as not having direct access to tile 0’s ports. The drivers, however, provide some additional initialization functions that can be used by the application to share the instance on tile 0 with tile 1. After this initialization is done, code running on tile 1 may use the instance with the same driver API as tile 0, almost as if it was actually running on tile 0.
The example application referenced above, as well as the RTOS driver documentation, should be consulted to see exactly how to initialize and share driver instances. Additionally, not all IO is capable of being shared between tiles directly through the driver API due to timing constraints.
The RTOS framework provides the ON_TILE(t) preprocessor macro. This macro may be used by applications to ensure certain code is included only on a specific tile at compile time. In the example application, there is a single task that is created on both tiles that starts the drivers and creates the remaining application tasks. While this function is written as a single function, various parts are inside #if ON_TILE() blocks. For example, consider the following code snippet found inside the i2c_init() function:
When this function is compiled for tile I2C_TILE_NO, only the first block is included. When it is compiled for the other tile, only the second block is included. When the application is run, tile I2C_TILE_NO performs the initialization of the the I2C master driver host, while the other tile initializes the I2C master driver client. Because the I2C driver instance is shared between the two tiles, it may in fact be set to either zero or one, providing a demonstration of the way that drivers instances may be shared between tiles.
The RTOS framework provides a single XC file that provides the main() function. This provided main() function calls main_tile0() through main_tile3(), depending on the number of tiles that the application requires and the number of tiles provided by the target xcore processor. The application must provide each of these tile entry point functions. Each one is provided with up to three channel ends that are connected to each of the other tiles.
The example application provides both main_tile0() and main_tile1(). Each one calls a common initialization function that initializes all the drivers for the interfaces specific to its tile. These functions also call the initialization functions to share these driver instances between the tiles. These initialization functions are found in the platform/platform_init.c source file.
Each tile then creates the startup_task() task and starts the FreeRTOS scheduler. The startup_task() completes the driver instance sharing and then starts all of the driver instances. The driver startup functions are found in the platform/platform_start.c source file.
Consult the RTOS driver documentation for the details on what exactly each of the RTOS API functions called by this application does.
XCORE ® -VOICE Solutions$$$RTOS Programming Guide$$$Tutorials$$$RTOS Application Design$$$Board Support Configurations£££modules/rtos/doc/programming_guide/tutorials/application_design.html#board-support-configurations
xcore leverages its architecture to provide a flexible chip where many typically silicon based peripherals are found in software. This allows a chip to be reconfigured in a way that provides the specific IO required for a given application, thus resulting in a low cost yet incredibly silicon efficient solution. Board support configurations (bsp_configs) are the description for the hardware IO that exists in a given board. The bsp_configs provide the application programmer with an API to initialize and start the hardware configuration, as well as the supported RTOS driver contexts. The programming model in this FreeRTOS architecture is:
.xn files provide the mapping of ports, pins, and links
bsp_configs specify, setup, and start hardware IO and provide the application with RTOS driver contexts
applications use the bsp_config init/start code as well as RTOS driver contexts, similar to conventional microcontroller programming models.
To support any generic bsp_config, applications should call platform_init() before starting the scheduler, and then platform_start() after the scheduler is running and before any RTOS drivers are used.
The bsp_configs provided with the RTOS framework in modules/rtos/modules/bsp_config are an excellent starting point. They provide the most common peripheral drivers that are supported by the boards that support RTOS framework based applications. For advanced users, it is recommended that you copy one of these bsp_config into your application project and customize as needed.
custom_config.cmake provides the CMake target of the configuration. This target should link the required RTOS framework libraries to support the configuration it defines.
custom_config_xn_file.xn provides various hardware parameters including but not limited to the chip package, IO mapping, and network information.
platform_conf.h provides default configuration of all header defined configuration macros. These may be overridden by compile definitions or application headers.
driver_instances.h provides the declaration of all RTOS drivers in the configuration. It may define XCORE hardware resources, such as ports and clockblocks. It may also define tile placements.
driver_instances.c provides the definition of all RTOS drivers in the configuration.
platform_init.h provides the declaration of platform_init(chanend_t other_tile_c) and platform_start(void)
platform_init.c provides the initialization of all drivers defined in the configuration through the definition of platform_init(chanend_t other_tile_c). This code is run before the scheduler is started and therefore will not be able to access all RTOS driver functionalities nor kernel objects.
platform_start.c provides the starting of all drivers defined in the configuration through the definition of platform_start(void). It may also perform any initialization setup, such as configuring the app_pll or setting up an on board DAC. This code is run once the kernel is running and is therefore subject to preemption and other dynamic scheduling SMP programming considerations.
One of these features if the -report option, which will Display a summary of resource usage. One of the outputs of this report is memory usage, split into the stack, code, and data requirements of the program. Unlike most XC applications, FreeRTOS makes heavy use of dynamic memory allocation. The FreeRTOS heap will appear as Data in the XTC Tools report. The heap size is determined by the compile time definition configTOTAL_HEAP_SIZE, which can be found in an application’s FreeRTOSConfig.h.
For AMP SMP FreeRTOS builds, which are created using the cmake macro merge_binaries(), there are actually multiple application builds, one per tile, which are then combined. While building a given AMP application, the console output will contain both of the individual tile build reports.
As an example, consider building the example_freertos_explorer_board target.
Because the tile 1 portion of the tile1 target build replaces the tile 1 portion in the tile0 target build.
The XTC Tools also provide a method to examine the resource usage of a binary post build. This method will only work if used on the intermediate binaries.
Note: Because the resulting example_freertos_explorer_board.xe binary was created by merging into tile0_example_freertos_explorer_board.xe, the results of xobjdump –resources example_freertos_explorer_board.xe will be the exact same as xobjdump –resources tile0_example_freertos_explorer_board.xe and not account for the actual tile 1 requirements.
Applications using the RTOS Framework are built using CMake. The RTOS framework provides many libraries, drivers and software services, all of which can be included by the application’s CMakeLists.txt file. The application’s CMakeLists can specify precisely which drivers and software services within the SDK should be included through the use of various CMake target aliases.
XCORE ® -VOICE Solutions$$$RTOS Programming Guide$$$Tutorials$$$Board Support Configurations£££modules/rtos/doc/programming_guide/tutorials/bsp_config.html#board-support-configurations
xcore leverages its architecture to provide a flexible chip where many typically silicon based peripherals are found in software. This allows a chip to be reconfigured in a way that provides the specific IO required for a given application, thus resulting in a low cost yet incredibly silicon efficient solution. Board support configurations (bsp_configs) are the description for the hardware IO that exists in a given board. The bsp_configs provide the application programmer with an API to initialize and start the hardware configuration, as well as the supported RTOS driver contexts. The programming model in this FreeRTOS architecture is:
.xn files provide the mapping of ports, pins, and links
bsp_configs specify, setup, and start hardware IO and provide the application with RTOS driver contexts
applications use the bsp_config init/start code as well as RTOS driver contexts, similar to conventional microcontroller programming models.
To support any generic bsp_config, applications should call platform_init() before starting the scheduler, and then platform_start() after the scheduler is running and before any RTOS drivers are used.
The bsp_configs provided with the RTOS framework in modules/rtos/modules/bsp_config are an excellent starting point. They provide the most common peripheral drivers that are supported by the boards that support RTOS framework based applications. For advanced users, it is recommended that you copy one of these bsp_config into your application project and customize as needed.
XCORE ® -VOICE Solutions$$$RTOS Programming Guide$$$Tutorials$$$Board Support Configurations$$$Creating Custom bsp_configs£££modules/rtos/doc/programming_guide/tutorials/bsp_config.html#creating-custom-bsp-configs
To enable hardware portability, a minimal bsp_config should contain the following:
custom_config.cmake provides the CMake target of the configuration. This target should link the required RTOS framework libraries to support the configuration it defines.
custom_config_xn_file.xn provides various hardware parameters including but not limited to the chip package, IO mapping, and network information.
platform_conf.h provides default configuration of all header defined configuration macros. These may be overridden by compile definitions or application headers.
driver_instances.h provides the declaration of all RTOS drivers in the configuration. It may define XCORE hardware resources, such as ports and clockblocks. It may also define tile placements.
driver_instances.c provides the definition of all RTOS drivers in the configuration.
platform_init.h provides the declaration of platform_init(chanend_tother_tile_c) and platform_start(void)
platform_init.c provides the initialization of all drivers defined in the configuration through the definition of platform_init(chanend_tother_tile_c). This code is run before the scheduler is started and therefore will not be able to access all RTOS driver functionalities nor kernel objects.
platform_start.c provides the starting of all drivers defined in the configuration through the definition of platform_start(void). It may also perform any initialization setup, such as configuring the app_pll or setting up an on board DAC. This code is run once the kernel is running and is therefore subject to preemption and other dynamic scheduling SMP programming considerations.
This driver provides the application with the boot partition and data partition layout of the flash used by the second stage bootloader. The driver provides a subset of the functionality of libquadflash enabling the application to use any transport method and the RTOS qspi flash driver to read the factory image, read/write a single upgrade image, and read/write the data partition.
unsignedaddr=rtos_dfu_image_get_factory_addr(dfu_image_ctx);unsignedsize=rtos_dfu_image_get_factory_size(dfu_image_ctx);unsignedchar*buf=pvPortMalloc(sizeof(unsignedchar)*size);rtos_qspi_flash_read(qspi_flash_ctx,(uint8_t*)buf,addr,size);// buf now contains the factory image contents
It is advised to perform this operation in blocks rather than full image size to reduce memory usage. Once the buffer is populated from flash, it can be sent over the desired transport method, such as USB, I2C, etc.
unsignedaddr=rtos_dfu_image_get_upgrade_addr(dfu_image_ctx);unsignedsize=rtos_dfu_image_get_upgrade_size(dfu_image_ctx);unsignedchar*buf=pvPortMalloc(sizeof(unsignedchar)*size);rtos_qspi_flash_read(qspi_flash_ctx,(uint8_t*)buf,addr,size);// buf now contains the upgrade image contents
It is advised to perform this operation in blocks rather than full image size to reduce memory usage. Once the buffer is populated from flash, it can be sent over the desired transport method, such as USB, I2C, etc.
// Assuming buf contains the image data// and size contains the size in bytesunsignedaddr=rtos_dfu_image_get_upgrade_addr(dfu_image_ctx);unsigneddata_partition_base_addr=rtos_dfu_image_get_data_partition_addr(dfu_image_ctx);unsignedbytes_avail=data_partition_base_addr-addr;size_tsector_size=rtos_qspi_flash_sector_size_get(qspi_flash_ctx);if(size<bytes_avail){unsignedchar*tmp_buf=pvPortMalloc(sizeof(unsignedchar)*sector_size);unsignedcur_offset=0;do{unsignedlength=(size-(cur_offset-addr))>=sector_size?sector_size:(size-(cur_offset-addr));rtos_qspi_flash_lock(qspi_flash_ctx);{rtos_qspi_flash_read(qspi_flash_ctx,tmp_buf,addr+cur_offset,sector_size);memcpy(tmp_buf,data+cur_offset,length);rtos_qspi_flash_erase(qspi_flash_ctx,addr+cur_offset,sector_size);rtos_qspi_flash_write(qspi_flash_ctx,(uint8_t*)tmp_buf,addr+cur_offset,sector_size);}rtos_qspi_flash_unlock(qspi_flash_ctx);cur_offset+=length;}while(cur_offset<(size-1));vPortFree(tmp_buf);}else{rtos_printf("Insufficient space for upgrade image\n");}
It is advised to perform this operation in blocks rather than full image size to reduce memory usage. The buffer can be populated over the desired transport method, such as USB, I2C, etc.
XCORE ® -VOICE Solutions$$$RTOS Programming Guide$$$Tutorials$$$RTOS Application DFU$$$Reading the Data Partition Image£££modules/rtos/doc/programming_guide/tutorials/application_dfu_usage.html#reading-the-data-partition-image
To read back the data partition image:
unsignedaddr=rtos_dfu_image_get_data_partition_addr(dfu_image_ctx);unsignedsize=rtos_qspi_flash_size_get(qspi_flash_ctx);unsignedchar*buf=pvPortMalloc(sizeof(unsignedchar)*size);rtos_qspi_flash_read(qspi_flash_ctx,(uint8_t*)buf,addr,size);// buf now contains the data partition image contents
It is advised to perform this operation in blocks rather than full image size to reduce memory usage. The data partition will likely be too large to read into SRAM in a read single operation. Once the buffer is populated from flash, it can be sent over the desired transport method, such as USB, I2C, etc.
XCORE ® -VOICE Solutions$$$RTOS Programming Guide$$$Tutorials$$$RTOS Application DFU$$$Writing the Data Partition Image£££modules/rtos/doc/programming_guide/tutorials/application_dfu_usage.html#writing-the-data-partition-image
To overwrite the current data partition image:
// Assuming buf contains the image data// and size contains the size in bytesunsignedaddr=rtos_dfu_image_get_data_partition_addr(dfu_image_ctx);unsignedend_addr=rtos_qspi_flash_size_get(qspi_flash_ctx);unsignedbytes_avail=end_addr-addr;size_tsector_size=rtos_qspi_flash_sector_size_get(qspi_flash_ctx);if(size<bytes_avail){unsignedchar*tmp_buf=pvPortMalloc(sizeof(unsignedchar)*sector_size);unsignedcur_offset=0;do{unsignedlength=(size-(cur_offset-addr))>=sector_size?sector_size:(size-(cur_offset-addr));rtos_qspi_flash_lock(qspi_flash_ctx);{rtos_qspi_flash_read(qspi_flash_ctx,tmp_buf,addr+cur_offset,sector_size);memcpy(tmp_buf,data+cur_offset,length);rtos_qspi_flash_erase(qspi_flash_ctx,addr+cur_offset,sector_size);rtos_qspi_flash_write(qspi_flash_ctx,(uint8_t*)tmp_buf,addr+cur_offset,sector_size);}rtos_qspi_flash_unlock(qspi_flash_ctx);cur_offset+=length;}while(cur_offset<(size-1));vPortFree(tmp_buf);}else{rtos_printf("Insufficient space for data partition image\n");}
It is advised to perform this operation in blocks rather than full image size to reduce memory usage. The buffer can be populated over the desired transport method, such as USB, I2C, etc.
Starts an RTOS GPIO driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before any of the core GPIO driver functions are called with this instance.
rtos_gpio_init() must be called on this GPIO driver instance prior to calling this.
Parameters:
ctx – A pointer to the GPIO driver instance to start.
Initializes an RTOS GPIO driver instance. There should only be one per tile. This instance represents all the GPIO ports owned by the calling tile. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before calling rtos_gpio_start() or any of the core GPIO driver functions with this instance.
Parameters:
ctx – A pointer to the GPIO driver instance to initialize.
RTOS_GPIO_ISR_CALLBACK_ATTR
This attribute must be specified on all RTOS GPIO interrupt callback functions provided by the application.
structrtos_gpio_isr_info_t
#include <rtos_gpio.h>
Struct to hold interrupt state data for GPIO ports.
The members in this struct should not be accessed directly.
structrtos_gpio_struct
#include <rtos_gpio.h>
Struct representing an RTOS GPIO driver instance.
The members in this struct should not be accessed directly.
Configures a port in drive mode. Output values will be driven on the pins. This is the default drive state of a port. This has the side effect of disabling the port’s internal pull-up and pull down resistors.
Parameters:
ctx – A pointer to the GPIO driver instance to use.
Configures a port in drive low mode. When the output value is 0 the pin is driven low, otherwise no value is driven. This has the side effect of enabled the port’s internal pull-up resistor.
Parameters:
ctx – A pointer to the GPIO driver instance to use.
Configures a port in drive high mode. When the output value is 1 the pin is driven high, otherwise no value is driven. This has the side effect of enabled the port’s internal pull-down resistor.
Parameters:
ctx – A pointer to the GPIO driver instance to use.
port_id – The GPIO port to set to drive mode high.
The following functions may be used to share a GPIO driver instance with other xcore tiles. Tiles that the
driver instance is shared with may call any of the core functions listed above.
Initializes an RTOS GPIO driver instance on a client tile. This allows a tile that does not own the actual driver instance to use a driver instance on another tile. This will be called instead of rtos_gpio_init(). The host tile that owns the actual instance must simultaneously call rtos_gpio_rpc_host_init().
Parameters:
gpio_ctx – A pointer to the GPIO driver instance to initialize.
rpc_config – A pointer to an RPC config struct. This must have the same scope as gpio_ctx.
host_intertile_ctx – A pointer to the intertile driver instance to use for performing the communication between the client and host tiles. This must have the same scope as gpio_ctx.
Performs additional initialization on a GPIO driver instance to allow client tiles to use the GPIO driver instance. Each client tile that will use this instance must simultaneously call rtos_gpio_rpc_client_init().
Parameters:
gpio_ctx – A pointer to the GPIO driver instance to share with clients.
rpc_config – A pointer to an RPC config struct. This must have the same scope as gpio_ctx.
client_intertile_ctx – An array of pointers to the intertile driver instances to use for performing the communication between the host tile and each client tile. This must have the same scope as gpio_ctx.
remote_client_count – The number of client tiles to share this driver instance with.
Configures the RPC for a GPIO driver instance. This must be called by both the host tile and all client tiles.
On the client tiles this must be called after calling rtos_gpio_rpc_client_init(). After calling this, the client tile may immediately begin to call the core GPIO functions on this driver instance. It does not need to wait for the host to call rtos_gpio_start().
gpio_ctx – A pointer to the GPIO driver instance to configure the RPC for.
intertile_port – The port number on the intertile channel to use for transferring the RPC requests and responses for this driver instance. This port must not be shared by any other functions. The port must be the same for the host and all its clients.
host_task_priority – The priority to use for the task on the host tile that handles RPC requests from the clients.
I2C RTOS Driver
This driver can be used to instantiate and control an I2C master or slave mode I/O interface on xcore in an RTOS application.
I2C Master RTOS Driver
This driver can be used to instantiate and control an I2C master I/O interface on xcore in an RTOS application.
I2C Master Initialization API
The following structures and functions are used to initialize and start an I2C driver instance.
Starts an RTOS I2C master driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before any of the core I2C master driver functions are called with this instance.
rtos_i2c_master_init() must be called on this I2C master driver instance prior to calling this.
Parameters:
i2c_master_ctx – A pointer to the I2C master driver instance to start.
Initializes an RTOS I2C master driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before calling rtos_i2c_master_start() or any of the core I2C master driver functions with this instance.
Parameters:
i2c_master_ctx – A pointer to the I2C master driver instance to initialize.
p_scl – The port containing SCL. This may be either the same as or different than p_sda.
scl_bit_position – The bit number of the SCL line on the port p_scl.
scl_other_bits_mask – A value that is ORed into the port value driven to p_scl both when SCL is high and low. The bit representing SCL (as well as SDA if they share the same port) must be set to 0.
p_sda – The port containing SDA. This may be either the same as or different than p_scl.
sda_bit_position – The bit number of the SDA line on the port p_sda.
sda_other_bits_mask – A value that is ORed into the port value driven to p_sda both when SDA is high and low. The bit representing SDA (as well as SCL if they share the same port) must be set to 0.
tmr – This is unused and should be set to 0. This will be removed.
kbits_per_second – The speed of the I2C bus. The maximum value allowed is 400.
structrtos_i2c_master_struct
#include <rtos_i2c_master.h>
Struct representing an RTOS I2C master driver instance.
The members in this struct should not be accessed directly.
I2C Master Core API
The following functions are the core I2C driver functions that are used after it has been initialized and started.
ctx – A pointer to the I2C master driver instance to use.
device_addr – The address of the device to write to.
buf – The buffer containing data to write.
n – The number of bytes to write.
num_bytes_sent – The function will set this value to the number of bytes actually sent. On success, this will be equal to n but it will be less if the slave sends an early NACK on the bus and the transaction fails.
send_stop_bit – If this is non-zero then a stop bit will be sent on the bus after the transaction. This is usually required for normal operation. If this parameter is zero then no stop bit will be omitted. In this case, no other task can use the component until a stop bit has been sent.
Return values:
<tt>I2C_ACK</tt> – if the write was acknowledged by the device.
ctx – A pointer to the I2C master driver instance to use.
device_addr – The address of the device to read from.
buf – The buffer to fill with data.
n – The number of bytes to read.
send_stop_bit – If this is non-zero then a stop bit. will be sent on the bus after the transaction. This is usually required for normal operation. If this parameter is zero then no stop bit will be omitted. In this case, no other task can use the component until a stop bit has been sent.
Return values:
<tt>I2C_ACK</tt> – if the read was acknowledged by the device.
This function will cause a stop bit to be sent on the bus. It should be used to complete/abort a transaction if the send_stop_bit argument was not set when calling the rtos_i2c_master_read() or rtos_i2c_master_write() functions.
Parameters:
ctx – A pointer to the I2C master driver instance to use.
This function writes to an 8-bit addressed, 8-bit register in an I2C device. The function writes the data by sending the register address followed by the register data to the device at the specified device address.
Parameters:
ctx – A pointer to the I2C master driver instance to use.
device_addr – The address of the device to write to.
reg_addr – The address of the register to write to.
data – The 8-bit value to write.
Return values:
<tt>I2C_REGOP_DEVICE_NACK</tt> – if the address is NACKed.
<tt>I2C_REGOP_INCOMPLETE</tt> – if not all data was ACKed.
<tt>I2C_REGOP_SUCCESS</tt> – on successful completion of the write.
This function reads from an 8-bit addressed, 8-bit register in an I2C device. The function reads the data by sending the register address followed reading the register data from the device at the specified device address.
Note that no stop bit is transmitted between the write and the read. The operation is performed as one transaction using a repeated start.
Parameters:
ctx – A pointer to the I2C master driver instance to use.
device_addr – The address of the device to read from.
reg_addr – The address of the register to read from.
data – A pointer to the byte to fill with data read from the register.
Return values:
<tt>I2C_REGOP_DEVICE_NACK</tt> – if the device NACKed.
<tt>I2C_REGOP_SUCCESS</tt> – on successful completion of the read.
I2C Master RPC Initialization API
The following functions may be used to share a I2C driver instance with other xcore tiles. Tiles that the
driver instance is shared with may call any of the core functions listed above.
Initializes an RTOS I2C master driver instance on a client tile. This allows a tile that does not own the actual driver instance to use a driver instance on another tile. This will be called instead of rtos_i2c_master_init(). The host tile that owns the actual instance must simultaneously call rtos_i2c_master_rpc_host_init().
Parameters:
i2c_master_ctx – A pointer to the I2C master driver instance to initialize.
rpc_config – A pointer to an RPC config struct. This must have the same scope as i2c_master_ctx.
host_intertile_ctx – A pointer to the intertile driver instance to use for performing the communication between the client and host tiles. This must have the same scope as i2c_master_ctx.
Performs additional initialization on an I2C master driver instance to allow client tiles to use the I2C master driver instance. Each client tile that will use this instance must simultaneously call rtos_i2c_master_rpc_client_init().
Parameters:
i2c_master_ctx – A pointer to the I2C master driver instance to share with clients.
rpc_config – A pointer to an RPC config struct. This must have the same scope as i2c_master_ctx.
client_intertile_ctx – An array of pointers to the intertile driver instances to use for performing the communication between the host tile and each client tile. This must have the same scope as i2c_master_ctx.
remote_client_count – The number of client tiles to share this driver instance with.
Configures the RPC for an I2C master driver instance. This must be called by both the host tile and all client tiles.
On the client tiles this must be called after calling rtos_i2c_master_rpc_client_init(). After calling this, the client tile may immediately begin to call the core I2C master functions on this driver instance. It does not need to wait for the host to call rtos_i2c_master_start().
i2c_master_ctx – A pointer to the I2C master driver instance to configure the RPC for.
intertile_port – The port number on the intertile channel to use for transferring the RPC requests and responses for this driver instance. This port must not be shared by any other functions. The port must be the same for the host and all its clients.
host_task_priority – The priority to use for the task on the host tile that handles RPC requests from the clients.
I2C Slave RTOS Driver
This driver can be used to instantiate and control an I2C slave I/O interface on xcore in an RTOS application.
I2C Slave API
The following structures and functions are used to initialize and start an I2C driver instance.
Function pointer type for application provided RTOS I2C slave start callback functions.
These callback functions are optionally called by an I2C slave driver’s thread when it is first started. This gives the application a chance to perform startup initialization from within the driver’s thread.
Param ctx:
A pointer to the associated I2C slave driver instance.
Param app_data:
A pointer to application specific data provided by the application. Used to share data between this callback function and the application.
Function pointer type for application provided RTOS I2C slave transmit start callback functions.
These callback functions are called when an I2C slave driver instance needs to transmit data to a master device. This callback must provide the data to transmit and the length.
Param ctx:
A pointer to the associated I2C slave driver instance.
Param app_data:
A pointer to application specific data provided by the application. Used to share data between this callback function and the application.
Param data:
A pointer to the data buffer to transmit to the master. The driver sets this to its internal data buffer, which has a size of RTOS_I2C_SLAVE_BUF_LEN, prior to calling this callback. This may be set to a different buffer by the callback. The callback must fill this buffer with the data to send to the master.
Return:
The number of bytes to transmit to the master from data. If the master reads more bytes than this, the driver will wrap around to the start of the buffer and send it again.
Function pointer type for application provided RTOS I2C slave transmit done callback functions.
These callback functions are optionally called when an I2C slave driver instance is done transmitting data to a master device. A buffer to the data sent and the actual number of bytes sent are provided to the callback.
The application may want to use this, for example, if the buffer that was sent was malloc’d. This callback can be used to free the buffer.
Param ctx:
A pointer to the associated I2C slave driver instance.
Param app_data:
A pointer to application specific data provided by the application. Used to share data between this callback function and the application.
Param data:
A pointer to the data transmitted to the master.
Param len:
The number of bytes transmitted to the master from data.
Function pointer type for application provided function to check bytes received from master individually.
This callback function is called once per byte received from the master device.
The application may want to use this, for example, to check byte by byte and force a NACK for an unexpected payload.
The user provided functions must be marked with RTOS_I2C_SLAVE_MASTER_SENT_BYTE_CHECK_CALLBACK_ATTR.
Param ctx:
A pointer to the associated I2C slave driver instance.
Param app_data:
A pointer to application specific data provided by the application. Used to share data between this callback function and the application.
Param data:
A copy of the most recent byte of data transmitted from the master.
Param cur_status:
A pointer to the current ACK/NACK response for this byte. The application may change this to I2C_SLAVE_ACK or I2C_SLAVE_NACK. If cur_status is returned as an invalid value, the driver will implicitly NACK.
Function pointer type for application provided function to alert application that there is a write transaction incoming from master
This allows an application to NACK if it is not ready for handling write requests.
The user provided functions must be marked with RTOS_I2C_SLAVE_WRITE_ADDR_REQUEST_CALLBACK_ATTR.
Param ctx:
A pointer to the associated I2C slave driver instance.
Param app_data:
A pointer to application specific data provided by the application. Used to share data between this callback function and the application.
Param cur_status:
A pointer to the current ACK/NACK response for this byte. The application may change this to I2C_SLAVE_ACK or I2C_SLAVE_NACK. If cur_status is returned as an invalid value, the driver will implicitly NACK. By default the driver will implicitly ACK.
Starts an RTOS I2C slave driver instance. This must only be called by the tile that owns the driver instance. It must be called after starting the RTOS from an RTOS thread.
rtos_i2c_slave_init() must be called on this I2C slave driver instance prior to calling this.
Parameters:
i2c_slave_ctx – A pointer to the I2C slave driver instance to start.
app_data – A pointer to application specific data to pass to the callback functions.
start – The callback function that is called when the driver’s thread starts. This is optional and may be NULL.
rx – The callback function to receive data from the bus master.
tx_start – The callback function to transmit data to the bus master.
tx_done – The callback function that is notified when transmits are complete. This is optional and may be NULL.
rx_byte_check – The callback function to check received bytes individually.
write_addr_req – The callback function to alert an incoming write request
interrupt_core_id – The ID of the core on which to enable the I2C interrupt.
priority – The priority of the task that gets created by the driver to call the callback functions.
Initializes an RTOS I2C slave driver instance. This must only be called by the tile that owns the driver instance. It should be called before starting the RTOS, and must be called before calling rtos_i2c_slave_start().
Parameters:
i2c_slave_ctx – A pointer to the I2C slave driver instance to initialize.
io_core_mask – A bitmask representing the cores on which the low level I2C I/O thread created by the driver is allowed to run. Bit 0 is core 0, bit 1 is core 1, etc.
p_scl – The port containing SCL. This must be a 1-bit port and different than p_sda.
p_sda – The port containing SDA. This must be a 1-bit port and different than p_scl.
device_addr – The 7-bit address of the slave device.
RTOS_I2C_SLAVE_BUF_LEN
The maximum number of bytes that a the RTOS I2C slave driver can receive from a master in a single write transaction.
RTOS_I2C_SLAVE_CALLBACK_ATTR
This attribute must be specified on all RTOS I2C slave callback functions provided by the application.
RTOS_I2C_SLAVE_RX_BYTE_CHECK_CALLBACK_ATTR
This attribute must be specified on all RTOS I2C slave rtos_i2c_slave_rx_byte_check_cb_t provided by the application.
RTOS_I2C_SLAVE_WRITE_ADDR_REQUEST_CALLBACK_ATTR
This attribute must be specified on all RTOS I2C slave rtos_i2c_slave_write_addr_request_cb_t provided by the application.
structrtos_i2c_slave_struct
#include <rtos_i2c_slave.h>
Struct representing an RTOS I2C slave driver instance.
The members in this struct should not be accessed directly.
I2S RTOS Driver
This driver can be used to instantiate and control an I2S master or slave mode I/O interface on xcore in an RTOS application.
Initializes an RTOS I2S driver instance in master mode. This must only be called by the tile that owns the driver instance. It should be called before starting the RTOS, and must be called before calling rtos_i2s_start() or any of the core I2S driver functions with this instance.
Parameters:
i2s_ctx – A pointer to the I2S driver instance to initialize.
io_core_mask – A bitmask representing the cores on which the low level I2S I/O thread created by the driver is allowed to run. Bit 0 is core 0, bit 1 is core 1, etc.
p_dout – An array of data output ports.
num_out – The number of output data ports.
p_din – An array of data input ports.
num_in – The number of input data ports.
p_bclk – The bit clock output port.
p_lrclk – The word clock output port.
p_mclk – Input port which supplies the master clock.
bclk – A clock that will get configured for use with the bit clock.
Initializes an RTOS I2S driver instance in master mode but that uses an externally generated bit clock. This must only be called by the tile that owns the driver instance. It should be called before starting the RTOS, and must be called before calling rtos_i2s_start() or any of the core I2S driver functions with this instance.
Parameters:
i2s_ctx – A pointer to the I2S driver instance to initialize.
io_core_mask – A bitmask representing the cores on which the low level I2S I/O thread created by the driver is allowed to run. Bit 0 is core 0, bit 1 is core 1, etc.
p_dout – An array of data output ports.
num_out – The number of output data ports.
p_din – An array of data input ports.
num_in – The number of input data ports.
p_bclk – The bit clock output port.
p_lrclk – The word clock output port.
bclk – A clock that is configured externally to be used as the bit clock
I2S Slave Initialization API
The following structures and functions are used to initialize and start an I2S slave driver instance.
Initializes an RTOS I2S driver instance in slave mode. This must only be called by the tile that owns the driver instance. It should be called before starting the RTOS, and must be called before calling rtos_i2s_start() or any of the core I2S driver functions with this instance.
Parameters:
i2s_ctx – A pointer to the I2S driver instance to initialize.
io_core_mask – A bitmask representing the cores on which the low level I2S I/O thread created by the driver is allowed to run. Bit 0 is core 0, bit 1 is core 1, etc.
p_dout – An array of data output ports.
num_out – The number of output data ports.
p_din – An array of data input ports.
num_in – The number of input data ports.
p_bclk – The bit clock input port.
p_lrclk – The word clock input port.
bclk – A clock that will get configured for use with the bit clock.
Function pointer type for application provided RTOS I2S send filter callback functions.
These callback functions are called when an I2S driver instance needs output the next audio frame to its interface. By default, audio frames in the driver’s send buffer are output directly to its interface. However, this gives the application an opportunity to override this and provide filtering.
These functions must not block.
Param ctx:
A pointer to the associated I2C slave driver instance.
Param app_data:
A pointer to application specific data provided by the application. Used to share data between this callback function and the application.
Param i2s_frame:
A pointer to the buffer where the callback should write the next frame to send.
Param i2s_frame_size:
The number of samples that should be written to i2s_frame.
Param send_buf:
A pointer to the next frame in the driver’s send buffer. The callback should use this as the input to its filter.
Function pointer type for application provided RTOS I2S receive filter callback functions.
These callback functions are called when an I2S driver instance has received the next audio frame from its interface. By default, audio frames received from the driver’s interface are put directly into its receive buffer. However, this gives the application an opportunity to override this and provide filtering.
These functions must not block.
Param ctx:
A pointer to the associated I2C slave driver instance.
Param app_data:
A pointer to application specific data provided by the application. Used to share data between this callback function and the application.
Param i2s_frame:
A pointer to the buffer where the callback should read the next received frame from The callback should use this as the input to its filter.
Param i2s_frame_size:
The number of samples that should be read from i2s_frame.
Param receive_buf:
A pointer to the next frame in the driver’s send buffer. The callback should use this as the input to its filter.
Starts an RTOS I2S driver instance. This must only be called by the tile that owns the driver instance. It must be called after starting the RTOS from an RTOS thread, and must be called before any of the core I2S driver functions are called with this instance.
i2s_ctx – A pointer to the I2S driver instance to start.
mclk_bclk_ratio – The master clock to bit clock ratio. This may be computed by the helper function rtos_i2s_mclk_bclk_ratio(). This is only used if the I2S instance was initialized with rtos_i2s_master_init(). Otherwise it is ignored.
mode – The mode of the LR clock. See i2s_mode_t.
recv_buffer_size – The size in frames of the input buffer. Each frame is two samples (left and right channels) per input port. For example, a size of two here when num_in is three would create a buffer that holds up to 12 samples.
send_buffer_size – The size in frames of the output buffer. Each frame is two samples (left and right channels) per output port. For example, a size of two here when num_out is three would create a buffer that holds up to 12 samples. Frames transmitted by rtos_i2s_tx() are stored in this buffers before they are sent out to the I2S interface.
interrupt_core_id – The ID of the core on which to enable the I2S interrupt.
RTOS_I2S_APP_SEND_FILTER_CALLBACK_ATTR
This attribute must be specified on all RTOS I2S send filter callback functions provided by the application.
RTOS_I2S_APP_RECEIVE_FILTER_CALLBACK_ATTR
This attribute must be specified on all RTOS I2S receive filter callback functions provided by the application.
structrtos_i2s_struct
#include <rtos_i2s.h>
Struct representing an RTOS I2S driver instance.
The members in this struct should not be accessed directly.
This function will block until new frames are available.
Parameters:
ctx – A pointer to the I2S driver instance to use.
i2s_sample_buf – A buffer to copy the received sample frames into.
frame_count – The number of frames to receive from the buffer. This must be less than or equal to the size of the input buffer specified to rtos_i2s_start().
timeout – The amount of time to wait before the requested number of frames becomes available.
Returns:
The number of frames actually received into i2s_sample_buf.
The samples are stored into a buffer and are not necessarily sent out to the I2S interface before this function returns.
Parameters:
ctx – A pointer to the I2S driver instance to use.
i2s_sample_buf – A buffer containing the sample frames to transmit out to the I2S interface.
frame_count – The number of frames to transmit out from the buffer. This must be less than or equal to the size of the output buffer specified to rtos_i2s_start().
timeout – The amount of time to wait before there is enough space in the send buffer to accept the frames to be transmitted.
Returns:
The number of frames actually stored into the buffer.
The following functions may be used to share a I2S driver instance with other xcore tiles. Tiles that the
driver instance is shared with may call any of the core functions listed above.
Initializes an RTOS I2S driver instance on a client tile. This allows a tile that does not own the actual driver instance to use a driver instance on another tile. This will be called instead of on of the RTOS I2S init functions. The host tile that owns the actual instance must simultaneously call rtos_i2s_rpc_host_init().
Parameters:
i2s_ctx – A pointer to the I2S driver instance to initialize.
rpc_config – A pointer to an RPC config struct. This must have the same scope as i2s_ctx.
host_intertile_ctx – A pointer to the intertile driver instance to use for performing the communication between the client and host tiles. This must have the same scope as i2s_ctx.
Performs additional initialization on a I2S driver instance to allow client tiles to use the I2S driver instance. Each client tile that will use this instance must simultaneously call rtos_i2s_rpc_client_init().
Parameters:
i2s_ctx – A pointer to the I2S driver instance to share with clients.
rpc_config – A pointer to an RPC config struct. This must have the same scope as i2s_ctx.
client_intertile_ctx – An array of pointers to the intertile driver instances to use for performing the communication between the host tile and each client tile. This must have the same scope as i2s_ctx.
remote_client_count – The number of client tiles to share this driver instance with.
Configures the RPC for a I2S driver instance. This must be called by both the host tile and all client tiles.
On the client tiles this must be called after calling rtos_i2s_rpc_client_init(). After calling this, the client tile may immediately begin to call the core I2S functions on this driver instance. It does not need to wait for the host to call rtos_i2s_start().
i2s_ctx – A pointer to the I2S driver instance to configure the RPC for.
intertile_port – The port number on the intertile channel to use for transferring the RPC requests and responses for this driver instance. This port must not be shared by any other functions. The port must be the same for the host and all its clients.
host_task_priority – The priority to use for the task on the host tile that handles RPC requests from the clients.
Starts an RTOS mic array driver instance. This must only be called by the tile that owns the driver instance. It must be called after starting the RTOS from an RTOS thread, and must be called before any of the core mic array driver functions are called with this instance.
rtos_mic_array_init() must be called on this mic array driver instance prior to calling this.
Parameters:
mic_array_ctx – A pointer to the mic array driver instance to start.
buffer_size – The size in frames of the input buffer. Each frame is two samples (one for each microphone) plus one sample per reference channel. This must be at least MIC_ARRAY_CONFIG_SAMPLES_PER_FRAME. Samples are pulled out of this buffer by the application by calling rtos_mic_array_rx().
interrupt_core_id – The ID of the core on which to enable the mic array interrupt.
Initializes an RTOS mic array driver instance. This must only be called by the tile that owns the driver instance. It should be called before starting the RTOS, and must be called before calling rtos_mic_array_start() or any of the core mic array driver functions with this instance.
Parameters:
mic_array_ctx – A pointer to the mic array driver instance to initialize.
io_core_mask – A bitmask representing the cores on which the low level mic array I/O thread created by the driver is allowed to run. Bit 0 is core 0, bit 1 is core 1, etc.
format – Format of the output data
structrtos_mic_array_struct
#include <rtos_mic_array.h>
Struct representing an RTOS mic array driver instance.
The members in this struct should not be accessed directly.
Receives sample frames from the PDM mic array interface.
This function will block until new frames are available.
Parameters:
ctx – A pointer to the mic array driver instance to use.
sample_buf – A buffer to copy the received sample frames into.
frame_count – The number of frames to receive from the buffer. This must be less than or equal to the size of the buffer specified to rtos_mic_array_start() if in RTOS_MIC_ARRAY_SAMPLE_CHANNEL mode. This must be equal to MIC_ARRAY_CONFIG_SAMPLES_PER_FRAME if in RTOS_MIC_ARRAY_CHANNEL_SAMPLE mode.
timeout – The amount of time to wait before the requested number of frames becomes available.
Returns:
The number of frames actually received into sample_buf.
The following functions may be used to share a microphone array driver instance with other xcore tiles. Tiles that the
driver instance is shared with may call any of the core functions listed above.
Initializes an RTOS mic array driver instance on a client tile. This allows a tile that does not own the actual driver instance to use a driver instance on another tile. This will be called instead of rtos_mic_array_init(). The host tile that owns the actual instance must simultaneously call rtos_mic_array_rpc_host_init().
Parameters:
mic_array_ctx – A pointer to the mic array driver instance to initialize.
rpc_config – A pointer to an RPC config struct. This must have the same scope as mic_array_ctx.
host_intertile_ctx – A pointer to the intertile driver instance to use for performing the communication between the client and host tiles. This must have the same scope as mic_array_ctx.
Performs additional initialization on a mic array driver instance to allow client tiles to use the mic array driver instance. Each client tile that will use this instance must simultaneously call rtos_mic_array_rpc_client_init().
Parameters:
mic_array_ctx – A pointer to the mic array driver instance to share with clients.
rpc_config – A pointer to an RPC config struct. This must have the same scope as mic_array_ctx.
client_intertile_ctx – An array of pointers to the intertile driver instances to use for performing the communication between the host tile and each client tile. This must have the same scope as mic_array_ctx.
remote_client_count – The number of client tiles to share this driver instance with.
Configures the RPC for a mic array driver instance. This must be called by both the host tile and all client tiles.
On the client tiles this must be called after calling rtos_mic_array_rpc_client_init(). After calling this, the client tile may immediately begin to call the core mic array functions on this driver instance. It does not need to wait for the host to call rtos_mic_array_start().
mic_array_ctx – A pointer to the mic array driver instance to configure the RPC for.
intertile_port – The port number on the intertile channel to use for transferring the RPC requests and responses for this driver instance. This port must not be shared by any other functions. The port must be the same for the host and all its clients.
host_task_priority – The priority to use for the task on the host tile that handles RPC requests from the clients.
Starts an RTOS QSPI flash driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before any of the core QSPI flash driver functions are called with this instance.
rtos_qspi_flash_init() must be called on this QSPI flash driver instance prior to calling this.
Parameters:
ctx – A pointer to the QSPI flash driver instance to start.
priority – The priority of the task that gets created by the driver to handle the QSPI flash interface.
Sets the core affinity for a RTOS QSPI flash driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, and should be called before any of the core QSPI flash driver functions are called with this instance.
Since interrupts are disabled during the QSPI transaction on the op thread, a core mask is provided to allow users to avoid collisions with application ISRs.
rtos_qspi_flash_start() must be called on this QSPI flash driver instance prior to calling this.
Parameters:
ctx – A pointer to the QSPI flash driver instance to start.
op_core_mask – A bitmask representing the cores on which the QSPI I/O thread created by the driver is allowed to run. Bit 0 is core 0, bit 1 is core 1, etc.
Initializes an RTOS QSPI flash driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before calling rtos_qspi_flash_start() or any of the core QSPI flash driver functions with this instance.
This function will initialize a flash driver using lib_quadflash for all operations.
Parameters:
ctx – A pointer to the QSPI flash driver instance to initialize.
clock_block – The clock block to use for the qspi_io interface.
cs_port – The chip select port. MUST be a 1-bit port.
sclk_port – The SCLK port. MUST be a 1-bit port.
sio_port – The SIO port. MUST be a 4-bit port.
spec – A pointer to the flash part specification. This may be set to NULL to use the XTC default
Initializes an RTOS QSPI flash driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before calling rtos_qspi_flash_start() or any of the core QSPI flash driver functions with this instance.
This function will initialize a flash driver using lib_quadflash for erase and writes, and lib_qspi_fast_read for reads. If calibration fails the driver will enable lib_quadflash for reads and allow the application to decide what to do about the failed calibration. The status of the calibration can be checked at runtime by calling rtos_qspi_flash_calibration_valid_get().
Parameters:
ctx – A pointer to the QSPI flash driver instance to initialize.
clock_block – The clock block to use for the qspi_io interface.
cs_port – The chip select port. MUST be a 1-bit port.
sclk_port – The SCLK port. MUST be a 1-bit port.
sio_port – The SIO port. MUST be a 4-bit port.
spec – A pointer to the flash part specification. This may be set to NULL to use the XTC default
read_mode – The transfer mode to use for port reads. Invalid values will default to qspi_fast_flash_read_transfer_raw
read_divide – The divisor to use for QSPI SCLK.
calibration_pattern_addr – The address of the default calibration pattern. This driver requires the default calibration pattern supplied with lib_qspi_fast_read and does not support custom patterns.
RTOS_QSPI_FLASH_READ_CHUNK_SIZE
structrtos_qspi_flash_struct
#include <rtos_qspi_flash.h>
Struct representing an RTOS QSPI flash driver instance.
The members in this struct should not be accessed directly.
Obtains a lock for exclusive access to the QSPI flash. This allows a thread to perform a sequence of operations (such as read, modify, erase, write) without the risk of another thread issuing a command in the middle of the sequence and corrupting the data in the flash.
If only a single atomic operation needs to be performed, such as a read, it is not necessary to call this to obtain the lock first. Each individual operation obtains and releases the lock automatically so that they cannot run while another thread has the lock.
This is a lower level version of rtos_qspi_flash_read() that is safe to call from within ISRs. If a task currently own the flash lock, or if another core is actively doing a read with this function, then the read will not be performed and an error returned. It is up to the application to determine what it should do in this situation and to avoid a potential deadlock.
This function may only be called on the same tile as the underlying peripheral.
This function uses the lib_quadflash API to perform the read. It is up to the application to ensure that XCORE resources are properly configured.
Note
It is not possible to call this from a task that currently owns the flash lock taken with rtos_qspi_flash_lock(). In general it is not advisable to call this from an RTOS task unless the small amount of overhead time that is introduced by rtos_qspi_flash_read() is unacceptable.
Parameters:
ctx – A pointer to the QSPI flash driver instance to use.
data – Pointer to the buffer to save the read data to.
address – The byte address in the flash to begin reading at. Only bits 23:0 contain the address. Bits 31:24 are ignored.
len – The number of bytes to read and save to data.
Return values:
0 – if the flash was available and the read operation was performed.
-1 – if the flash was unavailable and the read could not be performed.
This is a lower level version of rtos_qspi_flash_read() that is safe to call from within ISRs. If a task currently own the flash lock, or if another core is actively doing a read with this function, then the read will not be performed and an error returned. It is up to the application to determine what it should do in this situation and to avoid a potential deadlock.
This function may only be called on the same tile as the underlying peripheral.
This function uses the lib_qspi_fast_read API to perform the read. It is up to the application to ensure that XCORE resources are properly configured.
Note
It is not possible to call this from a task that currently owns the flash lock taken with rtos_qspi_flash_lock(). In general it is not advisable to call this from an RTOS task unless the small amount of overhead time that is introduced by rtos_qspi_flash_read() is unacceptable.
Parameters:
ctx – A pointer to the QSPI flash driver instance to use.
data – Pointer to the buffer to save the read data to.
address – The byte address in the flash to begin reading at. Only bits 23:0 contain the address. Bits 31:24 are ignored.
len – The number of bytes to read and save to data.
Return values:
0 – if the flash was available and the read operation was performed.
-1 – if the flash was unavailable and the read could not be performed.
This is a lower level version of rtos_qspi_flash_read_mode() that is safe to call from within ISRs. If a task currently own the flash lock, or if another core is actively doing a read with this function, then the read will not be performed and an error returned. It is up to the application to determine what it should do in this situation and to avoid a potential deadlock.
This function may only be called on the same tile as the underlying peripheral.
This function uses the lib_qspi_fast_read API to perform the read. It is up to the application to ensure that XCORE resources are properly configured.
Note
It is not possible to call this from a task that currently owns the flash lock taken with rtos_qspi_flash_lock(). In general it is not advisable to call this from an RTOS task unless the small amount of overhead time that is introduced by rtos_qspi_flash_read_mode() is unacceptable.
Parameters:
ctx – A pointer to the QSPI flash driver instance to use.
data – Pointer to the buffer to save the read data to.
address – The byte address in the flash to begin reading at. Only bits 23:0 contain the address. Bits 31:24 are ignored.
len – The number of bytes to read and save to data.
mode – The transfer mode for this read operation data.
Return values:
0 – if the flash was available and the read operation was performed.
-1 – if the flash was unavailable and the read could not be performed.
This writes data to the QSPI flash. The standard page program command is sent and only SIO0 (MOSI) is used to send the address and data.
The driver handles sending the write enable command, as well as waiting for the write to complete.
This function may return before the write operation is complete, as the actual write operation is queued and executed by a thread created by the driver.
Note
this function does NOT erase the flash first. Erase operations must be explicitly requested by the application.
Parameters:
ctx – A pointer to the QSPI flash driver instance to use.
data – Pointer to the data to write to the flash.
address – The byte address in the flash to begin writing at. Only bits 23:0 contain the address. The byte in bits 31:24 is not sent.
This erases data from the QSPI flash. If the address range to erase spans multiple sectors, then all of these sectors will be erased by issuing multiple erase commands.
The driver handles sending the write enable command, as well as waiting for the write to complete.
This function may return before the write operation is complete, as the actual erase operation is queued and executed by a thread created by the driver.
Note
The smallest amount of data that can be erased is a 4k sector. This means that data outside the address range specified by address and len will be erased if the address range does not both begin and end at 4k sector boundaries.
Parameters:
ctx – A pointer to the QSPI flash driver instance to use.
address – The byte address to begin erasing. This does not need to begin at a sector boundary, but if it does not, note that the entire sector that contains this address will still be erased.
len – The minimum number of bytes to erase. If address + len - 1 does not correspond to the last address within a sector, note that the entire sector that contains this address will still be erased.
The following functions may be used to share a QSPI flash driver instance with other xcore tiles. Tiles that the
driver instance is shared with may call any of the core functions listed above.
Initializes an RTOS QSPI flash driver instance on a client tile. This allows a tile that does not own the actual driver instance to use a driver instance on another tile. This will be called instead of rtos_qspi_flash_init(). The host tile that owns the actual instance must simultaneously call rtos_qspi_flash_rpc_host_init().
Parameters:
qspi_flash_ctx – A pointer to the QSPI flash driver instance to initialize.
rpc_config – A pointer to an RPC config struct. This must have the same scope as qspi_flash_ctx.
host_intertile_ctx – A pointer to the intertile driver instance to use for performing the communication between the client and host tiles. This must have the same scope as qspi_flash_ctx.
Performs additional initialization on a QSPI flash driver instance to allow client tiles to use the QSPI flash driver instance. Each client tile that will use this instance must simultaneously call rtos_qspi_flash_rpc_client_init().
Parameters:
qspi_flash_ctx – A pointer to the QSPI flash driver instance to share with clients.
rpc_config – A pointer to an RPC config struct. This must have the same scope as qspi_flash_ctx.
client_intertile_ctx – An array of pointers to the intertile driver instances to use for performing the communication between the host tile and each client tile. This must have the same scope as qspi_flash_ctx.
remote_client_count – The number of client tiles to share this driver instance with.
Configures the RPC for a QSPI flash driver instance. This must be called by both the host tile and all client tiles.
On the client tiles this must be called after calling rtos_qspi_flash_rpc_client_init(). After calling this, the client tile may immediately begin to call the core QSPI flash functions on this driver instance. It does not need to wait for the host to call rtos_qspi_flash_start().
qspi_flash_ctx – A pointer to the QSPI flash driver instance to configure the RPC for.
intertile_port – The port number on the intertile channel to use for transferring the RPC requests and responses for this driver instance. This port must not be shared by any other functions. The port must be the same for the host and all its clients.
host_task_priority – The priority to use for the task on the host tile that handles RPC requests from the clients.
Starts an RTOS SPI master driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before any of the core SPI master driver functions are called with this instance.
rtos_spi_master_init() must be called on this SPI master driver instance prior to calling this.
Parameters:
spi_master_ctx – A pointer to the SPI master driver instance to start.
priority – The priority of the task that gets created by the driver to handle the SPI master interface.
Initializes an RTOS SPI master driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before calling rtos_spi_master_start() or any of the core SPI master driver functions with this instance.
Parameters:
bus_ctx – A pointer to the SPI master driver instance to initialize.
clock_block – The clock block to use for the SPI master interface.
cs_port – The SPI interface’s chip select port. This may be a multi-bit port.
sclk_port – The SPI interface’s SCLK port. Must be a 1-bit port.
mosi_port – The SPI interface’s MOSI port. Must be a 1-bit port.
miso_port – The SPI interface’s MISO port. Must be a 1-bit port.
Initialize a SPI device. Multiple SPI devices may be initialized per RTOS SPI master driver instance. Each must be on a unique pin of the interface’s chip select port. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before calling rtos_spi_master_start() or any of the core SPI master driver functions with this instance.
Parameters:
dev_ctx – A pointer to the SPI device instance to initialize.
bus_ctx – A pointer to the SPI master driver instance to attach the device to.
cs_pin – The bit number of the chip select port that is connected to the device’s chip select pin.
cpol – The clock polarity required by the device.
cpha – The clock phase required by the device.
source_clock – The source clock to derive SCLK from. See spi_master_source_clock_t.
clock_divisor – The value to divide the source clock by. The frequency of SCLK will be set to:
(F_src) / (4 * clock_divisor) when clock_divisor > 0
(F_src) / (2) when clock_divisor = 0 Where F_src is the frequency of the source clock.
miso_sample_delay – When to sample MISO. See spi_master_sample_delay_t.
miso_pad_delay – The number of core clock cycles to delay sampling the MISO pad during a transaction. This allows for more fine grained adjustment of sampling time. The value may be between 0 and 5.
cs_to_clk_delay_ticks – The minimum number of reference clock ticks between assertion of chip select and the first clock edge.
clk_to_cs_delay_ticks – The minimum number of reference clock ticks between the last clock edge and de-assertion of chip select.
cs_to_cs_delay_ticks – The minimum number of reference clock ticks between transactions, which is between de-assertion of chip select and the end of one transaction, and its re-assertion at the beginning of the next.
structrtos_spi_master_struct
#include <rtos_spi_master.h>
Struct representing an RTOS SPI master driver instance.
The members in this struct should not be accessed directly.
structrtos_spi_master_device_struct
#include <rtos_spi_master.h>
Struct representing an RTOS SPI device instance.
The members in this struct should not be accessed directly.
Starts a transaction with the specified SPI device on a SPI bus. This leaves chip select asserted.
Note: When this is called, the servicer thread will be locked to the core that it executed on until rtos_spi_master_transaction_end() is called. This is because the underlying I/O software utilized fast mode and high priority.
Transfers data to and from the specified SPI device on a SPI bus. The transaction must already have been started by calling rtos_spi_master_transaction_start() on the same device instance. This may be called multiple times during a single transaction.
This function may return before the transfer is complete when data_in is NULL, as the actual transfer operation is queued and executed by a thread created by the driver.
Parameters:
ctx – A pointer to the SPI device instance.
data_out – Pointer to the data to transfer to the device. This may be NULL if there is no data to send.
data_in – Pointer to the buffer to save the received data to. This may be NULL if the received data is not needed.
len – The number of bytes to transfer in each direction. This number of bytes must be available in both the data_out and data_in buffers if they are not NULL.
If there is a minimum amount of idle time that is required by the device between transfers within a single transaction, then this may be called between each transfer where a delay is required.
This function will return immediately. If the call for the next transfer happens before the minimum time specified has elapsed, the delay will occur then before the transfer begins.
Note
This must be called during a transaction, otherwise the behavior is unspecified.
Note
Technically the next transfer will occur no earlier than delay_ticks after this function is called, so this should be called immediately following a transfer, rather than immediately before the next.
Parameters:
ctx – A pointer to the SPI device instance.
delay_ticks – The number of reference clock ticks to delay.
The following functions may be used to share a SPI master driver instance with other xcore tiles. Tiles that the
driver instance is shared with may call any of the core functions listed above.
Initializes an RTOS SPI master driver instance on a client tile, as well as any number of SPI device instances. This allows a tile that does not own the actual driver instance to use a driver instance on another tile. This will be called instead of rtos_spi_master_init() and rtos_spi_master_device_init(). The host tile that owns the actual instances must simultaneously call rtos_spi_master_rpc_host_init().
Parameters:
spi_master_ctx – A pointer to the SPI master driver instance to initialize.
spi_device_ctx – An array of pointers to SPI device instances to initialize.
spi_device_count – The number of SPI device instances to initialize.
rpc_config – A pointer to an RPC config struct. This must have the same scope as spi_master_ctx.
host_intertile_ctx – A pointer to the intertile driver instance to use for performing the communication between the client and host tiles. This must have the same scope as spi_master_ctx.
Performs additional initialization on a SPI master driver instance to allow client tiles to use the SPI master driver instance. Each client tile that will use this instance must simultaneously call rtos_spi_master_rpc_client_init().
Parameters:
spi_master_ctx – A pointer to the SPI master driver instance to share with clients.
spi_device_ctx – An array of pointers to SPI device instances to share with clients.
spi_device_count – The number of SPI device instances to share.
rpc_config – A pointer to an RPC config struct. This must have the same scope as spi_master_ctx.
client_intertile_ctx – An array of pointers to the intertile driver instances to use for performing the communication between the host tile and each client tile. This must have the same scope as spi_master_ctx.
remote_client_count – The number of client tiles to share this driver instance with.
Configures the RPC for a SPI master driver instance. This must be called by both the host tile and all client tiles.
On the client tiles this must be called after calling rtos_spi_master_rpc_client_init(). After calling this, the client tile may immediately begin to call the core SPI master functions on this driver instance. It does not need to wait for the host to call rtos_spi_master_start().
spi_master_ctx – A pointer to the SPI master driver instance to configure the RPC for.
intertile_port – The port number on the intertile channel to use for transferring the RPC requests and responses for this driver instance. This port must not be shared by any other functions. The port must be the same for the host and all its clients.
host_task_priority – The priority to use for the task on the host tile that handles RPC requests from the clients.
Function pointer type for application provided RTOS SPI slave start callback functions.
These callback functions are optionally called by a SPI slave driver’s thread when it is first started. This gives the application a chance to perform startup initialization from within the driver’s thread. It is a good place for the first call to spi_slave_xfer_prepare().
Param ctx:
A pointer to the associated SPI slave driver instance.
Param app_data:
A pointer to application specific data provided by the application. Used to share data between this callback function and the application.
Function pointer type for application provided RTOS SPI slave transfer done callback functions.
These callback functions are optionally called when a SPI slave driver instance is done transferring data with a master device.
An application can use this to be notified immediately when a transfer has completed. It can then call spi_slave_xfer_complete() with a timeout of 0 from within this callback to get the transfer results.
Param ctx:
A pointer to the associated SPI slave driver instance.
Param app_data:
A pointer to application specific data provided by the application. Used to share data between this callback function and the application.
Prepares an RTOS SPI slave driver instance with buffers for subsequent transfers. Before this is called for the first time, any transfers initiated by a master device with result in all received data over MOSI being dropped, and all data sent over MISO being zeros.
This only needs to be called when the buffers need to be changed. If all transfers will use the same buffers, then this only needs to be called once during initialization.
If the application has not processed the previous transaction, the buffers will be held, and default buffers set by spi_slave_xfer_prepare_default_buffers() will be used if a new transaction starts.
Parameters:
ctx – A pointer to the SPI slave driver instance to use.
rx_buf – The buffer to receive data into for any subsequent transfers.
rx_buf_ – The length in bytes of rx_buf. If the master transfers more than this during a single transfer, then the bytes that do not fit within rx_buf will be lost.
tx_buf – The buffer to send data from for any subsequent transfers.
tx_buf_len – The length in bytes of tx_buf. If the master transfers more than this during a single transfer, zeros will be sent following the last byte tx_buf.
Prepares an RTOS SPI slave driver instance with default buffers for subsequent transfers. Before this is called for the first time, any transfers initiated by a master device with result in all received data over MOSI being dropped, and all data sent over MISO being zeros.
This only needs to be called when the buffers need to be changed.
The default buffer will be used in the event that the application has not yet processed the previous transfer. This enables the application to have a default buffer to implement a sort of NACK over SPI in the event that the device was busy and had not yet finished handling the previous transaction before a new one started.
Parameters:
ctx – A pointer to the SPI slave driver instance to use.
rx_buf – The buffer to receive data into for any subsequent transfers.
rx_buf_ – The length in bytes of rx_buf. If the master transfers more than this during a single transfer, then the bytes that do not fit within rx_buf will be lost.
tx_buf – The buffer to send data from for any subsequent transfers.
tx_buf_len – The length in bytes of tx_buf. If the master transfers more than this during a single transfer, zeros will be sent following the last byte tx_buf.
Waits for a SPI transfer to complete. Returns either when the timeout is reached, or when a transfer completes, whichever comes first. If a transfer does complete, then the buffers and the number of bytes read from or written to them are returned via the parameters.
Note
The duration of this callback will effect the minimum duration between SPI transactions
Parameters:
ctx – A pointer to the SPI slave driver instance to use.
rx_buf – The receive buffer used for the completed transfer. This is set by the function upon completion of a transfer.
rx_len – The number of bytes written to rx_buf. This is set by the function upon completion of a transfer.
tx_buf – The transmit buffer used for the completed transfer. This is set by the function upon completion of a transfer.
tx_len – The number of bytes sent from tx_buf. This is set by the function upon completion of a transfer.
timeout – The number of RTOS ticks to wait before the next transfer is complete. When called from within the “xfer_done” callback, this should be 0.
Return values:
0 – if a transfer completed. All buffers and lengths are set in this case.
-1 – if no transfer completed before the timeout expired. No buffers or lengths are returned in this case.
Sets the driver to use callbacks for all default transactions. This will result in transfers done with the default buffer generating callbacks to the application to xfer_done. This will require default buffer transaction items to be processed with spi_slave_xfer_complete()
Note
This is the default setting
Parameters:
ctx – A pointer to the SPI slave driver instance to use.
Sets the driver to drop all default transactions. This will result in transfers done with the default buffer not generating callbacks to the application to xfer_done. This will also stop default buffer transaction items from being required to be processed with spi_slave_xfer_complete()
Parameters:
ctx – A pointer to the SPI slave driver instance to use.
Starts an RTOS SPI slave driver instance. This must only be called by the tile that owns the driver instance. It must be called after starting the RTOS from an RTOS thread.
rtos_spi_slave_init() must be called on this SPI slave driver instance prior to calling this.
Parameters:
spi_slave_ctx – A pointer to the SPI slave driver instance to start.
app_data – A pointer to application specific data to pass to the callback functions.
start – The callback function that is called when the driver’s thread starts. This is optional and may be NULL.
xfer_done – The callback function that is notified when transfers are complete. This is optional and may be NULL.
interrupt_core_id – The ID of the core on which to enable the SPI interrupt. This core should not be shared with threads that disable interrupts for long periods of time, nor enable other interrupts.
priority – The priority of the task that gets created by the driver to call the callback functions. If both callback functions are NULL, then this is unused.
Initializes an RTOS SPI slave driver instance. This must only be called by the tile that owns the driver instance. It should be called before starting the RTOS, and must be called before calling rtos_spi_slave_start().
For timing parameters and maximum clock rate, refer to the underlying HIL IO API.
Parameters:
spi_slave_ctx – A pointer to the SPI slave driver instance to initialize.
io_core_mask – A bitmask representing the cores on which the low level SPI I/O thread created by the driver is allowed to run. Bit 0 is core 0, bit 1 is core 1, etc.
clock_block – The clock block to use for the SPI slave.
cpol – The clock polarity to use.
cpha – The clock phase to use.
p_sclk – The SPI slave’s SCLK port. Must be a 1-bit port.
p_mosi – The SPI slave’s MOSI port. Must be a 1-bit port.
p_miso – The SPI slave’s MISO port. Must be a 1-bit port.
p_cs – The SPI slave’s CS port. Must be a 1-bit port.
RTOS_SPI_SLAVE_CALLBACK_ATTR
This attribute must be specified on all RTOS SPI slave callback functions provided by the application.
HIL_IO_SPI_SLAVE_HIGH_PRIO
Set SPI Slave thread to high priority
HIL_IO_SPI_SLAVE_FAST_MODE
Set SPI Slave thread to run in fast mode
structxfer_done_queue_item
#include <rtos_spi_slave.h>
Internally used struct representing an received data packet.
The members in this struct should not be accessed directly.
structrtos_spi_slave_struct
#include <rtos_spi_slave.h>
Struct representing an RTOS SPI slave driver instance.
The members in this struct should not be accessed directly.
Writes data to an initialized and started UART instance. Unlike the UART rx, an xcore logical core is not reserved. The UART transmission is a function call and the the function blocks until the stop bit of the last byte to be transmittted has completed. Interrupts are masked during this time to avoid stretching of the waveform. Consequently, the tx consumes cycles from the caller thread.
Parameters:
ctx – A pointer to the UART Tx driver instance to use.
Initialises an RTOS UART tx driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before calling rtos_uart_tx_start() or any of the core UART tx driver functions with this instance.
Parameters:
ctx – A pointer to the UART tx driver instance to initialise.
tx_port – The port containing the transmit pin
baud_rate – The baud rate of the UART in bits per second.
num_data_bits – The number of data bits per frame sent.
parity – The type of parity used. See uart_parity_t above.
stop_bits – The number of stop bits asserted at the of the frame.
tmr – The resource id of the timer to be used by the UART tx.
Starts an RTOS UART tx driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before any of the core UART tx driver functions are called with this instance.
rtos_uart_tx_init() must be called on this UART tx driver instance prior to calling this.
Parameters:
ctx – A pointer to the UART tx driver instance to start.
structrtos_uart_tx_struct
#include <rtos_uart_tx.h>
Struct representing an RTOS UART tx driver instance.
The members in this struct should not be accessed directly.
The following functions may be used to share a UART Tx driver instance with other xCORE tiles. Tiles that the
driver instance is shared with may call any of the core functions listed above.
Initializes an RTOS UART tx driver instance on a client tile. This allows a tile that does not own the actual driver instance to use a driver instance on another tile. This will be called instead of rtos_uart_tx_init(). The host tile that owns the actual instance must simultaneously call rtos_uart_tx_rpc_host_init().
Parameters:
uart_tx_ctx – A pointer to the UART tx driver instance to initialize.
rpc_config – A pointer to an RPC config struct. This must have the same scope as uart_tx_ctx.
host_intertile_ctx – A pointer to the intertile driver instance to use for performing the communication between the client and host tiles. This must have the same scope as uart_tx_ctx.
Performs additional initialization on an UART tx driver instance to allow client tiles to use the UART tx driver instance. Each client tile that will use this instance must simultaneously call rtos_uart_tx_rpc_client_init().
Parameters:
uart_tx_ctx – A pointer to the UART tx driver instance to share with clients.
rpc_config – A pointer to an RPC config struct. This must have the same scope as uart_tx_ctx.
client_intertile_ctx – An array of pointers to the intertile driver instances to use for performing the communication between the host tile and each client tile. This must have the same scope as uart_tx_ctx.
remote_client_count – The number of client tiles to share this driver instance with.
Configures the RPC for an UART tx driver instance. This must be called by both the host tile and all client tiles.
On the client tiles this must be called after calling rtos_uart_tx_rpc_client_init(). After calling this, the client tile may immediately begin to call the core UART tx functions on this driver instance. It does not need to wait for the host to call rtos_uart_tx_start().
uart_tx_ctx – A pointer to the UART tx driver instance to configure the RPC for.
intertile_port – The port number on the intertile channel to use for transferring the RPC requests and responses for this driver instance. This port must not be shared by any other functions. The port must be the same for the host and all its clients.
host_task_priority – The priority to use for the task on the host tile that handles RPC requests from the clients.
Function pointer type for application provided RTOS UART rx start callback functions.
This callback function is optionally (may be NULL) called by an UART rx driver’s thread when it is first started. This gives the application a chance to perform startup initialization from within the driver’s thread.
Param ctx:
A pointer to the associated UART rx driver instance.
Function pointer type for application provided RTOS UART rx receive callback function.
This callback functions are called when an UART rx driver instance has received data to a specified depth. Please use the xStreamBufferReceive(rtos_uart_rx_ctx->isr_byte_buffer, … to read the bytes.
Param ctx:
A pointer to the associated UART rx driver instance.
Function pointer type for application provided RTOS UART rx error callback functions.
This callback function is optionally (may be NULL_ called when an UART rx driver instance experiences an error in reception. These error types are defined in uart.h of the underlying HIL driver but can be of the following types for the RTOS rx: UART_START_BIT_ERROR, UART_PARITY_ERROR, UART_FRAMING_ERROR, UART_OVERRUN_ERROR.
Param ctx:
A pointer to the associated UART rx driver instance.
Param err_flags:
An 8b word containing error flags set during reception of last frame. See rtos_uart_rx.h for the bit field definitions.
Initializes an RTOS UART rx driver instance. This must only be called by the tile that owns the driver instance. It should be called before starting the RTOS, and must be called before calling rtos_uart_rx_start(). Note that UART rx requires a whole logical core for the underlying HIL UART Rx instance.
Parameters:
uart_rx_ctx – A pointer to the UART rx driver instance to initialize.
io_core_mask – A bitmask representing the cores on which the low UART Rx thread created by the driver is allowed to run. Bit 0 is core 0, bit 1 is core 1, etc.
rx_port – The port containing the receive pin
baud_rate – The baud rate of the UART in bits per second.
data_bits – The number of data bits per frame sent.
parity – The type of parity used. See uart_parity_t above.
stop_bits – The number of stop bits asserted at the of the frame.
tmr – The resource id of the timer to be used by the UART Rx.
Starts an RTOS UART rx driver instance. This must only be called by the tile that owns the driver instance. It must be called after starting the RTOS and from an RTOS thread.
rtos_uart_rx_init() must be called on this UART rx driver instance prior to calling this.
Parameters:
uart_rx_ctx – A pointer to the UART rx driver instance to start.
app_data – A pointer to application specific data to pass to the callback functions available in rtos_uart_rx_struct.
start – The callback function that is called when the driver’s thread starts. This is optional and may be NULL.
rx_complete – The callback function to indicate data received by the UART.
error – The callback function called when a reception error has occured.
interrupt_core_id – The ID of the core on which to enable the UART rx interrupt.
priority – The priority of the task that gets created by the driver to call the callback functions.
app_rx_buff_size – The size in bytes of the RTOS xstreambuffer used to buffer received words for the application.
UR_COMPLETE_CB_CODE
The callback code bit positions available for RTOS UART Rx.
UR_STARTED_CB_CODE
UR_START_BIT_ERR_CB_CODE
UR_PARITY_ERR_CB_CODE
UR_FRAMING_ERR_CB_CODE
UR_OVERRUN_ERR_CB_CODE
UR_COMPLETE_CB_FLAG
The callback code flag masks available for RTOS UART Rx.
UR_STARTED_CB_FLAG
UR_START_BIT_ERR_CB_FLAG
UR_PARITY_ERR_CB_FLAG
UR_FRAMING_ERR_CB_FLAG
UR_OVERRUN_ERR_CB_FLAG
RX_ERROR_FLAGS
RX_ALL_FLAGS
RTOS_UART_RX_BUF_LEN
The size of the byte buffer between the ISR and the appthread. It needs to be able to hold sufficient bytes received until the app_thread is able to service it. This is not the same as app_byte_buffer_size which can be of any size, specified by the user at device start. At 1Mbps we get a byte every 10us so 64B allows 640us for the app thread to respond. Note buffer is size n+1 as required by lib_uart.
RTOS_UART_RX_CALLBACK_ATTR
This attribute must be specified on all RTOS UART rx callback functions provided by the application to allow compiler stack calculation.
RTOS_UART_RX_CALL_ATTR
This attribute must be specified on all RTOS UART functions provided by the application to allow compiler stack calculation.
structrtos_uart_rx_struct
#include <rtos_uart_rx.h>
Struct representing an RTOS UART rx driver instance.
The members in this struct should not be accessed directly.
This driver can be used to instantiate and control a USB device interface on xcore in an RTOS application.
Unlike most other xcore I/O interface RTOS drivers, only a single USB driver instance may be started. It also does not require an initialization step prior to starting the driver. This is due to an implementation detail in lib_xud, which is what the RTOS USB driver uses at its core.
Requests a transfer on a USB endpoint. This function returns immediately. When the transfer is complete, the application’s ISR callback provided to rtos_usb_start() will be called.
Parameters:
ctx – A pointer to the USB driver instance to use.
endpoint_addr – The address of the endpoint to perform the transfer on.
buffer – A pointer to the buffer to transfer data into for OUT endpoints, or from for IN endpoints. For OUT endpoint, the buffer needs an additional +4 bytes of space, this additional data should not be reflected in the len parameter.
len – The maximum number of bytes to receive for OUT endpoints, or the actual number of bytes to send for IN endpoints.
is_setup – To be set when preparing for the transfer of a setup packet.
Return values:
XUD_RES_OKAY – if the transfer was requested successfully.
XUD_RES_RST – if the transfer was not requested and the USB bus needs to be reset. In this case, the application should reset the USB bus.
This function will complete a reset on an endpoint. The address of the endpoint to reset must be provided, and may be either direction (IN or OUT) endpoint. If there is an associated endpoint of the opposite direction, however, it will also be reset.
The return value should be inspected to find the new bus-speed.
Parameters:
endpoint_addr – IN or OUT endpoint address to reset.
Return values:
XUD_SPEED_HS – the host has accepted that this device can execute at high speed.
XUD_SPEED_FS – the device is running at full speed.
Sets the USB device’s bus address. This function must be called after a setDeviceAddress request is made by the host, and after the ZLP status is sent.
Parameters:
ctx – A pointer to the USB driver instance to use.
Stalls a USB endpoint. The stall is cleared automatically when a setup packet is received on the endpoint. Otherwise it can be cleared manually with rtos_usb_endpoint_stall_clear().
Parameters:
ctx – A pointer to the USB driver instance to use.
endpoint_addr – The address of the endpoint to stall.
Starts the USB driver instance’s low level USB I/O thread and enables its interrupts on the requested core. This must only be called by the tile that owns the driver instance. It must be called after starting the RTOS from an RTOS thread.
rtos_usb_init() must be called on this USB driver instance prior to calling this.
Parameters:
ctx – A pointer to the USB driver instance to start.
endpoint_count –
The number of endpoints that will be used by the application. A single endpoint here includes both its IN and OUT endpoints. For example, if the application uses EP0_IN, EP0_OUT, EP1_IN, EP2_IN, EP2_OUT, EP3_OUT, then the endpoint count specified here should be 4 (endpoint 0 through endpoint 3) regardless of the lack of EP1_OUT and EP3_IN. If these two endpoints were used, the count would still be 4.
If for whatever reason, the application needs to use a particular endpoint number, say only EP6 in addition to EP0, then the count here needs to be 7, even though endpoints 1 through 5 are unused. All unused endpoints must be marked as disabled in the two endpoint type lists
endpoint_out_type and endpoint_in_type.
endpoint_out_type – A list of the endpoint types for each output endpoint. Index 0 represents the type for EP0_OUT, and so on. See XUD_EpType in lib_xud. If the endpoint is unused, it must be set to XUD_EPTYPE_DIS.
endpoint_in_type – A list of the endpoint types for each input endpoint. Index 0 represents the type for EP0_IN, and so on. See XUD_EpType in lib_xud. If the endpoint is unused, it must be set to XUD_EPTYPE_DIS.
speed – The speed at which the bus should operate. Either XUD_SPEED_FS or XUD_SPEED_HS. See XUD_BusSpeed_t in lib_xud.
power_source – The source of the device’s power. Either bus powered (XUD_PWR_BUS) or self powered (XUD_PWR_SELF). See XUD_PwrConfig in lib_xud.
interrupt_core_id – The ID of the core on which to enable the USB interrupts.
sof_interrupt_core_id – The ID of the core on which to enable the SOF interrupt. Set to < 0 to disable the SoF interrupt if it is not needed.
Initializes an RTOS USB driver instance. This must only be called by the tile that owns the driver instance. It should be called prior to starting the RTOS, and must be called before any of the core USB driver functions are called with this instance.
This will create an RTOS thread that runs lib_xud’s main loop. This thread is created with the highest priority and with preemption disabled.
Note
Due to implementation details of lib_xud, it is only possible to have one USB instance per application. Functionally this is not an issue, as no xcore chips have more than one USB interface.
Note
If using the Tiny USB stack, then this function should not be called directly by the application. The xcore device port for Tiny USB takes care of calling this, as well as all other USB driver functions.
Parameters:
ctx – A pointer to the USB driver instance to start.
io_core_mask – A bitmask representing the cores on which the low level USB I/O thread created by the driver is allowed to run. Bit 0 is core 0, bit 1 is core 1, etc.
isr_cb – The callback function for the driver to call when transfers are completed.
isr_app_data – A pointer to application specific data to pass to the application’s ISR callback function isr_cb.
This function may be called to wait for a transfer on a particular endpoint to complete. This requires that the USB instance was initialized with rtos_usb_simple_init().
Parameters:
ctx – A pointer to the USB driver instance to use.
endpoint_addr – The address of the endpoint to wait for.
len – The actual number of bytes transferred. For IN endpoints, this will be the same as the length requested by rtos_usb_endpoint_transfer_start(). For OUT endpoints, it may be less.
timeout – The maximum amount of time to wait for the transfer to complete before returning.
Return values:
XUD_RES_OKAY – if the transfer was completed successfully.
XUD_RES_RST – if the transfer was not able to complete and the USB bus needs to be reset. In this case, the application should reset the USB bus.
XUD_RES_ERR – if there was an unexpected error transferring the data.
Initializes an RTOS USB driver instance. This must only be called by the tile that owns the driver instance. It should be called prior to starting the RTOS, and must be called before any of the core USB driver functions are called with this instance.
This initialization function may be used instead of rtos_usb_init() if the application is not using a USB stack. This allows application threads to wait for transfers to complete with the rtos_usb_simple_transfer_complete() function. The application cannot provide its own ISR callback when initialized with this function. This provides a similar programming interface as a traditional bare metal xcore application using lib_xud.
This will create an RTOS thread that runs lib_xud’s main loop. This thread is created with the highest priority and with preemption disabled.
Note
Due to implementation details of lib_xud, it is only possible to have one USB instance per application. Functionally this is not an issue, as no xcore chips have more than one USB interface.
Parameters:
ctx – A pointer to the USB driver instance to start.
io_core_mask – A bitmask representing the cores on which the low level USB I/O thread created by the driver is allowed to run. Bit 0 is core 0, bit 1 is core 1, etc.
RTOS_USB_ENDPOINT_COUNT_MAX
The maximum number of USB endpoint numbers supported by the RTOS USB driver.
RTOS_USB_ISR_CALLBACK_ATTR
This attribute must be specified on the RTOS USB interrupt callback function provided by the application.
structrtos_usb_ep_xfer_info_t
#include <rtos_usb.h>
Struct to hold USB transfer state data per endpoint, used as the argument to the ISR.
The members in this struct should not be accessed directly.
structrtos_usb_struct
#include <rtos_usb.h>
Struct representing an RTOS USB driver instance.
The members in this struct should not be accessed directly.
This driver can be used to instantiate an xscope-based trace module in an RTOS
application. The trace module currently supports both a demonstrative ASCII-mode
and Percepio’s Tracealzyer on FreeRTOS. Both modes are dependent on
RTOS-specific hooks/macros to handle the majority of RTOS event recording and
integration.
For general usage of the FreeRTOS trace functionality please refer to FreeRTOS’
documentation here:
RTOS Trace Macros
For basic information on printf debugging using xscope please refer to the tools
guide here:
XSCOPE debugging
The trace driver supports Percepio’s Tracealyzer, a feature rich tool for
working with trace files. This implementation supports Tracealyzer’s
streaming mode; currently, snapshot mode is not supported. The current
underlying trace recording implementation interfaces with the
xscope_core_bytes API function (on Probe 0).
To select Tracealyzer as the trace module’s event recorder, the following must
be set. This can be applied at the CMake project level:
In addition to the configuration steps outlined above, Percepio’s Tracealyzer
streaming mode needs additional function calls to start recording trace data. In
the most basic use-case, the following functions should be called on the XCORE
tile that is to record trace data:
xTraceInitialize();xTraceEnable(TRC_START);
Note
xTraceInitialize must be called before any RTOS interaction
(before any traced objects are being interacted with). It is advisable to
call it as soon as possible in the application.
The Percepio’s Tracealzyer C-unit outputs to a stream-able file format called
Percepio Streaming Format (PSF). The xscope2psf utility aids in the extraction
of the PSF file from the underlying xscope communication (making it readily
available on the host’s filesystem). This tool can be configured to read from a
VCD (value change dump) file that is generated when specifying the xgdb option
–xscope-port <ip:port>, or it can be configured as an xscope-endpoint when
specifying the –xscope-port <ip:port> option. Both options can be processed
by the Tracealyzer graphical tool either as a post processing step or live.
Note
xscope2psf currently resides in a Tracealyzer example application here:
example.
This is likely to change in the future. Refer to either the README or the
application’s help documentation for usage details.
Note
Currently, the only supported PSF Streaming target connection type is
File System. Ensure this connection type is specified under Tracealyzer’s
Recording Settings.
For general usage of Tracealyzer please refer to the Percepio’s documentation here:
Manual
The trace driver supports a basic ASCII mode that is primarily meant as an
example for expanding support to other tracing tools/frameworks. In this mode,
only the following FreeRTOS trace hooks are supported:
traceTASK_SWITCHED_IN
traceTASK_SWITCHED_OUT
This implementation will produce xscope logs for the RTOS task switching. The
underlying xscope API xscope_core_bytes is used for communicating this
information.
To select ASCII mode as the trace module’s event recorder, the following must
be set. This can be applied at the CMake project level:
#define USE_TRACE_MODE TRACE_MODE_XSCOPE_ASCII
Note
xcore_trace.h contains the definition for these modes.
To begin capturing ASCII mode traces, run xgdb with the –xscope-file
option. Task switching events will be recorded to the specified VCD (value
change dump) file.
Starts an RTOS clock control driver instance. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before any of the core clock control driver functions are called with this instance.
rtos_clock_control_init() must be called on this clock control driver instance prior to calling this.
Parameters:
ctx – A pointer to the clock control driver instance to start.
Initializes an RTOS clock control driver instance. There should only be one per tile. This must only be called by the tile that owns the driver instance. It may be called either before or after starting the RTOS, but must be called before calling rtos_clock_control_start() or any of the core clock control driver functions with this instance.
Parameters:
ctx – A pointer to the GPIO driver instance to initialize.
structrtos_clock_control_struct
#include <rtos_clock_control.h>
Struct representing an RTOS clock control driver instance.
The members in this struct should not be accessed directly.
Sets the tile clock PLL control register value on the tile that owns this driver instance. The value set is calculated from the divider stage 1, multiplier stage, and divider stage 2 values provided.
VCO freq = fosc * (F + 1) / (2 * (R + 1)) VCO must be between 260MHz and 1.3GHz for XS2 Core freq = VCO / (OD + 1)
Refer to the xcore Clock Frequency Control document for more details.
Note: This function will not reset the chip and wait for the PLL to settle before re-enabling the chip to allow for large frequency jumps. This will cause a delay during settings.
Note: It is up to the application to ensure that it is safe to change the clock.
Parameters:
ctx – A pointer to the clock control driver instance to use.
Gets the divider stage 1, multiplier stage, and divider stage 2 values from the tile clock PLL control register values on the tile that owns this driver instance.
Parameters:
ctx – A pointer to the clock control driver instance to use.
pre_div – A pointer to be populated with the value of R
mul – A pointer to be populated with the value of F
post_div – A pointer to be populated with the value of OD
Gets the local lock for clock control on the tile that owns this driver instance. This is intended for applications to use to prevent clock changes around critical sections.
Parameters:
ctx – A pointer to the clock control driver instance to use.
The following functions may be used to share a GPIO driver instance with other xcore tiles. Tiles that the
driver instance is shared with may call any of the core functions listed above.
Initializes an RTOS clock control driver instance on a client tile. This allows a tile that does not own the actual driver instance to use a driver instance on another tile. This will be called instead of rtos_clock_control_init(). The host tile that owns the actual instance must simultaneously call rtos_clock_control_rpc_host_init().
Parameters:
cc_ctx – A pointer to the clock control driver instance to initialize.
rpc_config – A pointer to an RPC config struct. This must have the same scope as cc_ctx.
host_intertile_ctx – A pointer to the intertile driver instance to use for performing the communication between the client and host tiles. This must have the same scope as cc_ctx.
Performs additional initialization on a clock control driver instance to allow client tiles to use the clock control driver instance. Each client tile that will use this instance must simultaneously call rtos_clock_control_rpc_client_init().
Parameters:
cc_ctx – A pointer to the clock control driver instance to share with clients.
rpc_config – A pointer to an RPC config struct. This must have the same scope as cc_ctx.
client_intertile_ctx – An array of pointers to the intertile driver instances to use for performing the communication between the host tile and each client tile. This must have the same scope as cc_ctx.
remote_client_count – The number of client tiles to share this driver instance with.
Configures the RPC for a clock control driver instance. This must be called by both the host tile and all client tiles.
On the client tiles this must be called after calling rtos_clock_control_rpc_client_init(). After calling this, the client tile may immediately begin to call the core clock control functions on this driver instance. It does not need to wait for the host to call rtos_clock_control_start().
cc_ctx – A pointer to the clock control driver instance to configure the RPC for.
intertile_port – The port number on the intertile channel to use for transferring the RPC requests and responses for this driver instance. This port must not be shared by any other functions. The port must be the same for the host and all its clients.
host_task_priority – The priority to use for the task on the host tile that handles RPC requests from the clients.
Starts an RTOS intertile driver instance. It may be called either before or after starting the RTOS, but must be called before any of the core intertile driver functions are called with this instance.
rtos_intertile_init() must be called on this intertile driver instance prior to calling this.
Parameters:
intertile_ctx – A pointer to the intertile driver instance to start.
Initializes an RTOS intertile driver instance. This must be called simultaneously on the two tiles establishing an intertile link. It may be called either before or after starting the RTOS, but must be called before calling rtos_intertile_start() or any of the core RTOS intertile functions with this instance.
This establishes a new streaming channel between the two tiles, using the provided non-streaming channel to bootstrap this.
Parameters:
intertile_ctx – A pointer to the intertile driver instance to initialize.
c – A channel end that is already allocated and connected to channel end on the tile with which to establish an intertile link. After this function returns, this channel end is no longer needed and may be deallocated or used for other purposes.
structrtos_intertile_t
#include <rtos_intertile.h>
Struct representing an RTOS intertile driver instance.
The members in this struct should not be accessed directly.
structrtos_intertile_address_t
#include <rtos_intertile.h>
Struct to hold an address to a remote function, consisting of both an intertile instance and a port number. Primarily used by the RPC mechanism in the RTOS drivers.
the buffer returned via msg must be freed by the application using rtos_osal_free().
Note
It is important that no other thread listen on this port simultaneously. If this happens, it is undefined which one will receive the data, and it is possible for a resource exception to occur.
Parameters:
ctx – A pointer to the intertile driver instance to use.
port – The number of the port to listen for data on. Only data sent to this port by the remote tile will be received.
msg – A pointer to the received data is written to this pointer variable. This buffer is obtained from the heap and must be freed by the application using rtos_osal_free().
timeout – The amount of time to wait before data become available.
Initializes the l2 cache for use by the RTOS l2 cache memory driver.
Cache buffer must be dword aligned
RTOS_L2_CACHE_DIRECT_MAP
Convenience macro that may be used to specify the direct map cache to rtos_l2_cache_init() in place of setup_fn and thread_fn.
RTOS_L2_CACHE_TWO_WAY_ASSOCIATIVE
Convenience macro that may be used to specify the two way associative cache to rtos_l2_cache_init() in place of setup_fn and thread_fn.
RTOS_L2_CACHE_BUFFER_WORDS_DIRECT_MAP
Convenience macro that may be used to specify the size of the cache buffer for a direct map cache. A pointer to the buffer of size RTOS_L2_CACHE_BUFFER_WORDS_DIRECT_MAP should be passed to the cache_buffer argument of rtos_l2_cache_init().
RTOS_L2_CACHE_BUFFER_WORDS_TWO_WAY
Convenience macro that may be used to specify the size of the cache buffer for a two way associative cache. A pointer to the buffer of size RTOS_L2_CACHE_BUFFER_WORDS_TWO_WAY should be passed to the cache_buffer argument of rtos_l2_cache_init().
structrtos_l2_cache_struct
#include <rtos_l2_cache.h>
Struct representing an RTOS l2 cache driver instance.
The members in this struct should not be accessed directly.
Services a software memory read request from within the software memory fill interrupt handler. This function may be provided by the application when the software memory driver is initialized with the RTOS_SWMEM_READ_FLAG flag. If the application code to satisfy a fill request requires being run from within an RTOS thread, then rtos_swmem_read_request() should be used instead. Both this handler and rtos_swmem_read_request() may be used together. If the ISR handler is able to satisfy the request it should return true. If it is not, but the request can be satisfied from within rtos_swmem_read_request(), then it should return false.
Parameters:
offset – The byte offset into the software memory of the cache line that has had a cache miss.
buf – This function must fill this with SWMEM_EVICT_SIZE_WORDS words of data. Where this data comes from is up to the application. One example is from a flash memory.
Return values:
true – if the fill request was satisfied.
false – if the fill request was not satisfied. This requires that rtos_swmem_read_request() also be provided.
Services a software memory write request from within the software memory fill interrupt handler. This function may be provided by the application when the software memory driver is initialized with the RTOS_SWMEM_WRITE_FLAG flag. If the application code to satisfy an evict request requires being run from within an RTOS thread, then rtos_swmem_write_request() should be used instead. Both this handler and rtos_swmem_write_request() may be used together. If the ISR handler is able to satisfy the request it should return true. If it is not, but the request can be satisfied from within rtos_swmem_write_request(), then it should return false.
Parameters:
offset – The byte offset into the software memory of the cache line that is being evicted.
dirty_mask – A bytewise dirty mask for the data in buf. The least significant bit corresponds to the lowest byte address in buf and each subsequent byte address corresponds to the next least significant bit.
buf – A pointer to a buffer containing SWMEM_EVICT_SIZE_WORDS words of data from the cache line being evicted. It is up to the application what it does with this data. One example is to write it to flash memory.
Return values:
true – if the evict request was satisifed.
false – if the evict request was not satisfied. This requires that rtos_swmem_write_request() also be provided.
Services a software memory read request from within the software memory RTOS thread. This function may be provided by the application when the software memory driver is initialized with the RTOS_SWMEM_READ_FLAG flag. If rtos_swmem_read_request_isr() is also implemented, then it will be called first. If it is unable to satisfy the request, then this handler will be called. See the description for rtos_swmem_read_request_isr().
Parameters:
offset – The byte offset into the software memory of the cache line that has had a cache miss.
buf – This function must fill this with SWMEM_EVICT_SIZE_WORDS words of data. Where this data comes from is up to the application. One example is from a flash memory.
Services a software memory write request from within the software memory RTOS thread. This function may be provided by the application when the software memory driver is initialized with the RTOS_SWMEM_WRITE_FLAG flag. If rtos_swmem_write_request_isr() is also implemented, then it will be called first. If it is unable to satisfy the request, then this handler will be called. See the description for rtos_swmem_write_request_isr().
Parameters:
offset – The byte offset into the software memory of the cache line that is being evicted.
dirty_mask – A bytewise dirty mask for the data in buf. The least significant bit corresponds to the lowest byte address in buf and each subsequent byte address corresponds to the next least significant bit.
buf – A pointer to a buffer containing SWMEM_EVICT_SIZE_WORDS words of data from the cache line being evicted. It is up to the application what it does with this data. One example is to write it to flash memory.
voidrtos_swmem_start(unsignedpriority)
Starts the RTOS software memory driver.
Parameters:
priority – The priority of the task that gets created by the driver to service the software memory.
voidrtos_swmem_init(uint32_tinit_flags)
Initializes the software memory for use by the RTOS software memory driver.
Parameters:
init_flags – A bitfield consisting of initialization flags.
RTOS_SWMEM_READ_FLAG enables swmem reads.
RTOS_SWMEM_WRITE_FLAG enables swmem writes.
unsignedintrtos_swmem_offset_get()
Return the offset from XS1_SWMEM_BASE to the start of the software memory.
RTOS_SWMEM_READ_FLAG
Flag indicating that software memory reads should be enabled. This should probably always be set when using software memory.
RTOS_SWMEM_WRITE_FLAG
Flag indicating that software memory writes should be enabled. This will not always need to be set, especially if flash is backing the software memory and intended to be read only.
The Device Control Service provides the ability to configure and control an XMOS device from a host over a number of transport layers.
Features of the service include:
Simple read/write API
Fully acknowledged protocol
Includes different transports including I2C and USB.
The table below shows combinations of host and transport mechanisms that are currently supported.
Adding new transport layers and/or hosts is straightforward where the hardware supports it.
This type is used to inform the control library the direction of a control transfer from the transport layer.
Values:
enumeratorCONTROL_HOST_TO_DEVICE
enumeratorCONTROL_DEVICE_TO_HOST
CONTROL_VERSION
This is the version of control protocol. Used to check compatibility
IS_CONTROL_CMD_READ(c)
Checks if the read bit is set in a command code.
Parameters:
c – [in] The command code to check
Returns:
true if the read bit in the command is set
Returns:
false if the read bit is not set
CONTROL_CMD_SET_READ(c)
Sets the read bit on a command code
Parameters:
c – [inout] The command code to set the read bit on.
CONTROL_CMD_SET_WRITE(c)
Clears the read bit on a command code
Parameters:
c – [inout] The command code to clear the read bit on.
CONTROL_SPECIAL_RESID
This is the special resource ID owned by the control library. It can be used to check the version of the control protocol. Servicers may not register this resource ID.
CONTROL_MAX_RESOURCE_ID
The maximum resource ID. IDs greater than this cannot be registered.
CONTROL_GET_VERSION
The command to read the version of the control protocol. It must be sent to resource ID CONTROL_SPECIAL_RESID.
CONTROL_GET_LAST_COMMAND_STATUS
The command to read the return status of the last command. It must be sent to resource ID CONTROL_SPECIAL_RESID.
DEVICE_CONTROL_HOST_MODE
The mode value to use when initializing a device control instance that is on the same tile as its associated transport layer. These may be connected to device control instances on other tiles that have been initialized with DEVICE_CONTROL_CLIENT_MODE.
DEVICE_CONTROL_CLIENT_MODE
The mode value to use when initializing a device control instance that is not on the same tile as its associated transport layer. These must be connected to a device control instance on another tile that has been initialized with DEVICE_CONTROL_HOST_MODE.
DEVICE_CONTROL_CALLBACK_ATTR
This attribute must be specified on all device control command handler callback functions provided by the application.
Function pointer type for application provided device control read command handler callback functions.
Called by device_control_servicer_cmd_recv() when a read command is received from the transport layer. The command consists of a resource ID, command value, and a payload_len. This handler must respond with a payload of the requested length.
Param resid:
[in] Resource ID. Indicates which resource the command is intended for.
Param cmd:
[in] Command code. Note that this will be in the range 0x80 to 0xFF because bit 7 set indicates a read command.
Param payload:
[out] Payload bytes of length payload_len that will be sent back over the transport layer in response to this read command.
Param payload_len:
[in] Requested size of the payload in bytes.
Param app_data:
[inout] A pointer to application specific data provided to device_control_servicer_cmd_recv(). How and if this is used is entirely up to the application.
Return:
CONTROL_SUCCESS if the handling of the read data by the device was successful. An error code otherwise.
Function pointer type for application provided device control write command handler callback functions.
Called by device_control_servicer_cmd_recv() when a write command is received from the transport layer. The command consists of a resource ID, command value, payload, and the payload’s length.
Param resid:
[in] Resource ID. Indicates which resource the command is intended for.
Param cmd:
[in] Command code. Note that this will be in the range 0x80 to 0xFF because bit 7 set indicates a read command.
Param payload:
[in] Payload bytes of length payload_len.
Param payload_len:
[in] The number of bytes in payload.
Param app_data:
[inout] A pointer to application specific data provided to device_control_servicer_cmd_recv(). How and if this is used is entirely up to the application.
Return:
CONTROL_SUCCESS if the handling of the read data by the device was successful. An error code otherwise.
Must be called by the transport layer when a new request is received.
Precisely how each of the three command parameters resid, cmd, and payload_len are received is specific to the transport layer and not defined by this library.
Parameters:
ctx – A pointer to the associated device control instance.
resid – The received resource ID.
cmd – The received command value.
payload_len – The length in bytes of the payload that will follow.
Return values:
CONTROL_SUCCESS – if resid has been registered by a servicer.
CONTROL_BAD_COMMAND – if resid has not been registered by a servicer.
Must be called by the transport layer either when it receives a payload, or when it requires a payload to transmit.
Parameters:
ctx – A pointer to the associated device control instance.
payload_buf – A pointer to the payload buffer.
buf_size – A pointer to a variable containing the size of payload_buf.
When \pdirectionisCONTROL_HOST_TO_DEVICE,nomorethanthisnumberofbyteswillbereadfromit.When \pdirectionisCONTROL_DEVICE_TO_HOST,thiswillbeupdatedtothenumberofbytesactuallywrittento \ppayload_buf.
direction – The direction of the payload transfer.
Must be called by the transport layer when it receives a payload and requires a payload to transmit, for example, in a SPI transfer. The error status returned by the servicer handling the command is updated in the first byte of the tx_buf.
Parameters:
ctx – A pointer to the associated device control instance.
rx_buf – A pointer to the receive payload buffer.
rx_size – A variable containing the size of rx_buf.
Nomorethanthisnumberofbyteswillbereadfromit.
tx_buf – A pointer to the transmitr payload buffer.
tx_size – A pointer variable containing the size of tx_buf.
This is called by servicers to wait for and receive any commands received by the transport layer contain one of the resource IDs registered by the servicer. This is also responsible for responding to read commands.
Parameters:
ctx – A pointer to the device control servicer context to receive commands for.
read_cmd_cb – The callback function to handle read commands for all resource IDs associated with the given servicer.
write_cmd_cb – The callback function to handle write commands for all resource IDs associated with the given servicer.
app_data – A pointer to application specific data to pass along to the provided callback functions. How and if this is used is entirely up to the application.
timeout – The number of RTOS ticks to wait before returning if no command is received.
Return values:
CONTROL_SUCCESS – if a command successfully received and responded to.
CONTROL_ERROR – if no command is received before the function times out, or if there was a problem communicating back to the transport layer thread.
This must be called on the tile that runs the transport layer for the device control instance, and has initialized it with DEVICE_CONTROL_HOST_MODE. This must be called after calling device_control_start() and before the transport layer is started. It is to be run simultaneously with device_control_servicer_register() from other threads on any tiles associated with the device control instance. The number of servicers that must register is specified by the servicer_count parameter of device_control_init().
Parameters:
ctx – A pointer to the device control instance to register resources for.
timeout – The amount of time in RTOS ticks to wait before all servicers register their resource IDs with device_control_servicer_register().
Return values:
CONTROL_SUCCESS – if all servicers successfully register their resource IDs before the timeout.
Registers a servicer for a device control instance. Each servicer is responsible for handling any number of resource IDs. All commands received from the transport layer will be forwarded to the servicer that has registered the resource ID that is found in the command.
Servicers may be registered on any tile that has initialized a device control instance. This must be called after calling device_control_start().
Parameters:
ctx – A pointer to the device control servicer context to initialize.
device_control_ctx – An array of pointers to the device control instance to register the servicer with.
device_control_ctx_count – The number of device control instances to register the servicer with.
resources – Array of resource IDs to associate with this servicer.
num_resources – The number of resource IDs within resources.
Starts a device control instance. This must be called by all tiles that have called device_control_init(). It may be called either before or after starting the RTOS, but must be called before registering the resources and servicers for this instance.
device_control_init() must be called on this device control instance prior to calling this.
Parameters:
ctx – A pointer to the device control instance to start.
intertile_port – The port to use with any and all associated intertile instances associated with this device control instance. If this device control instance is only used by one tile then this is unused.
priority – The priority of the task that will be created if the device control instance was initialized with DEVICE_CONTROL_CLIENT_MODE. This is unused on the tiles where this has been initialized with DEVICE_CONTROL_HOST_MODE. This task is used to listen for commands for a resource ID registered by a servicer running on this tile, but received by the transport layer that is running on another.
This must be called by the tile that runs the transport layer (I2C, USB, etc) for the device control instance, as well as all tiles that will register device control servicers for it. It may be called either before or after starting the RTOS, but must be called before calling device_control_start().
Parameters:
ctx – A pointer to the device control context to initialize.
mode – Set to DEVICE_CONTROL_HOST_MODE if the command transport layer is on the same tile. Set to DEVICE_CONTROL_CLIENT_MODE if the command transport layer is on another tile.
servicer_count – The number of servicers that will be associated with this device control instance.
intertile_ctx – An array of intertile contexts used to communicate with other tiles.
intertile_count – The number of intertile contexts in the intertile_ctx array.
When \pmodeisDEVICE_CONTROL_HOST_MODE,thismaybe0iftherearenoservicersonothertiles,uptooneperdevicecontrolinstancethathasbeeninitializedwithDEVICE_CONTROL_CLIENT_MODEonothertiles.When \pmodeisDEVICE_CONTROL_CLIENT_MODEthenthismustbe1,andtheintertilecontextmustconnecttoadevicecontrolinstanceonanothertilethathasbeeninitializedwithDEVICE_CONTROL_HOST_MODE.
Returns:
CONTROL_SUCCESS if the initialization was successful. An error status otherwise.
structdevice_control_t
#include <device_control.h>
Struct representing a device control instance.
The members in this struct should not be accessed directly.
structdevice_control_client_t
#include <device_control.h>
A device_control_t pointer may be cast to a pointer to this structure type and used with the device control API, provided it is initialized with DEVICE_CONTROL_CLIENT_MODE. This is not necessary to do, but will save a small amount of memory.
structdevice_control_servicer_t
#include <device_control.h>
Struct representing a device control servicer instance.
The members in this struct should not be accessed directly.
XCORE ® -VOICE Solutions$$$RTOS Programming Guide$$$API Reference$$$RTOS Services$$$Device Control$$$Transport protocol for control parameters£££modules/rtos/doc/programming_guide/reference/rtos_services/device_control/device_control_protocol.html#transport-protocol-for-control-parameters
Control parameters are converted to an array of bytes in network byte
order (big endian) before they’re sent over the transport protocol. For
example, to set a control parameter to integer value 305419896 which
corresponds to hex 0x12345678, the array of bytes sent over the
transport protocol would be {0x12, 0x34, 0x56, 0x78}. Similarly, a 4
byte payload {0x00, 0x01, 0x23, 0x22} read over the transport protocol
is interpreted as an integer value 0x00012322.
In addition to the control parameters values, commands include Resource
ID, the Command ID and Payload Length fields that must be communicated
from the host to the device. The Resource ID is an 8-bit identifier that
identifies the resource within the device that the command is for. The
Command ID is an 8-bit identifier used to identify a command for a
resource in the device. Payload length is the length of the data in
bytes that the host wants to write to the device or read from the
device.
The payload length is interpreted differently for GET_ and SET_
commands. For SET_commands, the payload length is simply the number of
bytes worth of control parameters to write to the device. For example,
the payload length for a SET_ command to set a control parameter of
type int32 to a certain value, would be set to 4. For GET_ commands the
payload length is 1 more than the number of bytes of control parameters
to read from the device. For example, a GET_ command to read a
parameter of type int32, payload length would be set to 5. The one extra
byte is used for status and is the first byte (payload[0]) of the
payload received from the device. In the example above, payload[0] would
be the status byte and payload[1]..payload[4] would be the 4 bytes that
make up the value of the control parameter.
The table below lists the different values of the status byte and the
action the user is expected to take for each status:
Values for returned status byte
Return code
Values
Description
ctrl_done
0
Read command successful. The payload bytes contain valid payload returned from the device
ctrl_wait
1
Read command not serviced. Retry until ctrl_done status returned
ctrl_invalid
3
Error in read command. Abort and debug
The GET_commands need the extra status byte since the device might not
return the control parameter value immediately due to timing
constraints. If that is the case the status byte would indicate the
status as ctrl_wait and the user would need to retry the command. When
returned a ctrl_wait, the user is expected to retry the GET_ command
until the status is returned as ctrl_done. The first GET_command is
placed in a queue and it will be serviced by the end of each 15ms audio
frame. Once the status byte indicates ctrl_done, the rest of the bytes
in the payload indicate the control parameter value.
Transporting control parameters over I2C
This section describes the I2C command sequence when issuing read and
write commands to the device.
The first byte sent over I2C after start contains the device address and
information about whether this is an I2C read transaction or a write
transaction. This byte is 0x58 for a write command or 0x59 for a read
command. These values are derived by left shifting the device address
(0x2c) by 1 and doing a logical OR of the resulting value with 0 for an
I2C write and 1 for an I2C read.
The bytes sequence sent between I2C start and stop for SET_ commands is
shown in the figure below.
For GET_ commands, the I2C commands sequence consists of a write
command followed by a read command with a repeated start between the 2
commands. The write command writes the resource ID, command ID and the
expected data length to the device and the read command reads the status
byte followed by the rest of the payload that makes up the control
parameter value. The figure below shows the I2C bytes sequence sent and
received for a GET_ command.
XCORE ® -VOICE Solutions$$$RTOS Programming Guide$$$API Reference$$$RTOS Services$$$Device Control$$$Transporting control parameters over USB£££modules/rtos/doc/programming_guide/reference/rtos_services/device_control/device_control_protocol.html#transporting-control-parameters-over-usb
Use the vendor_id 0x20B1, product_id 0x0020 and interface number 0 to
initialize for USB.
XCORE ® -VOICE Solutions$$$RTOS Programming Guide$$$API Reference$$$RTOS Services$$$Device Control$$$Floating point to fixed point (Q format) conversion£££modules/rtos/doc/programming_guide/reference/rtos_services/device_control/device_control_protocol.html#floating-point-to-fixed-point-q-format-conversion
Numbers with fractional parts can be represented as floating-point or
fixed-point numbers. Floating point formats are widely used but carry
performance overheads. Fixed point formats can improve system efficiency
and are used extensively within the XVF3610. Fixed point numbers have
the position of the decimal point fixed and this is indicated as a part
of the format description.
In this document, Q format is used to describe fixed point number
formats, with the representation given as Qm.n format where m
is the number of bits reserved for the sign and integer part of the
number and n is the number of bits reserved for the fractional part of
the number. The position of the decimal point is a trade-off between the
range of values supported and the resolution provided by the fractional
bits.
The dynamic range of Qm.n format is -2m-1 and
2m-1-2-n with a resolution of 2-n
To convert a floating-point format number to Qm.n format
fixed-point number:
Multiply the floating-point number by 2m
Round the result to the nearest integer
The resulting integer number is the Qm.n fixed-point
representation of the initial floating-point number
To convert a Qm.n fixed-point number to floating-point:
Divide the fixed-point number by 2m
The resulting decimal number is a floating-point representation of
the fixed-point number.
Converting a number into fixed point format and then back to a floating
point number may introduce an error of up to ±2-(n+1)
Example:
To represent a floating-point number 14.765467 in Q8.24 format, the
equivalent fixed-point number would be 14.765467 x 224 =
247723429.2 which rounds to 247723429.
To get back the floating-point number given the Q8.24 number 247723429,
calculate 247723429 ÷ 224 and get back the floating-point
number as 14.76546699. The difference of 0.00000001 is correct to with
the error bounds of ±2-25 which is ±0.00000003
The concurrency support sw_service contains a multiple reader single writer lock to support multitheaded applications that need to safely support shared access to a single hardware or software resource. This implementation supports either reader preferred or writer preferred locks.
The generic pipeline service provides a generic construct to create multithreaded pipelines. This can be used to create a variety of sequential operations on data, such as an audio processing pipeline.
The generic_pipeline_init() creates stage_count tasks. In the first stage the application provided input_data function pointer is called. The data then is passed to the first stage_function. After the first state function the data is passed by an RTOS queue to the subsequent stage function. Middle stage functions receive from the previous stage queue, call the stage function, and output to the next stage queue. The last stage function will receive from the previous stage queue, call the stage function, and then call the output_data function pointer.
This code snippet is an example of creating a pipeline to consume a buffer.
Example generic pipeline use
staticvoid*input_func(void*input_app_data){uint32_t*data=pvPortMalloc(100*sizeof(uint32_t));/* Populate some dummy data */for(inti=0;i<100;i++){data[i]=i;}returndata;}staticvoid*output_func(void*data,void*output_app_data){/* Use data here */for(inti=0;i<100;i++){rtos_printf("val[%d] = %d\n",i,(uint32_t*)data[i]);}return1;/* Return nonzero value for generic pipeline to implicitly free the packet */}staticvoidstage0(void*data){/* Perform operation on data here*/;}staticvoidstage1(void*data){/* Perform operation on data here*/;}staticvoidstage2(void*data){/* Perform operation on data here*/;}
The following structures and functions are used to initialize and start a generic pipeline instance.
typedefvoid*(*pipeline_input_t)(void*input_data)
Function pointer type for application provided generic pipeline input callback functions.
Called by the first generic_pipeline_stage() when the stage wants input data. This data pointer is provided to the first stage function to be processed.
This function will create a multistage pipeline, creating a task per stage and connecting them via queues. Each stage task follows the convention:
Get input data
Process data
Push output data
For the first stage, the input data are the provided by the input callback. For the final stage, the output data are provided to the output callback.
Parameters:
input – A function pointer called to get input data
output – A function pointer called to give output data
input_data – A pointer to application specific data to pass to the input callback function
output_data – A pointer to application specific data to pass to the output callback function
stage_functions – An array of stage function pointers
stage_stack_word_sizes – The stack size of each stage. Note: For the first stage must contain enough stack for the stage function + input function. Likewise, the last stage must contain enough stack for the stage function + output function.
pipeline_priority – The priority of all pipeline tasks
stage_count – The number of stages. The limit is 10 stages.
XCORE ® -VOICE Solutions$$$RTOS Programming Guide$$$FAQs$$$What is the memory overhead of the FreeRTOS kernel?£££modules/rtos/doc/programming_guide/faq.html#what-is-the-memory-overhead-of-the-freertos-kernel
The FreeRTOS kernel can be configured to require as little as 9kB of RAM (per tile). In a typical applicaiton, expect the requirement to be closer to 16kB of RAM (per tile).
XCORE ® -VOICE Solutions$$$RTOS Programming Guide$$$FAQs$$$How do I determine the number of words to allocate for use as a task’s stack?£££modules/rtos/doc/programming_guide/faq.html#how-do-i-determine-the-number-of-words-to-allocate-for-use-as-a-task-s-stack
Since tasks run within FreeRTOS, the RTOS stack requirement must be known at compile time. In FreeRTOS applications on most other microcontrollers, the general practice is to create a task with a large amount of stack, use the FreeRTOS stack debug functions to determine the worst case runtime usage of stack, and then adjust the stack memory value accordingly. The problem with this method is that the stack of any given thread varies greatly based on the functions that are called within, and thus a code or compiler optimization change result in the optimal task stack usage to have to be redetermined. This issue results in many FreeRTOS applications being written in such a way that wastes memory, by providing task with way more stack than they should need. Additionally, stack overflow bugs can remain hidden for a long time and even when bugs do manifest, the source can be difficult to pinpoint.
The XTC Tools address this issue by creating a symbol that represents the maximum stack requirement of any function at compile time. By using the RTOS_THREAD_STACK_SIZE() macro, for the stack words argument for creating a FreeRTOS task, it is guaranteed that the optimal stack requirement is used, provided that the function does not call function pointers nor can infinitely recurse.
If function pointers are used within a thread, then the application programmer must annotate the code with the appropriate function pointer group attribute. For recursive functions, the only option is to specify the stack manually. See Appendix A - Guiding Stack Size Calculation in the XTC Tools documentation for more information.
XCORE ® -VOICE Solutions$$$RTOS Programming Guide$$$FAQs$$$Can I use xcore resources like channels, timers and hw_locks?£££modules/rtos/doc/programming_guide/faq.html#can-i-use-xcore-resources-like-channels-timers-and-hw-locks
You are free to use channels, ports, timers, etc… in your FreeRTOS applications. However, some considerations need to be made. The RTOS kernel knows about RTOS primitives. For example, if RTOS thread A attempts to take a semaphore, the kernel is free to schedule other tasks in thread A’s place while thread A is waiting for some other task to give the semaphore. The RTOS kernel does not know anything about xcore resources. For example, if RTOS thread A attempts to recv on a channel, the kernel is not free to schedule other tasks in its place while thread A is waiting for some other task to send to the other end of the channel. A developer should be aware that blocking calls on xcore resources will block a FreeRTOS thread. This may be OK as long as it is carefully considered in the application design. There are a variety of methods to handle the decoupling of xcore and RTOS resources. These can be best seen in the various RTOS drivers, which wrap the realtime IO hardware imitation layer.
One easy to make mistake in FreeRTOS, is not providing enough stack space for a created task. A vast amount of questions exist online around how to select the FreeRTOS stack size, which the most common answer being to create the task with more than enough stack, force the worst case stack condition (not always trivial), and then use the FreeRTOS debug function uxTaskGetStackHighWaterMark() to determine how much you can decrease the stack. This method leaves plenty of room for error and must be done during runtime, and therefore on a build by build basis. The static analysis tools provided by The XTC Tools greatly simplify this process since they calculate the exact stack required for a given function call. The macro RTOS_THREAD_STACK_SIZE will return the nstackwords symbol for a given thread plus the additional space required for the kernel ISRs. Using this macro for every task create will ensure that there is appropriate stack space for each thread, and thus no stack overflow.
XCORE ® -VOICE Solutions$$$Build System User Guide£££modules/rtos/doc/build_system_guide/index.html#build-system-user-guide
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Build System£££build-system-guide.html#build-system
This document describes the CMake-based build system used by applications based on the XMOS RTOS framework. The build system is designed so a user does not have to be an expert using CMake. However, some familiarity with CMake is helpful. You can familiarize yourself by reading the CMake Tutorial or CMake documentation. Reviewing these is optional and the reader should feel free to save that for later.
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Build System$$$Overview£££build-system-guide.html#overview
An xcore RTOS project can be seen as an integration of several modules. For example, for a FreeRTOS application that captures audio from PDM microphones and outputs it to a DAC, there could be the following modules:
Several core modules (for debug prints, etc…)
The FreeRTOS kernel and drivers
PDM microphone array driver for receiving audio samples
I2C driver for configuring the DAC
I2S driver for outputting to the DAC
Application code tying it all together
When a project is compiled, the build system will build all libraries and source files required for the application. For this to happen, your CMakeLists.txt file will need to specify:
It is very common for target link alias libraries, like rtos::freertos in the snippet above, to include common sets of target link libraries. The snippet above could be simplified because the rtos::freertos alias includes many commonly used drivers and peripheral IO libraries as a dependency.
Application target link libraries can be further simplified using existing bsp_configs. These provide their dependent link libraries enabling applications to simplify their target link libraries list. The snippet above could be simplified because the rtos::bsp_config::xcore_ai_explorer alias includes core::general, rtos::freertos, and all required drivers and peripheral IO libraries used by the bsp_config. More information on bsp_configs can be found in the RTOS Programming Guide.
XMOS libraries and frameworks provide several target aliases. Being aware of the Targets will simplify your application CMakeLists.txt.
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Example CMakeLists.txt£££build-system-example-cmakelists.html#example-cmakelists-txt
CMake is powerful tool that provides the developer a great deal of flexibility in how their projects are built. As a result, CMakeLists.txt files can accomplish the same function in multiple ways.
Below is an example CMakeLists.txt that shows both required and conventional commands for a basic FreeRTOS project. This example can be used as a starting point for your application, but it is recommended to copy a CMakeLists.txt from an XMOS reference design or other example application that closely resembles your application.
## Specify your application sources by globbing the src folderfile(GLOB_RECURSEAPP_SOURCESsrc/*.c)## Specify your application include pathsset(APP_INCLUDESsrc)## Specify your compiler flagsset(APP_COMPILER_FLAGS-Os-report-fxscope-mcmodel=large${CMAKE_CURRENT_SOURCE_DIR}/src/config.xscope${CMAKE_CURRENT_SOURCE_DIR}/XCORE-AI-EXPLORER.xn)## Specify any compile definitionsset(APP_COMPILE_DEFINITIONSconfigENABLE_DEBUG_PRINTF=1PLATFORM_USES_TILE_0=1PLATFORM_USES_TILE_1=1)## Set your link librariesset(APP_LINK_LIBRARIESrtos::bsp_config::xcore_ai_explorer)## Set your link optionsset(APP_LINK_OPTIONS-report${CMAKE_CURRENT_SOURCE_DIR}/XCORE-AI-EXPLORER.xn${CMAKE_CURRENT_SOURCE_DIR}/src/config.xscope)## Create your targets## Create the target for the portion of application code that will execute on tile[0]set(TARGET_NAMEtile0_my_app)add_executable(${TARGET_NAME}EXCLUDE_FROM_ALL)target_sources(${TARGET_NAME}PUBLIC${APP_SOURCES})target_include_directories(${TARGET_NAME}PUBLIC${APP_INCLUDES})target_compile_definitions(${TARGET_NAME}PUBLIC${APP_COMPILE_DEFINITIONS}THIS_XCORE_TILE=0)target_compile_options(${TARGET_NAME}PRIVATE${APP_COMPILER_FLAGS})target_link_libraries(${TARGET_NAME}PUBLIC${APP_LINK_LIBRARIES})target_link_options(${TARGET_NAME}PRIVATE${APP_LINK_OPTIONS})unset(TARGET_NAME)## Create the target for the portion of application code that will execute on tile[1]set(TARGET_NAMEtile1_my_app)add_executable(${TARGET_NAME}EXCLUDE_FROM_ALL)target_sources(${TARGET_NAME}PUBLIC${APP_SOURCES})target_include_directories(${TARGET_NAME}PUBLIC${APP_INCLUDES})target_compile_definitions(${TARGET_NAME}PUBLIC${APP_COMPILE_DEFINITIONS}THIS_XCORE_TILE=1)target_compile_options(${TARGET_NAME}PRIVATE${APP_COMPILER_FLAGS})target_link_libraries(${TARGET_NAME}PUBLIC${APP_LINK_LIBRARIES})target_link_libraries(${TARGET_NAME}PRIVATE${APP_LINK_OPTIONS})unset(TARGET_NAME)## Merge tile[0] and tile[1] binaries into a single binary using an XMOS CMake macromerge_binaries(my_apptile0_my_apptile1_my_app1)## Optionally create run and debug targets using XMOS CMake macroscreate_run_target(my_app)create_debug_target(my_app)
For more information, see the documentation for each of the CMake commands used in the example above.
See Macros for more information on the XMOS CMake macros.
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Targets£££build-system-targets.html#targets
The following library target aliases can be used in your application CMakeLists.txt. An example of how to add aliases to your target link libraries is shown below:
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Targets$$$General£££build-system-targets.html#general
Several aliases are provided that specify a collection of libraries with similar functions. These composite target libraries provide a concise alternative to specifying all the individual targets that are commonly required.
Composite Target Libraries
Target
Description
core::general
Commonly used core libraries
io::general
Commonly used peripheral libraries
io::audio
Commonly used peripheral libraries for audio applications
rtos::freertos
Commonly used RTOS libraries
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Targets$$$Core£££build-system-targets.html#core
If you prefer, you can specify individual core library targets.
Core Libraries
Target
Description
framework_core_clock_control
Clock control API
framework_core_utils
General utilities used by most applications
framework_core_legacy_compat
For compatibility with XC
lib_xcore_math
VPU-optimized math library
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Targets$$$Peripherals£££build-system-targets.html#peripherals
If you prefer, you can specify individual peripheral libraries.
Peripheral Libraries
Target
Description
lib_i2c
I2C library
lib_spi
SPI library
lib_uart
UART library
lib_qspi_io
QSPI library
lib_xud
XUD USB library
lib_i2s
I2S library
lib_mic_array
Microphone Array library
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Targets$$$RTOS£££build-system-targets.html#rtos
Several aliases are provided that specify a collection of RTOS libraries with similar functions. These composite target libraries provide a concise alternative to specifying all the individual targets that are commonly required.
Composite RTOS Libraries
Target
Description
rtos::freertos
All libraries used my most FreeRTOS applications
rtos::drivers:all
All RTOS Driver libraries
rtos::freertos_usb
All libraries to support development with TinyUSB
rtos::sw_services::general
Most commonly used RTOS software service libraries
rtos::iot
All IoT libraries
rtos::wifi
All WiFi libraries
These board support libraries simplify development with a specific board.
Board Support Libraries
Target
Description
rtos::bsp_config::xcore_ai_explorer
xcore.ai Explorer RTOS board support library
If you prefer, you can specify individual RTOS driver libraries.
Individual RTOS Driver Libraries
Target
Description
rtos::drivers::uart
UART RTOS driver library
rtos::drivers::i2c
I2C RTOS driver library
rtos::drivers::i2s
I2S RTOS driver library
rtos::drivers::spi
SPI RTOS driver library
rtos::drivers::qspi_io
QSPI RTOS driver library
rtos::drivers::mic_array
Microphone Array RTOS driver library
rtos::drivers::usb
USB RTOS driver library
rtos::drivers::dfu_image
RTOS DFU driver library
rtos::drivers::gpio
GPIO RTOS driver library
rtos::drivers::l2_cache
L2 Cache RTOS driver library
rtos::drivers::clock_control
Clock control RTOS driver library
rtos::drivers::trace
Trace RTOS driver library
rtos::drivers::swmem
SwMem RTOS driver library
rtos::drivers::wifi
WiFi RTOS driver library
rtos::drivers::intertile
Intertile RTOS driver library
rtos::drivers::rpc
Remote procedure call RTOS driver library
If you prefer, you can specify individual software service libraries.
Individual Software Service Libraries
Target
Description
rtos::sw_services::fatfs
FatFS library
rtos::sw_services::usb
USB library
rtos::sw_services::device_control
Device control library
rtos::sw_services::usb_device_control
USB device control library
rtos::sw_services::wifi_manager
WiFi manager library
rtos::sw_services::tls_support
TLS library
rtos::sw_services::dhcp
DHCP library
rtos::sw_services::json
JSON library
rtos::sw_services::http
HTTP library
rtos::sw_services::sntpd
SNTP daemon library
rtos::sw_services::mqtt
MQTT library
The following libraries for building host applications are also provided by the SDK.
Host (x86) Libraries
Target
Description
rtos::sw_services::device_control_host_usb
Host USB device control library
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros£££build-system-guide-macros.html#macros
Several CMake macros and functions are provide to make building for XCORE easier. These macros are located in the file tools/cmake_utils/xmos_macros.cmake and are documented below.
To see what XTC Tools commands the macros and functions are running, add VERBOSE=1 to your build command line. For example:
make run_my_target VERBOSE=1
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Common Macros£££build-system-guide-macros.html#common-macros
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Common Macros$$$merge_binaries£££build-system-guide-macros.html#merge-binaries
merge_binaries combines multiple xcore applications into one by extracting a tile elf and recombining it into another binary. This is used in multitile RTOS applications to enable building unique instances of the FreeRTOS kernel and task sets on a per tile basis.
This macro takes an output target name, a base target, a target containing a tile to merge, and the tile number to merge.
This macro can be called in two ways. The 4 argument version is for when the
application has only 1 node and therefore only the core needs to be specified.
# create target OUT by replacing tile number 0 in BASE with tile 0 in OTHERmerge_binaries(${OUT}${BASE}${OTHER}0)
The 5 argument version is for multi-node applications. IMPORTANT: node number
is not the “Node Id” from the xn file, rather the index of the node in the
JTAGChain which is defined in the xn file.
# create target OUT by replacing tile 1 on node 0 in BASE with tile 1 on# node 0 in OTHERmerge_binaries(${OUT}${BASE}${OTHER}01)
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Common Macros$$$create_run_target£££build-system-guide-macros.html#create-run-target
create_run_target creates a run target for <TARGET_NAME> with xscope output.
create_run_target(<TARGET_NAME>)
create_run_target allows you to run a binary with the following command instead of invoking xrun--xscope.
make run_my_target
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Common Macros$$$create_debug_target£££build-system-guide-macros.html#create-debug-target
create_debug_target creates a debug target for <TARGET_NAME>.
create_debug_target(<TARGET_NAME>)
create_debug_target allows you to debug a binary with the following command instead of invoking xgdb. This target implicitly sets up the xscope debug interface as well.
make debug_my_target
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Common Macros$$$create_filesystem_target£££build-system-guide-macros.html#create-filesystem-target
create_filesystem_target creates a filesystem file for <TARGET_NAME> using the files in the <FILESYSTEM_INPUT_DIR> directory. <IMAGE_SIZE> specifies the size (in bytes) of the filesystem. The filesystem output filename will end in _fat.fs. Optional argument <OPTIONAL_DEPENDS_TARGETS> can be used to specify other dependency targets, such as filesystem generators.
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Common Macros$$$create_data_partition_directory£££build-system-guide-macros.html#create-data-partition-directory
create_data_partition_directory creates a directory populated with all components related to the data partition. The data partition output folder will end in _data_partition
Optional argument <OPTIONAL_DEPENDS_TARGETS> can be used to specify other dependency targets.
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Common Macros$$$create_flash_app_target£££build-system-guide-macros.html#create-flash-app-target
create_flash_app_target creates a debug target for <TARGET_NAME> with optional arguments <BOOT_PARTITION_SIZE>, <DATA_PARTITION_CONTENTS>, and <OPTIONAL_DEPENDS_TARGETS>. <BOOT_PARTITION_SIZE> specificies the size in bytes of the boot partition. <DATA_PARTITION_CONTENTS> specifies the optional binary contents of the data partition. <OPTIONAL_DEPENDS_TARGETS> specifies CMake targets that should be dependencies of the resulting create_flash_app_target target. This may be used to create recipes that generate the data partition contents.
create_flash_app_target allows you to flash a factory image binary and optional data partition with the following command instead of invoking xflash.
make flash_app_my_target
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Less Common Macros£££build-system-guide-macros.html#less-common-macros
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Less Common Macros$$$create_install_target£££build-system-guide-macros.html#create-install-target
create_install_target creates an install target for <TARGET_NAME>.
create_install_target(<TARGET_NAME>)
create_install_target will copy <TARGET_NAME>.xe to the ${PROJECT_SOURCE_DIR}/dist directory.
make install_my_target
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Less Common Macros$$$create_run_xscope_to_file_target£££build-system-guide-macros.html#create-run-xscope-to-file-target
create_run_xscope_to_file_target creates a run target for <TARGET_NAME>. <XSCOPE_FILE> specifies the file to save to (no extension).
create_run_xscope_to_file_target allows you to run a binary with the following command instead of invoking xrun--xscope-file.
make run_xscope_to_file_my_target
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Less Common Macros$$$create_upgrade_img_target£££build-system-guide-macros.html#create-upgrade-img-target
create_upgrade_img_target creates an xflash image upgrade target for a provided binary for use in DFU
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Less Common Macros$$$create_erase_all_target£££build-system-guide-macros.html#create-erase-all-target
create_erase_all_target creates an xflash erase all target for <TARGET_FILEPATH> target XN file. The full filepath must be specified for XN file
create_erase_all_target allows you to erase flash with the following command instead of invoking xflash.
make erase_all_my_target
XCORE ® -VOICE Solutions$$$Build System User Guide$$$Macros$$$Less Common Macros$$$query_tools_version£££build-system-guide-macros.html#query-tools-version
query_tools_version populates the following CMake variables:
This library provide a software defined UART (universal asynchronous receiver transmitter) allowing you to communicate with other UART enabled devices in your system. A UART is a single wire per direction communications interface allowing either half or full duplex communication. The components in this library are controlled via C and behave as a UART transmitter and/or receiver peripheral.
Various configuration options are available including baud rate (individually settable per direction), number of data bits (between 5 and 8), parity (EVEN, ODD or NONE) and number of stop bits (1 or 2). The UART does not support flow control signals. Only a single 1b IO port per UART direction is needed.
The Tx UART supports up to 1152000 baud unbuffered and 576000 baud buffered with a 75MHz logical core. The Rx UART supports up to 700000 baud unbuffered and 422400 baud buffered with a 75MHz logical core. Proportionally higher rates are achievable using a higher logical core MHz.
The UART receive supports standard error detection including START, PARITY and FRAMING errors. A callback mechanism is included to notify the user of these conditions.
The UART may be used in blocking mode, where the call to Tx/Rx does not return until the stop bit is complete. It may also be used in ISR/buffered mode where the UART Rx and/or Tx operates in background mode using a FIFO and callbacks to manage data-flow and error conditions. Cycles are stolen from the logical core which setup the interrupt. In ISR/buffered mode additional callbacks are supported indicating the UNDERRUN condition when the Tx buffer is empty and OVERRUN when the Rx buffer is full.
UART data wires
Tx
Transmit line controlled by UART Tx
Rx
Receive line controlled by UART Rx
All UART functions can be accessed via the uart.h header:
The following code snippet demonstrates the basic blocking usage of an UART Tx device.
#include<xs1.h>#include"uart.h"uart_tx_tuart;port_tp_uart_tx=XS1_PORT_1A;hwtimer_ttmr=hwtimer_alloc();uint8_ttx_data[4]={0x01,0x02,0x04,0x08};// Initialize the UART Txuart_tx_blocking_init(&uart,p_uart_tx,115200,8,UART_PARITY_NONE,1,tmr);// Transfer some datafor(inti=0;i<sizeof(tx_data);i++){uart_tx(&uart,tx_data[i]);}
The following code snippet demonstrates the usage of an UART Tx device used in ISR/Buffered mode:
#include<xs1.h>#include"uart.h"HIL_UART_TX_CALLBACK_ATTRvoidtx_empty_callback(void*app_data){int*tx_empty=(int*)app_data;*tx_empty=1;}voiduart_tx(void){uart_tx_tuart;port_tp_uart_tx=XS1_PORT_1A;hwtimer_ttmr=hwtimer_alloc();uint8_tbuffer[64+1]={0};// Note buffer size plus oneuint8_ttx_data[4]={0x01,0x02,0x04,0x08};volatileinttx_empty=0;// Initialize the UART Txuart_tx_init(&uart,p_uart_tx,115200,8,UART_PARITY_NONE,1,tmr,buffer,sizeof(buffer),tx_empty_callback,&tx_empty);// Transfer some datafor(inti=0;i<sizeof(tx_data);i++){uart_tx(&uart,tx_data[i]);}// Wait for it to completewhile(!tx_empty);
baud_rate – The baud rate of the UART in bits per second.
data_bits – The number of data bits per frame sent.
parity – The type of parity used. See uart_parity_t above.
stop_bits – The number of stop bits asserted at the of the frame.
tmr – The resource id of the timer to be used. Polling mode will be used if set to 0.
tx_buff – Pointer to a buffer. Optional. If set to zero the UART will run in blocking mode. If initialised to a valid buffer, the UART will be interrupt driven.
buffer_size_plus_one – Size of the buffer if enabled in tx_buff. Note that the buffer allocation and size argument must be one greater than needed. Eg. buff[65] for a 64 byte buffer.
uart_tx_empty_callback_fptr – Callback function pointer for UART buffer empty in buffered mode.
app_data – A pointer to application specific data provided by the application. Used to share data between this callback function and the application.
Define which sets the enum start point of RX errors. This is relied upon by the RTOS drivers and allows optimisation of error handling.
HIL_UART_TX_CALLBACK_ATTR
This attribute must be specified on the UART TX UNDERRUN callback function provided by the application. It ensures the correct stack usage is calculated.
HIL_UART_RX_CALLBACK_ATTR
This attribute must be specified on the UART Rx callback functions (both ERROR and Rx complete callbacks) provided by the application. It ensures the correct stack usage is correctly calculated.
structuart_tx_t
#include <uart.h>
Struct to hold a UART Tx context.
The members in this struct should not be accessed directly. Use the API provided instead.
The following code snippet demonstrates the basic usage of an UART Rx device where the function call to Rx returns after the stop bit has been sampled. The function blocks until a complete byte has been received.
#include<xs1.h>#include<print.h>#include"uart.h"HIL_UART_RX_CALLBACK_ATTRvoidrx_error_callback(uart_callback_code_tcallback_code,void*app_data){switch(callback_code){caseUART_START_BIT_ERROR:printstrln("UART_START_BIT_ERROR");break;caseUART_PARITY_ERROR:printstrln("UART_PARITY_ERROR");break;caseUART_FRAMING_ERROR:printstrln("UART_FRAMING_ERROR");test_abort=1;break;caseUART_OVERRUN_ERROR:printstrln("UART_OVERRUN_ERROR");break;caseUART_UNDERRUN_ERROR:printstrln("UART_UNDERRUN_ERROR");break;default:printstr("Unexpected callback code: ");printintln(callback_code);}}voiduart_rx(void){uart_rx_tuart;port_tp_uart_rx=XS1_PORT_1B;hwtimer_ttmr=hwtimer_alloc();chartest_rx[16];// Initialize the UART Rxuart_rx_blocking_init(&uart,p_uart_rx,115200,8,UART_PARITY_NONE,1,tmr,rx_error_callback,&uart);// Receive some datafor(inti=0;i<sizeof(rx_data);i++){test_rx[i]=uart_rx(&uart);}
The following code snippet demonstrates the usage of an UART Rx device used in ISR/Buffered mode:
#include<xs1.h>#include<print.h>#include"uart.h"HIL_UART_RX_CALLBACK_ATTRvoidrx_error_callback(uart_callback_code_tcallback_code,void*app_data){switch(callback_code){caseUART_START_BIT_ERROR:printstrln("UART_START_BIT_ERROR");break;caseUART_PARITY_ERROR:printstrln("UART_PARITY_ERROR");break;caseUART_FRAMING_ERROR:printstrln("UART_FRAMING_ERROR");test_abort=1;break;caseUART_OVERRUN_ERROR:printstrln("UART_OVERRUN_ERROR");break;caseUART_UNDERRUN_ERROR:printstrln("UART_UNDERRUN_ERROR");break;default:printstr("Unexpected callback code: ");printintln(callback_code);}}HIL_UART_RX_CALLBACK_ATTRvoidrx_callback(void*app_data){unsigned*bytes_received=(unsigned*)app_data;*bytes_received+=1;}voiduart_rx(void){uart_rx_tuart;port_tp_uart_rx=XS1_PORT_1A;hwtimer_ttmr=hwtimer_alloc();uint8_tbuffer[64+1]={0};// Note buffer size plus onevolatileunsignedbytes_received=0;// Initialize the UART Rxuart_rx_init(&uart,p_uart_rx,115200,8,UART_PARITY_NONE,1,tmr,buffer,sizeof(buffer),rx_callback,&bytes_received);// Wait for 16b of datawhile(bytes_received<15);// Get the datauint8_ttest_rx[NUM_RX_WORDS];for(inti=0;i<16;i++){test_rx[i]=uart_rx(&uart);}
baud_rate – The baud rate of the UART in bits per second.
data_bits – The number of data bits per frame sent.
parity – The type of parity used. See uart_parity_t above.
stop_bits – The number of stop bits asserted at the of the frame.
tmr – The resource id of the timer to be used. Polling mode will be used if set to 0.
rx_buff – Pointer to a buffer. Optional. If set to zero the UART will run in blocking mode. If initialised to a valid buffer, the UART will be interrupt driven.
buffer_size_plus_one – Size of the buffer if enabled in rx_buff. Note that the buffer allocation and size argument must be one greater than needed. Eg. buff[65] for a 64 byte buffer.
uart_rx_complete_callback_fptr – Callback function pointer for UART rx complete (one word) in buffered mode only. Optionally NULL.
uart_rx_error_callback_fptr – Callback function pointer for UART rx errors The error is contained in cb_code in the uart_rx_t struct.
app_data – A pointer to application specific data provided by the application. Used to share data between this callback function and the application.
Initializes a UART Rx I/O interface. This API is fixed to blocking mode which is where the call to uart_rx returns as soon as the stop bit has been sampled.
baud_rate – The baud rate of the UART in bits per second.
data_bits – The number of data bits per frame sent.
parity – The type of parity used. See uart_parity_t above.
stop_bits – The number of stop bits asserted at the of the frame.
tmr – The resource id of the timer to be used. Polling mode will be used if set to 0.
uart_rx_error_callback_fptr – Callback function pointer for UART rx errors The error is contained in cb_code in the uart_rx_t struct.
app_data – A pointer to application specific data provided by the application. Used to share data between the error callback function and the application.
The members in this struct should not be accessed directly. Use the API provided instead.
I2C Library
A software defined I2C library that allows you to control an I2C bus via xcore ports. I2C is a two-wire hardware serial interface, first developed by Philips. The components in the library are controlled via C and can either act as I2C master or slave.
The library is compatible with multiple slave devices existing on the same bus. The I2C master component can be used by multiple tasks within the xcore device (each addressing the same or different slave devices).
The library can also be used to implement multiple I2C physical interfaces on a single xcore device simultaneously.
All signals are designed to comply with the timings in the I2C specification.
Note that the following optional parts of the I2C specification are not supported:
Multi-master arbitration
10-bit slave addressing
General call addressing
Software reset
START byte
Device ID
Fast-mode Plus, High-speed mode, Ultra Fast-mode
I2C consists of two signals: a clock line (SCL) and a data line (SDA). Both these signals are open-drain and require external resistors to pull the line up if no device is driving the signal down. The correct value for the resistors can be found in the I2C specification.
All I2C functions can be accessed via the i2c.h header:
#include<i2c.h>
I2C Master
I2C Master Usage
The following code snippet demonstrates the basic usage of an I2C master device.
#include<xs1.h>#include"i2c.h"i2c_master_ti2c_ctx;port_tp_scl=XS1_PORT_1A;port_tp_sda=XS1_PORT_1B;uint8_tdata[1]={0x99};// Initialize the masteri2c_master_init(&i2c_ctx,p_scl,0,0,p_sda,0,0,100);// Write some datai2c_master_write(&i2c_ctx,0x33,data,1,NULL,1);// Shutdowni2c_master_shutdown(&i2c_ctx);
I2C Master API
The following structures and functions are used to initialize and start an I2C master instance.
device_addr – The address of the device to write to.
buf – The buffer containing data to write.
n – The number of bytes to write.
num_bytes_sent – The function will set this value to the number of bytes actually sent. On success, this will be equal to n but it will be less if the slave sends an early NACK on the bus and the transaction fails.
send_stop_bit – If this is non-zero then a stop bit will be sent on the bus after the transaction. This is usually required for normal operation. If this parameter is zero then no stop bit will be omitted. In this case, no other task can use the component until a stop bit has been sent.
Returns:
I2C_ACK if the write was acknowledged by the device, I2C_NACK otherwise.
device_addr – The address of the device to read from.
buf – The buffer to fill with data.
n – The number of bytes to read.
send_stop_bit – If this is non-zero then a stop bit. will be sent on the bus after the transaction. This is usually required for normal operation. If this parameter is zero then no stop bit will be omitted. In this case, no other task can use the component until a stop bit has been sent.
Returns:
I2C_ACK if the read was acknowledged by the device, I2C_NACK otherwise.
This function will cause a stop bit to be sent on the bus. It should be used to complete/abort a transaction if the send_stop_bit argument was not set when calling the i2c_master_read() or i2c_master_write() functions.
Implements an I2C master device on one or two single or multi-bit ports.
Parameters:
ctx – A pointer to the I2C master context to initialize.
p_scl – The port containing SCL. This may be either the same as or different than p_sda.
scl_bit_position – The bit number of the SCL line on the port p_scl.
scl_other_bits_mask – A value that is ORed into the port value driven to p_scl both when SCL is high and low. The bit representing SCL (as well as SDA if they share the same port) must be set to 0.
p_sda – The port containing SDA. This may be either the same as or different than p_scl.
sda_bit_position – The bit number of the SDA line on the port p_sda.
sda_other_bits_mask – A value that is ORed into the port value driven to p_sda both when SDA is high and low. The bit representing SDA (as well as SCL if they share the same port) must be set to 0.
kbits_per_second – The speed of the I2C bus. The maximum value allowed is 400.
This function disables the ports associated with the I2C master and deallocates its timer if it was not provided by the application.
If subsequent reads or writes need to be performed, then i2c_master_init() must be called again first.
Parameters:
ctx – A pointer to the I2C master context to shut down.
structi2c_master_struct
#include <i2c.h>
Struct to hold an I2C master context.
The members in this struct should not be accessed directly.
I2C Slave
I2C Slave Usage
The following code snippet demonstrates the basic usage of an I2C slave device.
#include<xs1.h>#include"i2c.h"port_tp_scl=XS1_PORT_1A;port_tp_sda=XS1_PORT_1B;// Setup callbacks// NOTE: See API or SDK examples for more on using the callbacksi2c_callback_group_ti_i2c={.ack_read_request=(ack_read_request_t)i2c_ack_read_req,.ack_write_request=(ack_write_request_t)i2c_ack_write_req,.master_requires_data=(master_requires_data_t)i2c_master_req_data,.master_sent_data=(master_sent_data_t)i2c_master_sent_data,.stop_bit=(stop_bit_t)i2c_stop_bit,.shutdown=(shutdown_t)i2c_shutdown,.app_data=NULL,};// Start the slave device in this thread// NOTE: You may wish to launch the slave device in a different thread.// See the XTC Tools documentation reference for lib_xcore.i2c_slave(&i_i2c,p_scl,p_sda,0x3c);
I2C Slave API
The following structures and functions are used to initialize and start an I2C slave instance.
enumi2c_slave_ack
I2C Slave Response
This type is used to describe the I2C slave response.
i2c_cbg – The I2C callback group pointing to the application’s functions to use for initialization and getting and receiving frames. Also points to application specific data which will be shared between the callbacks.
p_scl – The SCL port of the I2C bus. This should be a 1 bit port. If not, The SCL pin must be at bit 0 and the other bits unused.
p_sda – The SDA port of the I2C bus. This should be a 1 bit port. If not, The SDA pin must be at bit 0 and the other bits unused.
device_addr – The address of the slave device.
I2C_CALLBACK_ATTR
This attribute must be specified on all I2C callback functions provided by the application.
structi2c_callback_group_t
#include <i2c.h>
Callback group representing callback events that can occur during the operation of the I2C slave task. Must be initialized by the application prior to passing it to one of the I2C tasks.
I2C Registers
I2C Register API
The following structures and functions are used to read and write I2C registers.
enumi2c_regop_res_t
This type is used by the supplementary I2C register read/write functions to report back on whether the operation was a success or not.
Values:
enumeratorI2C_REGOP_SUCCESS
The operation was successful.
enumeratorI2C_REGOP_DEVICE_NACK
The operation was NACKed when sending the device address, so either the device is missing or busy.
enumeratorI2C_REGOP_INCOMPLETE
The operation was NACKed halfway through by the slave.
This function reads from an 8-bit addressed, 8-bit register in an I2C device. The function reads the data by sending the register address followed reading the register data from the device at the specified device address.
Note
No stop bit is transmitted between the write and the read. The operation is performed as one transaction using a repeated start.
Parameters:
ctx – A pointer to the I2C master context to use.
device_addr – The address of the device to read from.
reg – The address of the register to read from.
result – Indicates whether the read completed successfully. Will be set to I2C_REGOP_DEVICE_NACK if the slave NACKed, and I2C_REGOP_SUCCESS on successful completion of the read.
This function reads from an 16-bit addressed, 8-bit register in an I2C device. The function reads the data by sending the register address followed reading the register data from the device at the specified device address.
Note
No stop bit is transmitted between the write and the read. The operation is performed as one transaction using a repeated start.
Parameters:
ctx – A pointer to the I2C master context to use.
device_addr – The address of the device to read from.
reg – The address of the register to read from.
result – Indicates whether the read completed successfully. Will be set to I2C_REGOP_DEVICE_NACK if the slave NACKed, and I2C_REGOP_SUCCESS on successful completion of the read.
This function reads from an 8-bit addressed, 16-bit register in an I2C device. The function reads the data by sending the register address followed reading the register data from the device at the specified device address.
Note
No stop bit is transmitted between the write and the read. The operation is performed as one transaction using a repeated start.
Parameters:
ctx – A pointer to the I2C master context to use.
device_addr – The address of the device to read from.
reg – The address of the register to read from.
result – Indicates whether the read completed successfully. Will be set to I2C_REGOP_DEVICE_NACK if the slave NACKed, and I2C_REGOP_SUCCESS on successful completion of the read.
This function reads from an 16-bit addressed, 16-bit register in an I2C device. The function reads the data by sending the register address followed reading the register data from the device at the specified device address.
Note
No stop bit is transmitted between the write and the read. The operation is performed as one transaction using a repeated start.
Parameters:
ctx – A pointer to the I2C master context to use.
device_addr – The address of the device to read from.
reg – The address of the register to read from.
result – Indicates whether the read completed successfully. Will be set to I2C_REGOP_DEVICE_NACK if the slave NACKed, and I2C_REGOP_SUCCESS on successful completion of the read.
This function writes to an 8-bit addressed, 8-bit register in an I2C device. The function writes the data by sending the register address followed by the register data to the device at the specified device address.
Parameters:
ctx – A pointer to the I2C master context to use.
device_addr – The address of the device to write to.
This function writes to a 16-bit addressed, 8-bit register in an I2C device. The function writes the data by sending the register address followed by the register data to the device at the specified device address.
Parameters:
ctx – A pointer to the I2C master context to use.
device_addr – The address of the device to write to.
This function writes to an 8-bit addressed, 16-bit register in an I2C device. The function writes the data by sending the register address followed by the register data to the device at the specified device address.
Parameters:
ctx – A pointer to the I2C master context to use.
device_addr – The address of the device to write to.
This function writes to a 16-bit addressed, 16-bit register in an I2C device. The function writes the data by sending the register address followed by the register data to the device at the specified device address.
Parameters:
ctx – A pointer to the I2C master context to use.
device_addr – The address of the device to write to.
A software defined library that allows you to control an I2S (Inter-IC Sound) bus via xcore ports. I2S is a digital data streaming interfaces particularly appropriate for transmission of audio data. TDM is a special case of I2S which supports transport of more than two audio channels and is partially included in the library at this time. The components in the library are controlled via C and can either act as I2S master, I2S slave or TDM slave.
Note
TDM is only currently supported as a TDM16 slave Tx component. Expansion of this library to support master or slave Rx is possible and can be done on request.
I2S is a protocol between two devices where one is the master and one is the slave which determines who drives the clock lines. The protocol is made up of four signals shown in I2S data wires.
I2S data wires
MCLK
Clock line, driven by external oscillator. This signal is optional.
BCLK
Bit clock. This is a fixed divide of the MCLK and is driven
by the master.
LRCLK (or WCLK)
Word clock (or word select). This is driven by the master.
DATA
Data line, driven by one of the slave or master depending on
the data direction. There may be several data lines in
differing directions.
All I2S functions can be accessed via the i2s.h header:
#include"i2s.h"
TDM is a protocol between two devices similar to I2S where one is the master and one is the slave which determines who drives the clock lines. The protocol is made up of four signals shown in TDM data wires.
TDM data wires
MCLK
Clock line, driven by external oscillator. This signal is optional.
BCLK
Bit clock. This is a fixed divide of the MCLK and is driven
by the master.
FSYCNH
Frame synchronization. Toggles at the start of the TDM data frame. This is driven by the master.
DATA
Data line, driven by one of the slave or master depending on
the data direction. There may be several data lines in
differing directions.
Currently supported TDM functions can be accessed via the i2s_tdm_slave.h header:
#include"i2s_tdm_slave.h"
I2S Common API
I2S Instances
The macro I2S_DATA_WIDTH may be set as a compile flag (e.g.
-DI2S_DATA_WIDTH=16) to alter the number of bits per word for both the I2S
Master and I2S Slave components; this defaults to 32 bits per word. This
value may be set to any value between 1 and 32. Correct operation of the I2S
components has only currently been verified at 16 and 32 bits per word.
The following structures and functions are used by an I2S master or slave instance.
enumi2s_mode
I2S mode.
This type is used to describe the I2S mode.
Values:
enumeratorI2S_MODE_I2S
The LR clock transitions ahead of the data by one bit clock.
enumeratorI2S_MODE_LEFT_JUSTIFIED
The LR clock and data are phase aligned.
enumi2s_slave_bclk_polarity
I2S slave bit clock polarity.
Standard I2S is positive, that is toggle data and LR clock on falling edge of bit clock and sample them on rising edge of bit clock. Some masters have it the other way around.
Values:
enumeratorI2S_SLAVE_SAMPLE_ON_BCLK_RISING
Toggle falling, sample rising (default if not set)
enumeratorI2S_SLAVE_SAMPLE_ON_BCLK_FALLING
Toggle rising, sample falling
enumi2s_restart
Restart command type.
Restart commands that can be signalled to the I2S or TDM component.
Values:
enumeratorI2S_NO_RESTART
Do not restart.
enumeratorI2S_RESTART
Restart the bus (causes the I2S/TDM to stop and a new init callback to occur allowing reconfiguration of the BUS).
enumeratorI2S_SHUTDOWN
Shutdown. This will cause the I2S/TDM component to exit.
Standard I2S is positive, that is toggle data and LR clock on falling edge of bit clock and sample them on rising edge of bit clock. Some masters have it the other way around.
The I2S component will call this when it first initializes on first run of after a restart.
This will contain the TDM context when in TDM mode.
Param app_data:
Points to application specific data supplied by the application. May be used for context data specific to each I2S task instance.
Param i2s_config:
This structure is provided if the connected component drives an I2S bus. The members of the structure should be set to the required configuration. This is ignored when used in TDM mode.
This callback will be called when a new frame of samples is read in by the I2S task.
Param app_data:
Points to application specific data supplied by the application. May be used for context data specific to each I2S task instance.
Param num_in:
The number of input channels contained within the array.
Param samples:
The samples data array as signed 32-bit values. The component may not have 32-bits of accuracy (for example, many I2S codecs are 24-bit), in which case the bottom bits will be arbitrary values.
This callback will be called when the I2S task needs a new frame of samples.
Param app_data:
Points to application specific data supplied by the application. May be used for context data specific to each I2S task instance.
Param num_out:
The number of output channels contained within the array.
Param samples:
The samples data array as signed 32-bit values. The component may not have 32-bits of accuracy (for example, many I2S codecs are 24-bit), in which case the bottom bits will be arbitrary values.
I2S_MAX_DATALINES
I2S_CHANS_PER_FRAME
I2S_CALLBACK_ATTR
This attribute must be specified on all I2S callback functions provided by the application.
structi2s_config
#include <i2s.h>
I2S configuration structure.
This structure describes the configuration of an I2S bus.
structi2s_callback_group_t
#include <i2s.h>
Callback group representing callback events that can occur during the operation of the I2S task. Must be initialized by the application prior to passing it to one of the I2S tasks.
The TDM component will call this after it first initializes the ports. This gives the app the chance to make adjustments to port timing which are often needed when clocking above 15MHz.
Param i2s_tdm_ctx:
Points to i2s_tdm_ctx_t struct allowing the resources to be modified after they have been enabled and initialised.
I2S_TDM_MAX_POUT_CNT
I2S_TDM_MAX_PIN_CNT
I2S_TDM_MAX_CH_PER_FRAME
TDM_CALLBACK_ATTR
This attribute must be specified on the TDM callback function provided by the application.
structi2s_tdm_ctx_t
#include <i2s_tdm_slave.h>
Struct to hold an I2S TDM context.
The members in this struct should not be accessed directly.
I2S Master
I2S Master Usage
The following code snippet demonstrates the basic usage of an I2S master device.
#include<xs1.h>#include"i2s.h"port_tp_i2s_dout[1];port_tp_bclk;port_tp_lrclk;port_tp_mclk;xclock_tbclk;i2s_callback_group_ti2s_cb_group;// Setup ports and clocksp_i2s_dout[0]=PORT_I2S_DAC_DATA;p_bclk=PORT_I2S_BCLK;p_lrclk=PORT_I2S_LRCLK;p_mclk=PORT_MCLK_IN;bclk=I2S_CLKBLK;port_enable(p_mclk);port_enable(p_bclk);// NOTE: p_lrclk does not need to be enabled by the caller// Setup callbacks// NOTE: See API or SDK examples for more on using the callbacksi2s_cb_group.init=(i2s_init_t)i2s_init;i2s_cb_group.restart_check=(i2s_restart_check_t)i2s_restart_check;i2s_cb_group.receive=(i2s_receive_t)i2s_receive;i2s_cb_group.send=(i2s_send_t)i2s_send;i2s_cb_group.app_data=NULL;// Start the master device in this thread// NOTE: You may wish to launch the slave device in a different thread.// See the XTC Tools documentation reference for lib_xcore.i2s_master(&i2s_cb_group,p_i2s_dout,1,NULL,0,p_bclk,p_lrclk,p_mclk,bclk);
I2S Master API
The following structures and functions are used to initialize and start an I2S master instance.
This task performs I2S on the provided pins. It will perform callbacks over the i2s_callback_group_t callback group to get/receive frames of data from the application using this component.
The task performs I2S master so will drive the word clock and bit clock lines.
Parameters:
i2s_cbg – The I2S callback group pointing to the application’s functions to use for initialization and getting and receiving frames. Also points to application specific data which will be shared between the callbacks.
p_dout – An array of data output ports
num_out – The number of output data ports
p_din – An array of data input ports
num_in – The number of input data ports
p_bclk – The bit clock output port
p_lrclk – The word clock output port
p_mclk – Input port which supplies the master clock
bclk – A clock that will get configured for use with the bit clock
This task differs from i2s_master() in that bclk must already be configured to the BCLK frequency. Other than that, it is identical.
This task performs I2S on the provided pins. It will perform callbacks over the i2s_callback_group_t callback group to get/receive frames of data from the application using this component.
The task performs I2S master so will drive the word clock and bit clock lines.
Parameters:
i2s_cbg – The I2S callback group pointing to the application’s functions to use for initialization and getting and receiving frames. Also points to application specific data which will be shared between the callbacks.
p_dout – An array of data output ports
num_out – The number of output data ports
p_din – An array of data input ports
num_in – The number of input data ports
p_bclk – The bit clock output port
p_lrclk – The word clock output port
bclk – A clock that is configured externally to be used as the bit clock
I2S Slave
I2S Slave Usage
The following code snippet demonstrates the basic usage of an I2S slave device.
#include<xs1.h>#include"i2s.h"// Setup ports and clocksport_tp_bclk=XS1_PORT_1B;port_tp_lrclk=XS1_PORT_1C;port_tp_din[4]={XS1_PORT_1D,XS1_PORT_1E,XS1_PORT_1F,XS1_PORT_1G};port_tp_dout[4]={XS1_PORT_1H,XS1_PORT_1I,XS1_PORT_1J,XS1_PORT_1K};xclock_tbclk=XS1_CLKBLK_1;port_enable(p_bclk);// NOTE: p_lrclk does not need to be enabled by the caller// Setup callbacks// NOTE: See API or SDK examples for more on using the callbacksi2s_callback_group_ti_i2s={.init=(i2s_init_t)i2s_init,.restart_check=(i2s_restart_check_t)i2s_restart_check,.receive=(i2s_receive_t)i2s_receive,.send=(i2s_send_t)i2s_send,.app_data=NULL,};// Start the slave device in this thread// NOTE: You may wish to launch the slave device in a different thread.// See the XTC Tools documentation reference for lib_xcore.i2s_slave(&i_i2s,p_dout,4,p_din,4,p_bclk,p_lrclk,bclk);
I2S Slave API
The following structures and functions are used to initialize and start an I2S slave instance.
This task performs I2S on the provided pins. It will perform callbacks over the i2s_callback_group_t callback group to get/receive data from the application using this component.
The component performs I2S slave so will expect the word clock and bit clock to be driven externally.
Parameters:
i2s_cbg – The I2S callback group pointing to the application’s functions to use for initialization and getting and receiving frames. Also points to application specific data which will be shared between the callbacks.
p_dout – An array of data output ports
num_out – The number of output data ports
p_din – An array of data input ports
num_in – The number of input data ports
p_bclk – The bit clock input port
p_lrclk – The word clock input port
bclk – A clock that will get configured for use with the bit clock
The following code snippet demonstrates the basic usage of a TDM slave Tx device.
#include<xs1.h>#include"i2s_tdm_slave.h"// Setup ports and clocksport_tp_bclk=XS1_PORT_1A;port_tp_fsync=XS1_PORT_1B;port_tp_dout=XS1_PORT_1C;xclock_tclk_bclk=XS1_CLKBLK_1;// Setup callbacks// NOTE: See API or sln_voice examples for more on using the callbacksi2s_tdm_ctx_tctx;i2s_callback_group_ti_i2s={.init=(i2s_init_t)i2s_init,.restart_check=(i2s_restart_check_t)i2s_restart_check,.receive=NULL,.send=(i2s_send_t)i2s_send,.app_data=NULL,};// Initialize the TDM slavei2s_tdm_slave_tx_16_init(&ctx,&i_i2s,p_dout,p_fsync,p_bclk,clk_bclk,0,I2S_SLAVE_SAMPLE_ON_BCLK_FALLING,NULL);// Start the slave device in this thread// NOTE: You may wish to launch the slave device in a different thread.// See the XTC Tools documentation reference for lib_xcore.i2s_tdm_slave_tx_16_thread(&ctx);
i2s_cbg – The I2S callback group pointing to the application’s functions to use for initialization and getting and receiving frames. For TDM the app_data variable within this struct is NOT used.
p_dout – The data output port. MUST be a 1b port
p_fsync – The fsync input port. MUST be a 1b port
p_bclk – The bit clock input port. MUST be a 1b port
bclk – A clock that will get configured for use with the bit clock
tx_offset – The number of bclks from FSYNC transition to the MSB of Slot 0
slave_bclk_pol – The polarity of bclk
tdm_post_port_init – Callback to be called just after resource init. Allows for modification of port timing for >15MHz clocks. Set to NULL if not needed.
This task performs I2S TDM slave on the provided context which was initialized with i2s_tdm_slave_tx_16_init(). It will perform callbacks over the i2s_callback_group_t callback group to get data from the application using this component.
This thread assumes 1 data output port, 32b word length, 32b channel length, and 16 channels per frame.
The component performs I2S TDM slave so will expect the fsync and bit clock to be driven externally.
A software defined SPI (serial peripheral interface) library that allows you to control a SPI bus via the xcore GPIO hardware-response ports. SPI is a four-wire hardware bi-directional serial interface. The components in the library are controlled via C and can either act as SPI master or slave.
The SPI bus can be used by multiple tasks within the xcore device and (each addressing the same or different slaves) and is compatible with other slave devices on the same bus.
The SPI protocol requires a clock, one or more slave selects and either one or two data wires.
SPI data wires
SCLK
Clock line, driven by the master
MOSI
Master Output, Slave Input data line, driven by the master
MISO
Master Input, Slave Output data line, driven by the slave
SS
Slave select line, driven by the master
All SPI functions can be accessed via the spi.h header:
The following code snippet demonstrates the basic usage of an SPI master device.
#include<xs1.h>#include"spi.h"spi_master_tspi_ctx;spi_master_device_tspi_dev;port_tp_miso=XS1_PORT_1A;port_tp_ss[1]={XS1_PORT_1B};port_tp_sclk=XS1_PORT_1C;port_tp_mosi=XS1_PORT_1D;xclock_tcb=XS1_CLKBLK_1;uint8_ttx[4]={0x01,0x02,0x04,0x08};uint8_trx[4];// Initialize the master devicespi_master_init(&spi_ctx,cb,p_ss[0],p_sclk,p_mosi,p_miso);spi_master_device_init(&spi_dev,&spi_ctx,1,SPI_MODE_0,spi_master_source_clock_ref,0,spi_master_sample_delay_0,0,0,0,0);// Transfer some dataspi_master_start_transaction(&spi_ctx);spi_master_transfer(&spi_ctx,(uint8_t*)tx,(uint8_t*)rx,4);spi_master_end_transaction(&spi_ctx);
This callback function will be called when the SPI master has asserted this slave’s chip select.
The input and output buffer may be the same; however, partial byte/incomplete reads will result in out_buf bits being masked off due to a partial bit output.
Param app_data:
A pointer to application specific data provided by the application. Used to share data between
This callback function will be called when the SPI master has de-asserted this slave’s chip select.
The value of bytes_read contains the number of full bytes that are in in_buf. When read_bits is greater than 0, the byte after the last full byte contains the partial bits read.
Param app_data:
A pointer to application specific data provided by the application. Used to share data between
Param out_buf:
The buffer that had been provided to be sent to the master
Param bytes_written:
The length in bytes of out_buf that had been written
Param in_buf:
The buffer that had been provided to be received into from the master
Param bytes_read:
The length in bytes of in_buf that has been read in to
Note: To guarantee timing in all situations, the SPI I/O interface implicitly sets the fast mode and high priority status register bits for the duration of SPI operations. This may reduce the MIPS of other threads based on overall system setup.
Initialize a SPI device. Multiple SPI devices may be initialized per SPI interface. Each must be on a unique pin of the interface’s chip select port.
Parameters:
dev – The context representing the device to initialize.
spi – The context representing the SPI master interface that the device is connected to.
cs_pin – The bit number of the chip select port that is connected to the device’s chip select pin.
cpol – The clock polarity required by the device.
cpha – The clock phase required by the device.
source_clock – The source clock to derive SCLK from. See spi_master_source_clock_t.
clock_divisor – The value to divide the source clock by. The frequency of SCLK will be set to:
(F_src) / (4 * clock_divisor) when clock_divisor > 0
(F_src) / (2) when clock_divisor = 0 Where F_src is the frequency of the source clock.
miso_sample_delay – When to sample MISO. See spi_master_sample_delay_t.
miso_pad_delay – The number of core clock cycles to delay sampling the MISO pad during a transaction. This allows for more fine grained adjustment of sampling time. The value may be between 0 and 5.
cs_to_clk_delay_ticks – The minimum number of reference clock ticks between assertion of chip select and the first clock edge.
clk_to_cs_delay_ticks – The minimum number of reference clock ticks between the last clock edge and de-assertion of chip select.
cs_to_cs_delay_ticks – The minimum number of reference clock ticks between transactions, which is between de-assertion of chip select and the end of one transaction, and its re-assertion at the beginning of the next.
The following code snippet demonstrates the basic usage of an SPI slave device.
#include<xs1.h>#include"spi.h"// Setup callbacks// NOTE: See API or SDK examples for more on using the callbacksspi_slave_callback_group_tspi_cbg={.slave_transaction_started=(slave_transaction_started_t)start,.slave_transaction_ended=(slave_transaction_ended_t)end,.app_data=NULL};port_tp_miso=XS1_PORT_1A;port_tp_cs=XS1_PORT_1B;port_tp_sclk=XS1_PORT_1C;port_tp_mosi=XS1_PORT_1D;xclock_tcb=XS1_CLKBLK_1;// Start the slave device in this thread// NOTE: You may wish to launch the slave device in a different thread.// See the XTC Tools documentation reference for lib_xcore.spi_slave(&spi_cbg,p_sclk,p_mosi,p_miso,p_cs,cb,SPI_MODE_0);
The CS to first clock minimum delay, sometimes referred to as setup time, will vary based on the duration of the slave_transaction_started callback. This parameter will be application specific. To determine the typical value, time the duration of the slave_transaction_started callback, and add 2000ns as a safety factor. If slave_transaction_started has a non-deterministic runtime, perhaps due to waiting on an XCORE resource, then the application developer must decide an appropriate CS to first SCLK specification.
The minimum delay between consecutive transactions varies based on SPI mode, and if MISO is used.
Note
Verified at 25000 kbps, with a 2000ns CS assertion to first clock in all modes.
structspi_slave_callback_group_t
#include <spi.h>
Callback group representing callback events that can occur during the operation of the SPI slave task. Must be initialized by the application prior to passing it to one of the SPI slaves.
XCORE ® -VOICE Solutions$$$Indices and tables£££structspi__slave__callback__group__t.html#indices-and-tables