Linux | Academic OSS ELC

Finding Sources of Latency on your Linux System

Tue, 30 Jun 2020 00:00:00 +0000

In today’s computer systems the level of complexity has risen such that when a task or response to an event takes longer than expected, it is not easy knowing what the culprit is. The Linux operating system contains several utilities that allows a user to see where things may be held up. This talk will cover many of these utilities and briefly explain how to use them. From the hardware latency detector to the latency tracers. It will also discuss the new synthetic event interface that allows users to create a histogram on the time it takes any two events to occur.

I have previously discussed the latency tracers but they are soon to be obsolete, and the new synthetic event interface will be replacing them. There is still a lot more development going on with the synthetic events and this talk will briefly go over some of the new features that are yet to arrive.

Notes

Latency from Hardware
- System Mangement Interrupt (SMI)
Cache miss
branch prediction
Hyper-threading
Hardware Latency detector (HWLAT_TRACER)
fedora 31 not debian
tracefs

Measuring latency from interrupts

Linux on Open Source Hardware with RISC-V

Tue, 30 Jun 2020 00:00:00 +0000

Want to run Linux on open hardware? This talk will explore Open Source Hardware projects capable of that task, and explore how RISC-V and free software FPGA projects can be leveraged to create libre systems.

This talk will explore Open Source Hardware projects relevant to Linux, including boards like BeagleBone, Olimex OLinuXino, the Reform laptop and more.

I will also talk about the importance of the open RISC-V instruction set and free software FPGA toolchains. I will explain how myself and others at Hackaday Supercon teamed up to get Linux running on RISC-V core in the ECP5 FPGA badge. I will explain what LiteX is and how it enabled us to quickly build a SoC capable of running Linux.

Finally, I will explore the landscape of open source chip design projects and the Linux-capable RISC-V silicon chips on horizon for 2020.

Notes

What’s required to be “open hardware”

schematic
board layouts

RISC V: The instruction set for everything?

ISA=instruction set architecture
ARM requires commercial licening

What’s different about RISC-V

simple
clean slate design
modular
- extensively/specialization
- small standards base ISA, with multiple standard extensions
stable
community design

Ecosystem

opensource software
commercial software
open-source cores

Hardware

commercial cores
commercial core provisders
inhouse cores

Industry

Most people are just licensing cores from ARM
open source implementation
- BOOM, Rocket, PULP, SweRV

Chips

lowRISC

not-for-profit organisation
OpenTitan project with Google

SiFive

SiFive FE310 microcontroller
HiFive1: Arguino Compatible RISC-V dev kit

Linux on RISV-V

HiFive Unleashed
- best core if you want to run RISC-V Linux
- mostly just a proof of concept

Kendryte K210

8 mb of SRAM
$13
https://hackaday.com/2019/02/14/new-part-day-a-risc-v-cpu-for-eight-dollars/
Has MMU but too old to be supported by linux
will be suported in Linux 5.8

PolarFire

CORE-V Chassis - tapeout 2H 2020
Hard RISC-V with FPGA fabric

Open Source FPGA Tools

Linux on Badge

has ECP5 FPGA
- Supported by open source FPGA tool
Linux running on Supercon badge
2019 Supercon badget
32 MB of SDRAM to the Hackaday badge

Migen

HDL written in python
https://github.com/litex-hub/fpga_101

LiteX

Litex used to build cores, create SoCs and full FPGA designs
LiteX Hub - Collaborative FPGA projects around LiteX
Based on Migen

Linux on LiteX

https://github.com/litex-hub/linux-on-litex-vexriscv

What baords could you use?

ULX3S
- ECP FPGA dev board

Quickstart - FPGA

Fomu
- RGB LED
- MicroPython
- Verilog
- LiteX

Quickstart -No Hardware

Renode

Linux Stateless Video Decoder Support

Tue, 30 Jun 2020 00:00:00 +0000

While it has been under development for years, the support for video CODEC accelerators has gain a lot of traction in past year. A formal specification has now been merge into Linux Media subsystem and staging control APIs and drivers now exist. This allow for blob free HW accelerated decoding on popular SoC like Allwinner, i.MX8 and Rockchip.

In this talk, Nicolas will give an overview of the decoding process using such hardware accelerators along with an overview of the user space API and how it’s used within multimedia frameworks. Nicolas will also explain the relation between this and accelerators attached to modern GPUs. This presentation would not be complete without mentioning the development of FFMPEG and GStreamer native support and their major role in the development of the the new Open Source drivers.

This talk is addressed to multimedia enthusiasts and developers curious about video decoding and the upstream effort effort to make that available to users.

Notes

State-full Decoder

Begining of Linux Codecs

What is state-full?

stateful is a blackbox
blackbox gets bitstream

V4L2 M2M

output -> VPU -> capture

V4L2 output queue is used for the bitstream
capture queue is used for the decoded pictures
capability to seek and end stream

Pro/cons

pro: minimal per codec code needed
con: require a firmware
con: harder to multiplex

CODA Driver

Enabling iMX51 and iMX6
reversed enginerring from imx binary blobs

Begining of State-Less

2015

Rockchip
new type of CODEC hardware
Rockchip VDPU

What is State-Less

no processor
expose “accelarators”
1. inputs:
- reference
- bitstream
- parameters
1. accerator
1. outputs
- pictures
- may not be in order

What are GPU?

Crafting command stream is HW specific
only implement in userspace drivers (mesa)
commands are schedule by the GPU driver
multiple GPU hardware in the same application remains tedius

This is not what google did!

V4L2 M2M + Request (2015)

Instead, extend stateless: V4L2 M2M + Request

Added components

CODEC Controls
Request
Topology
Request API
- way to queue parameters, controls, and bitstream buffers
- added to media controller API

H.264

NALU Sequence
- SPS, PPS, IDR Slice, P Slice, B Slice
- P previous decoded pictures
- both past and future references
Annex B NALU
- Start-Code
- HDR, Paylod
AVCc NALU
- Size
- HDR, Payload

Decoding Process

MIA… Missed one slide
Fill SPS/ PPS, Decode Parameters, Slice parameters
Modify reference list
Decode the slide/frame
Do DPB management as per spec
- Display Picture management
Output frames that could be re-ordered

V4L2

Allocate a Request (an FD)
set per frame
queue a v4l2_buffer
queue the request
poll the request FD for completion

Continue in time

MediaTek VPU (State-full)

tiled output only (requires HW converter)

Qualcomm Venus

Stateful
lots of CODECS

Upstreaming Stalled

Could not settle on the Request/Job API
Low knowledge of CODEC decoing process by the linux-media maintainers
only one hardware to test the API design (Only rockchip)
No formal specific (not that state-full CODEC had one either)

Allwinner VPU support Kickstarter by Bootlin (2018)

Request API is finalized
MPEG2 Support in Staging
H.264 support was progressing (but only sliced based)
reverse engineered from binary userspace
VAAPI userspace drivers

2019

formal specification was merged
RK3288 driver was mainlined
H264, VP8, HEVC uAPI added as staging control API
RK3288 driver was renamed?

The Hantro Driver

I.MX8M Quad, using Hantro G1/G2
Registry compatible with the RK3288
Give email and you get specification

Hantro Company

Hantro Visibility Better
on2 technology
Google
- VP8 and VP9 royalty free
VeriSilicon

Other story of API rename

stmmac is an ethernet driver, that was thought to be STM design
later found to be DesignWare design, shared across numerous SoC
Since API was released it was unable to be renamed

Testing and Fixing

LibreELEC - just enough OS for KODI
- FFMPEG support
- bug fixing
- interlaced Content Support

2020

RK3399 JPEG, MPEG2, H.264, and VP9 support
Start having base class for Stateless
- DXVA2 and NVDEC support
VA V4L2 Request driver was abandoned

demo

linux kernel 5.7rc2
gstreamer has tool called ‘gst-build’ built on mesin
- allow you to run plugin
- gst-play-1.0
video layers are an overlay
prerolling.
three different streams
one two iMx8

Questions

performance stateless vs state-full
- have to wake up processor more
- more control over queuing
converting them to state-less is complex

PipeWire: The New Multimedia Service, Now Ready for Automotive

Mon, 29 Jun 2020 00:00:00 +0000

PipeWire: The New Multimedia Service, Now Ready for Automotive - Julian Bouzas, Collabora

PipeWire is the new emerging open source project that aims to greatly improve both audio and video handling on Linux systems, both desktop and embedded. It was recently adopted by Automotive Grade Linux as the core audio framework because its design’s flexibility makes it possible to address automotive requirements, replacing entirely previous solutions and addressing new use cases such as achieving ultra low latency with zero-copy media exchange, and allowing external session managers to define device policy logic.

In this talk, Julian is going to present the PipeWire project, how it evolved to overcome the automotive industry use cases, and what the current upstream status is. Julian will also present the WirePlumber policy management framework, which makes it easy to write use-case specific policy systems.

Notes

Intro

Fresh multimedia service for Linux

originally mean for video only: PulseAudio for Video (PulseVideo)
video capture
- cameras
- graphic sources (wayland, vulkan, OpenGL)
Audio playback and capture
- microphone and speaker
- bluetooth device

Why do we need Pipewire

Unified both pulseaudio and Jack audio servers
Permissions, support for container
Low Latency
- designed for small buffer sizes
- (1-2ms of latency)
Flexible: external session manager adaptable to any use cases
- can write your own session manager

Architecture and Design

Multimedia stack

DRM
PipeWire: Layer between Kernel and Applications
DRM -> Wayland compositor -> PipeWire
Kernel devices: DRM, V4L2, Bluetooth, Alsa
PipeWire Sessions Manager

Compatibilities APIs on top of PipeWirte

Alsa
Pulse AUdio
JACK

Architecture and Design

Modular with Plugin
Graph based like GStreamer: Nodes, ports and Links
Multi-Process
- external session manager configures and links the nodes
- daemon processes most of the data
Fully based on internal and simple plugin API library (SPA)
- extremely simple lightweight generic purpose multimedia libraries
- mostly header-only C library

Pipewire is very performant

Uses modern Linux Kernel APIs

Security

External session manager grants permission

Permissions

read - Visable, capture data
write, play data
execute: allow executing methods on object (setup format on nodes)

Adopted

Version 0.3.5 released in May 2020 and distributed in Fedora 32
Adopted by AGL

pipewire dot tool

Generates DOT graphs pw-dot pw.dot

WirePlumber Design

application that loads modules that were written with that library
allows external session manager
Plan to add PYthon/Rust bindings

Wire plumber Versions

v0.3.0 (2020 June) first version with desktop support
pavucontrol can still be used to control audio

Tutorial: Introduction to the Embedded Boot Loader U-boot

Mon, 29 Jun 2020 00:00:00 +0000

U-Boot is the universal bootloader used on a vast majority of embedded systems, development kits, products and so on. This session is an introduction into the U-Boot bootloader, including a hands-on part, and covers practical topics like identifying that the board is running U-Boot, accessing and exploring the U-Boot shell, including advanced scripting techniques to make life easier, obtaining information about the current hardware, accessing buses and storage and finally booting the kernel. Furthermore, since every embedded project has it’s a unique set of requirements, U-Boot customization topics are briefly touched at the end of the session.

Notes

SPL - Secondary program Loader
- Just enough of U-Boot to load something else
Tertiary program loader
- smaller than SPL
- uncommon

U-boot Shell

original shell
HUSH shell
similar to bourne shell
You can get help for ‘usb’: help usb

contact

irc mailing list

Memory access

support for read/write mem and reg
support for byte/word/long/quad
memory modification interactivly mm, mn
cp copy memory
cmp compare

U-boot environment

env key value storage
default to compile time default
can have defaults read from somewhere
can be saved in memory
printenv print all env
env print arch
env set fookey barvaluse
env set foonowempty similar to unset foonowempty
saveenv for env persistence
run for running scripts. chain commands with ‘;’
The script environment is evaluate when set. You can escape $ to evaluate later

Special variables

ver
stdin, stdou, stderr
loadaddr
filesize
bootargs
bootcmd
preboo
ipaddr, netmask, serverip, gatewayip - network settings,
ethaddr, eth1addr

Other commands

setexpr - environment manipulation multitool

logical and conditional expression

if, ||, &&
can’t use ! in conditional.
while

u-boot data loading

SD/MMC mmc, usb, sata, nand
Uiversal FS access ls, load
Squash FS not supported

Example with SD card

mmc rescan
mmc part
# device and partition
ls mmc 0:1
load mmc 0:1 $loadaddr ID.txt
md.b $loadaddr $filesize

Loading from network

U-Boot network stack is UDP-only (no TCP)

support for TFTP, NFS, DHCP/BOOTP
ping
tftp
dhcp (obtain settings then tftp)

env set
ping $serverip
tftp $loadaddr $serverip:somefile
dhcp $loadaddr $serverip:somefile

Load over serial

X/Y modem
Srecord
kermit protocol
loady

Screen

loady
ctrl-a:exec !! sb -T yourbinary.bin

Booting the kernel

(z)Image
- Linux binary (with decompressor)
- no protection gainst bitrot
- just set up registers and jump
- optional separate device tree
uImage (originally support, now legacy)
- Legacy since forever
- wrapper around binary
fitImage - multi component image
- based on device tree
- support multiple files
- configurable checksum algorithm per entry
- support digital signatures
- like tar files

Booting

boot commands: bootz, booti, bootm

env set bootargs console=ttyO0,115200
load mmc 0:1 boot/dtbs/4.9.../am335x.dtb
load mmc 0:1 0x880000 boot/dtbs/4.9.../am335x.dtb

Device Tree

Data structure describing the hardware
asyclic graph
can link to any items

snippet

`images { kernel@1 { data
data = /incbin/("./arch/arm/boot/dts/...dtb")
}}

mkimage -f fitimage.its fitImage

bootm $fitimageaddr
iminfo imxtract

fdt

fdt command can maniuplate flattend device tree

fdt addr
fdt resize
fdt print
fdt set

gpio

gpio input
gpio set
gpio clear
gpio toggle

i2c command

i2c bus
i2c dev
i2c md
i2c mw - write registers to I2C device
i2c probe
i2c speed

qemu

qemu-system-arm -M virt -bios u-boot.bin

Demo

env set bootargs root= rootfstype= rootwait console=ttyO0,115200
mmc rescan
load mcc...
load mmc...

Task 7 build u-boot in sandbox mode

ALlows you to run u-boot as user application

make sandbox_defconfig

Other task

env ask mac 'MAC address ?'