Getting Started: The Yocto Project Overview
The Yocto Project is an open source collaboration project that helps developers create custom Linux-based systems for embedded products and other targeted environments, regardless of the hardware architecture. The project provides a flexible set of tools and a space where embedded developers worldwide can share technologies, software stacks, configurations and best practices which can be used to create tailored Linux and RTOS images for embedded devices.
The project provides a standard to deliver hardware support and software stacks, allowing the interchange of software configurations and builds. This could be between suppliers and consumers of hardware or to allow collaboration in and around a software ecosystem for example. The tools allow users to build and support customizations for multiple hardware platforms and software stacks in a maintainable and scalable way.
Historically the project grew from, and works with the OpenEmbedded Project which is the build system the project uses and shares components with.
The Yocto Project combines and maintains several key elements:
- The OpenEmbedded build system, co-maintained with the OpenEmbedded Project consisting of OpenEmbedded-Core and BitBake.
- A reference/example embedded Linux configuration used for testing (called Poky)
- Extensive testing infrastructure through our Buildbot based autobuilder
- Integrated tools to make working with embedded Linux successful:
- tools for automated building and testing
- processes and standards for board support definitions and interchange
- tools security analysis and license compliance, software manifests (SBoM) support in SPDX
There are many different open source components and tools within the Yocto Project umbrella.
Poky, the reference embedded OS is actually a working BUILD EXAMPLE which will build a small embedded OS with the included build system (BitBake, the build engine and OpenEmbedded-Core, the core build system metadata).
The build system is downloaded with Poky build instruction “files” called recipes and layers (defined below). You can alter, copy, extend, or use the Poky build specifics in any way you might need to in order to create your custom embedded Linux.
The Layer Model – The Key To Customization
Yocto Project has a development model for embedded Linux creation which distinguishes it from other simple build systems. It is called the Layer Model.
The Layer Model is designed to support both collaboration and customization at the same time. Layers are repositories containing related sets of instructions which tell the build system what to do. Users can collaborate, share, and reuse layers. Layers can contain changes to previous instructions or settings at any time.
This powerful override capability is what allows you to customize previous collaborative or community supplied layers to suit your product requirements.
Use different layers to logically separate information in your build. As an example, you could have a BSP layer, a GUI layer, a distro configuration, middleware, or an application. Putting your entire build into one layer limits and complicates future customization and reuse. Isolating information into layers, on the other hand, helps simplify future customizations and reuse. Use BSP layers from silicon vendors when possible.
Familiarize yourself with the curated (tested) YOCTO PROJECT COMPATIBLE LAYER INDEX . There is also the OpenEmbedded layer index which contains more layers but the content is less universally validated.
Components And Tools Maintained By The Yocto Project
There is a collection of components and tools that are used by the project to maintain and update the actual project. There are also those used directly by the project itself. Finally, other components and tools are used by developers to create their custom OSs. These components and tools are open source projects themselves, or metadata. Both are separate from the reference distribution and build system and most are downloaded separately.
Terms For Reference
Configuration Files: Files which hold global definitions of variables, user defined variables and hardware configuration information. They tell the build system what to build and put into the image to support a particular platform.
Recipe: The most common form of metadata. A recipe will contain a list of settings and tasks (instructions) for building packages which are then used to build the binary image. A recipe describes where you get source code and which patches to apply. Recipes describe dependencies for libraries or for other recipes, as well as configuration and compilation options. They are stored in layers.
Layer: A collection of related recipes. Layers allow you to consolidate related metadata to customize your build, and isolate information for multiple architecture builds. Layers are hierarchical in their ability to override previous specifications. You can include any number of available layers from the Yocto Project and customize the build by adding your layers after them. The Layer Index is searchable for layers within Yocto Project.
Metadata: A key element of the Yocto Project is the meta-data which is used to construct a Linux distribution, contained in the files that the build system parses when building an image. In general, Metadata includes recipes, configuration files and other information referring to the build instructions themselves, as well as the data used to control what things get built and to affect how they are built. The meta-data also includes commands and data used to indicate what versions of software are used, and where they are obtained from, as well as changes or additions to the software itself (patches or auxiliary files) which are used to fix bugs or customize the software for use in a particular situation. OpenEmbedded Core is an important set of validated metadata.
OpenEmbedded-Core: oe-core is meta-data composed of foundation recipes, classes and associated files that are meant to be common among many different OpenEmbedded-derived systems, including the Yocto Project. It is a curated subset of an original repository developed by the OpenEmbedded community which has been pared down into a smaller, core set of continuously validated recipes resulting in a tightly controlled and a quality-assured core set of recipes.
Poky: A reference embedded distribution and a reference test configuration created to 1) provide a base level functional distro which can be used to illustrate how to customize a distribution, 2) to test the Yocto Project components, Poky is used to validate Yocto Project, and 3) as a vehicle for users to download Yocto Project. Poky is not a product level distro, but a good starting point for customization. Poky is an integration layer on top of oe-core.
Build System – “Bitbake”: a scheduler and execution engine which parses instructions (recipes) and configuration data. It then creates a dependency tree to order the compilation, schedules the compilation of the included code, and finally, executes the building of the specified, custom Linux image (distribution). BitBake is a make-like build tool. BitBake recipes specify how a particular package is built. They include all the package dependencies, source code locations, configuration, compilation, build, install and remove instructions. Recipes also store the metadata for the package in standard variables. Related recipes are consolidated into a layer. During the build process dependencies are tracked and native or cross-compilation of the package is performed. As a first step in a cross-build setup, the framework will attempt to create a cross-compiler toolchain (Extensible SDK) suited for the target platform.
Packages: The output of the build system used to create your final image.
Extensible Software Development Kit (ESDK): A custom SDK for application developers that allows them to incorporate their library and programming changes back into the image to make their code available to other apps developers.
Image: A binary form of a Linux distribution (operating system) intended to be loaded onto a device.
The General “Workflow” — How It All Works
- To begin, developers specify architecture, policies, patches and configuration details.
- The build system then fetches and downloads the source code from where ever specified. The project supports standard methods such as tarballs or source code repositories systems such as git.
- Once downloaded, the sources are extracted into a local work area where patches are applied and common steps for configuring and compiling the software will be run.
- The software is then installed into a temporary staging area where the binary package format you select (deb, rpm, or ipk) will be used to roll up the software.
- Different QA and sanity checks are run throughout entire build process.
- After the binaries are created, a binary package feed is generated which is then used to create the final root file image.
- The file system image is generated.
When using Yocto Project, this workflow will change depending on what components and tools are actually being used.
Development Environment — Host Setup
Most developers use a Linux development host as described in the Yocto Project Quick Build.
Reference Embedded Distribution (Poky)
“Poky” is the name of the Yocto Project’s reference distribution or Reference OS Kit. It contains the build system (BitBake and OpenEmbedded Core) as well as a set of metadata to get you started building your own distro.
To use the Yocto Project tools, you can download Poky and use it to bootstrap your own distribution. Note that Poky does not contain binary files – it is a working example of how to build your own custom Linux distribution from source.
Features
Apart from inherent capabilities, the Yocto Project has project features which are introduced in one release, and generally expanded on and completed over the course of a few releases. In-depth information about the features can be found in the “readme” files associated with releases and tools.