Compulab cm-x270 kernel updated to 2.6.29 in OE

The cm-x270 kernel support in OpenEmbedded has just been updated to version 2.6.29.


Who is using OpenEmbedded?

Having used OpenEmbedded for a number of Embedded Linux projects over the past 5 years, it is interesting watching new users come on board.  The most recent announcement came from Koan that the KaeilOS distribution will be joining the OpenEmbedded project.  In the announcement, they summed up my thoughts well:

We at KOAN wanted to avoid waste of energy converging KOAN efforts into a stable and solid Open Source project like OE is.

“Avoid waste of energy” — isn’t this what OSS is all about?  Life is way too short to re-invent every wheel.  Considering OpenEmbedded has dozens of active contributors, it would be very difficult for any one company to replicate the functionality of OpenEmbedded. Although there are many ways to build embedded Linux distributions, and some tools do some things better than OE, there still does not seem to be any other embedded Linux build tool that fulfills the following requirements as well as OE:

  1. build from scratch on any x86 Linux system
  2. large number of contributors
  3. large number of packages
  4. support a large number of architectures
  5. Can modify images and integrate custom applications in a consistent way as part of the build system, and not a post processing hack.
  6. Open Source

Some other companies using OpenEmbedded include:

  • TI
  • AMD
  • Intel (was OpenedHand)
  • Gumstix
  • Bug Labs
  • OpenMoko
  • Atmel
  • Techsol
  • Compulab
  • Toradex
  • 4G-Systems
  • Dream Multimedia TV
  • M&N Solutions
  • emQbit
  • Siemens
  • Embedded Alley
  • and the list goes on …. (If you are using OpenEmbedded, please add your name to this list.)

I’m looking forward to seeing the KaeilOS improvements such as mdev, and boot time improvements.


Socketcan utils and test apps added to OpenEmbedded

The Socketcan test applications and utilities are now available in OpenEmbedded.  The socketcan kernel modules already exist in OE.  To build and deploy:

  • get the latest version of OE metadata
  • bitbake socketcan-utils-test
  • scp <oedir>/build/angstrom-2008.1/tmp/deploy/glibc/ipk/<machine>/socketcan-utils-test_0.0+svnr917-r0_armv5te.ipk root@<target IP address>:

And then on the target system:

  • opkg install socketcan-utils-test_0.0+svnr917-r0_armv5te.ipk
  • opkg files socketcan-utils-test

An impressive array of utilities!


Socketcan CAN-bus drivers added to OpenEmbedded

I just added a recipe to OpenEmbedded to build the Socketcan kernel modules from the socketcan SVN.  So if you are using the latest OpenEmbedded metadata, you can:

  • bitbake socketcan-modules
  • scp <oedir>/build/angstrom-2008.1/tmp/deploy/glibc/ipk/<machine>/socketcan-modules_0.0+svnr917-r0_cm-x270.ipk  root@<target IP address>:

and then on the target system:

  • opkg install socketcan-modules_0.0+svnr917-r0_cm-x270.ipk
  • opkg files socketcan-modules
Package socketcan-modules (0.0+svnr917-r0) is installed on root and has the following files:

Then to use the modules:

  • depmod
  • modprobe mcp251x
  • modprobe can-bcm

What is CAN?  Can stands for Controller-area network and is popular in industrial and automotive applications.  A convenient way to add CAN to your Embedded Linux system is with the Microchip MCP2515.  This device connects to a SPI bus which means it can be interfaced with a number of popular SOC’s such as the PXA270, OMAP3, 91SAM9xxx, etc.

More on Socketcan later …

BTW, this recipe really illustrates how easy it is to compile kernel modules outside a kernel in OpenEmbedded:

DESCRIPTION = "Socketcan kernel modules"
SECTION = "kernel/modules"
DEPENDS = "virtual/kernel"
PV = "0.0+svnr${SRCREV}"

SRC_URI = "svn://;module=kernel;proto=svn"

S = "${WORKDIR}/kernel/2.6"

inherit module

This recipe tells OE to download the source, cross-compile it against your target kernel build dir, and then package it for easy install on the device.  This is the way things should work — no messing around figuring out make options, kernel source paths, compiler env variables, etc.


Mono 2.2 RC1 added to Openembedded

I recently added Mono 2.2 RC1 to OpenEmbedded.  The last usable version of Mono in Openembedded was 1.2.6, so this is a big step up.  Lots of interesting things have been happening since then (like Full Static Compilation) that continue to make Mono interesting for embedded.  There is still plenty to do yet such as clean up the packaging, and update the other related recipes such as mono-xsp, and gtk-sharp.


Benefits of OpenEmbedded switching to Git

Recently the OpenEmbedded project has switched to the Git version control system.  This is good news for many reasons.  The obvious reasons are Git is faster than Monotone, handles branching better, has lots of nice features, larger user base, etc.  Monotone has served us well, but as new tools become available, it is time to change.  For those of us helping customers use OE for embedded projects, Git will make using OE much easier.  Companies want to be able to track OE with minimal effort and typically maintain a few minor tweaks to OE.  As most Embedded Linux projects now implement Git infrastructure for kernel development, the OE source tree can be maintained using the same infrastructure.  Branching is very natural in Git, so it is fairly easy to create a OE “topic branch” and periodically merge with upstream changes as needed.  The ability for easy repo hosting and branching allows OE users to lock down a version of OE for periods of project development, and still maintain a connection to the upstream OE repository for easily updating to new versions, or cherrypicking changes.  So a big thank you to all who helped make this change happen.


Linux Input Testing and Debugging

The Linux input layer has made a lot of progress in recent years.  When writing a new input driver (such as keyboard, trackball, etc), it is useful to be able to monitor input events using a test application.  This article describes two ways to accomplish this using kernel input debugging, and the evtest utility.

kernel input debugging

The Linux kernel includes some support for printing input events as debug messages.  To use, set the following in the kernel config:


Because the print messages are at the KERN_DEBUG level (7), we typically need to change the debug level:

echo 8 > /proc/sys/kernel/printk

At this point, you will see messages like the following on the console:

evbug.c: Event. Dev: gpio-keys/input0, Type: 1, Code: 108, Value: 1
evbug.c: Event. Dev: gpio-keys/input0, Type: 0, Code: 0, Value: 0
evbug.c: Event. Dev: gpio-keys/input0, Type: 1, Code: 103, Value: 1
evbug.c: Event. Dev: gpio-keys/input0, Type: 1, Code: 106, Value: 1
evbug.c: Event. Dev: gpio-keys/input0, Type: 0, Code: 0, Value: 0
evbug.c: Event. Dev: gpio-keys/input0, Type: 1, Code: 103, Value: 0
evbug.c: Event. Dev: gpio-keys/input0, Type: 0, Code: 0, Value: 0
evbug.c: Event. Dev: gpio-keys/input0, Type: 1, Code: 103, Value: 1
evbug.c: Event. Dev: gpio-keys/input0, Type: 0, Code: 0, Value: 0
evbug.c: Event. Dev: <NULL>, Type: 1, Code: 6, Value: 1
evbug.c: Event. Dev: <NULL>, Type: 0, Code: 0, Value: 0
evbug.c: Event. Dev: <NULL>, Type: 1, Code: 6, Value: 0
evbug.c: Event. Dev: <NULL>, Type: 0, Code: 0, Value: 0
evbug.c: Event. Dev: <NULL>, Type: 1, Code: 6, Value: 1
evbug.c: Event. Dev: <NULL>, Type: 0, Code: 0, Value: 0
evbug.c: Event. Dev: <NULL>, Type: 1, Code: 6, Value: 0
evbug.c: Event. Dev: <NULL>, Type: 0, Code: 0, Value: 0


The linux-input project ( includes a neat little tool named evtest that is very handy for testing input devices.  If you are using OpenEmbedded, simply:

bitbake linux-input

If you are not using OpenEmbedded, you can glean enough details about how to get and build the this package from the OE recipe:;a=blob;f=packages/linux-input/

Install the resulting package on your device.   To use, run evtest with a /dev/input/eventx argument:

root@machine1:~$ evtest /dev/input/event0
Input driver version is 1.0.0
Input device ID: bus 0x19 vendor 0x1 product 0x1 version 0x100
Input device name: "gpio-keys"
Supported events:
  Event type 0 (Sync)
  Event type 1 (Key)
    Event code 103 (Up)
    Event code 105 (Left)
    Event code 106 (Right)
    Event code 108 (Down)
    Event code 116 (Power)
Testing ... (interrupt to exit)
Event: time 1009.433605, type 1 (Key), code 108 (Down), value 0
Event: time 1009.433737, -------------- Report Sync ------------
Event: time 1010.735605, type 1 (Key), code 108 (Down), value 1
Event: time 1010.735740, -------------- Report Sync ------------
Event: time 1011.075586, type 1 (Key), code 106 (Right), value 1
Event: time 1011.075720, -------------- Report Sync ------------
Event: time 1011.103379, type 1 (Key), code 108 (Down), value 0
Event: time 1011.103550, -------------- Report Sync ------------
Event: time 1011.333924, type 1 (Key), code 108 (Down), value 1
Event: time 1011.334058, -------------- Report Sync ------------
Event: time 1011.339880, type 1 (Key), code 106 (Right), value 0

Notice that evtest lists the keymap when it is initially run which is very handy, and then lists key events as they occur.  The key descriptions are also included with events so you don’t have to manually decode the keycodes.


Compulab cm-x270 kernel update to 2.6.24

The kernel support in OpenEmbedded for the Compulab cm-x270 has been updated to version 2.6.24.  The 2.6.24 kernel opens up several possibilities including better real time and high resolution timer functionality, an improved SD Card driver, and a full SDIO stack with support for SDIO Wifi devices.  Also included is a patch to set the default framebuffer depth to 16 for several commonly used displays.  Xfbev really does not like when you give it a 8-bit framebuffer when it is expecting 16, and typically returns a very unhelpful error message like: “error: Invalid argument”.


Mono 1.2.6 added to OpenEmbedded

I recently updated the Mono recipes in OpenEmbedded to include the Mono 1.2.6 version.  Several critical bugs that were present in 1.2.5 have been fixed, and its looking very good.  More on this later …


C# on embedded ARM Linux systems is now practical

Update: Michael Dominic K. has kindly provided feedback on this article and has some additional points about Mono and embedded systems.

Mono has included support for ARM systems for some time, but with the release of Mono 1.2.6, and its inclusion in OpenEmbedded, it is now practical and easy to build and run C# applications on your everyday ARM Linux system.  Several ARM issues in 1.2.5 have been fixed, and thus far it seems the average program runs quite well.  In this article, we’ll explore what advantages C# offers over more traditional languages such as C or Python, and how OpenEmbedded simplifies adding C# support to your embedded system.

Why C#

Why is C# an attractive language for embedded Linux systems?  There are many reasons including memory protection and exception handling, native types, good threading support, extensive library, and developer productivity.  In the past, I’ve typically used C or Python for implementing large applications on embedded Linux systems.  This section will compare C# to my experiences with both languages.

A typical experience when implementing a large application in an embedded system is to deploy the system for field testing (before it is ready) and start getting reports that the application is crashing.  Somewhere in the mass of C code your team just wrote there is a stray pointer clobbering something it should not.  Of course this does not happen in the lab where you have access to the machine, but in some piece of industrial equipment many miles away.  And, you cannot reproduce the problem in the lab because with embedded systems, simulating a real world situation is challenging because it is difficult to simulate real I/O.  This is less of a problem with desktop applications where you are less dependent on external I/O.  Sound familiar?  Even if you can reproduce the problem on a machine you have access to (perhaps even remotely), the problem can be caused by a stray pointer corrupting data in a completely unrelated piece of code.  So while the debugger may tell you what got clobbered, you still don’t know exactly who did it.  If you have lots of time and a team to do lots of testing, this is less of an issue, but available resources and time to market sometimes don’t give you all you need to do the job right before it gets to the field.  One solution to this problem is use a language that has exception handling (I’ve used Python a lot in the past), and log all exceptions.  If something bad happens in the field, I ask for the log.  With the stack trace in the log, and some focused testing I can usually find the problem relatively quickly.  Add in the memory protection and things get a lot easier.  For small applications this is not so much of an issue, but lets face it, now that we have access to very capable hardware at a low cost, applications in embedded systems tend to get large and complex.  C# gives you exception handling and memory protection that makes problems much easier to find and debug; especially during field testing.

While its hard to beat Python or Ruby for developer productively and ease of use, there are performance issues like threading support, and support for static types that must be weighed in the balance.  With Python, everything is an object or a reference.  The checking is done at run time to make sure a variable is of the correct type.  With C#, this is done at compile time (much like C).  While it is generally more work to write code in C# than Python, the compiler tends to catch more errors for you.  There is probably also a performance advantage to being able to implement simple variables statically.  Threading support is another consideration and is often critical in embedded systems because you are usually tending to I/O, processing data, running a user interface, etc.  This type of problem is well suited to a threaded application.  Threading in Python is implemented using a global interpreter lock mechanism that is inefficient in some situations.  C# threading support is more similar to C.  While it may require you to do more manual locking than Python, it gives you more control.

When it comes to developer productivity, in my experience C# is more work to write than Python or Ruby, but less work than C.  So it seems to fall somewhere in the middle of the spectrum.

So, C# seems to fall nicely into the middle ground between C and very high level languages such as Python and Ruby.  It offers many of the advantages at both ends of the spectrum and seems to be a nice balance for programming embedded systems.


What is Mono?  From the Mono web site :

Mono provides the necessary software to develop and run .NET client and server applications on Linux, Solaris, Mac OS X, Windows, and Unix.

Mono includes all the necessary components to build and run C# applications on Linux systems including embedded ARM Linux systems.   The Mono source tree is fairly clean and is nicely segmented between native and managed code.  This is critical when it comes to cross compiling.  The Mono team has also stuck with traditional build tools such as autotools which makes the project a lot easier to cross compile for an embedded target.

Mono includes a very extensive C# library that is comparable to Microsoft’s implementation.  Most of what you might ever need is built right into the Mono class library; much like Python or Java.  With C or C++, you typically need to add a number of additional libraries to the system.

Mono Support in OpenEmbedded

OpenEmbedded includes comprehensive support for Mono including crosscompiling support, granular packaging of the Mono class library, and automatic run-time dependency generation for Mono applications.  In this section, we’ll see how OpenEmbedded can make your life a lot easier.

Mono is typically cross compiled in two steps :

  • The managed portions are generated by compiling Mono for the host machine architecture.  A very nice feature of the Mono build process is that it can bootstrap itself.
  • The native only portion of Mono is then cross compiled over top of the above output and the native host binaries will then get replaced with the cross compiled target binaries.

To implement this, there are two recipes implemented in OpenEmbedded.  The mono-mcs-intermediate is used to generate all the managed binaries, while the mono recipe builds only the native portion of Mono.  The first step when building the mono cross recipe is to extract the results of the mono-mcs-intermediate in the install directory and then proceed to overlay the native binaries produced by the mono recipe in that same directory.  As mentioned before, the use of standard tools like autotools makes all this work pretty well.  All of this is automated in OpenEmbedded and happens when you run “bitbake mono”.

One issue with embedded systems is you typically have limited flash/disk space.  So you only want to include that pieces of Mono in your target image that your application needs.  The OE build of Mono makes this very easy by packaging up the Mono class library into a number of different packages.  The dependencies between packages is automatically tracked so that if you add a Mono package to an OE build, its dependencies will automatically be included with no extra work on your part.  This mechanism uses the monodis tool to figure out what each Mono library provides and the dependencies of each library.  This saves a huge amount of very tedious manual work to have all this automated.

Building and Running your Mono application

Now that we have Mono running on our target system, there are a number of ways to build Mono applications to run on the target.  The quick-n-dirty method is to simply build the mono binary on any system, copy it to your embedded system and run it.  Building and running your applications is easy as Mono binaries are platform independent and will run on any system that supports Mono.  So, you don’t need to worry about cross compiling, toolchains, etc.

The quick-n-dirty method works well for getting started, experimenting, etc, but there are several advantages to integrating your Mono application into an OpenEmbedded build, even though you technically don’t need to worry about cross compiling.  The first reason is packaging.  A very simple OpenEmbedded recipe provides an easy way to automatically fetch your project from a separate version control system, build it, and package it properly into the OS image for your target.  It also gives you the ability to deploy updates to target systems using the ipkg package manager.   The second reason is automatic run-time dependencies.  If you inherit the mono OpenEmbedded class, all of the Mono components your application needs are automatically added to the image.  An example OpenEmbedded recipe for a Mono application is shown below:

DESCRIPTION = "My Application"

DEPENDS = "mono"

PV = "1.0+svnr${SRCREV}"
PR = "r1"

SRC_URI = "svn://;module=myapplication;proto=https"

S = ${WORKDIR}/myapplication
inherit autotools mono

The above recipe assume you have set up your application to be built with autotools, which is a good idea.  This page provides lots of good information on building Mono applications using autotools.  You can also look at a number of open source applications such as Tomboy for additional clues about how to use autotools with Mono.  For a simple application that I built, the following run time dependencies where automatically determined by OpenEmbedded: mono, libmono0, libmono2.0-cil, libmono-corlib2.0-cil. Each of these dependencies may have additional dependencies, and in this case a dozen or so mono packages ended up being installed.


As embedded systems become larger and more capable, we can expect that higher level languages such as C# and Java will start to become more common.  In the past, the focus on embedded systems programming was often efficiency, minimizing resource utilization, etc.  Now that very capable hardware is becoming cheaper, the focus in some systems is shifting to features and time to market.  Operating systems like Linux and languages like C# help us reach those goals and develop competitive products     quickly with small teams.  C# provides the features needed to rapidly develop, deploy, and debug complex applications that still perform reasonably well.  OpenEmbedded provides a way to manage builds, so that you can focus on real work instead of tedious build issues.


Linux 2.6.23 for the Compulab cm-x270

Updated 2.6.23 kernel patches for the Compulab cm-x270 are now available in OpenEmbedded:

The cm-x270 support in OpenEmbedded has undergone a lot of clean-up in recent weeks.  Now is a good time to give OpenEmbedded a try if you need a full featured Linux distribution for the cm-x270.


Howto load Openembedded on the Compulab cm-x270 computer module

There are quite a few people using (or would like to use) OpenEmbedded on the Compulab cm-x270.  The cm-x270 is a high performance, low cost computer module that can be used with a custom baseboard in embedded systems (see for a review).  I get a lot of questions on how to load OpenEmbedded on this system, so this howto attempts to detail a procedure that can be used to load OpenEmbedded.

The NAND Flash wrinkle

The Linux solution provided by compulab includes a proprietary NAND flash driver.  Most developers are interested in using standard Linux MTD drivers with the JFFS2 filesystem, but its not readily obvious how to flash a JFFS2 NAND image using the cm-x270 bootloader.  Fortunately, you can boot a small Linux image stored in NOR flash, and then use standard MTD utilities to then flash the NAND jffs2 image.

Creating the needed images

For this method, you need to generate two images: a small image to run from NOR flash, and a larger full featured image that will run from NAND flash.  Because you only have about 2.3MB of NOR flash, you want to use uclibc in the NOR image as it is much smaller.  A base Linux image that will boot into a shell using uclibc and contains a few utilities is about 1MB in size, which leaves plenty of space for custom apps should you need them in this partition.  The local.conf settings to build this image are:

MACHINE = "cm-x270"
DISTRO = "angstrom-2008.1"
ANGSTROM_MODE = "uclibc"
IMAGE_FSTYPES = "jffs2 tar cpio.gz"

At this point, “bitbake angstrom-minimal-image-with-mtd-utils”, and OE will generate a “minimalist-image-mtdutils-cm-x270.cpio.gz” image that can be programmed into the NOR ramdisk partition using the cm-x270 bootloader, and tftp download.  A kernel will also be built that should be programmed into the kernel partition.

To build the NAND image, simply remove the ANGSTROM_MODE line, and “bitbake angstrom-console-image”.  This will generate a “console-image-cm-x270.rootfs-summary.jffs2” that can be programmed into NAND flash.

Programming Images

Loading the software onto the cm-x270 is a multi-step process.  The first step is to the load the kernel image and the minimalist-image into NOR flash using the cm-x270 bootloader and tftp.  After this is accomplished, run the following commands to boot into the NOR flash rootfs:

  • ramdisk on
  • bootos

If successful, you will be presented with a prompt — log in as user root, and no password.

You now need to download the jffs2 NAND image into RAM on the cm-x270.  One way to do this is to use the tftp download feature in busybox:

tftp -g -r console-image-cm-x270.rootfs-summary.jffs2 <tftp server IP>

At this point you can program the jffs2 image into NAND flash:

  • flash_eraseall -j /dev/mtd3
  • nandwrite /dev/mtd3 console-image-cm-x270.rootfs-summary.jffs2

After programming completes, reboot the system and enter the following in the cm-x270 bootloader:

  • ramdisk off (this will disable the initramfs in NOR flash)
  • bootos

At this point, your system should boot into the main NAND rootfs.  With a little imagination, it should be obvious that you can do a lot of neat things with this type of setup by extending the functionality in the NOR rootfs including field upgrades and system recovery.


Compulab cm-x270 NAND flash eraseblock sizes

If you are having trouble mounting a JFFS2 file system on a Compulab cm-x270 module, you may have a device that has a NAND flash with 16KiB eraseblocks.  All of the devices I have personally used to date have had 128KiB eraseblocks, but I just helped a cm-x270 user through some flash issues and after much pain discovered the eraseblock size was 16KiB on his module.  A variable has been added to the cm-x270.conf file in OE to set the eraseblock size that can be overridden in your local.conf file if you are using OpenEmbedded to generate jffs2 images:;a=blob;f=conf/machine/cm-x270.conf;h=3c21c546b1320914c9b5d2c44e7332782fc58748


Mono support in OpenEmbedded and Openmoko packages

Original article published 2007-10-04

As of today, the Mono build in OpenEmbedded is working pretty well.  Mono support for ARM and other embedded architectures should add some interesting capabilities for embedded Linux devices.  Having written a number of large embedded Linux applications, I can really appreciate the advantages of high level languages for exception handling.  It is also interesting to note that Java support in OpenEmbedded is making progress as well (  Windows CE developers have already taken to the .net compact framework in a big way, so its nice to see similar capabilities for embedded Linux systems.

A feed has been set up for installing mono packages on your Openmoko phone.  Add the following line to a /etc/ipkg/*-feed.conf file:

src/gz mono-armv4t

And then:

ipkg update
ipkg install mono

The packaging is fairly granular, so look at the listing in the above URL to see a complete list of what is available.


The correct way to add packages to an OpenEmbedded Image

Update 2007-10-14: use IMAGE_INSTALL in image recipe

Update 2007-10-22: a few corrections, added full path for include and comments

Update 2010-06-07: use recipes instead of packages directory

As more and more OpenMoko developers are coming on-line, it is becoming obvious that my previous post ( about adding packages to OpenEmbedded could use some improvements.  This post suggested using the DISTRO_EXTRA_RDEPENDS variable, which was intended to only be used in distro.conf files.  A better approach is to create a custom image recipe that includes the packages you want.  This article covers how to create a custom image recipe.

What are the problems with the DISTRO_EXTRA_RDEPENDS approach?

As detailed by Marcin Juszkiewicz in his blog post “Why using of DISTRO/MACHINE variables in local.conf is wrong“, and from discussions with OE developers, the following issues surfaced:

  • You have to remember to rebuild task-base every time you update the DISTRO_EXTRA_RDEPENDS variable.
  • The image is no longer angstrom-console-image or whatever you have extended, so when asking for support, other developers do not realize exactly what you are building.  If it is a custom image, then that is obvious by the image name.
  • Changing DISTRO_EXTRA_RDEPENDS in local.conf requires that you bitbake reparse the entire recipe tree (which takes minutes).  Changing a custom image recipe only requires one file to be reparsed.

Creating a custom image file

Creating a custom image file is quite easy, and works much better than modifying DISTRO_EXTRA_RDEPENDS in your local.conf file.  A custom image is created by simply copying or including an image file closest to what we want to use, and then adding additional packages to IMAGE_INSTALL variable.  In the following example, we require and extend it with a few lines:

require recipes/images/

        xset xserver-kdrive-fbdev xinit \
        kernel \
        kernel-module-mmc-block \
        kernel-module-mmc-core \
        kernel-module-pxamci \
        kernel-module-ac97-bus \

export IMAGE_BASENAME = "my-custom-image"

Note, this method requires your custom image file to be in the same location as the original image recipe. If you use a bbcollections overlay, then you may want to copy the original recipe.

Note the use of a full path specified in the require statement.  This allows you to have your custom image recipe in a bbcollections overlay, and bitbake will still find the required recipe in the main OE tree.

Thats it!  With the many advantages to this method, there is no reason not to create your own image recipes — it works better.