GESBC-9302E kernel update to 2.6.24, and reboot fix

I recently updated the GESBC-9302 machine support in OpenEmbedded to include the 2.6.24 released kernel.  Also, Glomation has kindly provided me with a patch for the software reboot problem which I have integrated into the OpenEmbedded build for the GESBC.   The patch files are part of the OpenEmbedded dev branch or can be obtained here:

http://cgit.openembedded.net/cgit.cgi?url=openembedded/tree/recipes/linux/linux-2.6.24/gesbc-9302

Other related OpenEmbedded bits:

Future features to be integrated include RTC driver configuration, and support for loading Initramfs images from NOR flash.

Glomation GESBC-9302E Review

Recently, I’ve had the opportunity to work with a Glomation GESBC-9302E single board computer.  This SBC contains a Cirrus Logic EP9302 ARM processor and options for plenty of RAM and Flash memory which means you have many options for writing your applications including high level languages like C# and Python.  In this review, I’ll cover some of basic features of this board and where this board might be useful.

Why the GESBC-9302?

One of the most attractive features of the GESBC-9302E is cost.  The base model with 32MB of RAM, and no NAND flash costs $95 at quantities of 1 unit.  For a model with 64MB of RAM, and 128MB of flash, the cost is $120.  Looking at the board, its simplicity is very evident.  There is basically a CPU, flash, ram, a few linear power supplies, and a very limited amount of support circuitry.  There really is not much to it.

gesbc_9302e_1

We are helping a customer use the GESBC-9302E in a project where it is functioning as a gateway between a local Zigbee wireless network, and a server located on the internet.  The customer had originally considered using a Linksys NSLU2 for the gateway device during the alpha testing, but upon consideration it seemed like the GESBC-9302E is a better choice in this application for the following reasons:

  • the GESBC does not cost a lot more than a NSLU2
  • you can buy the GESBC with plenty of NAND flash.
  • the GESBC provides two serial ports
  • the GESBC provides many signals on headers than can be used with cables or a daughter card.

Having raw NAND flash on the board is a significant advantage in my opinion.  I’ve run into considerable hassles with consumer grade flash devices in the past (SD, USB Flash, etc).  Although there are industrial grade flash devices, they tend to be more expensive.  My preference at this point for reliable storage in an embedded Linux system is JFFS2 on a 64-128MB raw NAND flash device.  With 128MB of NAND flash, there is plenty of space for a full featured Linux distribution.

With 2 USB ports, plenty of general pupose I/O pins (GPIO), and lots of memory, there is not a lot you can’t do with the GESBC in your typical headless control or gateway type application.  Being able to interface to a number of USB peripherals, and support for high level languages makes this SBC+Linux ideal for implementing a device that may need to talk web services or other complex network protocols.

Software Support

Most of the GESBC functionality is fairly well supported in the mainstream Linux kernel.  Simply configure the kernel for the EDB9302 machine and most things will just work.  There are several pieces missing from the mainstream kernel for the GESBC that are useful: NAND flash support, and a change needed to make the Ethernet work.  For the NAND flash, I extracted a MTD driver from the Glomation kernel source code.  The Ethernet change is a small tweak to the platform files.  These changes are available for the 2.6.24 kernel in a public git repository at: http://dev.bec-systems.com/linux-2.6-gesbc-9302.git.

The GESBC uses Redboot for the bootloader.  No complaints with the bootloader so far — it does everything I need including loading images from a TFTP server.  The bootloader does not support writing images to NAND flash, so to load an OS image into NAND flash you typically have to boot a Linux OS from NOR flash, USB flash, or NFS root, and then use the userspace mtdutils to write an image into NAND flash.

Glomation provides a sample Linux image, but I prefer to use images generated by OpenEmbedded so that I have support for things like C# , Python, and thousands of other packages.  Support for the GESBC has been added to OpenEmbedded and I’m in the process of updating the kernel support to 2.6.24.

Summary

Overall I’m very pleased with this SBC.  Glomation has been very responsive to requests and has been a good company to work with. I still need to apply a fix for a known soft reboot issue with the 2.6 kernel and enable a driver for the RTC on the SBC.    Future articles will detail how to load Linux images built with OpenEmbedded on this device.

Compulab EM-X270 Review

The EM-X270 is a full featured computer board from Compulab (http://www.compulab.co.il/x270em/html/x270-em-datasheet.htm) designed for handheld/mobile applications.  The board includes options for about anything you might need in a portable system including a PXA270 processor, GSM radio, GPS, Wifi, Bluetooth, Audio, SD/MMC, USB, battery circuitry, and the list goes on.  The EM-X270 is designed to give you a big head start in designing a handheld computing device, and can significantly reduce time to market and design costs.  This article provides a high level overview of this product, how it might be used, and lots of photos.

EM-X270 Overview

The EM-X270 is a computer board about the size of a display you might find in a typical PDA.  While the size is probably a little large to be used in consumer devices like phones and PDAs, it seems like a nice size for slightly larger products used in vertical industries.

em-x270_1

As already mentioned, the EM-X270 includes about all the base features you might need in a mobile computing device.  An expansion connector is also included that allows you to easily add custom circuitry to the system.  The stack-up between the EM-X270 and an expansion board is shown below:

em-x270_2

The EM-X270 differers from most other embedded computer modules in that it is a little more application specified, and quite a bit more circuitry is included in the design such as power supplies, battery charging circuitry, radios, etc.

Why EM-X270?

Why are we interested in the EM-X270?  If all you need is a generic handheld computer, there are plenty of options from PDAs to off-the-shelf industrial handhelds.  However, if you need a handheld that provides some additional features not typically found in existing devices, then you can use the EM-X270 plus a custom baseboard and packaging to add your high value features.  Depending what you need to add, the expansion board can be a simple 4 layer board which is very easy to design and manufacture.  All of the complex PCB design and assembly is already done for you on the EM-X270 module.  This allows low to moderate volume manufactures design products where they would not otherwise have the time and budget for designing a full custom processor board.   Some example applications might be handheld products for the medical, industrial, and test and measurement industry.

Compulab’s pricing model is also very interesting.  Compulab will custom build EM-X270 modules with just the options you need.  The base price without any extra features (such as radios) is $122 for 1000 pieces.

The EB-X270 Evaluation Kit

Compulab provides a EB-X270 Evaluation kit for getting started with EM-X270 development.  Like their other Evaluation klits, is is very full featured and very well done.   An extender board is provided that adds Ethernet, JTAG, and several other useful connectors for development.

em-x270_3

The eval kit also includes all necessary cables, a display, battery, etc.

em-x270_4

Software for the EM-X270

Compulab provides software support for both Windows CE and Linux.  This includes very up-to-date support for kernel version 2.6.23.  EM-X270 machine support has also been included in the OpenEmbedded project for building full featured Linux images.  This combination will drastically reduce development costs compared to solutions provided by many other vendors.  All too often, single board computer vendors provide an ancient kernel, a hacked up root file system and call it a “Linux BSP”.  Actually, any time the term “Linux BSP” is used, you should probably be a bit wary.  A much more attractive solution for those of us building products using Linux is for processor and module vendors to say “our products are supported in the mainstream kernel and OpenEmbedded sources”.  Compulab is doing it right.

OpenEmbedded provides many interesting options for devices like the EM-X270.  One of the most interesting might be the ability to run the OpenMoko software on industrial devices.

Compulab provides a Linux demo image built with OpenEmbedded that runs the GPE Palmtop Environment.  The method for loading this image is interesting, and similar to the method I have been using for some time with systems based on the cm-x270 module (http://bec-systems.com/web/content/view/82/9/) in that it used mtd-utils from a running Linux system to erase and write to the NAND flash.  The bootloader in the EM-X270 is able to boot a kernel and rootfs directly from a USB Flash disk.  This “LiveDisk” image, as Compulab calls it, contains a small utility that programs an image from the USB flash disk directly into NAND flash.

As with any modern Linux based embedded system, there is a lot going on in a very small package.  Managing this complexity effectively requires some amount of experience.  Stay tuned for future articles that discuss using OpenEmbedded to build software for the EM-X270.

Additional Photos

em-x270_5

em-x270_6

em-x270_7

The Embedded WiFi module Quest

original article written in Sept, 2007

How does one implement WiFi functionality in vertical, low volume portable products?  This is a good question, and one I’ve been struggling with for the past 3 months.  I have a customer who is designing a portable data acquisition system based on a AT91SAM9260 processor, and needs WiFi functionality.  The fundamental problem is that no one has time to deal with low volume customers, and the task of implementing WiFi functionality is obviously complex.  At volumes of 5000 units a year or less, it makes a lot of sense to go with a WiFi module rather than trying to integrate the WiFi chipset on the board.  This article covers some of the options we have looked at and some of the possible solutions.

How is WiFi done on x86 systems?

With desktop Windows, most devices are well supported by device manufacturers.  With x86 Linux some of the devices are well supported by OSS drivers, but these tend to be PCI or USB devices.  Some of the devices that are not supported can still be used by running the Windows NDIS drivers inside of ndiswrapper, which allows you to run the Windows driver on a x86 Linux system, but this obviously does not work very well on non-x86 processors such as ARM.

WiFi solutions for portable, embedded systems

While there are many off-the-shelf solutions for PCs, Notebooks, etc, the current solutions are generally in the USB formfactor.  The ideal solution for portable systems running non-x86 processors are the small modules that connect to the host processor using the SDIO or SPI bus — like the modules you would find in newer cell phones and PDAs.  Older modules tend to use the Compact Flash interface, but these modules tend to be larger and are not packaged as nicely for deeply embedded applications.  So Ideally we want:

  • a modules with a SDIO or SPI interface
  • packaging options such as solder down for robust packaging

The scope of this article is limited to modules that implement a SPI or SDIO interface.  Why are we interested in modules?  Putting a WiFi chip down on a custom PCB is a lot of work.  Because the WiFi silicon includes a processor, if things do not work, it may be very difficult to debug.  There is also the issue of factory calibration which usually requires proprietary PC based tools that can communicate directly with the WiFi silicon.  If the WiFi silicon is buried in your product, getting all this to work can be a challenge.  The test equipment required for calibration is expensive.  Add to this the difficulty in getting support from anyone who makes WiFi silicon …

Complexity

It should be noted that WiFi modules are fairly complex devices.  They are typically based on a highly integrated IC that includes an ARM processor, Flash memory, and radio circuitry.  The fact that these modules include a significant amount of firmware on the module contributes significantly to the complexity of getting WiFi solutions to work properly.  Hopefully no bugs are encountered in the module firmware, because I can only imagine how difficult it would be to get these fixed for a small customer.   The fact that the module firmware is involved in many of the WiFi functions like authentication further complicates the problem with the abundance of authentication and encryption options available for WiFi.   The worst possible problem is the case where you must design an embedded WiFi system that must operate at the enterprise level in every environment.  It is one thing to design a system that works most of the time in most environments where WiFi is more of a convenience (like a Cell phone or PDA).  It is another thing to design an industrial grade system where it must work all the time and in all environments.

The Fundamental Problems with low volume embedded WiFi

I have never worked with a technology with so many dead ends as embedded WiFi modules.  Many of the companies I email or call never even bother to return emails and phone calls.  This is certainly true of the chipset manufactures.  From what I can gather, the fundamental problems are:

  • WiFi implementation is difficult; therefore, a manufacturer’s scarce resources are dedicated to high volume customers.  Low volume customers are a significant distraction.
  • WiFi companies are very busy right now — they are not hungry for business from small customers.
  • WiFi solutions are very competitive and highly proprietary.  Many companies are very secretive and will not release driver source code.  This is certainly true at the module firmware level.

There are many WiFi module manufacturers and resellers out there, but very few of them offer any type of driver solution for Windows CE or Linux other than binary modules that are supposed to work on perhaps one reference platform.  Sorry, this is not going to cut it.  So, with plenty of hardware available, the gating item is the availability of software drivers and knowledge of how to use the software.  It would seem to me that this is the perfect opportunity for chipset vendors to open source some drivers and develop vibrant support communities so that their modules can be used without a lot of hand-holding by the chipset vendors.

So What are the options?

Atheros AR6001

The Atheros AR6001 seems like a nice solution.  The driver situation for the AR6001 is progressing.  For Windows CE, there is an opensource WinCE 6.0 driver available at:  http://www.codeplex.com/CEWifiDriverAR6000/.  Unfortunately, this project appears to be a snapshot of code and does not have any significant amount of community activity or development.  Perhaps once more AR6001 modules are available, this will change.

For Linux, there are several options:

  • Atheros has released a driver that is available:  http://sourceforge.net/project/showfiles.php?group_id=186068.  There are several issues with this driver in that it is written for a SDIO stack that will likely never be part of the mainstream kernel.
  • A driver is being developed as part of the OpenMoko project.

Embedded Works supplies Atheros based modules and development boards: http://www.embeddedworks.net/wlan/oem_sdio_80211g.html.    AR6001 development boards are also available from Cardaccess: http://www.cardaccess-inc.com/products/index.php?a=wlan_sdio.

Marvell

The Marvell 88W8385 seems to be a very popular IC in WiFi modules available from a number of different companies.  OSS Linux drivers are in progress and are reportedly somewhat functional at this point, so it is probably just a matter of time before these devices are well supported in Linux.  This chipset/driver is often referred to as “libertas”.  The SDIO stack being developed for the Linux mainline (http://lwn.net/Articles/242747/) is where support for this device is being developed.

For Windows CE, it seems the only option is to get the drivers from Marvell, which is a very difficult and time consuming process — at least for low volume customers.  First you have to find a module reseller that can get you the source code from Marvell, and then the process takes about 3 months.

Other Options

There are a number of other WiFi Silicon manufacturers and module vendors, but the driver options from them seem to be very limited in the form of binary only drivers that will only work with certain processors/operating systems.  As noted before there is almost no hope of getting anyone to even talk with you if your volumes are low.  So your only hope is finding a module vendor that can do any driver work for a NRE fee, obtain the source from the silicon manufacturer, or use OSS drivers.

Summary

The Embedded WiFi situation is changing fast, so by the time you read this, it is probably already out of date.  We are currently at the phase where it is very difficult to deeply embedded WiFi in low volume products.  I expect this will change during the next year.  Eventually, WiFi modules re-sellers will figure out that supplying hardware is just one side of the equation.  As things get more and more complex, the software availability and support is becoming the gating item.

Many thanks to James Nahra, Dave Anders, and Erik Strack for sharing information and their WiFi experiences with me over the past few months.

The Embedded Industry Transformation

I just ran across an interesting article written by Doug Gaff titled “Is the embedded industry dead?“. The article points out the transformation that is happening in the “embedded” industry. For many systems, we are no longer constrained by minimal resources (http://bec-systems.com/web/content/view/67/9/). Embedded Systems are no longer isolated systems, but parts of larger systems.

A few of my own observations:

There is more to modern embedded systems than control and logic. We now have to worry about data. In the past the task might be to get a microcontroller to run a state machine, control motors, etc. Today, we need to present a calendar to the user through a web interface, collect information about the location of the Sun, upload statistics to a remote server, support a rich GUI application on a VGA color display. The control aspect is suddenly the small part of the problem, and managing data is often the primary concern.

Embedded system development now encompasses many disciplines including OS development, web services, advanced application development in high level languages, and the traditional hardware debugging and low level coding tasks. It is becoming less about writing code and more about integration of the right pieces. As Embedded Systems become more complex, access to specialized knowledge is becoming more important. There is just too much for one person (or even one company) to know. Companies developing competitive products in the future will be the ones who have access to experts.
Is your company taking advantage of this transformation?

Compulab cm-x270 PXA270 module review

We are in the process of supporting a customer who is designing a product that uses the Compulab cm-x270 “Computer-on-module”.  The cm-x270 is a small computer module 66x44x7mm that contains an Intel PXA270 ARM processor similar to those found in many PDAs or smart phones.  This review provides an overview of the cm-x270, why it was selected, and how we are using it.

Project Needs

For this project, we needed a low cost embedded computer that could drive a VGA (640×480) color display.  The graphical user interface for the device was fairly advanced, so we were also looking at a system that could run a modern OS with an advanced graphical toolkit.  After discussing the display requirements with the customer, we concluded that we needed a 32-bit processor with a LCD display controller.  As there was some custom circuitry needed, a configuration where the customer could design a custom baseboard and use an off-the-shelf computer module seemed optimal.  The schedule was also very aggressive and we did not have the time or budget to develop a full custom processor board.

CM-X270

After researching a number of computer modules, we chose the cm-x270 from Compulab.  The cm-x270 is a module that includes the following components:

  • Intel PXA270 ARM processor (up to 520Mhz)
  • 802.11b Interface
  • up to 128MB SDRAM
  • 128-512MB NAND flash
  • up to 4MB NOR flash
  • PCMCIA
  • PCI
  • Serial Ports
  • Touch panel controller (UCB1400)
  • Host and Slave USB
  • and other features …

For more details visit the compulab web site (http://http://www.compulab.co.il ).  The cm-x270 provides a very impressive number of features in a very small space.

Below is a picture of the cm-x270 mounted on the Compulab SBC-X270 baseboard with a VGA display.  The system is running the GPE Palmtop Environment built with OpenEmbedded.

cm-x270-002

Low Cost

Perhaps the most impressive feature of the cm-x270 is the cost.  When ordering in volume, Compulab will build the module with only the components you need.  For prototyping, you can order a standard offering with all components populated.  Starting a price of $50, it is the lowest cost module we found.  The configuration we will be using costs around $98 in 2K quantities (64MB DRAM, 4MB NOR Flash, 512MHz + 128MB NAND Flash + Audio/Touchscreen + RTC).  Eventually, we may be able to reduce some of the features and get the price down around $70 per module.

At this price, the cm-x270 provides a very cost effective way to develop a low volume product (a few thousand per year) with advanced features provided by the PXA270 processor.

Documentation

The bootloader and hardware design of the cm-x270 is proprietary.  Compulab does not supply schematics for the design or source code to the bootloader.  The documentation provided is adequate and provided all the information we needed to complete the design.  Compulab also provides schematics for their development baseboard, which is helpful.

Bootloader

The cm-x270 bootloader provides typical features found in an embedded processor bootloader, including: a serial console, NAND and NOR flash programming, download images from a tftp server, downloading images over a USB connection to a PC, and a number of other configuration commands.  The tftp download works very well and is fast.  The USB download is much slower, but is convenient for systems that do not have a network connection.  The USB download requires a windows PC with a specific version of Microsoft ActiveSync installed, which is unfortunate.  A more general solution that uses something like libusb (http://libusb.sourceforge.net/) instead of ActiveSync would be preferred as it could then run on both Windows and Linux computers.

Linux Support

Compulab provides a fairly complete port of the 2.6.16 Linux kernel for the cm-x270 in the form of a kernel patch.  Most of the peripherals we tested seem to work with the following exceptions:

  • The SD card driver is not preempt safe.  In this application, we need kernel preemption turned on to meet certain real-time requirements.
  • The Touch driver contains a small bug that does not allow applications using it to exit properly

Overall, support is pretty good and enabled us to get started with development quickly.

NAND Flash Driver

As the Compulab module only provides 4MB of NOR flash, using the NAND flash is a requirement when implementing a full featured Linux system with an advanced graphical user interface.  Compulab provides a binary module that is linked into a kernel driver to support the NAND flash.  This driver provides a block interface on which standard filesystems like ext2 can be used.  The NAND driver has worked fairly well, although we have seen several cases where we get I/O errors if we remove power to the device without shutting down the system properly.  The way to recover is to do a low level format the NAND flash.  This should not be an issue with this product as we will have a backup battery that will be used to power the system during shutdown if power is removed.  We may eventually move to an open source JFFS2 flash driver as time permits.

The proprietary NAND driver presents an additional wrinkle in that the driver cannot be linked into the kernel due to GPL licensing requirements.  One solution is to boot a small root file system from NOR flash that includes the NAND driver module, and then proceed to boot from NAND flash.

GNU/Linux Root File System

Compulab provides an example root file system (I think based on Debian) and suggests using scratchbox to build applications.  We looked at it briefly, but then decided to build our own rootfs using OpenEmbedded for the following reasons:

  • We can easily add additional packages from the extensive OpenEmbedded catalog as needed.
  • Allows us to have an rootfs independent of the module — we can more easily move to other processor modules or architectures in the future.
  • OpenEmbedded provides us with mechanisms for package management and building complete images that include our applications and tweaks to the OS.

For more information on OpenEmbedded, read our “Introduction to OpenEmbedded “.  With OpenEmbedded, we were able to quickly build a toolchain and a root file system that included the kdrive version of Xwindows and GTK+ libraries.  This rootfs fits in less than 32MB of flash disk space, which is much smaller than Compulab’s OS image.

We have also added machine configuration files to the OpenEmbedded project to support the cm-x270: http://www.openembedded.org/repo/org.openembedded.dev/conf/machine/compulab-pxa270.conf

Design Support

Compulab seems mainly interested in customers who are interested in purchasing modules in volume.  They do not sell low cost development boards such as LogicPD’s $500 board.  Compulab requires you to purchase a fairly high cost development system ($1900) in order to get started.  Part of this cost is to pay for Compulab’s support costs.  For companies who do not have experience designing systems with PXA270 class processors, we highly recommend getting some up-front help from someone who has experience with these systems.  Even though the module takes care of a lot of the design complexity, it helps to have a detailed understanding of the PXA270 and associated components, as you still need to interact with them.  Consulting companies like BEC Systems can provide assistance in the form of hardware design reviews and Linux OS support to accelerate projects and help avoid costly mistakes and delays.  As Compulab designs only the module, there is only so much they can do when you run into problems — as shown in the next section.

Issues Encountered

Overall, the project using the cm-x270 has gone well.  The only real challenge we have faced is a latch-up issue with the ITE PCI controller on the cm-x270 module.  It appears that this part is very sensitive to the characteristics of the 3.3V supply to the module.  After a considerable amount of debugging and testing, we were able to solve this problem with some filtering and slowing down the rise time of the 3.3V supply.  This is a good example of where a project can get difficult very quickly when things don’t work as expected.

Summary

So far, the cm-x270 has worked very well in this project.  With its low cost, it is a very attractive option for companies wanting to implement advanced functionality in their products, but don’t have the time, sales volumes, or experience to justify a full processor board design.  Feel free to post comments about your experiences using embedded computer modules.

Do you have an Exit Strategy?

Some time ago, eWeek published in interesting article by Scott McNealy named “Technology’s Barriers to Exit.  In this article, Scott makes an interesting point about considering the cost of technology.  There are three costs to consider:

  • cost of acquisition
  • ongoing cost
  • cost to exit

Scott states that most people analyze the first two costs fairly well.  What is not thought about as much is the the cost to exit — how do I move on to the next thing.  I think these basic questions apply very much to embedded technology decisions.

Most of the time we carefully consider our requirements, think about ongoing manufacturing costs, etc.  There may be some thought given to upgrading (the exit), but we encounter several difficulties when planning for the future with complex embedded systems:

  • we often can’t predict what technologies will be available in the future
  • if may be difficult to predict our product requirements in the future

As an example, we may have a product with an ethernet networking interface, and two years after product launch we have an opportunity that requires a wireless network interface.  How much is it going to cost to add wireless support to our product?  If the product is based on technologies that do not fundamentally support wireless networking, we are looking at very costly exit strategy.  Another example is application-specific, off-the-shelf solutions.  While these are great for many problems and provide a quick way to get something working, what happens when we want to upgrade?  Do we have to re-implement the entire system?

How does one hedge against the unknowns in the future?  One way is to try to anticipate every possible requirement and design it in.  I think this is bad strategy because you usually end up spending a lot of time working on things you don’t really need and the quality of the product suffers.  A better approach is make sure your product does what it needs to do well and is based on a flexible platform where significant parts of your design and software can be re-used in future versions.  This approach has many parallels to recent trends in software development and manufacturing such as Agile Software Development, and Lean Manufacturing.

Follow Scott’s advice and make technology selection decisions with the following questions in mind:

  • Is the technology based on open standards?
  • Are programming interfaces and protocols open, published, and been adopted by industry?
  • Is there a community process behind the technology (not just one company)?
  • Can the same technology be sourced from multiple companies?

The use of Linux in embedded systems is an excellent example of a technology that meets the above criteria.  You can develop your prototype on a x86 PC.  Move it to an embedded ARM single board computer (SBC) for the first production run.  Switch to a different SBC or design your own board when you run into new product requirements.  Sure, it will cost you more to develop the product initially due to the complexity of a system like Linux, but you now have options to expand and change your product while directly re-using the work you have already done.

But wait, as long as I write my application in C, it is portable — right?  Most embedded systems support development in C so I will just port my application to a new platform when needed.  C is a great language and the language is more or less portable between systems.  But as systems get more and more complex, embedded engineers find themselves spending less time writing original code and spending more time putting together pieces that already exist.  A modern embedded application is often little more than some glue between various libraries and system functions.  The fact that you can develop in C for a particular system does not really mean a lot.  What about the threading model, driver model, library APIs, etc.  I just spent a significant amount of time recently porting a large C application from a proprietary platform to new hardware platform as the original system was going obsolete.  It was a huge amount of work.  The driver model was completely different, APIs were different.  Changing platforms can be very expensive.

It rarely pays to be short sighted when choosing technologies for complex embedded systems.  The best strategy is to stay flexible and choose a platform with a future.

Opencores SPI Controller Review

Recently, I had the opportunity to use a SPI controller from Opencores in a customer design based on an Intel PXA255 processor and a Xilinx CPLD.  The customer did not want to change the processor or the CPLD family, but needed an extra SPI interface in the device.  I had my doubts, but mentioned I would look into a SPI controller from Opencores.  To my surprise, the core fit in a Xilinx CPLD and worked out very well.  We have since upgraded to a Altera MAXII CPLD.  This review details this effort. Continue reading “Opencores SPI Controller Review”

Embedded is a Service Industry

Linuxdevices.com has recently published an article titled: Let’s call Embedded what it is — a Service Industry (http://linuxdevices.com/articles/AT6989225710.html). In this article, Curt Schacker discusses how little growth there has been in the Embedded Systems industry. He claims that most embedded companies are trying to establish themselves as product companies, when the market is really a services market. Having worked for two Embedded companies in the past that went out of business, I think he is right. I have often heard the argument that service companies are too much work, they don’t scale, etc. But, to me services has always been a more natural fit for myself and an embedded systems company — one of the reasons BEC Systems exists.


Specialization or not

I have often wondered about specialization.  We can often find ourselves in ruts if we specialize too much.  But on the other hand, specialization can lead efficiencies in some cases.  In my work as a consultant, I often find myself in new situations where I have to learn fast.  As one of my associates states — problem solving is the key skill.  Kevin Barnes article Three theories on how to use developers efficiently is a good read about the subject.

Economics and Finding Outsourcing Talent

Economics In One Lesson by Henry Hazlitt presents an interesting and logical view of economics.  The book presents many ideas that seem right (like saving) and provides very logical arguments to back them up.  Even though the book was originally published in 1946, the topics covered seem very applicable to today (The Blessings of Destruction, Public Works Mean Taxes, Taxes Discourage Production, Credit Diverts Production, The Curse of Machinery, Spread-the-Work Schemes, Disbanding Troops and Bureaucrats, The Fetish of Full Employment, Who’s “Protected” by Tariffs?, The Drive for Exports, “Parity” Prices, Saving the X Industry, How the Price System Works, “Stabilizing” Commodities, Government Price-Fixing, What Rent Control Does, Minimum Wage Laws, Do Unions Really Raise Wages?, “Enough to Buy Back the Product”, The Function of Profits, The Mirage of Inflation, The Assault on Saving)

The book can be found:

The following article presents some of the realities of finding talent in places like India.  It is encouraging that the basic economic laws of supply and demand still work.

Finding coders on the subcontinent

How to Implement a Web Application Framework in an Embedded Linux System

As devices are increasingly more networked, an embedded web server is becoming a standard way for users to interact with and configure an embedded device using a standard web browser. As an example, most pieces of networking equipment (such as routers and wireless access points) are configured this way. There are many ways to implement a web server. One way is to just write a monolithic program that handles the requests and outputs HTML using print statements. Without a lot of planning, you usually end up with something where logic is intermixed with presentation. The disadvantage of such an approach is that it quickly becomes very difficult to maintain your web application as it grows and changes. The current best practice with web application frameworks (such as Ruby on Rails) splits a web application into 3 distinct components (Model, View, Controller — MVC) so that changes to one component can be made with minimal impact to others. This article details a solution used in a recent project to implement such an architecture using the following open source components: SQLite, Clearsilver, and Python. The challenge was to find a solution that performed acceptably on a resource constrained 130MHz ARM-Linux system.

Components Used

The requirements for the web application framework for this system are:

  • Footprint must be fairly small — less than 10MiB.
  • Must enable us to implement a clean MVC type architecture.
  • Must support a high level language like python for rapid development.
  • Includes a database.
  • Reasonable performance — less than 1 second to render a typical page.

After researching and testing several options, the following components were chosen to implement the framework:

  • Web Server: Cherokee (1MiB)
  • Model: SQLite  (290KiB)
  • View: Clearsilver  (170KiB)
  • Controller: Python (2-3MiB)

web_app_framework1

Total size for the above components is about 4MiB.

The following sequence occurs during a typical web transaction:

  • An HTTP request is received by Cherokee
  • If the request matches the URI for the web application, Cherokee forwards the request to the web application framework via CGI.
  • Clearsilver parses the HTTP headers (including CGI GET and POST parameters) and provides the data to python in a easy to access HDF format.
  • Python looks at the URI and dispatches the request to the appropriate function.  Data is extracted from the SQLite database and an HDF datastructure is created.
  • Clearsilver is called to render the appropriate template.  Data from the HDF datastructure is used in the template to provide the dynamic content in the page.
  • The rendered HTML is passed back to Cherokee, and is then returned to the user’s web browser.

Each component is discussed in more detail in the following sections.

Performance

One of the issues with modern web frameworks is that most require a lot of processing power.  Many frameworks are written in an interpreted language which tend to be not very efficient on embedded systems such as the 130MHz ARM-Linux system used in this project.  As most of the application development for this project is done in Python, I tried several other Python based solutions with the following results:

  • Django: takes 10 seconds to render a page
  • webpy: takes 5 seconds to render a page
  • Clearsilver/Python: takes about 1 second to render a page

Anything over 1 second is very slow for a web interface.  With fastcgi, webpy could probably have been sped up to be acceptably fast, but I don’t think there is much hope for frameworks like django on this type of system.  I have read of similar experiences trying to run Ruby-on-Rails on ARM-Linux systems.  As I’m currently just using a basic CGI interface, the Clearsilver solution could also be sped up significantly as well, if the Python portion could be kept running between requests with fastcgi or a similar mechanism.

Cherokee Web Server

A web application typically consists of a web server (like Apache) and a program that provides the dynamic web content.  Some of the functions of a web server are:

  • receives HTTP requests
  • handles the request by serving static files/images or routes the request to a program that outputs dynamic content
  • authentication
  • encryption (SSL, TLS)

There are many web servers that can be used in embedded devices.  Some of them are:

Cherokee was chosen for this application because it provides a good balance between the size and functionality I am looking for.  It is also included in the Openembedded (http://openembedded.org ) build system I am using.  Other systems will have different requirements — use the one that fits your application best.

Clearsilver

Clearsilver (http://www.clearsilver.net/ ) is the real gem that was discovered during this exercise.   Clearsilver is a language-neutral HTML template system written in C.  It is used as the templating system for many high volume sites such as Google Groups 2.  Clearsilver also provides CGI handling functions and bindings to several languages including Python.  The fact that it is written in C and is fast for high volume sites also makes it fast enough on slower embedded systems.  Because Clearsilver is written in C, the templating system is already fast.  Parts or all of the application can also be moved to C as needed to get the required performance.  Having a performance upgrade path is nice.  Clearsilver also forces a strict separation of application logic and presentation templates, which keeps things in line with the MVC architecture.

SQLite

SQLite (http://www.sqlite.org/ ) is a small C library that implements a self-contained, embeddable, zero-configuration SQL database engine.  SQLite works as expected and also has bindings to a number of languages including Python.  A SQLite database is just a single file that requires no configuration, making it very easy to use.  The database engine runs in the same process as the application using it, which increases performance because there is no context switch when running database operations.   Because SQLite does not have its own process, it does not handle concurrency as well as other databases such as MySQL, but in an embedded system you seldom have more than one process using the database.  SQLite provides some support for concurrency with reader/writer locks for the entire database.  If your application is not real busy, this is often adequate if you need two processes to share some data.

Python

Python is the glue that ties things together in this web application framework.  It sits between Clearsilver and the database.  The advantages of using a language like Python are rapid development and an extensive library.  Python is considerably slower than C, but there is always the option to move parts of the application into C as needed.

Openembedded Build System

How does one put together a system with all these components?  The answer is the Openembedded Build System (OE) (http://openembedded.org).  OE includes support for all of the components discussed in this article.

Summary

The combination of Python, Clearsilver, and SQLite provides a compelling solution for a web application framework in an Embedded Linux System.  The solution is reasonably small, performs well, and provides a path to improve performance if needed.  We also get a clean architecture to create clean, maintainable web applications.


About the author — Cliff Brake owns BEC Systems, a consulting operation that helps customers utilize modern computer technologies in their products.  BEC offers a range of services to make your embedded project a success including technology selection, development, troubleshooting, and training.  Please visit our website (http://bec-systems.com ) for more information and other free resources.