How to capture source changes to an OpenEmbedded package

One task that is a often confusing to new OpenEmbedded users is how to capture changes to the source code for a package/recipe.  First, lets review the progression in tools use to capture source code changes.

In days of yore, developers often used gnu diff to capture modifications to a source tree.  The typical practice was to create two source trees: a pristine, and a dirty tree.  Modifications were made in the dirty tree, and periodically gnu diff was run between the two trees to capture the changes.  Then came quilt which allowed you to capture changes without a pristine tree — kind of a cheap version control system.  The typical flow was:

• cd <source dir>
• quilt new my_changes.patch
• quilt add file1.c
• edit file1.c
• quilt refresh (now my_changes.patch records the edits to file1.c)
• debug
• edit file2.c
• (oops forget to quilt add file2.c — now what ….)
• restore file2.c from pristine source
• quilt add file2.c
• edit file2.c
• quilt refresh
• (now my_changes.patch records edits to file1.c and file2.c)

As you can see, things are much better than plain diff, but you still have to keep track of the files you edit.  With git, we can do essentially the same things, only git keeps track of the files you edit, plus it is easier to organize patch files.  The flow with git is something like this:

• cd <source dir>
• git init
• git add *
• git commit (now the entire source tree is recorded by git)
• edit file1.c
• edit file2.c ….
• git status (shows you files that were modified)
• git add file1.c file2.c
• git commit
• git log (shows you changes you have made)
• git format-patch HEAD~1 (outputs the patch created by the last commit)

Once your source code is in git, then you can do all kinds of fancy things like merging, re-ordering, and re-naming patch files with the git-rebase –interactive command.

One of the uses of this technique is when you need to record a small change to a package source in a build system such as OpenEmbedded, and apply that change for subsequent builds.  In this case you might do something like the following:

• bitbake -c devshell <recipe_to_patch>
• in devshell capture changes in git as shown above to create my_changes.patch
• cp my_changes.patch <oe_tree>/recipes/<recipe_to_change>/files/
• edit <oe_tree>/recipes/<recipe_to_change>/<recipe_to_change>.bb
  • tell bitbake to apply the patch when building the recipe: SRC_URI += “file://my_changes.patch;patch=1”
  • increment the PR variable so that bitbake knows to rebuild the recipe.

Figure 2 illustrates the above steps.

While the above steps might seem overly complex, you must consider what you are trying to do.  Building a Linux distribution is not a simple task.  We are dealing with large amounts of source code from many different projects.  Putting all this source code in one tree is not an option, because then we would be defeating the very reasons open source is attractive in the first place: flexibility, largly decoupled projects, advanced technology, fast bug fixes, many options to choose from, developer support, etc. Each open source project must remain a self contained project that is largly decoupled from other projects so it can develop at its own pace.  It is for these reasons that open source technologies are used to give products a competitive edge.  So, like any complex process or task, there must be some understanding of how it works, an appreciation for what it can do for you, and an investment in tools and techniques required to be effective.

update 2023-09-23: recipetool and devtool are updated tools that can be used to make and capture changes.