webOS Nation Forums >  Homebrew >  WebOS Internals > Next Generation Kernel Development and Packaging
Next Generation Kernel Development and Packaging
  Reply
Like Tree1Likes

 
Thread Tools Display Modes
Old 04/12/2010, 08:33 AM   #1 (permalink)
Developer
 
rwhitby's Avatar
 
Posts: 10,559
WebOS Internals has a charter of exploring the boundaries of low-level development for webOS-based devices, and then making the results of those explorations easily and safely installable for the mainstream webOS device user.

Now that the Preware and Save/Restore have mature 1.0 versions, it is time to turn our attention to the issue of next generation kernel development and packaging.

The Linux kernel is a very special item in the Linux operating system upon which webOS sits. The Linux kernel is always running, and is intended to be the sole interface to the underlying hardware. There is only ever one Linux kernel running on a webOS device, and it is the root upon which the security and functionality of the device is built. The kernel cannot be easily changed while the device is running (it requires a reboot to take effect), therefore users are forced to choose one kernel, and one kernel alone to meet their needs.

It is for these reasons that the processes for kernel development and kernel packaging are of critical importance. Some would argue that the processes for kernel development and packaging are even more important than the results obtained by the output of such processes ...

In this thread, we will outline the principles and criteria that we believe must be taken into account when developing and packaging replacement Linux kernels for webOS devices.
  1. The kernel must be built from publicly available source code.
    This is a key requirement of the GNU Public License (the license that the Linux kernel uses). It is also critical to ensuring the continual sharing of, and building upon, improvements to the kernel.
  2. The kernel build process must be transparent to all, and must be repeatable by any third party using a publicly available build system.
    This is a key requirement to ensure that the security and functionality of a replacement kernel can be trusted, and to ensure that no one single party can take a strangle-hold on kernel development (and that includes making sure that WebOS Internals does not have such a position).
  3. A replacement kernel must be uniquely identifiable.
    Users and utilities must be able to determine at run-time the precise origin and pedigree of a replacement kernel, without relying on a user's memory of the installation process, or installation database files on volatile storage.
  4. A replacement kernel must not differ from the stock kernel at boot.
    Any differences must be enabled by scripts, services or applications. New capabilities may be built into a replacement kernel, but they must be disabled by default. This requirement ensures a fail-safe scenario for kernels in the presence of OTA updates and partial erasure mechanisms. The requirement also allows multiple independent improvements to the kernel to be rolled into a single package and independently enabled at run-time as per the user's wishes.
  5. The stock vendor kernel must be available at all times as an installable option.
    The user must be able to safely revert to the vendor kernel, even in the presence of bugs in the uninstall mechanisms for replacement kernels.
  6. A replacement kernel must verify that the files it is replacing have not been previously modified in any way.
    This is to ensure that the new kernel is being installed into a known environment. This also applies when putting files back on uninstall. An exception to this principle is a recovery kernel, which is designed to forcibly overwrite any kernel files to ensure a correct recovery.
  7. A replacement kernel must verify that it is being installed on the correct version of webOS, and must refuse to install or correctly uninstall if the version is different.
    Since each webOS version is likely to kernel changes, and the rest of the operating system is likely to assume that those changes are in place, it is important for fail-safe characteristics for a replacement kernel to refuse to install or safely uninstall on a different webOS version.
  8. I'm sure we'll think of, and get good suggestions for, others as we go down this path ... and we'll update this list if we do.

Based on these principles, WebOS Internals is embarking on the development of a number of replacement kernels for webOS devices.
  • Palm Kernel
    This kernel is simply the standard unmodified vendor kernel binaries, extracted directly from the webOS Doctor, available as an installable option for reverting back to the standard kernel when all other uninstallation options fail. It will forcibly install over the top of any other kernel, removes any other installed WebOS Internals custom kernels, and does nothing when you remove the package.
  • UberKernel
    A collection of all the best (in the subjective opinion of the WebOS Internals kernel team) kernel improvements for the Palm Pre that comply with the above principles, all rolled into a single OTA-safe installation package.

As well as developing these kernels, we have also upgraded the rest of the open standard homebrew ecosystem to understand and work with replacement kernels.

To use the UberKernel, simply install the Govnah application and the UberKernel from Preware.

Those people who are interested in participating in the development of replacement kernel design and packaging solutions should make themselves known in the #webos-internals IRC channel.

Thanks,

-- Rod (on behalf of the WebOS Internals kernel team)
__________________
WebOS Internals and Preware Founder and Developer
You may wish to donate by Paypal to donations @ webos-internals.org if you find our work useful.
All donations go back into development.
www.webos-internals.org twitter.com/webosinternals facebook.com/webosinternals

Last edited by rwhitby; 08/16/2010 at 07:24 PM.
rwhitby is online now   Reply With Quote
Liked by jcmarcos likes this.
Old 04/12/2010, 08:56 AM   #2 (permalink)
Member
 
Posts: 116
"A collection of all the best (in the subjective opinion of the WebOS Internals kernel team) kernel improvements for the Palm Pre"

Such aaaaas... ?
Baryn is offline   Reply With Quote
Old 04/12/2010, 12:07 PM   #3 (permalink)
Member
 
GodShapedHole's Avatar
 
Posts: 1,831
At times like these, I feel that even if the worst happens and Palm went bankrupt tomorrow and all official updates or support for webOS ceased, the Homebrew teams would just take over almost seamlessly (at least from the viewpoint of the consumer). Thank you!
GodShapedHole is offline   Reply With Quote
Old 04/14/2010, 04:28 AM   #4 (permalink)
Developer
 
rwhitby's Avatar
 
Posts: 10,559
Quote:
Originally Posted by rwhitby View Post
Note that this thread is not about any particular existing replacement kernel for webOS devices. There are a number of other threads related to the current generation of overclocking replacement kernels being released by caj2008 and others, and you should use those threads if you are posting messages that are relevant to those replacement kernels - this is not the thread for such discussion.
I want to clearly state (so there is no confusion), that the kernels referenced in the post previous to this one are the first generation kernels currently in release by caj2008 and others, and as such are off-topic for this particular thread.

They are not the next generation kernels that are the topic of this thread, and do not comply with the design and packaging principles outlined in the first post of this thread.

Whilst many people are having great success in using these first generation kernels specifically for overclocking purposes, they will not be the basis of the next generation kernels that are the topic of this thread.

In particular, they do not comply with principles 2, 3, 4, 5, 6 and 7 - and are not representative of the three kernel types referred to in the first post of this thread.

-- Rod

Last edited by rwhitby; 04/14/2010 at 03:50 PM.
rwhitby is online now   Reply With Quote
Old 04/14/2010, 03:06 PM   #5 (permalink)
Developer
 
rwhitby's Avatar
 
Posts: 10,559
I'll repeat items from the first post, since they seem to have been missed:

Quote:
Originally Posted by rwhitby View Post
THIS IS NOT AN OVERCLOCKING THREAD. YOU WILL NOT FIND DOWNLOADABLE END-USER INSTALLABLE ITEMS HERE. THIS IS NOT AN ANNOUNCEMENT FOR THINGS THAT YOU CAN INSTALL.
...
It is for these reasons that the processes for kernel development and kernel packaging are of critical importance. Some would argue that the processes for kernel development and packaging are even more important than the results obtained by the output of such processes ...

In this thread, we will outline the principles and criteria that we believe must be taken into account when developing and packaging replacement Linux kernels for webOS devices.
The key point is that this is an open development process. Anyone who is interested in the performance of next generation kernels at this stage of the development process can participate in the development, and measure that performance themselves. There are no items that are published for release to end-users, so we would not want to interfere with the current first generation kernel release work by posting vapour-ware performance figures.

-- Rod
__________________
WebOS Internals and Preware Founder and Developer
You may wish to donate by Paypal to donations @ webos-internals.org if you find our work useful.
All donations go back into development.
www.webos-internals.org twitter.com/webosinternals facebook.com/webosinternals
rwhitby is online now   Reply With Quote
Old 04/14/2010, 03:29 PM   #6 (permalink)
Developer
 
rwhitby's Avatar
 
Posts: 10,559
In the last couple of days since we started this thread, we have added a couple more principles to the list:

A replacement kernel must verify that the files it is replacing have not been previously modified in any way.
This is to ensure that the new kernel is being installed into a known environment. An exception to this principle is a recovery kernel, which is designed to forcibly overwrite any kernel files to ensure a correct recovery.
A replacement kernel must verify that it is being installed on the correct version of webOS, and must refuse to install if the version is different.
Since each webOS version is likely to kernel changes, and the rest of the operating system is likely to assume that those changes are in place, it is important for fail-safe characteristics for a replacement kernel to refuse to install on a different webOS version.

A member of the kernel team has also found the open source code from later Linux kernel versions that supports the internal CPU temperature sensor, and we're working on back-porting that code to the version of Linux that Palm uses for the Pre.

Work continues first on the Palm Kernel, which is a kernel that anyone can use to return their device's kernel back to original webOS Doctor state.

-- Rod
__________________
WebOS Internals and Preware Founder and Developer
You may wish to donate by Paypal to donations @ webos-internals.org if you find our work useful.
All donations go back into development.
www.webos-internals.org twitter.com/webosinternals facebook.com/webosinternals
rwhitby is online now   Reply With Quote
Old 04/16/2010, 08:10 AM   #7 (permalink)
Developer
 
rwhitby's Avatar
 
Posts: 10,559
Time for an update on our progress towards robust OTA-safe next-generation kernel development and packaging processes.

At the start of this thread, we outlined a number of principles that we believe any replacement kernel should meet to be considered "well-formed".

Here is how the WebOS Internals kernels under development currently meet these criteria:
  1. The kernel must be built from publicly available source code.

    All WebOS Internals packages are built from publicly available source code, and the kernels are no different. The majority of the source code is found on our git.webos-internals.org site, but the build process clearly states exactly which files are used from which upstream URLs (including patches), and a copy of all files used is stored on our public source archive site for at least 5 years into the future.
  2. The kernel build process must be transparent to all, and must be repeatable by any third party using a publicly available build system.

    All WebOS Internals packages are built using an automated build system, the source code of which is also available publicly on the git.webos-internals.org site. This automated build system can be replicated by any third-party using this public source code, removing any reliance on a single-point of failure for homebrew kernel development and release. Full build logs of the latest and previous build runs are available for public scrutiny at all times, to ensure full transparency of the build process for correctness and security review. It is a key release criteria that the release candidate package be able to be built independently by a third party automatically from original source code.
  3. A replacement kernel must be uniquely identifiable.

    All WebOS Internals kernels are uniquely marked with a version (e.g. 1.4.1-6) and a package identifier (e.g. org.webosinternals.kernels.uber-kernel-pre). These values are built into the kernel at compile time, and can be retrieved at run-time by examining the /proc/version file. These values also match the values in the package metadata as displayed in Preware or the webOS user interface.
  4. A replacement kernel must not differ from the stock kernel at boot.

    All additional features of WebOS Internals kernels are deactivated or passive at boot, and then enabled at run-time via a supplementary patch, script or application. This allows a single unified modular kernel to be used for many different selectable features (since only a single kernel can be installed at any one time) instead of having multiple disjoint kernels for slightly different run-time settings.
  5. The stock vendor kernel must be available at all times as an installable option.

    The official Palm Kernel is available now in the WebOS Internals feed in Preware in the recovery section. You must be running Preware 0.9.36 or later to use this recovery kernel. This original GPL binary vendor kernel will assist users who wish to recover a device that has another third-party kernel installed if the removal instructions for that third-part kernel fail to correctly remove it from the device, or leave the kernel-related files in a state which does not match the original pristine state.
  6. A replacement kernel must verify that the files it is replacing have not been previously modified in any way.

    All WebOS Internals kernels perform sophisticated cyptographic-grade checksums on all files before replacement. If a kernel-related file is not in it's original pristine state, then the WebOS Internals replacement kernel will refuse to install (and will abort before modifying any files). A recovery Palm Kernel can be used in this instance to return all the kernel-related files to an original pristine state.
  7. A replacement kernel must verify that it is being installed on the correct version of webOS, and must refuse to install if the version is different.

    All WebOS Internals kernels validate the operating webOS version for compatibility before attempting to install or uninstall a replacement kernel. This is particularly important when a kernel is left in place across an OTA update (something which is not recommended for the existing released first-generation kernels, but which will be supported by the next-generation kernels from WebOS Internals).

Since the build system for the WebOS Internals Palm Kernel, Stock Kernel and Uber-Kernel is available publicly, anyone with the appropriate skills and knowledge can build the kernel from original source code and verify compliance with the above principles for themselves, and we encourage all kernel developers to do so as part of this development phase before we consider entering an alpha testing phase for these kernels.

-- Rod
__________________
WebOS Internals and Preware Founder and Developer
You may wish to donate by Paypal to donations @ webos-internals.org if you find our work useful.
All donations go back into development.
www.webos-internals.org twitter.com/webosinternals facebook.com/webosinternals
rwhitby is online now   Reply With Quote
Old 04/19/2010, 01:59 AM   #8 (permalink)
Developer
 
rwhitby's Avatar
 
Posts: 10,559
Quote:
Originally Posted by rwhitby View Post
A member of the kernel team has also found the open source code from later Linux kernel versions that supports the internal CPU temperature sensor, and we're working on back-porting that code to the version of Linux that Palm uses for the Pre.
The first two patches that we found didn't work.

But today I was able to extract a working OMAP temperature sensor from a Droid custom rom git kernel source repo, and successfully back-port it to the Pre kernel.

And it seems to be reporting internal CPU temperatures nicely ... 30degC at 500MHz idle, 38degC at 600MHz SDL game play.

-- Rod
__________________
WebOS Internals and Preware Founder and Developer
You may wish to donate by Paypal to donations @ webos-internals.org if you find our work useful.
All donations go back into development.
www.webos-internals.org twitter.com/webosinternals facebook.com/webosinternals
rwhitby is online now   Reply With Quote
Old 04/19/2010, 03:11 AM   #9 (permalink)
Developer
 
rwhitby's Avatar
 
Posts: 10,559
Note that we've also offered the new internal CPU temperature patch to caj2008 for him to use in the current generation of kernels:

http://forums.precentral.net/showthread.php?p=2390684

-- Rod
__________________
WebOS Internals and Preware Founder and Developer
You may wish to donate by Paypal to donations @ webos-internals.org if you find our work useful.
All donations go back into development.
www.webos-internals.org twitter.com/webosinternals facebook.com/webosinternals
rwhitby is online now   Reply With Quote
Old 04/20/2010, 12:52 AM   #10 (permalink)
Member
 
Posts: 109
To build the kernel from the webos-internals git repository, what cross-compile build environment is required?

Is it the same Scratchbox 2 environment running on Ubuntu 9.10 as described in the webos-internals wiki for the WebOS Internals PDK?

Thanks... plee3
plee3ac is offline   Reply With Quote
Old 04/20/2010, 01:12 AM   #11 (permalink)
Developer
 
rwhitby's Avatar
 
Posts: 10,559
Quote:
Originally Posted by plee3 View Post
To build the kernel from the webos-internals git repository, what cross-compile build environment is required?

Is it the same Scratchbox 2 environment running on Ubuntu 9.10 as described in the webos-internals wiki for the WebOS Internals PDK?

Thanks... plee3
No, it's just a cross-compile toolchain. "make toolchain" in build.git will download and install it.

-- Rod
__________________
WebOS Internals and Preware Founder and Developer
You may wish to donate by Paypal to donations @ webos-internals.org if you find our work useful.
All donations go back into development.
www.webos-internals.org twitter.com/webosinternals facebook.com/webosinternals
rwhitby is online now   Reply With Quote
Thanked By: plee3
Old 04/27/2010, 10:31 PM   #12 (permalink)
Member
 
mamouton's Avatar
 
Posts: 5,719
rob I installed the recovery kernel from Preware How can I make sure that there are no other remnants of other kernels?
__________________
My contributions to customizing the Palm Pre.

Browser Mega Mix|Email Mega Mix|Additional Sync Days|
Disable Various Phone Sounds|Priority Discard Buttons|
mamouton is offline   Reply With Quote
Old 04/27/2010, 11:10 PM   #13 (permalink)
Developer
 
rwhitby's Avatar
 
Posts: 10,559
Quote:
Originally Posted by mamouton View Post
rob I installed the recovery kernel from Preware How can I make sure that there are no other remnants of other kernels?
Well, it depends if you're talking about kernel files, or if you're talking about other binary patches that other groups put in the same package as kernel files.

For kernel files specifically, a recovery kernel will do the job.

For other binary patches and kernel installer services and cpu scaling applications, you really need to talk to the maintainers of those packages, or do a webOS Doctor if you don't get a good answer.

-- Rod
__________________
WebOS Internals and Preware Founder and Developer
You may wish to donate by Paypal to donations @ webos-internals.org if you find our work useful.
All donations go back into development.
www.webos-internals.org twitter.com/webosinternals facebook.com/webosinternals
rwhitby is online now   Reply With Quote
Thanked By: mamouton
Old 04/27/2010, 11:12 PM   #14 (permalink)
Member
 
mamouton's Avatar
 
Posts: 5,719
ok thanks the recovery kernel did the trick
__________________
My contributions to customizing the Palm Pre.

Browser Mega Mix|Email Mega Mix|Additional Sync Days|
Disable Various Phone Sounds|Priority Discard Buttons|
mamouton is offline   Reply With Quote
Old 05/29/2010, 07:19 PM   #15 (permalink)
Member
 
Posts: 2
Can anyone assist me in finding the kernel files???
OneBadDude is offline   Reply With Quote
Old 05/29/2010, 08:04 PM   #16 (permalink)
Developer
 
rwhitby's Avatar
 
Posts: 10,559
Quote:
Originally Posted by OneBadDude View Post
Can anyone assist me in finding the kernel files???
UberKernel and Govnah in Preware.

-- Rod
rwhitby is online now   Reply With Quote
Thanked By: pjjohn73
Old 06/08/2010, 02:31 AM   #17 (permalink)
Member
 
Posts: 316
Quote:
Originally Posted by rwhitby View Post
UberKernel and Govnah in Preware.

-- Rod
Thanks Rod, I managed to find the link already, and just installed it. So far so good. Was using the Super PreKernel. Worked just fine also, but I see it was not being supported any longer, so it is nice to find an updated kernal that is being supported. Thanks!
pjjohn73 is offline   Reply With Quote
Old 06/09/2010, 06:49 AM   #18 (permalink)
Member
 
Posts: 16
Thanks Rod for the GREAT Kernel!!!
ZsZolee is offline   Reply With Quote
Old 08/20/2010, 12:13 AM   #19 (permalink)
Member
 
Posts: 499
Quote:
Originally Posted by rwhitby View Post
UberKernel and Govnah in Preware.

-- Rod
I can't find it either, is there a specific feed I need to use? I even typed "UBER" in search as requested in PreWare and don't know how to search the .git repository. I was thinking it might have been pulled with 1.4.5 out, but that doesn't make much sense. Govnah was there. Thanks.
copasetic is offline   Reply With Quote
Old 09/11/2010, 03:17 AM   #20 (permalink)
Member
 
Posts: 7
oh herrooooo... just wonderin... i saved, and am running a profile i saved as a performance governor
min freq 1g
max freq 1g
could that be bad for anything but battery life?
noneuknow is offline   Reply With Quote
Reply

 

Thread Tools
Display Modes



 


Content Relevant URLs by vBSEO 3.6.0