PlayDeb base is ready

Hello,
we have done some major work on apt-portal/playdeb on the last days.
We are using a custom Model-View-Controller framework using elixir for the Model, mako for the View and CherryPy for the Controller .
The core site design is done and the register/email auth/login functions are already working.

You can test the current design/code on your system by following this instructions.

GetDeb/PlayDeb Testing Repository

Hello,
I have been setting up the test repository, the testing repository will be used to test packages before they get into the main repository.
You SHOULD NOT use this repository in a production system.

To setup the repository check the instructions at:
http://wiki.getdeb.net/TestingRepository

Please note there is no main at this time, the only option right now is to be a tester :)

Working on GetDeb v2

It has been 3 years since getdeb was started. During that time, we developed our own content management system using PHP+MySQL which has been sufficient for the hundreds of application that we have provided.

With the increasing maturity of the project though, we have defined a set of improvements to be implemented on several areas:

Organization
- Game packages and related contents will be moved to the PlayDeb project - the games category has always been a super category and serves a specific audience.
Security:
- Automatic package updates
- Package authenticity and integrity
Usability:
- Single click installs for multi package software

These changes cannot be accomplished with the current getdeb code and structure, so we will need to develop a new set of tools and a new web content management system. Because we are a small spare time working team these changes, it will take several months - but we have already some work done.
The new architecture will use components developed in self contained open source projects with sufficient documentation and generic interfaces so that they can be used for any open sources projects in general. The distribution will be done using an APT repository with 2 components, apps and games (getdeb/playdeb) - along with apt-url for the web integration.

What needs to be done:

Package building & test repository populating (https://launchpad.net/debfactory)
This is a set of scripts required for managing the source packages and building. Most of the scripts are already developed, however they need to be improved to support multiple components (apps/games) - Task 1
The scripts that are needed:
To be run on the building server:
check_incoming.py: Verifies uploaders authenticity and authority, checks the source files integrity and moves the files to the pre_build queue
check_pre_build.py: Checks the pre_build_queue for packages that need to be build, moves them to the post_build queue after a sucesfull build
To be run on the master server:
check_post_build.py: Checks the post_build_queue and moves packages to the testing reprepro repository
To Be Defined: Promoting a package from testing to final will be done manually (archive admin) ?

Package & Applications information importing and user presentation (https://launchpad.net/apt-portal)
APT-Portal will provide the web frontend for the repository. It will be a generic APT repository web content management system, and at the end we will have 2 apt-portal instances for getdeb and playdeb. The only difference for those should be configuration, theme and plugins. - Task 2

APT-Portal - GetDeb - Task 3
Theme & Plugins: To be defined

APT-Portal - PayDeb - Task 4
Theme & Plugins: To be defined - The major work here was already done (https://code.launchpad.net/playdeb), it will need to be ported to APT-Portal theme/plugins

Current status:
Task 1 - I am working on it
Task 2 - Some initial db import scripts were created, we need task 1 done and a few packages in the repository to do the real development here
Task 3 - Static HTML/CSS with the expected design could be done before Task 2 is finished, required for the real implementation
Task 4 - Depends on Task 2

If you're familiar with python / php / mysql and would like to help us, get in touch at #getdeb on irc.freenode.net.

How to build and test packages for GetDeb

This article is addressed to those planning to build packages for GetDeb or which just want to understand how we do it.
Building a package may require a lot of development libraries which are not needed for your regular desktop use - and you also probably don't want to overwrite current working applications with testing packages. For this reason, the recommended procedure is to use an isolated minimal system install that you can use for both building and testing.

This can be achieved by using a virtualization application, however we will be presenting a much simpler and lighter approach using schroot and sbuild.

Getdeb's debfactory is a set of tools designed to assist in package building for developers. The first thing you will need is to get the latest version of the debfactory tools, you can retrieve them with:
bzr co lp:debfactory

Now we need to create the schroot images that will contain the minimal packages for the releases we plan to build for.
To create the schroot images execute:
sudo debfactory/bin/schroot_build.py -b
The -b indicates we will be using the schroots not only for testing but also for building, sbuild will be installed.

You will need to add yourself to the sbuild group with:
sudo usermod -a -G sbuild $USER
Re-login for the change to take effect, you can see the groups your user belongs to with the "id" command.

Now let's manually build a package for isomaster
# Get the isomaster source and debian building rules from getdeb with:
dget -ux http://archive.getdeb.net/getdeb/ubuntu/jaunty/is/isomaster_1.3.5-1~getdeb1.dsc
# Switch to the schroot for the building with:
schroot -c jaunty.i386 -p

# Let's go into the source directory and install the build dependencies defined for the package
cd isomaster-1.3.5
sudo ~/debfactory/bin/satisfydepends
# Now let's build it
debuild -us -uc
# You should now have the file ../isomaster_1.3.5-1~getdeb1_i386.deb that you can install, but first let's exit the schroot with the exit command. All the changes done on the schroot except for /home will be lost, we want to enter the schroot again in it's initial state so that we can check if the package will install and run without problems on a minimal system.
schroot -c jaunty.i386 -p
# Now let's try to install the package
sudo dpkg -i isomaster_1.3.5-1~getdeb1_i386.deb
# Install missing dependencies
sudo apt-get install -f
# Run the application for testing
isomaster
# It's is everything ok ? so just exit

# What if you wanted an automated build after the dget ? just use sbuild:
sbuild -c jaunty.i386 isomaster_1.3.5-1~getdeb1.dsc

That's all for this article - I will present some other getdeb tools and processes in the future.

Preparing the getdeb/playdeb repository

Hello,
after a lot of reading and testing I have acquired the skills required to establish the future APT repository for both getdeb and playdeb. Here is the plan:

The implementation will be done using a standard apt repository structure, using a single archive pool with two distributions(per release) and two components.

Distributions
jaunty-getdeb-testing
Packages automatically imported after a successful build on ABS, it will be used for testing packages before promoting them into the -getdeb repository.
It will be strongly discouraged to use this repository on production/stable systems.

jaunty-getdeb
Packages promoted from *-testing, ready for global distribution.

Components
apps - application packages (to be available on www.getdeb.net)
games - game packages (to be available on www.playdeb.net)

Getting ready for Jaunty

Hello,
the new Ubuntu release (9.04 Jaunty) will be available on the next days so it's time to update the support policy for the next cycle.
The current policy of supporting the current plus the LTS version had a very positive result, there are clearly two main segments of users: the "current" users which always prefer the current version and the "LTS" users which like to update their packages but keeping an LTS kernel/core tools.
We will keep the current policy, for the next 6 months we will be providing packages mainly for Jaunty and supporting Hardy in a best effort basis.

Effectively today Ubuntu 8.10 Intrepid is set "Available", the archive will be kept online but no updates will be applied.
Ubuntu 9.04 Jaunty was configured on the database as "Supported", applications releases will be available on the next days.

Moving to an APT repository - Request For Comments

One of the most frequently requested features for getdeb.net is the ability to install the packages from a repository. Despite the clear advantages of using a repository there were some issues that made it unsuitable for us.

On the last couple of months most of those issues have been resolved with changes/improvements that will be available on Ubuntu 9.04. With 2 months left for its release this is the right time to reevaluate the change to an APT repository distribution method.

Advantages
- Security / integrity verification for packages (GPG signed repository)
- Automatic updates
- Install multi-package applications with a single click
- Provide packages with additional dependencies not available on the official repositories

Disadvantages
- Updates will be recommended for all packages making hard to apply only specific packages updates
- Faulty packages will have a wider impact
- Installing a package forces a repositories info update (to ensure you will get the latest version installed)
- Reverting to a previous installed version is harder (requires an unusual force version command or Synaptic)

Implementation
The implementation will require some technical changes that will need to be implemented on the next 2 months:
- Database model needs to be simplified (instead of listing files we only need to list package names and provide APT urls)
- A server side mirror selection script must be implemented to redirect APT file requests to available/updated mirrors
- A debian package must be provided to setup the repository, add custom APT config and install the GPG keyring

The decision to change to a repository or keep with the current (.deb) system must be taken, as providing both methods is not an option (release management would be much harder, not enough human resources to handle it).

I would like to see your opinions/suggestions.