Wednesday, May 13, 2009

Working on GetDeb v2

It has been 3 years since getdeb was started. During that time, we developed our own content management system using PHP+MySQL which has been sufficient for the hundreds of application that we have provided.

With the increasing maturity of the project though, we have defined a set of improvements to be implemented on several areas:

Organization
- Game packages and related contents will be moved to the PlayDeb project - the games category has always been a super category and serves a specific audience.
Security:
- Automatic package updates
- Package authenticity and integrity
Usability:
- Single click installs for multi package software

These changes cannot be accomplished with the current getdeb code and structure, so we will need to develop a new set of tools and a new web content management system. Because we are a small spare time working team these changes, it will take several months - but we have already some work done.
The new architecture will use components developed in self contained open source projects with sufficient documentation and generic interfaces so that they can be used for any open sources projects in general. The distribution will be done using an APT repository with 2 components, apps and games (getdeb/playdeb) - along with apt-url for the web integration.

What needs to be done:

Package building & test repository populating (https://launchpad.net/debfactory)
This is a set of scripts required for managing the source packages and building. Most of the scripts are already developed, however they need to be improved to support multiple components (apps/games) - Task 1
The scripts that are needed:
To be run on the building server:
check_incoming.py: Verifies uploaders authenticity and authority, checks the source files integrity and moves the files to the pre_build queue
check_pre_build.py: Checks the pre_build_queue for packages that need to be build, moves them to the post_build queue after a sucesfull build
To be run on the master server:
check_post_build.py: Checks the post_build_queue and moves packages to the testing reprepro repository
To Be Defined: Promoting a package from testing to final will be done manually (archive admin) ?

Package & Applications information importing and user presentation (https://launchpad.net/apt-portal)
APT-Portal will provide the web frontend for the repository. It will be a generic APT repository web content management system, and at the end we will have 2 apt-portal instances for getdeb and playdeb. The only difference for those should be configuration, theme and plugins. - Task 2

APT-Portal - GetDeb - Task 3
Theme & Plugins: To be defined

APT-Portal - PayDeb - Task 4
Theme & Plugins: To be defined - The major work here was already done (https://code.launchpad.net/playdeb), it will need to be ported to APT-Portal theme/plugins

Current status:
Task 1 - I am working on it
Task 2 - Some initial db import scripts were created, we need task 1 done and a few packages in the repository to do the real development here
Task 3 - Static HTML/CSS with the expected design could be done before Task 2 is finished, required for the real implementation
Task 4 - Depends on Task 2

If you're familiar with python / php / mysql and would like to help us, get in touch at #getdeb on irc.freenode.net.

9 comments:

Vincent said...

Please tell me I can find the PlayDeb results on the GetDeb site... I probably won't remember to look at PlayDeb when I'm searching for a game - especially for applications of which I'm not sure if they're a game (e.g. Robocode is in the Games category at my PC but might as well be Programming)...

yekcim said...

Why don't you use ppa? That could be a easier way...

João Pinto said...

Vincent,
we could create a cross-search function, that is not planned at this time.

yekcim, PPAs do not provide the flexibility that we required, for example we need to perform certain validation on the building server, also some of our packages could not be published on PPA's due to license restrictions.

Jean-Philippe said...

What about adding a category with payable debs? Something where you have to pay with paypal or something like that to be able to download the deb. It would be good for companies to make software for ubuntu. I'm sure 2D Boy would love that. http://2dboy.com/games.php

João Pinto said...

Jean,
a software shop for commercial apps would require a legal company, which are not :)

care3g said...

Are you planing to use Drupal for the portal? It would fasten things a lot.

It could also be used https://alioth.debian.org/projects/php-apt-parser/ to read the APT repository in a real easy way and it is already prepared to deal with diferent distributions ;)

João Pinto said...

care3g,
we are not going to use standard CMS like drupal, it does not suite our needs.

We already developed our python parser, apt2sql (using python).

We are still selecting the technology, it will be most likely what we have now available from the apt-portal bzr branch.
CheryPy for the web server, Mako for the moddeling and python-elixir for the database model.

Thanks for the suggestions.

willson said...
This comment has been removed by a blog administrator.
Ruedii said...

I have been having trouble accessing the site the past few days.

I know there is an issue with servers using certain SPI firewalls, My ISP and NAT routing. I haven't gotten it all figured out, but I am going to contact my ISP's security team (who is in charge of the QoS filters that are probably causing the issue.)