Sunday, October 18, 2009

GetDeb v2 Beta Available

GetDeb v2 Beta is a major milestone on the GetDeb project development, it addresses most of the technical limitations of the current system/engine that we have been using for the last 3 years.
The major improvements are:

* Software installation using a GPG signed APT repository with mirror distribution
* Single click installation even for multi-package installs
* Automatic software updates (from the repository)
* New design - for the first time not based on a Ubuntu offical site
* Web based applications publishing


These changes will increase our ability to provide more and better packages in a safely manner.
Most of the effort was invested at the APT-Portal project, which is shared with Playdeb and because is open source it can be used by by other teams to deploy their own repository based portals.

Both the design and code base -rewritten in python- still need a lot of work, if you have any skills with Python/HTML/CSS/AJAX please join the developers team.

http://beta.getdeb.net
http://wiki.getdeb.net/apt-portal

Saturday, October 3, 2009

Preparing for Karmic

Karmic hit Beta and we still have a lot to do, fortunately I will am on vacations next week so I hope to be able to complete the major tasks just in time.

Here is the list:
- Setup the automated building system for karmic
- Finish the design and code for the getdeb based on apt-portal
- Test and move existing games on playdeb from jaunty to karmic

Tuesday, July 21, 2009

PlayDeb Beta 2 available for testing

It has been several months since we started working on the new version of playdeb, we started with a cool design but there was a lot of work to do on the function part.
There is sill a lot to be done but the core functions are there, we will now start populating it with more games.
Hope you enjoy.

Check http://www.playdeb.net

Wednesday, June 24, 2009

PlayDeb base is ready

Hello,
we have done some major work on apt-portal/playdeb on the last days.
We are using a custom Model-View-Controller framework using elixir for the Model, mako for the View and CherryPy for the Controller .
The core site design is done and the register/email auth/login functions are already working.

You can test the current design/code on your system by following this instructions.

Sunday, May 24, 2009

GetDeb/PlayDeb Testing Repository

Hello,
I have been setting up the test repository, the testing repository will be used to test packages before they get into the main repository.
You SHOULD NOT use this repository in a production system.

To setup the repository check the instructions at:
http://wiki.getdeb.net/TestingRepository

Please note there is no main at this time, the only option right now is to be a tester :)

Wednesday, May 13, 2009

Working on GetDeb v2

It has been 3 years since getdeb was started. During that time, we developed our own content management system using PHP+MySQL which has been sufficient for the hundreds of application that we have provided.

With the increasing maturity of the project though, we have defined a set of improvements to be implemented on several areas:

Organization
- Game packages and related contents will be moved to the PlayDeb project - the games category has always been a super category and serves a specific audience.
Security:
- Automatic package updates
- Package authenticity and integrity
Usability:
- Single click installs for multi package software

These changes cannot be accomplished with the current getdeb code and structure, so we will need to develop a new set of tools and a new web content management system. Because we are a small spare time working team these changes, it will take several months - but we have already some work done.
The new architecture will use components developed in self contained open source projects with sufficient documentation and generic interfaces so that they can be used for any open sources projects in general. The distribution will be done using an APT repository with 2 components, apps and games (getdeb/playdeb) - along with apt-url for the web integration.

What needs to be done:

Package building & test repository populating (https://launchpad.net/debfactory)
This is a set of scripts required for managing the source packages and building. Most of the scripts are already developed, however they need to be improved to support multiple components (apps/games) - Task 1
The scripts that are needed:
To be run on the building server:
check_incoming.py: Verifies uploaders authenticity and authority, checks the source files integrity and moves the files to the pre_build queue
check_pre_build.py: Checks the pre_build_queue for packages that need to be build, moves them to the post_build queue after a sucesfull build
To be run on the master server:
check_post_build.py: Checks the post_build_queue and moves packages to the testing reprepro repository
To Be Defined: Promoting a package from testing to final will be done manually (archive admin) ?

Package & Applications information importing and user presentation (https://launchpad.net/apt-portal)
APT-Portal will provide the web frontend for the repository. It will be a generic APT repository web content management system, and at the end we will have 2 apt-portal instances for getdeb and playdeb. The only difference for those should be configuration, theme and plugins. - Task 2

APT-Portal - GetDeb - Task 3
Theme & Plugins: To be defined

APT-Portal - PayDeb - Task 4
Theme & Plugins: To be defined - The major work here was already done (https://code.launchpad.net/playdeb), it will need to be ported to APT-Portal theme/plugins

Current status:
Task 1 - I am working on it
Task 2 - Some initial db import scripts were created, we need task 1 done and a few packages in the repository to do the real development here
Task 3 - Static HTML/CSS with the expected design could be done before Task 2 is finished, required for the real implementation
Task 4 - Depends on Task 2

If you're familiar with python / php / mysql and would like to help us, get in touch at #getdeb on irc.freenode.net.

Wednesday, April 29, 2009

How to build and test packages for GetDeb

This article is addressed to those planning to build packages for GetDeb or which just want to understand how we do it.
Building a package may require a lot of development libraries which are not needed for your regular desktop use - and you also probably don't want to overwrite current working applications with testing packages. For this reason, the recommended procedure is to use an isolated minimal system install that you can use for both building and testing.

This can be achieved by using a virtualization application, however we will be presenting a much simpler and lighter approach using schroot and sbuild.

Getdeb's debfactory is a set of tools designed to assist in package building for developers. The first thing you will need is to get the latest version of the debfactory tools, you can retrieve them with:
bzr co lp:debfactory

Now we need to create the schroot images that will contain the minimal packages for the releases we plan to build for.
To create the schroot images execute:
sudo debfactory/bin/schroot_build.py -b
The -b indicates we will be using the schroots not only for testing but also for building, sbuild will be installed.

You will need to add yourself to the sbuild group with:
sudo usermod -a -G sbuild $USER
Re-login for the change to take effect, you can see the groups your user belongs to with the "id" command.

Now let's manually build a package for isomaster
# Get the isomaster source and debian building rules from getdeb with:
dget -ux http://archive.getdeb.net/getdeb/ubuntu/jaunty/is/isomaster_1.3.5-1~getdeb1.dsc
# Switch to the schroot for the building with:
schroot -c jaunty.i386 -p

# Let's go into the source directory and install the build dependencies defined for the package
cd isomaster-1.3.5
sudo ~/debfactory/bin/satisfydepends

# Now let's build it
debuild -us -uc
# You should now have the file ../isomaster_1.3.5-1~getdeb1_i386.deb that you can install, but first let's exit the schroot with the exit command. All the changes done on the schroot except for /home will be lost, we want to enter the schroot again in it's initial state so that we can check if the package will install and run without problems on a minimal system.
schroot -c jaunty.i386 -p
# Now let's try to install the package
sudo dpkg -i isomaster_1.3.5-1~getdeb1_i386.deb
# Install missing dependencies
sudo apt-get install -f
# Run the application for testing
isomaster
# It's is everything ok ? so just exit

# What if you wanted an automated build after the dget ? just use sbuild:
sbuild -c jaunty.i386 isomaster_1.3.5-1~getdeb1.dsc

That's all for this article - I will present some other getdeb tools and processes in the future.

Monday, April 27, 2009

Preparing the getdeb/playdeb repository

Hello,
after a lot of reading and testing I have acquired the skills required to establish the future APT repository for both getdeb and playdeb. Here is the plan:

The implementation will be done using a standard apt repository structure, using a single archive pool with two distributions(per release) and two components.

Distributions
jaunty-getdeb-testing
Packages automatically imported after a successful build on ABS, it will be used for testing packages before promoting them into the -getdeb repository.
It will be strongly discouraged to use this repository on production/stable systems.

jaunty-getdeb
Packages promoted from *-testing, ready for global distribution.

Components
apps - application packages (to be available on www.getdeb.net)
games - game packages (to be available on www.playdeb.net)

Monday, April 20, 2009

Getting ready for Jaunty

Hello,
the new Ubuntu release (9.04 Jaunty) will be available on the next days so it's time to update the support policy for the next cycle.
The current policy of supporting the current plus the LTS version had a very positive result, there are clearly two main segments of users: the "current" users which always prefer the current version and the "LTS" users which like to update their packages but keeping an LTS kernel/core tools.
We will keep the current policy, for the next 6 months we will be providing packages mainly for Jaunty and supporting Hardy in a best effort basis.

Effectively today Ubuntu 8.10 Intrepid is set "Available", the archive will be kept online but no updates will be applied.
Ubuntu 9.04 Jaunty was configured on the database as "Supported", applications releases will be available on the next days.

Wednesday, March 4, 2009

Moving to an APT repository - Request For Comments

One of the most frequently requested features for getdeb.net is the ability to install the packages from a repository. Despite the clear advantages of using a repository there were some issues that made it unsuitable for us.

On the last couple of months most of those issues have been resolved with changes/improvements that will be available on Ubuntu 9.04. With 2 months left for its release this is the right time to reevaluate the change to an APT repository distribution method.

Advantages
- Security / integrity verification for packages (GPG signed repository)
- Automatic updates
- Install multi-package applications with a single click
- Provide packages with additional dependencies not available on the official repositories

Disadvantages
- Updates will be recommended for all packages making hard to apply only specific packages updates
- Faulty packages will have a wider impact
- Installing a package forces a repositories info update (to ensure you will get the latest version installed)
- Reverting to a previous installed version is harder (requires an unusual force version command or Synaptic)

Implementation
The implementation will require some technical changes that will need to be implemented on the next 2 months:
- Database model needs to be simplified (instead of listing files we only need to list package names and provide APT urls)
- A server side mirror selection script must be implemented to redirect APT file requests to available/updated mirrors
- A debian package must be provided to setup the repository, add custom APT config and install the GPG keyring

The decision to change to a repository or keep with the current (.deb) system must be taken, as providing both methods is not an option (release management would be much harder, not enough human resources to handle it).

I would like to see your opinions/suggestions.

Thursday, January 22, 2009

Playdeb status

Hi all,

You may have noticed lately that most of Playdeb's games are yet to be updated for Ubuntu 8.04 - and we're still on the temporary website. The reason for this is that we're waiting on Ubuntu to fix the dreaded 400 uri failure bug.

The bug happens only when the Playdeb website is down - and when that happens, the user cannot do any updates on his system because this error blocks them all. This is potentially dangerous, because someone can miss out on security updates because of this - so until this bug is fixed, we don't quite want to continue with this.

After the problem is fixed though, we'll update Playdeb for the tons of 8.10 and new games that we have ready. Our friends at Nexuiz Ninjas have designed a fabulous new website for Playdeb - which will go live after we get the ball rolling again too.

Additionally - we're looking for people with PHP, smarty experience to help us out. Contact us if you're interested!

Tuesday, January 13, 2009

Development Plans for 2009

There are couple of projects/features that for different reasons are not fully developed yet, here is a short list of those I would like to see ready during 2009.

PlayDeb - The gamers repository
We already got a nice design/layout that needs to be ported to Smarty templates with the getdeb CMS alike engine. Right now the project is pending on some Ubuntu (APT) bugs resolution, let's hope they get fixed for Ubuntu 9.04 .

GetDeb Quick Update Release System
Right now releasing a package is one man blocked task, we need to develop a web based system to allow participants to test, post results and ultimately release packages, this with the proper recording/auhtorization schemes required for Quality Assurance.

GetDebI - Ability to provide multi-packages install with a single click
This can be either implemented with a client application (gdebi alike) capable of handling our own file format describing which files to fetch and install, or, once the bugs affecting playdeb get resolved we could evalute moving to a repository based install solution.

GetDeb Supporter Services - This system is expected to provide higher priority services for members supporting getdeb, financially or labor. - This should be a major funding source - important to speedup all the other projects development.

I don't believe we will be able to have all them properly implemented by the end of this year, major development on all of them would be nice.

These projects cover a wide area of knowledge, from PHP/Web development to Python/GUI and application testers, let's hope we can attract more developers to our project.

Sunday, January 11, 2009

Changed to apache

Hello,
as some of you may have noticed on the last week of 2008/starting of 2009 we had some major stability issues. The main site was unavailable for several hours on a daily basis.

After our major hardware upgrade last year, the main server is running VMWARE ESXi to host multiple virtual servers. During this time we have experienced several hangs on the web VM (100% CPU), the problem was so severe that we where unable to shutdown the specific VM, the entire VM server had to be restarted.
This events happened randomly, there were no system logs for the crash and because the console was not responding we were unable to request kernel dumps.

There are multiple VMs on this server and none of them presented this problem, so it was something specific to our configuration. After several days testing change and moving services to a different VM we have found that the problem happened with the lighttpd/fastcgi services.

As a workaround we have changed the http server software to apache, with this change the server as been running stable.