Wednesday, March 4, 2009

Moving to an APT repository - Request For Comments

One of the most frequently requested features for getdeb.net is the ability to install the packages from a repository. Despite the clear advantages of using a repository there were some issues that made it unsuitable for us.

On the last couple of months most of those issues have been resolved with changes/improvements that will be available on Ubuntu 9.04. With 2 months left for its release this is the right time to reevaluate the change to an APT repository distribution method.

Advantages
- Security / integrity verification for packages (GPG signed repository)
- Automatic updates
- Install multi-package applications with a single click
- Provide packages with additional dependencies not available on the official repositories

Disadvantages
- Updates will be recommended for all packages making hard to apply only specific packages updates
- Faulty packages will have a wider impact
- Installing a package forces a repositories info update (to ensure you will get the latest version installed)
- Reverting to a previous installed version is harder (requires an unusual force version command or Synaptic)

Implementation
The implementation will require some technical changes that will need to be implemented on the next 2 months:
- Database model needs to be simplified (instead of listing files we only need to list package names and provide APT urls)
- A server side mirror selection script must be implemented to redirect APT file requests to available/updated mirrors
- A debian package must be provided to setup the repository, add custom APT config and install the GPG keyring

The decision to change to a repository or keep with the current (.deb) system must be taken, as providing both methods is not an option (release management would be much harder, not enough human resources to handle it).

I would like to see your opinions/suggestions.

30 comments:

Galv_BL said...

I prefer a repository

PS:Thanks for all the good work

markusf21 said...

sounds great to me. I would love a repository

Masiosare said...

As you already know, I have been running a repository version of the getdeb packages with very good results

http://getdeb.masio.com.mx/

I haven't received complaints yet and I use it everyday.

so it shouldn't be very hard and if you need some help just let me know.

Regards :)

BlondieGirl said...

I would love an oficial repository. Thanks!

KarelZ said...

I prefer a repository too.

Cheers
Karel

muhammad_wanas said...

I would like to use the official repo it would be great
and thanks for ur hard work I really love ur work

Arthur Reeder said...

You blokes do an amazing job, by the way. A repository would be amazing. Especially for your builds of Audour and Audacity alone.

Ge! said...

Hello Getdeb Team!

I don't know if it's possible or not, but maybe a system that you have some script that generates repos by the user id. Ex: deb://apt.getdeb.net/userid/XXXX intrepid

Why am I suggesting this? Well, this way every registered user can "mark" some package in the getdeb page like a favourite package, and that way you could have updates of only selected packages. So, if I want to update Pidgin but no Gimp, for example, I just have to add Pidgin to my fav-list.

I don't think that's to hard to implement, given that you already have a user system and are planning on generate HTTP debs repo...

dandanio said...

I have been creating repository-like Packages.gz from your mirrored content for a year or more now. Thanks for all the hard work and I am strongly behind a repo version. As a side note and your concern over the quality of broken packages, I think it is a good thing, it will make you test harder. :) +1 repo!

João Pinto said...

Ge,
I already had the deb://apt.getdeb.net/userid/XXXX intrepid idea some time ago, it is not that easy to implement, it would require the Packages.gz for that particular user to be automatically regenerated (and signed) every time a new package is added/removed to the user preferred package list.
It can be done, but is not simple.

Hated On said...

I have no problem with getdeb adding a repository, but I think it would be a bad idea to get rid of the current format.

When I was new to Ubuntu, the format of getdeb made it easy to download and install programs. Even with a repository I would still primarily use the old format because I like reading the descriptions and comments about programs, following the links, and saving the packages to the hard drive.

Yes, I know when installing from a repository packages are saved, but you have to go searching in admin areas to find them.

Also, I know you can use a web browser to explore and download specific packages from a repository, but it is not as clearly organized or descriptive as getdeb is.

I hope if you guys add a repository you keep the old format as well.

getdeb's current format is a great resource for people new to ubuntu.

João Pinto said...

Hated On,
we will keep the web portal format, using apturls for the installation.

goat said...

Joao,

Firstly, +1 on the repo, this site is awesome, and the repo capability would be even awesomer!

In regards to your comment about the ease of having an personal repo system, couldn't the package.gz and signature be pre-created and then symlinked to per user?
For example, you have apps A, B, C, D, and E. you could create the indexes that contain A,B,C,D,E or combinations of each and store them somewhere, with a table or db that keeps everything in order. Then if user 1 selects A,B, and E, he gets packageABE.gz symlinked to his user account, If he selects another, the db recreates the symlink. You wouldn't be having resource burdens based on each change of a user.

I'm not sure what the package to user ratio is, but this could be beneficial. The disk burden wouldn't be too much, I imagine, as even some of the bigest package.gz files are in the 10's of MB.
I would guess the hardest part would be to create a system that, when package A is updated, it regen's all the packageAxx.gz that include this.

Is my idea too bonkers?

tzunder said...

Thanks for website. I'd like a repository but I am staying with 8.04 for now. Can you manage a repository for 8.04 and 8.10 and 9.04?

João Pinto said...

goat,
there are millions of possible combinations for packages installations, creating a link representing each combination is not doable.

tzunder,
we can't provide a repository for <9.04 because re rely on a feature only available from 9.04 for the mirror selection.

papukaija said...

I prefer a repository too.

Jonathan said...

I think there are good sides and bad sides to this. In effect, getdeb could become an "unstable" repo for Ubuntu, packaging and supplying all the newest releases of software without having to add various teams PPAs to your Software Sources. Many people would really enjoy how easy it is to keep up with the latest updates in 3rd party stuff like Pidgin, AWN, and Gnome-Do. At the same time, that instability could present a jarring view of Linux to new people. They might wonder why their software update introduced such a significant change. There's also the chance that upgrades could not go smoothly and people's configurations could be lost or misapplied. Since you say a web interface will still be available, I think new users should be encouraged to use that unless they fully understand the consequences of such a repository. The problem I foresee is a new user just wanting the latest Pidgin, so they add the repo, and all of a sudden half of their applications get an upgrade, some of which have changed things in ways they're not ready for. As long there is an appropriate instability warning (i.e. you understand what you're doing by adding this repo), I think it's a great idea.

E_rulez said...

Disadvantage:
- Updates will be recommended for all packages making hard to apply only specific packages updates

Solution:
You can use similar system to one that backports.org uses (http://www.backports.org/dokuwiki/doku.php?id=instructions):
e.g. All getdeb packages are deactivated by default. If you want to install something from getdeb run:
apt-get -t getdeb install “package”

omega13a said...

An official repository is a very good idea. The one at http://ubuntu.org.ua/getdeb/ has a bad habit of mixing hardy and intrepid packets together resulting in all kinds of dependency problems when new versions come out.

bigdavesr said...

I lke the idea of a repo.What I like most is the idea of dependecies.I have run into dependecies on updates in packages such as pidgen. A repo should help with that preoblem.Have used your site from the start. Keep up the good work. Thanks for being here.

Laibeus Lord said...

Stay with the current web-based system ;)

A repo is good, I once wished that you have one. However, as you already mentioned, it will be harder to use a different version from another repo.

Maybe in the future, if (X/K)Ubuntu's Synaptic (at least) gets a feature to separate each repo, so that we can see which is from which. And easily switch between different installers.

I did encounter apps wherein the GetDeb version works; or the GetDeb version doesn't work and a PPA one works; and so on. Usually, since I also use 3 other PPAs, I have to delete and refresh just so Synaptic will fully reload. Disabling simply doesn't work, it still loads the repo of the disabled repo.

If GetDeb will add to that, then that surely will be a mess to handle. In my opinion, and this is the first time I am going to say this "ever", don't fix what isn't broken :p

Anyway, that's just me. Use a repo once we can Synaptic can show separately each repo in its list. ^_^

OR at least will respect "priority" levels for each repo (which I've tried and doesn't work either).

manish mahabir said...

can we have two repos for getdeb...1)proposed...containing all the releases first day first show and 2)stable.
packages which have been tested in the proposed repo for some time and which do not have any issues could be moved to the stable repo.

ilembitov said...

Great idea. But keep in mind, that pretty often you're maintaining an app that can already have an official or semi-official deb available from the project homepage. Furthermore, some packages may even have official or semi-official (but still regulary updated) PPAs. For example, Deluge, Transmission, Banshee, GNOME Do and many more have official PPAs, so you don't need to maintain those packages. That way you can reduce resourses needed, achieve higher quality and cover more packages that are interesting, but don't have an Ubuntu repo.

gartuz said...

But there should be a way to reward the effort made by the human resources Getdeb.net and I think it sounds pretty repositories but most people should be considered and click on the Ads to help maintain or Getdeb.net repositories if they want they can contribute at least $ 1 or a minimum of U.S. dollars.

Crew said...

+1 repository... ;)

roadboy said...

+1 for repository.

kyle said...

Repo please!

It makes it so much easier for me at least! Then the software installation is just like ubuntus and even better, the packages can update.

AYKUT ÖZDEMİR said...

ı prefer a repository for jaunty, too

typhos said...

+1 for repository

Hollow said...

The repository is necessary.
There two ways, the Web end the repository, and the user should option the way better for him.