Saturday, July 17, 2010

Status update and repository users count

We have managed to get the site/archive in an usable state after some improvements with infrastructure architecture changes. We are already publishing updates but we still have bandwidth constrains that may affect our availability.

I have been looking into the repository access log statistics and we have pretty impressive numbers.
Yesterday, there were 25144 unique IPs requesting the packages Index for lucid, 7272 for karmic. This was a single day count, it means that currently we have more than 30k users with the GetDeb repository.

I expect a lot of people will keep on Lucid (LTS) when Maverick is out, needing some upgrade, so this numbers are likely to increase. We will need to increase our current capacity.

4 comments:

Geekoid said...

Hi João,

Can you give me some information on your current bandwidth use, requirements and approximate traffic? I may be able to offer you a better place to host your main repositories, for free.

Please let me know.

G

Geekoid said...

One other thing, João...

I noticed you were a commenter on this very bug, which I was researching as an option to suggest to you:

https://bugs.launchpad.net/debian/+source/apt/+bug/18645

I was going to suggest you configure a HTTP 301 redirect to an alternate mirror when your server can't take the strain. I see the bug has been fixed for some time now. What do you think?

Thanks

G

João Pinto said...

Hello G,
until last week our main server bandwidth usage was 1-2Mbits/s, it had a major impact on the site/archive response time.
This was, sitehtml/css+screenshots+rsync service+repository archive. Later
we moved the screenshots and rsync to different servers, the current usage is 0.5-1MBits/s.
The current hosting link was upgrade to 3Mbits/s, I believe that we the current configuration we might be able to support the demand without major impacts.
About the HTTP redirects, we have been using it since it was implemented on APT. We depend on it for the mirror selection, the repository is currently provided from 8 mirrors. We need to start some mirror recruit initiative because we really need more mirrors, with the gimp updates some of our mirrors were providing several gigabytes per hour.

Wise Ferret said...

Trying to update using the apt repositories I keep getting the following message:

Err http://getdeb.avaneya.com/ubuntu/ lucid-getdeb/games violetland 0.3.0-1~getdeb2
403 Forbidden
Err http://getdeb.avaneya.com/ubuntu/ lucid-getdeb/games violetland-data 0.3.0-1~getdeb2
403 Forbidden
Failed to fetch http://getdeb.avaneya.com/ubuntu/pool/games/v/violetland/violetland_0.3.0-1~getdeb2_amd64.deb 403 Forbidden
Failed to fetch http://getdeb.avaneya.com/ubuntu/pool/games/v/violetland/violetland-data_0.3.0-1~getdeb2_all.deb 403 Forbidden
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?