, 23 2012

rpm caching proxy for yum, zypper, kiwi, similar to apt-cacher using nginx

Good day,
As some of you may know, in my spare time I try to maintain a little linux live cd called Petite Linux.

Ok, so I skipped version 12.1 (based on the omonymous openSUSE version).
There's two reasons 12.1 never got released.

  • Lack of free time, seeing as my son now requires most of it and
  • RSI (i.e., it hurt to type, so I didn't). (Solution)
So since openSUSE 12.2 is due soon, I decided it was time to start getting prepared for a new Petite Linux version.
I'm going to do things in a different fashion this time, mainly to make my own job easier.
As few custom scripts as possible and my own rpm repository.

Somewhere around here, I got fed up with Suse Studio.I could go on and moan about the stuff that cheesed me off, but I don't see the point.
So I'm gunna be using kiwi which is a local build system, pretty straight forward.... however

To build a live cd, you obviously have to install the necessary packages each time you change something and rebuild.
For my own petite linux, the download size is around the 500MB mark currently, which on my 2Mbps line will take an annoyingly long time to download.

Now obviously, if you only have to download the packages once, well ok, it can't be avoided.
So I went looking for something similar to apt-cacher for openSUSE.
In case you don't know, apt-cacher is a proxy that stores .deb packages, to avoid downloading the same package more than once.
To my horror, there was nothing similar for openSUSE.

After a little googling, I got nothing but bad ideas, like

  • telling zypper to store packages and then use them to create your own repo
  • creating a mirror of the entire openSUSE repos
  • Lump it and download the packages every time
I mean, cmon now, seriously ?
So since I didn't find anything ready, I decided to do it myself.

Please note that the following is a real basic guide, not meant to be super optimal or anything.

Nginx is a web server with a small memory footprint which supposedly outperforms apache.
It is also a very capable reverse proxy, which is exactly what I needed here.
The configuration I used is pretty simple (openSUSE).

The configuration directory is /etc/nginx and it contains some regular useless crap.
I pretty much commented out everything useless in nginx.conf apart from the:
include conf.d/*.conf;
I created proxy.conf which contains the following:
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 100m;
client_body_buffer_size 1m;
proxy_connect_timeout 900;
proxy_send_timeout 900;
proxy_read_timeout 900;
proxy_buffers 32 4k;
proxy_cache STATIC;
proxy_cache_valid 365d;
proxy_ignore_headers X-Accel-Expires Expires Cache-Control;

I created the dir conf.d and the config file cacher.conf which contains the following
proxy_cache_path /home/nginx/cache levels=1 keys_zone=STATIC:50m inactive=200d max_size=12g;
server {
listen 80;
server_name opensuse.local ;
location ~ .rpm$ {
proxy_pass http://download.opensuse.org ;
include /etc/nginx/proxy.conf;
location / {
proxy_pass http://download.opensuse.org ;
server {
#Same as above, different hosts, etc

Yup, that's it.
Two whole tiny files...

Nginx documentation is pretty awesome so you can tweak this to your needs, I'm sure you get the general gist of it.
So if you mirrored the whole openSUSE repos, I'm sure you're feeling like an ass right now :)

How bout using it?
Well as you can see, I named my local repo(s) something.local.
To make use of these, I added entries to /etc/hosts as such opensuse.local packman.local petite.local etc...

Next I changed my kiwi config and changed the repos to something like this:

And there you have it.

Now you only have to download each package once :)

, 31 2012

SSDs on HP servers with e200, p400 and p410 controllers

I think this experience is worth sharing with other system administrators across the globe.

Our little data center has an array of mostly HP servers.

Our awesome R&D have created a distributed Key Value technology which we call CloudDS.

It is similar to cassandra, and similarly all writes are sequential, and the benefits of SSDs zero seek time are huge.

Cassandra and SSDs

Being that we are not giants, the idea of spending huge amounts of money for the HP ssds had us sceptical.

Having searched high and low, we decided to give it a shot and get some cheap ssds to see if they work.

Our first purchases were Intel 520s.

These drives are not enterprise level and don't have heat sensors.

When plugging them into our e200i's in our bl460c enclosure, everything went smoothly, apart from the fact that they were detected as sata1 with no Native Command Queueing. Neither of these are issues for our particular needs (and ssd's don't have moving parts) :)

(Sample hpacucli output)

physicaldrive 1I:1:2

Status: OK

Interface Type: SATA

Size: 120 GB

Firmware Revision: 400i


SATA NCQ Capable: False

PHY Transfer Rate: 1.5GBPS

When plugging them into our dl3xx G5's with p400 controllers, again sata was at 1.5Gbps although this time NCQ was detected and enabled.

physicaldrive 1I:1:8

Status: OK

Drive Type: Data Drive

Interface Type: SATA

Size: 120 GB

Firmware Revision: 400i


SATA NCQ Capable: True

SATA NCQ Enabled: True

PHY Transfer Rate: 1.5GBPS

So far so good... :)

Next up was the p410i on our dl3xx g7s

This is where things went bananas.

After plugging in the drive, the red controller light came on and the fans spun up to 100%.

I didn't waste much time, having read before hand that the servers would possibly shut down, I unplugged the hard drive and things returned to normal.

Unfortunately the p410 rejects drives that don't have a good thermal sensor reading.

According to one post we came across there is a bios option that means the server won't shut down, but fans will run at 50% all the time.

Increased Cooling

One option wasIntel's 710sthat come with thermal sensors, however the drives cost something like 5x more, which was too much.

We came across aRelated Hp Forums Postand opted for the ocz Deneva 2 disks which were reasonably priced AND came with a thermal sensor.

The end result was a success, the drives work perfectly on our p410 controllers :)

physicaldrive 2I:1:8

Status: OK

Interface Type: Solid State SATA

Size: 120 GB

Firmware Revision: 2.15

Model: ATA D2CSTK251A10-012

SATA NCQ Capable: True

SATA NCQ Enabled: True

Current Temperature (C): 30

Maximum Temperature (C): 30

PHY Transfer Rate: 3.0GBPS

All in all we purchased 7 drives for approximately the price of a single HP ssd drive.

Success !

, 22 2011

Linux Mint 11 Katya

So you know linux exists, you've heard of ubuntu, and you discovered distrowatch...

Wait, what's this:

1 Ubuntu 2292
2 Mint 2188
3 Fedora 1566
4 Debian 1485
5 openSUSE 1304


- The green stuff brits mix with vinegar and use as a sauce on roast lamb (it's actually amazing)

- The most common chewing gum flavour

- The second most popular linux distro on distrowatch, and not second by much !


So, it's green ubuntu ? Nope, it's green ubuntu+ with emphasis on the +

After having tried a few distros, it becomes hard to set anything apart as special.

yeah, so it's old gnome, yada yada, comes with some software, has a menu, a green theme blah blah.

The easy part usually is pointing out what went wrong.

With Linux Mint, well... nothing went wrong.

Everything was like, obvious, and in the right places, or easy to find.


I like.


If there's something special, it's the software manager.

I suppose the main reason I generaly use the terminal for software management is because of the general suckage of software managers.

Mint's software manager makes you want to use it and explore the available software.

It's like... not linux.


Aw cmon, there must be something nasty to say.

Honestly ? not by me, although my brief usage stints are hardly enough to cover everything that could go wrong.


Not as fast as e17

Not as pretty as opensuse kde

Overall great exprience, ideal for windows users who want a gentle first contact and existing linux users who are fed up with crap.

RSS: ,
Powered by Pathfinder blogs