MPlayer, Matroska and you

2009-08-17 11:49 This issue was fixed a few hours ago. I have bumped MPlayer in our tree to SVN rev. 29532

I bumped MPlayer tonight. Aside from a new VDPAU option and some miscellaneous fixes, the most important news item is that this bump most likely will be built for you by GCC 4.4, at least if you’re up to date. This is going to break playback support of files that use the Matroska container format (these files tend to use the .mkv file extension). The relevant bug in MPlayer’s Bugzilla is Bug #1467. This bug is present in all versions of MPlayer in ::media, including those I removed tonight. There are a couple of ways to work around it:

  • Use -demuxer lavf. The bug is in the libmpdemux Matroska demuxer so you can use the one from libavformat
  • Build MPlayer with the new debug-option. The bug disappears when you enable debugging support (yay!)
  • Build MPlayer with an older compiler. GCC 4.3 is known to work

It is possible this bug won’t affect you if your system is amd64.
You can grab a sample Matroska file if you want or need to test.

The state of the Exherbo Onion

Two perly things have happened in Exherbo recently. First of all, I pushed an exheres for 5.10.1-RC1 last night. I have tested to ensure that it should be possible to up- and downgrade between 5.10.0 and 5.10.1 without any issues, so feel free to test it. You will however need to rebuild anything that links against libperl.so (reconcilio should take care of that for you) so keep an eye out. Yes, this includes vim[perl] ;). RC1 and any further RCs will stay masked for testing indefinitely. 5.10.1 will introduce some changes, notably in features introduced in the 5.10-series, that are not backwards compatible with 5.10.0 so if you feel courageous, install it and see if you can make any packages break.

The second news is perl6. Ingmar has done some bitchin’ awesome work on getting parrot and rakudo in a state where they actually compile and install! As a result you can now play with the current version of rakudo, compiled straight from SCM:

$ perl6 -e'say "Hello, world!"'
Hello, world!
$

Remember to buy Ingmar a beer if you meet him :-)

ハ, パ… what’s the difference?

We get a lot of patches from many people in our main #exherbo channel. When someone is around and have the time to pick them up it works very well but some patches tend to drown in the general noise when we are all too lazy or drunk to do something with them.

Well, not anymore! Our new patchbot ‘hacchi’ (ハッチ) lets you queue a patch for later review, remove a single patch or several at once and get a list of all the currently pending patches. It accepts three different commands:

!patchqueue http://dpaste.com/12345/plain ::arbor
Enqueues the patch.

!patchdone 12345
Removes all patches matching ’12345′, so ‘!patchdone arbor’ would remove all patches for arbor.

!patchlist
Prints the current number of waiting patches to the channel and privmsgs the list to whoever ran the commands. hacchi also responds to this command in private.

The shorthand notations !pq, !pd & !pl also exists.

If there are any patches in the queue, hacchi will nag the channel about them once an hour to ensure they won’t be forgotten.

This is not a substitute for leaving a decent comment with an explaination that states what the patch is supposed to fix!

You should obviously still feel free to highlight a specific dev if you think your patch should be reviewed and applied by that person.

Topic repositories! Use them!

During his various talks Bryan has mentioned the use of unofficial topic repositories in Exherbo – repositories hosting packages related to a specific topic that are either mostly or completely maintained by non-developers (and by non-developers I mean people who cannot push to the official Exherbo repositories). So far, we haven’t really seen any of those mainly because most people have just kept packages in their own repositories. It gets tedious to add a lot of personal repositories just to get the latest and greatest packages people maintain though and at the end of the day it’s easier to collaborate and coordinate efforts if we can keep the related packages together, so we’ve (and by we I really mean Ali Polatel) created a media-unofficial repository hosted at github. So far alip, replica and I have pushed our media-related packages there and the repository currently contains stuff like Audacity, ncmpcpp and envtag.

Now, the point of a topic repository isn’t to give any and all push access – we want to exert a proper level of QA, even if it is an unofficial repo. But as with the official ones, pinging one of us with a decent git format-patch is all you need to contribute stuff. If you want to contribute a new media-related package, please consider whether it would be better off in -unofficial. Keeping the number of packages that we (and by we I mean Exherbo developers) have to maintain is essential if we want to continue keeping the number of developers (and by developers I mean non-non-developers) near-constant (and we want that).

The name media-unofficial does not imply that the packages are of a lesser quality than those in media.git, it does not imply that Exherbo developes want nothing to do with them, it does not imply that the packages are of dubious legality. The packages in that repository are maintained by smart people (including current Exherbo developers and current and former Gentoo developers) and we expect the quality to be as good as any official Exherbo repository.

I like the idea of topic repositories when it’s possible to draw a somewhat clear line between relevant and irrelevant packages. The whole notion of restricting who can directly touch a set of packages should help keep people from stepping on each others toes while still letting those who want to do really crazy stuff with their packages (and keep them private thankyouverymuch) do that just by copying the exheres’ to their own repositories. I hope we will see more of these as Exherbo grows – repositories maintained by groups of users and (optionally) a couple of Exherbo devs. They do all the work and we take all the credit!

Oh, and regarding that last sentence… anyone who wants to start an unofficial scientific repository and add all those things I’ve been too lazy to package? Pretty please?

Exherbo in numbers

Lies, damn lies and statistics…

At the time of writing (Wed Feb 11 11:22:48 CET 2009), the following numbers might be of interest:

  • 269 days since official announcement
  • We’ve been working on Exherbo for longer than that but serious work on packages and “userspace” stuff began around the time of the announcement.

  • 30 official package repositories
  • This includes individual developers’ supplemental repositories. Thanks to Paludis’ excellent support for multiple repositories and in particular the unavailable repository type, keeping track of packages in 30 different repositories is easy as an end-user (my laptop/server currently uses 16 of those repositories). Does any other distribution make it as easy to keep track of many small end-user oriented repositories? The Debian/Apt approach is definitely more painful than this.

  • 7 unofficial package repositories
  • These are repositories belonging to Exherbo users rather than official developers. The difference between these and the official ones is that these aren’t hosted on Exherbo infrastructure and they’re listed in unavailable-unofficial rather than unavailable.

  • 1313 packages in official repositories
  • Believe it or not but chances are you will find most of what you need here. There’s a long way to the tens of thousands of packages in distributions such as Gentoo and Debian but we don’t really have to go there. Adding packages with importare is easy and there are whole groups of packages that doesn’t really have to be hosted in official repositories. If a group of people want to add robotics-related packages to Exherbo they can just create an unofficial repository (could be anything, $scm repo, tarball, rsync) and get it listed in unavailable-unofficial. We really want to keep the number of packages we have to maintain low to allow us to spend time developing the system and its related tools. Maintaining packages doesn’t require any magic cloaks.

  • 76 packages in unofficial repositories
  • This isn’t a lot, in part because some people prefer to provide packages as patches for official repositories but mainly because Exherbo is just now crawling out of its infancy, taking its first steps towards being used by a wider audience. As this audience grows, we expect the number of unofficial packages to grow as well.

    Unofficial is actually a somewhat misleading term. It’s true that we don’t maintain them, but we do glance over the repositories before they are added to unavailable and if we hear of major breakages from these we can quite easily remove them again. Most (if not all) of the people with unofficial repositories listed in unavailable tend to hang around in #exherbo on IRC, so submitting patches to them is simple.

  • 20 developers
  • As currently listed on the wobsite. You will notice this number has been near-constant over the last year (I think one developer joined since the announcement). There’s no real reason to let this number grow at the moment, since all package maintenance can be done without push-access to the official repositories.

  • 20 contributors to official repositories
  • Over the last year, patches from 20 people against the official repositories have been accepted and pushed. Several packages are currently maintained by non-developers. Switching to Git early on was a great decision since it makes the whole patch-dance incredibly easy. We owe a great deal of thanks to these people.

  • 90 people in #exherbo
  • This is also one of the near-constant numbers. A majority of these people idles in the channel, but if you ping a developer with a link to a git format-patch, it’s likely to get pushed or rejected with suggestions for improvements quickly. Keep in mind that most Exherbo developers lives in Europe and some of us (usually not Ingmar) also sleep every now and then.

  • 50 kicks & 27 current bans in #exherbo
  • Given the potential for flames and the number of assholes on the internets these numbers seem fairly low. It’s usually a peaceful channel.

  • 3 platforms (x86, amd64, ppc64 [unofficial])
  • Exherbo currently has stages for three different platforms though not all packages has been tested on all platforms. Paludis will refuse to install untested packages and mark them as ‘Masked by platform’. Feel free to submit a git format-patch if you find that a masked package works well on your platform (you can override the platform mask in /etc/paludis/platforms.conf).

  • Activity in 12 official repositories over the last 24 hours
  • Even though it’s been nine months since the initial announcement we’re still doing lots of work all over the place.

  • 6896 commits in official repositories ~ 56 minutes between commits
  • The most important number of them all!!! This comes from ohloh and is a little more than a day old. It’s not too bad for forty people working on this project in their spare-time.

  • 31 open bugs / 120 closed bugs
  • We usually only use bugzilla when we need to keep track of issues for an extended period of time. Most issues are quickly resolved over IRC.
    If you file a bug, remember to attach a git format-patch!

  • 1174 quips
  • Don’t say we’re not funny! If you know a bit about Gentoo, you should recognise many of the quoted people.

  • 1 supported PM
  • No-one has stepped up to provide an alternative implementation of the exheres-0 EAPI, written in Haskell. Patches are welcome (in /dev/null, that is).

  • 1 (unwanted) slashdotting
  • That certainly turned what should’ve been a low-key announcement in a few relevant places into a lot more attention than what we wanted (or was prepared to recieve at that point in time). Still, it was a fun couple of days.

  • 0 flamewars
  • That’s right. It’s hard to start a serious flamewar among 20 people.

  • 1 awesome mascot
  • ‘Nuff said

  • 1 deleted wikipedia article
  • Someone wrote a wikipedia article just after the announcement, which was swiftly deleted as Exherbo didn’t meet the notability guidelines for wikipedia back then. Since wikipedia is all about verifiable, third-party information it will probably take a while before we’ll meet that requirement. But that’s all for the better – the people browsing wikipedia in search of their next distribution probably aren’t the kind of people we need/want right now.

  • 3 talks (only 1 video)
  • Bryan gave the talk on Genesis at FOSS Aalborg. It was this talk that prompted the announcement of Exherbo, and at the moment, this is the only talk we have a video from.
    Bryan then gave a talk about the visions and long-term goals of Exherbo last October at Open Source Days. This talk was a lot more interesting in general but unfortunately no video recording has been made available so far.
    Lastly, Bryan was invited to talk at FOSDEM four days ago. The video isn’t out yet, but it probably will be soon.

  • >9000 people pissed off by our honesty
  • For some reason a lot of people felt offended by the fact that we didn’t want them to waste their time on a half-working, most-likely-broken distribution. If you tell them to use it, they will complain that it’s broken. If you tell them not to, they will complain that you’re an arrogant bastard. Even so, scaring these people away was a lot better the wasting time on supporting them. We’re reaching a point where we feel Exherbo is usable by a wider audience, but we certainly weren’t there nine months ago. If you look a the frontpage you will see that its content was changed a short while ago, to better reflect the current state of Exherbo.

  • Too many offers of community management/PR/HR help
  • We still don’t need that thankyouverymuch!

That’s what I could think of at the moment. Are there any other interesting Exherbo-related numbers out there?

The worst thing…

The worst thing about Open Source Days [danish] isn’t choosing which talks to attend (I never have time for talks anyway). The worst thing is doing weird LaTeX hacks to get their logo placed correctly in the agenda
Turns out there’s this cool environment for wrapping figures in text named ‘wrapfig’ (gee, who thinks up all these names) that you can use for positioning an image a little more intelligently than the default, left-aligned placement. Alas, said environment (or something else – I am by no means a LaTeX guru), doesn’t play nicely with tables and as the majority of the page is made up by the actual agenda in a table, that means the logo ends up whereever it fits; in this case smack in the middle of the page. A lost cause? Of course not! <leonidas>This is LaTeX!</leonidas>. Take a look at this beauty:


\AddToShipoutPicture{
 \put(0,0){
  \parbox[b][420mm]{530mm}{
   \vfill
   \centering
   \includegraphics[width=60mm]{logo2008_small}
   \vfill
  }
 }
}

[courtesy of the same guy who wrote the aforementioned blagpost, Jesper Nyerup]

Of course, you might have to fiddle with the actual placement of the parbox but as far as I could tell from my experiments, this will place the image whereever you want it, no matter what is above or below it layerwise. At the point where Jesper pasted that, I had been driven to the nastier corners, fiddling with boundingboxes and another parbox-hack, none of which worked remotely as well as this solution.
An ugly hack, nonetheless, don’t use it if you can avoid it.

Also, this requires you to include the eso-pic package in your preamble.

Also, if you happen to attend Open Source Days on saturday, by all means come by the Exherbo user stand where we will be happy to act like the elistist bastards people assume we are and tell you exactly why you shouldn’t use Exherbo just yet.

Once upon a time…

Storytime guys and girls!

Once upon a time there was a prosperous kingdom built on the greatest hills, the finest plains and the mightiest forests, a kingdom where cunning wizards and valiant knights worked together to build the most magnificent country ever. There was peace and harmony and the people were happy. Time went by and the inhabitants grew with their kingdom. Marvellous inventions and beautiful works of art saw their origin there.

However, as time went by the kingdom started to feel its age. Some of the knights presented new ideas and visions but the kingdom were unable to follow. Here and there weed started to creep in and the once so magnificent cities and castles slowly began to tear and wither. These knights wanted reforms to the very structure of the kingdom and they thought that the high council of the druids were the place to ask. But alas, their ideas were turned down and their visions mocked and ridiculed. These knights fought for a long time but eventually they decided to leave the kingdom and seek their fortune under an alien sky.

Three knights left the kingdom and for years and years they travelled seeking new lands where they might turn their vision to reality. After seven long years of toils and hardships they finally came upon uninhabited lands, a place where they could start over. Delighted they began their work sending messengers back to their former home with news and pleas for help. More people decided to set out, some out of interest, some out of spite for their former throne. Some decided to stay back and maintain contact between the two realms working towards a greater good for both of them.
But they were not welcome anymore. The council of the ancient kingdom found that they posed a threat to their stability and set out on a campaign to defame and harass these ambassadors of peace. Slowly, one after the other, these people were having their most priced possessions confiscated and they found themselves thrown out of the kingdom without warning losing everything they owned. Thoroughly disappointed with their former masters they left for the new skies where they were welcomed, even though their mission of cooperation had failed. The newfound kingdom, now ruled by the three original knights along with their followers, could not understand why they were shown such hostility. After all they had left because their ideas weren’t welcome in their former homeland. They had left after they had given up on their proposed reforms. They had decided that it would be wiser to found a new realm where they were free to pursue the path they wanted, rather than letting the other kingdom succumb to internal struggles. How could this be a bad thing?

However, even in spite of their attempts to separate their experimental ideas from the olden realm, they were not left in peace. Rogue elements decided to use their mightiest tools in an unprecedented attempt to once and for all crush what they saw as a rebellion. Conjuring up their mightiest servants and using their most powerful spells, they launched a full-scale attack on the upcoming nation of the three valiant knights. The attack hit hard and swift: several gulls flew past the highest tower in the knights’ castle and dropped their excrements over it…

To be continued, once we find out how this is going to end.

Squashing some rumours

It’s been a little more than a week since Bryan announced Exherbo. The general interest was quite overwhelming and along with it came a lot of questions that had to be answered. I think most of it was covered quite decently but of course a certain amount of FUD was spread as well — both by people who should at least be somewhat informed and by people who didn’t quite get what we talked about. I’ll try to answer some of the common misunderstandings and hopefully help reduce the FUD a bit. I won’t delve into any technical aspects as Bryan, Ciaran and others are far more capable of explaining those.

If you want the short version of this, Mike Kelly already wrote it.

We don’t need yet another distribution
Quite the contrary. You might not need yet another distribution if you’ve already found one that suits you but those of us working on Exherbo definitely do. We want a distribution that fulfills the goals we stated on the website and to our knowledge, no other distribution does that. We’re not really interested in being bombarded with “Ah! But then please try distribution X – it will even cure cancer!”-messages at the moment, as we’re already past that stage. Even if no-one else finds our work useful it will still be useful to us and that’s what matters really. I hope that once we reach a somewhat stable stage, the result will be usable by others as well, but a closer study of our goals will show that “a billion users” and “a large thriving community” aren’t really something we aim towards.
Lastly, it’s also our hope that other distributions might benefit from the experiences we make. We are currently toying with a lot of different approaches to some common problems and hopefully some of our solutions will turn out to work better than what’s currently on the market. People are obviously welcome to look at what we’re doing and grab ideas from us (this being an Open Source world an’ all).
Oh, and a personal rant: Why is it the knee-jerk reaction to all new initiatives is “why do we need yet another”? There’s a reason we’re not still using Multics, there’s a reason we’re not still writing on stone tablets and that’s because someone was tired of carrying the stone tablets and thought he could do something smarter.

We don’t need yet another package manager/package format
This is mostly stated by people outside the gentoo community. Paludis has supported the gentoo ebuild format for a very long time and as such, Paludis has had lots of real world testing and is already technically superior to pretty much any other package mangler out there, at least in our opinion. Paludis is capable of supporting the rapid development process we’re currently in, where our exheres-format tends to change quite a bit as we introduce new features, deprecate others and modify the remaining lot just to see if we can do things even better than we currently are.
The exheres format isn’t really a package format in the way rpm, deb and the likes are. Exheres, like gentoo ebuilds, are advanced bash-scripts conforming to a certain EAPI that the package manager uses to download, configure, compile and install the packages. The exheres-0 EAPI is a fluent EAPI that will never be locked and can always change. But when at some point, we feel comfortable with what we got, we’ll create an exheres-1 EAPI that is locked and well-defined. This will allow other package managers to implement the EAPI if they want to and it will serve as a source of documentation for those writing exheres-1 exheres. This has another subtle but very nice side-effect: we can update our format without breaking existing implementation. A new exheres-2 EAPI could be made and rather than having old package managers choke on it and screw up systems, they can just bail out with a warning or error message. To gentoo people, this won’t sound too unfamiliar as the EAPI-concept is also somewhat working in ebuilds.
The ideas used the in the exheres-0 EAPI are based on some very smart peoples’ longtime experiences with gentoo and its shortcomings. So in conclusion, we’re not really trying to reinvent the wheel here, we’re trying to take a square wheel and turn it into an octogonal shape. It won’t run perfect right now, but at least it’s better than what’s currently out there. Remember that Exherbo is a source-based distribution so using debs, rpms and the like doesn’t really make sense.

We don’t need yet another init system
I’ll save the details and the fun for Bryan and his talk at FOSS Aalborg, but yes – we do need another init system. One that can accommodate modern computers and their needs. Check out Bryan’s blog after the 4th of June and judge his ideas there. Until then, kindly stfu.

Exherbo is all about pissing gentoo people off
Not at all! Pissing certain people off was a completely unintentional side effect.
In seriousness, we’re not really trying to make anyone mad, but if people take offense by the views we present then we can’t help it – that’s their problem and not ours.

Exherbo hates me! It says so on the website
No, what it says on the website is “It’s not that we hate you (unless we do)” which is quite the opposite. We don’t hate those who just want to try out Exherbo or those who are interested in the development of Exherbo. Nor do we hate those who doesn’t give a flying fuck about Exherbo. Those we do hate should be well aware of it and those who act like morons in our IRC-channels or on the mailing lists will swiftly receive some hate. But as the general rule of thumb you are not (explicitly) hated.

Exherbo isn’t a real distribution, you can’t install it yet
On the contrary! I’m writing this from Opera in Fluxbox on my Exherbo install. It’s true that we don’t provide installation mediums but those aren’t really required anyway. And Ciaran actually wrote a bit about installing Exherbo. But as we repeatedly state Exherbo isn’t fit for general use (yet) and we won’t support you. Questions on installing Exherbo will usually be ignored and in rare cases frowned upon or put up for public ridicule on our quotes page along with all the stupid things we’ve said :-).
That being said, we have a few useful packages in our repositories, but we’re still experimenting with more fundamental things at the moment, so it’s not a focus. Even once we reach a somewhat stable EAPI, it’s likely that we’ll hesitate to add a lot of generally useless things to our main repositories. We have no intention of growing an enormous developer base and adding tons of stuff only a few people will ever use. We have no intention of supporting 12000+ different packages but we want to make it easy to add new packages so that the general user and interested subgroups can maintain things they need themselves.

Exherbo isn’t fit for general use
Quite a few people have looked at our website and said this as if it was some huge revelation they reached after meticulously studying the content of the site. Allow me to quote: Exherbo is not, at the moment, a user-targeted distribution. There, it’s right there on the site, even on the front page. So how come some people see this as a well-hidden truth they’ve uncovered? It’s true — Exherbo isn’t fit for general use and that is why we want you to go and use ubuntu or gentoo or something else. It’s not because we hate you (unless we do) but we really have nothing to offer a general user right now and general users have nothing to offer us.