Skip navigation

In response to:

I usually don’t want to be the one sending out stop energy but TFA is so ridiculously crack-inspired one has to wonder if those involved are on the same planet as we are.

This seriously reminds of Douglas Adams and The Hitchhikers Guide to the Galaxy: a planet(a giant computer ran by rats) was built to find the answer to Life, The Universe, and Everything. After eons of calculations an answer was found, but none could remember what the problem was, so another planet was built to find the question that matched the solution found by the other planet. Unfortunately immediately prior to the successful completion of this project that planet was destroyed to make way for a hyperspace bypass.

Just a couple of notes in passing:

1) Differences in package management, repo structures and the tools involved in installing and removing software are the primary differentiators of Linux distributions. If this “problem” was solved, there would be remarkably little left to differentiate the distributions. And the only people interested in having such a “solution” are the one’s who already think that the LSB has already provided the answer to Life, The Universe and Everything.

2) On a superficial level there are ample opportunities for distros to reach practical consensi(pl?). XDG has been achieving this and making some nice progress for some time. These solutions do not try force distros to neuter themselves but rather to work together where there is practical benefit for all involved.

3) There will never be a single unitary package management system for Linux. Each of the various package management systems have advantages and disadvantages, each have their own problems, and each enable certain things which other systems do not. This is not merely chaos or mayhem- there is method to this madness: The organizational, social, and economic structures of the various distributions are mirrored in the particular package management systems of these distributions-in a sense, and very important sense really, these factors pre-condition each other. In other words the package management system *is* the distribution, ontologically speaking. The reasons for these differences reach into the raison d’etre of the distributions themselves-which ultimately devolves into questions of the viability of the distribution itself -whether socially, or economically.

4) Never use force to achieve what people are inclined to do anyway. With each version change in the fundamental libraries which make up the Linux ecosystem new functionality is exposed and form the basis for advances in the system itself. The individual distributions strive to take advantage of this infrastructure and utilize it for their own profit. This means that despite the rather large differences in the package management systems of the various distros that there is rather high degree of homogeneity in the version numbers of the fundamental core libraries which constitute Linux across the vast majority of distros. The differences in which actual versions are used are primarily a function of time-if the window of comparison is based on a one year time frame this homogeneity is *almost* universal, whereas if one takes 6 months as the basis there is a minor degree of diversity. Massive breaks in ABI are seldom and are surrounded by years where this is not an issue-far less often that Microsoft brings out new versions of it’s own OS.

5) the moment that any third party software mandates that my system conforms in some non-superficial way to some standard of which the distribution I am using is not part of is the moment that that software dictates which distribution I must use in order to have the priveledge of using that software. This is already the case for software like Oracle-I must use specific distributions which are endorsed by Oracle if I want to use Oracle software. Changing the distro I use for something which costs me many thousands of dollars is probably not unrealistic-but for me and the vast majority of Linux users there is no demand for extremely pricey proprietary software which would justify such. Back here, in reality, third party proprietary apps have to play ball according to our game. This does not mean that a lot cannot be done to make it easier for third party proprietary apps to integrate well within Linux-and I am all for such steps, but the LBA does not represent Linux-it represents Redhat and SUSE and to a lesser extent Debian and that’s it.

If you, as a third party propietary app developer, want to get wide scale adoption of your software in Linux then hire people who will get your software into the repos of the major distributions and provide tar balls for rest of us. Make use of the XDG tools available, target a rather actual core of main libraries (ie. not gcc-2.95) and target the infrastructure behind KDE or GNOME. If your software is enterprisy target Redhat, and SUSE(SLED) and perhaps Mandriva if you are feeling liberal-forget about the rest. If your software is for mere mortal beings target fedora, opensuse and debian/ubuntu. If you make software FOSS the community will do the work for you if your software is good enough to convince the community to support it.

If i have failed to address what the the author is looking to solve, please help me to see the problem, because I cannot find the problem which is being addressed by this solution.



    • Rufus
    • Posted June 26, 2008 at 2:00 pm
    • Permalink

    Your post proves that you have no clue on what you’re talking about. There are quite a lot of problems with centralized packaging under Linux today.

    First, there are applications that need immediate updates. For example, I needed to install a new version of an open source tax-related application: The old version just worked for 2007, but I needed to report taxes for 2008. The repositories were useless because nobody packaged the new version. I had to compile from scratch.

    Second, quite a lot of potential interesting applications are not in the repositories. Just recently, somebody gave me a list of multimedia applications that were supposed to be useful. A quick check revealed that roughly 40 % of them were not in my repositories so I couldn’t install them without hassle. What use is an application one cannot install?

    Third, getting in the repositories is also a marketing problem for open source application developers: A new release is boring if there’s no easy way to test the new version — that means to be able to install it. Boring releases don’t get promotion, thus the applications remains small and may ceases to improve.

    Forth, another problem is the lack of proper CoverCDs for Linux journals: Journalists either have to waste their time describing boring technical installation procedures or waste their reader’s time by letting each of them find out themselves. Additionally, new applications don’t get much promotion.

    Sixth, even if there’s a package in your distribution’s format, it may still be hard to install because you’d need some updated dependencies, as well. That may require the de-installation of older versions which in turn may break your distribution or any of its functionality.

    You say, the differences in package management constitutes a distribution. That may hold for some minority distributions such as Gentoo. Or it may hold for distributions dedicated to closely defined areas, say Web servers or Mail servers.

    However, package management should just work for the mainstream desktop distributions. If package-kit makes it in those mainstream distributions, which user will note differences in the underlying package system?

    However, what people are likely note is the amount of pre-packaged applications for a single distribution. Unless a user knows what applications he will install in the next 4 or 5 years, he better chooses the distribution with the most momentum. Suppose every users argues this way, then the majority will use the distribution with the most momentum. Additionally, application developers will package for the distribution with the most momentum, too. That, in turn, proves the argumentation of the users.

    The result will be a Betamax-effect: Small desktop distributions will die, only one large distribution is likely to survive.

    You already noted the problem in your point 5) — people are interested in real solutions such as Oracle. They want iTunes, Photoshop, Skype, etc. Linux distributions without ISV support are bound to remain small and irrelevant, while one large distribution will rule the rest. You may not bother about Oracle and Photoshop, but you’re part of a minority, here.

    In other words: the fragmentation of package management solutions will just result in a monopoly of a single desktop distribution — a small Microsoft, if you like.

    I call this a problem.

    • iwbcman
    • Posted June 27, 2008 at 1:58 pm
    • Permalink


    Firstly I wish to thank you for responding to my post-your response if the first I have received and I am looking forward to actually engaging in conversations with people as opposed to simply posting monologues.

    Obviously are perspectives differ greatly on this-yet be assured I *do* have an idea about what I am talking about. As is the case any time one writes something one cannot fully disclose the context within which something is written-and this context is usually pivotal for understanding the perspective taken in the piece written.

    Let me give you a bit of background as to why I took this stance and then I shall try to respond to some of the points you made. Part of what triggered my response was the the name of the project proposal: LSB package API.

    I am not a fan of the LSB and although I do see it’s utility in regard to an extremely narrow user market I rather fundamentally disagree with the way the LSB has been managed and many of decisions the LSB has made over the years.

    My main problem with the LSB, however, is that the success of the LSB bodes ill for the vast majority of distributions which are not now and never will be LSB compliant distributions. The LSB has to date mandated RPM as *the* package format for standardization purposes. For years the LSB refused to include QT into the standard.

    I will freely admit that I am not up to date on what is included in the latest LSB spec. Perhaps the situation has somewhat improved-but I seriously doubt it. The goal of the LSB is almost rather noble-but I fear the price of attainment of this goal is a threat to the nature of Free Software.

    10 years ago there was much more in the way of common infrastructure shared between the various distributions than there is now. The tremendous diversity in the Free Software community is what, in my mind, which makes this community so viable and dynamic. The LSB wishes to *bless* certain core aspects which define an OS and to standardize on these to present a highly conformant OS environment which is supposed to be distribution neutral. The fact, however, remains that only a handful of distributions are even in a position to implement the LSB and those that are, are conveniently the same groups which have proffered the LSB. For the “enterprise” market niche LSB *is* the answer. But I would contend the this niche is 1) not what Linux is about 2) not representative of the majority of Linux users and 3) something which actually threatens the diverse ecosystem of Free Software.

    Now I am pragmatic, to an extent. To the extent which various distributions can make superficial changes to their distributions in order to ease compatibility with the LSB I am all for it. Things like FHS are wonderful things-even if a) no two distros really implement this the same way and b) major proponents of the LSB like SUSE have the most crack-inspired FHS implementation known to mankind(just look at the horrific duplication of files-links going everywhere-a veritable nightmare).

    So for me the question becomes: how can one reach reasonable consensi which would enable greater inter-distro compatibility. LSB strives for ABI compatibility-Application Binary Interface. So they want binary compatibility.

    For me source compatibility is the necessary prerequisite for any question regarding ABI. If the compilation toolchain and how this toolchain is compiled varies too greatly then ABI compatibility becomes impossible.

    Traditionally SUSE and Redhat have made liberal use of statically compiled binaries-in many cases shunning the rich environment of shared libraries already available in their distributions. Statically compiled binaries are far more portable than their shared-lib counterparts-the higher one climbs in the dependency stack the more difficult it becomes to provide binary compatibility.

    If a 3rd party app only uses libc, openssl, and perhaps QT, then it is very, very easy for this app to be highly portable between various distributions. Yet if the same app uses all of the libraries which provide for meaningful desktop integration it becomes quite difficult-for each distro must provide each of the libraries upon whih this app is dependent and they must be compiled similiarly (which ./configure options) using a tool chain(gcc/glibc/binutils etc) which itself is similiarly compiled. If distros compile their software with the –as-needed linker flag this too facilitates inter-distro compatibility. But the real way to compatibility is to get the distros to commit to coherent toolchains-to solid build infrastructure, where any given program can simply be ./configure /make /make install -ed.

    So why am I talking about source when the LSB is all about binary compatibility ? There are many reasons:

    If 3rd party apps are not using the rich diversity of shared libs available in a given distro that app has no chance of actually integrating with the rest of the applications and the desktop itself. We do not want 3rd party apps to be aliens in the midst of our cozy ecosphere

    Things like printing, multimedia support, internationalization, support for people with disabilities are all great strengths of our system-apps that cannot or do not take advantage of this infrastructure -and contribute to this infrastructure are at best leeches which beset users with usability nightmares.

    To date the majority of 3rd party apps( apps not bound to distributions or bound to desktop projects) have been a continuous nightmare in terms of usability and a serious impediment to establishing coherent and cohesive integrated desktop environments.

    And the reason for this are simple: these apps have not used the available shared libs-they have reinvented the wheel time and time again- and why? Because they were statically compiled in order to minimize dependency issues and maximize compatibility.

    I honestly believe given the complexities hidden in the question concerning universal package management that a complete rethinking is required which goes far, far beyond anything that the LSB can or should address.

    Firstly I think the community needs to make clear what it actually wants-do we want 3rd party apps merely in order to have that which is not provided by Free Software or do we want 3rd party apps to be integral parts of our system, which make use of and contribute to the rich ecosphere of available infrastructure and which compliment the cohesive and coherent integrated desktop experience. If we choose the latter then this has consequences and poses certain challenges. This means that these app developers must be encouraged to make use of existing Free Software to interface with the rest of our desktop and applications. If they believe that their product contains important IP, black magic, then there code should be made in such a way that they reuse available shared-libs for the infrastructure of their app(ie. dialogues, printing, gstreamer/PA etc.) and find ways of encapsulating their black magic within this Free Software infrastructure. If the 3rd party app developers would be prepared to do this and actually use the Free Software available the community could and would provide much more than these 3rd parties will ever do themselves: we will translate their interfaces, we will make sure their software is usable by those with disabilities, we will work with them so that their software become first class citizens in the FOSS world-and given this kind of community interaction there is no question about such software being included in the repos which form the basis of the distro package management.

    This is the approach that major 3rd party apps should take(adobe flash/pdf reader, skype and similiar desktop applications).

    Secondly, there exists a FOSS technology known as Conary. This tech makes it possible to reproduce a coherent cohesive build environment inside of any existing distribution and could provide a kind of binary base which is far more rigorous and self-coherent than any thing the LSB could even dream of. Granted Conary would need to be endorsed by the major distributions to make this a reality(Novell is already working with them). But if there was significant uptake of this tech 3rd party app developers could simply provide conary recipes and thanks to the wonders of automagic these app developers could take advantage of a universally available ABI environment for producing apps that would be guaranteed to run on any distribution. Not all of the pieces to make such happen are in place now. But the work required to make such happen is infinitesimally small in contrast to the approach taken by the LSB which will never be all inclusive. Oracle could produce a conary recipe for their software and it would run identically on any machine/distro it was installed.

    Lastly if distributions would implement sane build environments(ie. build environments capable of recompiling themselves) and engage their creative minds it would be simply to produce a system which for small software packages-like a tax preparation tool-could completely automate downloading the source and compiling and installing it-if the software in question uses the existing Free Software infrastructure. Nvidia has been doing this successfully with their driver modules for years now.

    Ok let me sum up and then try to touch on the points you raised.

    1) we need and desire for 3rd party apps to properly integrate and make use of existing infrastructure-every app that does so is a net contribution to FOSS-even if their application includes bits of black magic.

    2) with some update by Redhat/fedora, SUSE, Debian/Ubuntu it would be possible to use tech like Conary to provide a sandboxed environment which was 100% ABI compatible across distros. If the 3rd app developers would use this system for creating their apps their compatibility concerns would be neglible-and there would be no concerns about getting their software into existing repos.

    3) For smaller apps, if the distro would provide decent build environments it would be simple to completely automate the compilation and installation process.

    Your first point is addressed in my 3rd point. On a recent machine, say of the last 4 years, it probably takes less than 3 minutes to download, compile and install-if the distro itself does not make this insanely difficult. Conary provides tools which developers can use to identify dependencies in source code- using such in conjunction with mapping such dependencies to the distro’s repos-this could be made to work so automatically that for smaller apps you would never even notice they were being compile -you simply click on a link, the source is downloaded, a tool analyzes the dependencies, triggers the distro package management to install that which is available and compile that which is not and recurse this procedure until the app appears in your menu.

    Regarding your 2nd point: if the app developer used conary to build their software they can self-publish their software to a rpath repo-they could a link on their web page which the user would click on enabling their self-published repo in rpath-the app itself and all of its deps would be dynamically resolved and installed. Prerequiste for this is more uptake by the major distros to facilitate cohabitation of a conary sandbox inside of their distros.

    Your third point would also vanish if the app developers could self-publish their own rpath repos for use with conary.

    Your fourth point is even simpler-simply include the conary build environment on the cd-one click the environment and the apps dependent upon it appear on your hard drive and in your menus.

    Your last point: if there was more widespread uptake of conary within various distros you could have multiple versions of any given library side by side without any conflicts-mitigating the dependency hell which is common in all binary distros(which is why I avoid binary distros in the first place and use Gentoo- a source based distro).

    Package kit simply obscures the realities of package management behind a pretty UI. I actually like it and would have no problem if it was to become widespread. What I said about package management *being* the distro is not merely a matter of perspective.

    Their are social, economic and technical realities behind each package management system. Ubuntu does not have the finances to implement the same package management system which Redhat and Novell have.

    Central servers containing tens of thousands of binary packages -the bandwidth and server storage for such centralized repos is something which only large corporations can afford. Gentoo uses tiny text files which describe how to compile apps-anybody can write their own, and anyone can host their own repo-things which make Gentoo possible.

    What you or I think about the command syntax of the package management system or the relative merits of RPM vs. Apt-Get is completely orthogonal to social, economical and technical structures which distros embody. You say you want things to “just work”-my answer to you pertains to how one can create this illusion in such a way that such a system is viable-technically, socially and in the last instance econcomically.

    For me the LSB represents an attempt to create a mini-Microsoft- it excludes the majority of distros, for most distros will never have the manpower to implement conformity to their spec. The LSB works against the diversity which makes the FOSS world so wonderful and rich. The approach taken by the LSB will not create a “just works” experience for anyone who is not using an “enterprise distro” of which there are only 2 or 3.

    In my post to which you responded I argued that this is a solution looking for a problem. I stand by this- what is a problem for you as a user is completely different from the problems facing 3rd party app developers. Any approach which does address both of these perspectives will fail one or other other and in all likelihood both. Automating the process of clicking a link on a web page to install software is easy-it is so easy that it is not worth arguing about. But creating an infrastructure which app devs can use to create their software and which users can use in order to have access to such apps is an entirely different thing.

    If 3rd party apps do not integrate with our existing infrastructure they cripple our desktops and beset our users with usability nightmares and make our admins cry when they go to sleep at night. If these devs are not enabled to contribute to our existing infrastructure they are dangerous parasites which impede the FOSS communities ability to innovate and make progress. If app devs do not have easy access to tech to enable to reuse existing infrastructure and contribute to it we will not see high quality 3rd apps in Linux.

    Oracle mandates that you use Oracles version of Linux or 1 or 2 versions of 1 or 2 distros. This basically means they dictate what their users can use.

    Flash does not support 64bit, does not support non-x86 systems, and has only begun to take advantage of the audio API’s which are in use on modern Linux distros. Look at how our distros are forced to bend over backwards, with incredibly complex system hacks, and endless bug reports just to get their distros flash enabled. This is insane-with friends like that who needs enemies.

    If it was not for the demand for these apps comming from users no distro in their right mind would ever go to such trouble to support it-especially given that these 3rd party app developers themselves do not really support their own products. If they did the distros would not be forced to put themselves through such hell just to get things to work. The hell that these 3rd party developers face producing apps which are inter-distro compatible is small fry compared to the hell the individual distros go through in order to support their oh so compatible apps.

    Their will never be a single universal solution to this “problem”- there are multiple problems, and the “who” for which these problems exist are manifold(users,devs, admin, distro maintainers, translators etc.). Distros should work together where it makes sense in order to facilitate inter-operability and compatibility. Tech like conary is one way of approaching a variety of these problems(consistent build environment, self-publish of repos, fully automated building of software with high guarantees of ABI conformity).

    But we need custom solutions for custom problems-what is needed for Oracle is not the same as is needed for Adobe Flash which is not the same as what is needed for the tax preparation program you spoke of. If we can find solutions for all of the participants we can easily create “just work” illusions so that end-users never even have to know whats going on.

    • iwbcman
    • Posted June 27, 2008 at 2:03 pm
    • Permalink

    minor correction:
    Any approach which does *not* address both of these perspectives will fail one or other other and in all likelihood both.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: