Github: too many forks?

I recently created a Github account (check me out!) and have become familiar with the site because it hosts a lot of Django-based modules.  At first blush it seems like a programmer utopia:  the code is open source and easily viewable, forking is encouraged and there is a somewhat intuitive interface for following them, wikis and issues are automatically created for each project, and, thanks to git, forks are (usually) easy to merge.  There's a lot of users, a lot of activity, and a lot of prolific and talented programmers on github. 

So what's not to love?

Really, Github does seem to be the cat's meow of the open source world at the moment, with a nod to BitBucket, where Mercurial-based projects are hosted.  As I've said I've only recently begun to use Github, so I've discovered it has some significant oddities for someone just joining the ecosystem.  I'm surprised there hasn't been more discussion of this, a quick search of the interwebs only really turned up this post by Andrew Wilkinson.

I'll summarize his points and then add some spice.

• Coders are "rock stars" that are emphasized over their projects and the interface is designed from a contributor point of view.  He goes on to later point out that projects of any decent size typically have more users than contributors, so the interface is a bit quirky for someone just wanting to use the project.
• Each fork gets its own issues and wiki, making it confusing where to discuss the project.
• Determining which fork to use is not trivial

The interface issues of the first I don't have much of an issue with because it sounds like Github is working on it; they've even made some recent improvements on this front.

The second is definitely a problem, but I think is a necessary one.  A fork potentially has its own features and bugs, so these need to be put somewhere.  However, anything not specific to the fork should be put in the wiki/issue tracker/whatever of the main project.  I think this would be less of an issue if the next point were fixed.

Finally, I come to the heart of my Github confusion:  which fork does one use?  By 'use' I mean either fork and contribute to or, as a user, download and install.  The developers have said they are working on a way to better identify the "main" project, but the solution is yet to be seen.

I'll show how I find the "main" project, and then examine the issue itself.  There are two "find main" methods that probably need to be used together.  The Network graph (read up on it here, you really need to understand these graphs to understand the my later figures) is not the place I start because it is relative to the current project, it will be used in a bit.  What I mean by "relative to the current project" is, if you're looking at a project that is say the fork of the original project, all the commits for the original up to the point that the fork occurred are put into the forked project's timeline.  Thus, I first try to find the "grandfather" or original project that started the chain.  I do this by following the "forked from" links until I get up to the one that is not a fork of another.

Currently looking at dcramer's version of the project, a fork of robhudson's, which happens to be the grandfather.

After getting to the grandfather, I then look at the Network graph, which I consider clearer now because all forks show only their own commits.  Typically the grandfather is the "main" version of the project, but occasionally a grandfather will become dormant and a fork will become the main line of development.  Look to see whether the grandfather is still being updated, forked, and other forks are merging back.  If not, see if another fork has taken over this position.  If not, find the fork with the bug fixes and improvements that seem best, as it will probably become the "main" version as other people make the same conclusion as you (hopefully).  Another thing that may be helpful is checking the number of watchers of a project/fork, this can give you an idea of its popularity.

Okay, so hopefully you can see this is about as clear as mud and a rather inexact science.  Wilkinson's "rock star" description is apt:  projects are first identified first by the coder--robhudson's django-debug-toolbar--rather than the project itself.  Admittedly this makes sense based on how git and forks work, but it leaves the interface muddied.  Whom do I trust?  robhudson or dcramer?  Side note: I'm glad people generally use their names or sensible nicknames as identifiers, if "l33tskillz393" had a fork I don't think I'd even give it the time of day.

To further clarify lets look at some pictures of the Network graphs for a couple of projects I've looked at recently.

(django-pagination)

What I have identified as the grandfather and main branch is the line on the top.  There are more forks not shown here, but none below hgrimelid's have any "recent" commits.  It is pretty clear here that the grandfather branch is the "main" version of the project:  past forks have either died or merged their changes back in (merges can be seen on the blue and neon green lines in the upper left).  There are some recent forks off the latest grandfather commit, possibly with important bug changes or features, so it makes the decision a bit less clear.  Go with the main branch and assume important changes will be merged in a future version, or go with a fork and hope it doesn't turn into a dead end?

Lets look at another with a slightly different situation.

(django-sorting)

The grandfather is again on top.  But this time the grandfather looks a bit outdated, no updates in months!  Looks like a very active project though, plenty of forks--and forks of forks!--being made and updated.  I do not think there is a clear choice.

Github has highlighted an unforeseen problem with distributed version control when the participants aren't under some guiding light, such as working for the same company.  The traditional model of a project is that there is some entity--a person, a committee, a company--that determines a project's versions, features, etc.  Distributed source control may be used to develop the project, but at some point someone says "this is the next version" and everyone trusts that authority.  This can be seen commercially in, say, how Microsoft releases new versions of Windows every so often.  In the more complicated world of distributed version control, look at what git was created for developing in the first place: the kernel.  It gets all kinds of forks and such but in the end the idea is that they get merged back in to the mainline kernel, Torvalds baptizes it, and distributions push this authoritative version out to users.  In this traditional model, users and programmers really only care about the project as a whole, not the forks that went into it.

Github flips this on its head, I assume because of the "rock star" approach.  Each programmer is given equal stage and there is no definitive project.  Without an authority people go on their merry way and we get these spider webs of Network graphs.

I'm going to eagerly await Github's "find the main branch" solution.  My admittedly rather warm-and-fuzzy suggestion is to strongly encourage merging forks.  Traditionally forks have been a big deal because they split a project's development due to legal reasons or vision disagreements or whatever.  But the assumption is there is no other recourse and no reconciliation.  Perhaps this trained behavior is one reason for the lack of merges?  Anyway, forks usually make significant changes to a project that are not necessarily meant to play nice with the original project.  In the Github world forks are THE way to update projects.  Thus forks are not splits, they are the way you do even small-scale things like fix bugs and add minor features.  These are things that should bubble up to the main project, not languish in a soon-to-be-forgotten fork.  It seems from my own experience that people fork, fix the bugs they need for their personal use, and then forget about the project altogether.  If you look at the figures I've provided you see an abundance of forks, but merges are rare.

If Github could convince all the forkers to be mergers it would be a much happier place.

Converting color spaces in C++

I recently posted my first bit of open source code on my website. It came about because I've had to do some converting of RGB colorspace into HSV for purposes of histogram matching for my research. There is code out there that can be found through The Googles but I've never been satisfied with any of them for various reasons. I've tried to make my code much more readible, straightforward, and clean than whizbang-look-at-all-the-c++-tricks-I-used code obscurity. I've also added the ability to convert RGB to HSL and back. I didn't see much use in having HSL<->HSV but this would not be hard to get by using RGB as a middle man. Have a look at it, use it, improve it, but make sure you follow the license! RGB-HSL-HSV converter project or the direct c++ code.

Web needs a paradigm shift

I was browsing the [H]ardForum earlier today and something dawned on me. There's all kinds of buzz amongst web-heads about the new CSS/HTML/whatever standards and Apple is doing custom CSS additions to allow for animation in WebKit (bad Apple! follow the standards!). My realization is that CSS cannot be the future of the web.

CSS is an extremely powerful way of expressing design and layout for a site. However, from personal experience and seeing a bazillion posts asking for CSS help, it is just too complicated or confusing to consistently get correct. Long ago we learned tables were bad for layout, we should use div instead. However, there seems to be no agreement as to the best way to use them and sites are often accused of overusing them. That's just one example, forums are littered with people asking why they can't get item x to overlap, float, stay put, or any other number of things on their pages and it seems the solution usually ends up being something obscure or unintuitive.

Some stalwarts would write this off as people not knowing what they are doing. But isn't computer science about making things easier on ourselves? Programming languages have moved from the miserable days of low-level languages to the immense spectrum of high-level ones we have today. Why can't the web do the same thing and unburden some of the mundane work? Anyone who has looked at a CSS file from a large website cannot claim that it's easy to get the gist of immediately with a straight face. One example of a CSS failing may be that there's too many ways to do things. It should be moving more towards Pythonic thinking than C++. Yes allowing for 1, 2, or 4 arguments for, say, padding space is a good shorthand but it's not obvious as to which order they are in or how the 2 maps to handling the other two dimensions.

Right now I cannot propose a new solution or ways to fix this. But I do know that we can do better, much better.

Python tip: splitting strings with whitespace

This has gotten me so many times now I figure I need to write it down. Plus, it'll be here for everyone else's benefit.

Examine the following Python code:
>>> s = "foo bar spam eggs" # two spaces between bar-spam, three between spam-eggs
>>> s.split( " " )
['foo', 'bar', '', 'spam', '', '', 'eggs']
>>> s.split( )
['foo', 'bar', 'spam', 'eggs']

Note that if you split on " " as in the first command, Python will split only on the first space in a span of whitespace. After the first, subsequent spaces are placed as empty strings into the returned list. However, if you use the split with no arguments, each split eats all connected whitespace.