Analyzing HaskellĀ stability []

Stability - according to gpt4o

I joined the Haskell Stability Working Group about a year ago. I was freshly scarred from a GHC upgrade, so I decided to try improving the situation. I think my first contribution was making an example for GHC nightly builds on GitHub Actions. This allows downstream1 maintainers to build GHC nightly to spot issues early.2

Recently, we’ve been curious about what Haskell ecosystem and GHC changes cause the most breakage. A suggestion was made to do a quantitative analysis of “head.hackage”. “head.hackage” is a repository of patches for Hackage. GHC engineers use these to test out new GHC builds on a wide range of Hackage packages without having to upstream3 a patch, which can take time. Instead, they can put the patch in “head.hackage” and immediately test it on a wide range of packages. We can analyze this repository of patches and categorize them according to the GHC Stability State of Play I volunteered for that effort. Here is a preliminary result of the “head.hackage” patches required to make around 485 packages4 build.

From this, we can see that we’re not doing too bad for stability in GHC directly. Many of these patches are for older changes for example. Although, from first hand experience, I know we can definitely improve! We’re going to extend this work by labeling the exact cause of breakage for each patch (many do overlap). This will allow us to see more clearly which changes caused most issues and help us prevent similar situations in the future.

Do you have any opinions on this document? do you feel stability could be better? or do you have a specific frustration which grinds your gears around stability? Please reach out! We’re always interested in hearing from the community and are in fact actively recruiting members. You don’t need to join; you can also email or use the comment box below.

  1. Downstream in this case means people who use GHC and are dependent on it. ↩

  2. At first, I thought these mechanisms were great. However, at some point upstream GHC started breaking for some reason and I learned that the availability was ‘best effort’, causing me a bunch of issues. So I turned it off. But I applied these “GitHub Actions” techniques I learned for various other repositories. I learned that for projects that have little activity, github likes to disable automations. Since 99% of my projects are like that, it essentially makes all of GitHub Actions kinda useless for me. I had a pretty bad experience with GitHub Actions before all this, and this made me even more estranged. It’s unreliable compared to the Nix CI pipelines I’m used to. ↩

  3. Upstreaming is the process of sending a patch to the “maintainers” of an open-source project. The maintainers will then make the patch ‘official’ by merging it. In principle, the process is simple, but in practice, the burden of proof (especially for larger projects) is on the person who submitted the patch. They have to convince the maintainers that the patch is useful, which takes time in the form of communication. ↩

  4. The estimate of 485 packages is done based on this Grafana dashboard. The number changes over time. this measurement was made on 2024-07-28. ↩

Haskell programming

Recent stuff

Tags

arubaaustraliabuild-toolscssdatabasedevopsfrpHaskellindonesiajakartajoblinuxmysqlnixnixosopinionpainpragmatic-haskellprogrammingprojectrealestatereflexreleaseservantstacktech-proposaltesttimetoolstraveltutorialvirtualizationwebsitewindowswork