Skip to main content

Search

Items tagged with: xz


If FLOSS is built on the four freedoms, and FLOSS has created an environment that is brittle, then perhaps it’s time for FLOSS to similarly augment the four freedoms.

We have to address this in a fundamental way. The alternative may well be the (eventual) end of FLOSS as we know it.

interpeer.io/blog/2024/04/in-s…

#xz #FLOSS #FourFreedoms #sustainability #robots


Here's a thorough analysis of all the commits by "Jia Tan" from 2023-08 through 2024-03, showing the many legitimate code changes done before the introduction of the #xz #backdoor:

tukaani.org/xz-backdoor/review…


Excellent summary by Solar Designer on oss-security of what's happened in the last two weeks in response to the #xz #backdoor:

openwall.com/lists/oss-securit…

Noteworthy:
- #OpenSSH implemented systemd notification
- #systemd moves to dlopen(3) for some dependencies
- another detailed timeline at research.swtch.com/xz-timeline
- similar social engineering takeover attempts suspected in #OpenJS and #OpenSSF


At Sovereign Tech Fund, we're following the #xz incident closely and listening to the many voices in the #FOSS maintainer community.

What's clear to us is that the xz incident shows the need for structural change:

sovereigntechfund.de/news/xz-s…

#foss #xz


I am getting tired of reading about the #xz #security issue as if it is all about issues within #opensource. It is much bigger than that, and those takes conflate the problem with the solution.

So I wrote "The xz issue isn't about Open Source" here: changelog.complete.org/archive…


"You have to understand, we’re responsible for taxpayer money here. We can’t just make a donation to your open source project."

— a national government who relies on #Matrix when being asked to support it financially

Read more about the problem and some initiatives that are responding to it:

matrix.org/blog/2024/04/open-s…

#FreeSoftware #OpenSource #FOSS #FLOSS #funding #xz #sustainability


Open source infrastructure *must* be a publicly funded service, and funders need to support maintenance – not just new feature development 📣

This is on our minds this week in the wake of the #xz news, and as we continue to seek funding to support #Matrix.

Read the latest from project lead, @matthew: matrix.org/blog/2024/04/open-s…

#OpenSource #FOSS #OpenStandards


"What it means is that there is no supply chain here. Because there is no supplier. I am not providing you something that you bought from me. There is no relationship. I put something online because I wanted to. The fact you made your product depend on it is your responsibility. Not mine. Not the one of the providers. We provide libraries. We do not supply them. You cannot apply rules to me. […] So all your Software Supply Chain ideas? You are not buying from a supplier, you are a raccoon digging through dumpsters for free code. So I would advise you to put these rules in the same dumpster."


🔥🔥🔥

softwaremaxims.com/blog/not-a-…

#xz

#xz


Any experienced C developers among my followers? #BoostsWelcome.

Expat, arguably the world's most popular #XML parser, is understaffed and without funding. As #xz has shown, situations like this are dangerous.

Last month, maintainer Sebastian Pipping put up a plea for help at github.com/libexpat/libexpat/b…

(I would help myself, but my C skills barely surpass "Hello, World".)

Found via @timbray - cosocial.ca/@timbray/112203547…

#libexpat
#SoftwareSupplyChainSecurity #OpenSource #OpenSourceMaintainer
#C


I think the #xz incident is teaching us that our infrastructure is dangerously fragile in the face of well-organized/funded attackers. The response isn’t “try harder” or “donate to your OSS project”, it needs to be institutional, professional, and at scale.

So, here’s my proposal, called “OSQI”, aimed at starting a how-to discussion: tbray.org/ongoing/When/202x/20…



Jia Tan's history of commits on #xz suggests that every png file in gcc or apache or whatever is a possible attack vector now.

joeyh.name/blog/entry/reflecti…

#xz


I think the #xz incident is teaching us that our infrastructure is dangerously fragile in the face of well-organized/funded attackers. The response isn’t “try harder” or “donate to your OSS project”, it needs to be institutional, professional, and at scale.

So, here’s my proposal, called “OSQI”, aimed at starting a how-to discussion: tbray.org/ongoing/When/202x/20…

#xz


While the #xz backdoor has everyone focusing on ways to make and keep open source sustainable, let's talk about the systemic abuse reinforcement mechanism that is the CVE database. Case in point: CVE-2023-45853.

This is a "vulnerability" that is reported for an _example_ source code file included in the zlib package. NIST has inexplicably classified this as a 9.8 out of 10. They fail to attribute the report: nvd.nist.gov/vuln/detail/CVE-2…

#xz


Stuff I already wrote that other people might be open to reading this week, because of the #xz incident:

harihareswara.net/posts/2021/s… Four Non-Dev Ways To Support Your Upstreams (Pass this along to executives who are asking "how can we prevent this in our dependencies?")

harihareswara.net/posts/2023/u… Potential cross-project #opensource tools and practices that you/we can implement to help lighten the load on each other

1/n


Three years ago, #FDroid had a similar kind of attempt as the #xz #backdoor. A new contributor submitted a merge request to improve the search, which was oft requested but the maintainers hadn't found time to work on. There was also pressure from other random accounts to merge it. In the end, it became clear that it added a #SQLinjection #vuln. In this case, we managed to catch it before it was merged. Since similar tactics were used, I think its relevant now

gitlab.com/fdroid/fdroidclient…


@kurtseifried and I recorded an #osspodcast episode about the #xz saga last night.

As much as I wish this was an April 1 joke, it's not

We try to break things down as simple as we can, and we end the show on a high note. There's no reason to lose hope, but there's a lot of hard work to do if we want to fix this.

Good luck to you all this week, it's going to be a wild ride

opensourcesecurity.io/2024/04/…


🤯 The level of sophistication of the XZ attack is very impressive! I tried to make sense of the analysis in a single page (which was quite complicated)!

I hope it helps to make sense of the information out there. Please treat the information "as is" while the analysis progresses! 🧐 #infosec #xz


I've found the best #meme about #xz #backdoor.


For anyone wanting to read up on the #xz clusterfuck: tukaani.org/xz-backdoor/ is currently the best starting point and will likely remain the best starting point
#xz


The abusive behavior that was being used to manipulate Lasse Collin into bringing on more maintainers for #xz went unnoticed because abusive behavior in Open Source communities is so pervasive. In context, we can clearly see it was part of an orchestrated operation. Out of context, it looks like just another asshole complaining about stuff they have no right to complain about. robmensching.com/blog/posts/20…
#xz


So, Philipp Kern dropped by asking if we could do some #ReproducibleBuilds verifications of recent Debian Security updates, given, well the whole #xz mess... and that our build infrastructure may have run compromised code at some point...

So I did a quick pass at a handful of updates and everything verified ok so far, though I skipped some of the probably more juicy targets such as chromium and firefox:

lists.reproducible-builds.org/…

Debian is reproducible enough to at least try this sort of thing!


boehs.org/node/everything-i-kn…

I have begun a post explaining this situation in a more detailed writeup. This is updating in realtime, and there is a lot still missing.

#security #xz #linux


Repeated the #compression #benchmark with the same file on a beefier machine (AMD Ryzen 9 5950X), results are quite identical, except faster overall.

This plot is also interesting:

- #gzip and #lz4 have fixed (!) and very low RAM usage across levels and compression/decompression
- #xz RAM usage scales with the level from a couple of MBs (0) to nearly a GB (9)
- #zstd RAM usage scales weirdly with level but not as extreme as #xz

#Python #matplotlib


First let's look only at the non-fancy options (no --fast or multithreading) and make log-log plots to better see what's happening in the 'clumps' of points. Points of interest for me:

- #gzip has a *really* low memory footprint across all compression levels
- #zstd clearly wins the decompression speed/compression ratio compromise!
- #xz at higher levels is unrivalled in compression ratio
- #lz4 higher levels aren't worth it. #lz4 is also just fast.

#zstd #xz #gzip #lz4


Interestingly, #xz at low compress levels actually compresses better than #zstd with lower RAM usage!
#zstd #xz


⏱️ Compression vs Decompression speed. Interesting that #zstd has some settings that actually decompress slower than they compress. Might just be my machine, though... #xz is still a turtle 🐢, #gzip is not much better, #lz4 decompresses much faster than is compressed, #zstd again has something for everybody.
#zstd #xz #gzip #lz4


First results of my #compression algorithm benchmark run on a 72MB CSV file. It seems #zstd really has something for everybody, though it can't reach #xz's insane (but slow) compression ratios at maximum settings.

This chart includes multithreaded runs for #zstd.

Very interesting! 🧐

gitlab.com/nobodyinperson/comp…

#Python #matplotlib #Jupyter #JupyterLab


My conclusion after all this is that I'll probably use #zstd level 1 (not the default level 3!) for #compression of my #CSV measurement data from now on:

- ultra fast compression and decompression, on par with #lz4
- nearly as good a compression ratio as #gzip level 9
- negligible RAM usage

When I need ultra small files though, e.g. for transfer over a slow connection, I'll keep using #xz level 9.