Who??
notice
javascript required to view this site
why
measured improvement in server performance
awesome incremental search
Boo! Just give me the text!
Edit: It’s long, but here’s the opening section, at least:
In 1984 KenThompson was presented with the ACM TuringAward. Ken’s acceptance speech Reflections On Trusting Trust (http://cm.bell-labs.com/who/ken/trust.html) describes a hack (in every sense), the most subversive ever perpetrated, nothing less than the root password of all evil.
Ken describes how he injected a virus into a compiler. Not only did his compiler know it was compiling the login function and inject a backdoor, but it also knew when it was compiling itself and injected the backdoor generator into the compiler it was creating. The source code for the compiler thereafter contains no evidence of either virus.
Ken wrote, In demonstrating the possibility of this kind of attack, I picked on the C compiler. I could have picked on any program-handling program such as an assembler, a loader, or even hardware microcode. As the level of program gets lower, these bugs will be harder and harder to detect. A well installed microcode bug will be almost impossible to detect.
Ken does not mean bug in the sense of error, but in the sense of listening device. And it is “almost” impossible to detect because TheKenThompsonHack easily propagates into the binaries of all the inspectors, debuggers, disassemblers, and dumpers a programmer would use to try to detect it. And defeats them. Unless you’re coding in binary, or you’re using tools compiled before the KTH was installed, you simply have no access to an uncompromised tool.
In fact, given the amenability of microcode to the KTH, not even then.
All manner of controls and monitors could be secreted this way in the OSes of all the devices we all use day to day. It isn’t very far fetched to suggest that the hack, in software, can create an updatable backdoor. This way every piece of software on the planet can be KTH bugged without any possibility of detection by any mortal engineer anywhere.
Well, maybe with the diligent use of an electron microscope.
Given last week’s horrifying revelations concerning the US government’s TotalInformationAwareness of every US domestic phone call, it is difficult to imagine that the ThreeLetterAgency’s KTH-hacked binaries are not omnipresent. I mean, can you really imagine AdmiralPoindexter would pass up an ability like this?
Amazing story, i loved it.
lol wow this is gold
Dev’s what?
Dev is screwed.
For the uninitiated: who compiles the compilers?
If I remember correctly, they used KenC to bootstrap the Go compiler.
I’ve had it on my todo for years to work through ddc and trusting trust.
Which is a method to verify a compiler is matching its source and thus trustworthy.An orthogonal approach is reproducible builds, which among many benefits can make sure a few people verifying things benefit everyone who can then see they have the same verified binaries.
What does that even mean. Whoever said that just uttered some empty but smart sounding catch phrase. Such is all the talk about the wonders of Ai
Linus Torvalds
Not exactly, but something along those lines.
We are talking about a guy still using mailing lists and patch files to conduct development on one of the largest codebases in the world, not exactly someone who jumps on any new shiny thing just to sound smart.
https://thenewstack.io/torvalds-ai-programming-productivity/
I think it’s supposed to be that how AI turns high level instructions into code is compared to how compilers turn code into assembly. Implying that using AI is just a natural extension of the handing off work to the computers that we’ve already been doing.
I wonder if you gave different AI models some c++ or something and told them to write assembly based on it how well they would do compared to an actual compiler
The very fact that Anthropic is now injecting a kind of watermark into every output, is solid proof that such a Ken Thompson hack is a inevetable risk
Ugh. Fuck “AI” and fuck Anthropic. But please read how the “watermarks” work. TL;TR its like a seeded run in a video game. With the seed and pseudo rng, you get the outcome i.e. the extruded text. In the watermark its the reverse, outcome + prng = seed. The result will be the same “quality” extruded garbage as before.
They already have a solution to the Trusting trust attack thanks to DDC and live-bootstrap’s work.
https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/
At work, I use AI for some things. Right now I’m rewriting some legacy spaghetti code that’s had a bunch of things hacked into it over the years. I spoke to the person most familiar with the expected behaviour and used AI to combine his info plus the existing code and unit/integration tests into a list of requirements.
I wrote the new code and tests based on the requirements rather than based on the old code. After each commit, I used AI to check for parity between the old and new code, and it keeps a Google Sheet up to date with the progress (which features were fully implemented, and which ones were missing or had gaps). I had AI write some tests cases too - given the list of requirements, write integration tests for them based on the style of a few tests I wrote by hand.
It has some quirks (eg for tests it loves over-mocking even though our skills tell it to mock as little as possible) but it definitely speeds things up.
I use AI for small side projects at work too. Tweaking and adding features I want to shared libraries, internal tools to help our team debug stuff and automate triaging of bug reports (they’re all still reviewed by a human), etc.
The entire reason I can trust its code is because I can read it and tweak it myself. I sometimes need to go through a few iterations to get AI code into an acceptable state. AI writing machine code directly, like what’s been talked about recently and what this post is referencing, is such a dumb idea.
There’s other people at work that use AI for absolutely everything. Writing code, reading code, writing posts in our internal groups, etc. That’s something I don’t understand. Some people that are all-in on AI produce so much low-quality AI slop.
Devs should be “dev managers and executives”. Real developers know LLMs are basically just a tool for finding examples and helping with syntax. Sure they’re useful, but I’d never let them write code, much less compile it. Who knows what they’d inject into a build.
It’s a tool, use it where it works and don’t where it doesn’t.
That saying (or, dare I say, thought-terminating cliché) glosses over the consideration of risks and costs that result from its uncritical usage. No sane person should trust it without serious reservations just because it fits the purpose, and that’s true whether or not it bears the latest combination of letters peddled by tech bros. It’s a dangerous, irresponsible mentality. It’s a tool the same way a sledgehammer made out of plutonium is a tool.
Plutonium sledgehammer sounds fucking awesome though
Pretty effective because of its high density, too. It’s a shame it’s brittle and would give you heavy metal poisoning.
🤘
As ways to die go that also sounds pretty rad
Oh, that would be rad alright
Unless you pick a particularly short-lived isotope, the radiation wouldn’t have time to hurt you before you died of regular heavy metal poisoning like you’d get from something boring like lead or mercury.
I mean in an old nuclear bomb that’s pretty much the situation. Use a conventional explosive to slam some plutonium like a sledgehammer and nukes have proven to be a strong deterrent when a country had them. Still a tool with a use case
Your comment is a sweeping statement that is thought terminating itself.
No sane person should trust it without serious reservations just because it fits the purpose
Using a tool for something it’s intended for automatically involves some thought process. And this thought process also involves its limitations and potential for danger.
Your criticism stems from your dislike for AI and your opinion that it’s generally not useful. An opinion that is shared widely here, but with little to no proof at all.
Na, fuck that shit. Just don’t use it.
Why? Do you have that feeling towards other algorithms?
And then your boss says: “Use it for everything or you’re fired. Why are you using less tokens than anyone else? You must be more productive or else…”
People probably say that about asbestos too.
And they’d be right
Of course, it still has uses. Just not in home insulation.
Your comment is so fit for purpose, displaying such a massive willful ignorance about the world in general. Because the biggest problem with asbestos wasn’t using it for home insulation. It killed millions of workers in manufacturing plants and mines. Its biggest industrial use case was for electrical insulation of high voltage transmission lines. Where it also killed hundreds of thousands of technicians in the us alone. Before that it was used for lamp wicks. Where it killed at least a couple of millions more over a century. Today, asbestos kills 250 thousand people a year, 12 to 15 thousand in the us every year.
I supposed any tool can be considered useful if you’re ignorant enough about it or if you’re willing to lie without remorse.
It’s a tool, use it where it works and don’t where it doesn’t.
But that doesn’t work with the people creating AI as they are totally dependent on the believe that AI can do absolutely everything (and an artificial general intelligence is just moments away…) to justify they insane investments. So they will make up a million stupid narratives why some people are “actually” not using AI as it obviously can’t be because of AI shortcomings…
If the tool don’t work then don’t use it. I wouldn’t try to unscrew something with chopsticks
You are probably also one of those insane ideologues that refuse to hammer in a screw for some reason, although you know how well that hammer worked on nails… 😂