FYI, your "8‿64 •bit._cast -⟜@ raw" in bqn/util.bqn results in passing to •bit._cast integers in the range 0..255, whereas it expects integers ¯128..127 (and does arbitrary platform-specific things on integers outside that for the int16→int8 narrow); removing the "-⟜@" makes it work properly, and 2.5-5x faster while at it from not going to intermediate int16. (•FBytes → •file.MapBytes also probably improves things, at the cost of the obvious mmap problem of issues if the file changes, as BQNs arrays are immutable). This is also what caused the problem at https://github.com/ClickHouse/ClickBench/pull/939/changes/40... (there's been some discussion about what to do about invalid •bit._cast before, considering it's rather perf-sensitive; it's the only place in CBQN with such quiet very-wrong results on invalid inputs)
The default CBQN "make o3" on x86-64 also results in it only using SSE2 (utilizing function multiversioning for AVX2 impls is on the ever-infinite TODO list, though somewhat-low on it considering it's strictly-unnecessary in any specific situation; there's also AVX-512 usage on a branch, though not mainline CBQN)
That all said, CBQN doesn't currently do any loop fusion, so being significantly-slower for sequences of operations over larger-than-cache arrays would kinda just be expected. BQN also just isn't particularly intended for database work anyway.
(didn't look much at the specific query impls, though "Pair" in utils.bqn is at least an overlong version of "Pair ← ⋈¨"; and some if not all of those Pairs would be better as "≍˘" to avoid nested arrays and ensuing pointer chasing)
BQN & CBQN are absolutely wonderful pieces of code.
L is mac/lin but linux is avx512 only specifically to try to deal with that problem. The compute on compressed algos helps fit more in those cache lines! https://lv1.sh/blog/compute-on-compressed/
I vibecoded https://punkx.org/apl/learn.html [1] its good slow entry into gnu-apl, at least it got me to be able to make tictactoe, vibecoded or not, it helped me to start.
I am amazed how quickly APL changed the way I think.
Also strongly recommend watching Aaron Hsu on youtube.
There is no better time to re-learn programming, try APL, Forth, LISP, z80 machine code, UXN TAL, just try new things.
"Men are born soft and supple; dead they are stiff and hard. Plants are born tender and pliant; dead, they are brittle and dry. Thus whoever is stiff and inflexible is a disciple of death. Whoever is soft and yielding is a disciple of life. The hard and stiff will be broken. The soft and supple will prevail."
Quite cool, but for a new runtime of an existing language it might make sense to compare to, y'know, the other[0] runtimes of that language? Even if one has to omit the best, closed ones for lack of access / permission to benchmark?
that is fast! Are you going to do the STAC-M3 benchmarks? Kx use them as evidence that they are the fastest, which is a fair claim as nobody else participates in that particular test :-)
Not sure why all the hate (sure site may be vibecoded, not all of us are front-end masters and it's at least not spartan)... I've always found k to be fascinating, and cool idea to try and roll a new one. Wish it were wholly open source, but cool to have a new variant that seems to bench well.
benchmarks at https://github.com/l-labs
unlike klong/ngn/bqn et al (which are GREAT) this has the goal of full production database compatibility (and full language compatibility).
q is the programming language underpinning kdb+ by Kx Systems - often claimed (by Kx) to be "the fastest database in the world". Possibly also the most expensive?
L is an independent implementation of a q interpreter. To those who reluctantly commercially license kdb+ - this will be a welcome alternative.
and fusion (e.g. f g h x has no intermediates e.g. mutates in place) is new
as is compute on compressed vectors (very helpful performance unlock) https://lv1.sh/blog/compute-on-compressed/
We'd like to see some benchmarks against open and closed k interpreters please! I'm curious how well a vibe coded k/q interpreter stands up to Shakti or whatever Mr. Whitney is letting out the door right now.
if you k you k ;-)
for the uninitiated, this looks like some wallstreet quant's new startup. Initially I thought it was the rebrand of shakti, Arthur Whites most recent rewrite of an array language. It's purpose built tooling for computing tick data for financial markets, but the best way I can describe it is codegolf for experienced programmers who don't want to give up the keyboard. these tools combine dataaccess and the ability to compute against that data with as few abstractions as possible.
As someone who does not know what k4, qSQL, or q are, reading through the landing page of this website was giving me mild schizophrenia. And then I tried to search for these things in the old way, and received incredibly dry technical sites that still don’t tell me what it is, and all these names are wildly SEO unfriendly. So I had Claude give me context and it’s apparently the database Wall Street uses for tick data. Sounds cool but, jeez.
K4 is the K programming language, Q is a language built on top of K. They are the practical over achieving members of the array family of programming languages.
Flagged, lots of unexplained grandiose thinking like “k and q made the vector the unit of thought.” K and q are unexplained and unlinked and “unit vector of thought” is pseudoscientific language
Extremely likely to be AI, though I’m not sure that matters for rules re: submissions
The language and its ideas come from Ken Iverson's famous paper "Notation as a Tool of Thought". This is a common understanding for people in the APL, K/J/Q etc array language communities, who are likely this website and product's intended audience.
> lots of unexplained grandiose thinking like “ k and q made the vector the unit of thought.”
That much makes sense in context: K is an array language, like APL, J, etc. From what little experience I have with J, ‘vector as a unit of thought’ seems like a reasonable description.
That’s fair: The most fair-to-submitter reading of your comment is my concerns are unfounded. If you didn’t intend that, that’s fine.
For moderators, I’d suggest that for the community, it’s spam. That’s one example, there are many more like it. The individual statement may be defensible but is still pseudoscientific language. This sort of content is a massive burden to community. Unanswerable anrguments about AI writing, whether the ability for an individual to have a parse-able reading is the same as writing being parable. The net effect is negative experiences for many and copy-editing for someone who did not do copy editing.
The 'unit of thought' thing is a nod to the Ken Iverson paper on APL that won him the Turing Award ("Notation as a Tool of Thought"), also referenced by Dyalog's tag line "The tool of thought for software solutions"). Variations of that phrase are endemic in the array languages space.
I see how jarring it is for you, but I want to comment-vouch against your flag. If you slow down a little, you can spot some meaning there. Its oddly phrased, but it does make sense that vectors (as in: simd vectors) are the building blocks of execution (to put it closer to how I might say it).
I’ve had similar ideas in the past: clearly simd is the way to get the most out of your cpu. Can we design a language where all operations are automatically simd, and it takes effort to do anything in scalars?
And I guess these array languages are what you might get.
It’s not ‘unit vector of thought,’ btw, which is weirder than what it says.
>If a story is spam or off-topic, flag it. Don't feed egregious comments by replying; flag them instead. If you flag, please don't also comment that you did.
yeah the site's clearly vibecoded and isn't opensource, but i also think this is a genuinely interesting design space and more people should be building in it. APL (https://www.dyalog.com/), BQN (https://mlochbaum.github.io/BQN/), J/Jd (https://code.jsoftware.com/wiki/Jd/Overview), Klong (https://news.ycombinator.com/item?id=10586872), Kerf (https://news.ycombinator.com/item?id=9782520), RayforceDB (https://news.ycombinator.com/item?id=45889607), k/q (https://kx.com/) glad there's a new entrant.
I've recently tested BQN on ClickBench (a benchmark for OLAP databases), and the results are not great: https://benchmark.clickhouse.com/#system=+N|liH&type=-&machi...
If anyone is curious how queries in this language look, you can see it here: https://github.com/ClickHouse/ClickBench/pull/939/changes#di...
FYI, your "8‿64 •bit._cast -⟜@ raw" in bqn/util.bqn results in passing to •bit._cast integers in the range 0..255, whereas it expects integers ¯128..127 (and does arbitrary platform-specific things on integers outside that for the int16→int8 narrow); removing the "-⟜@" makes it work properly, and 2.5-5x faster while at it from not going to intermediate int16. (•FBytes → •file.MapBytes also probably improves things, at the cost of the obvious mmap problem of issues if the file changes, as BQNs arrays are immutable). This is also what caused the problem at https://github.com/ClickHouse/ClickBench/pull/939/changes/40... (there's been some discussion about what to do about invalid •bit._cast before, considering it's rather perf-sensitive; it's the only place in CBQN with such quiet very-wrong results on invalid inputs)
The default CBQN "make o3" on x86-64 also results in it only using SSE2 (utilizing function multiversioning for AVX2 impls is on the ever-infinite TODO list, though somewhat-low on it considering it's strictly-unnecessary in any specific situation; there's also AVX-512 usage on a branch, though not mainline CBQN)
That all said, CBQN doesn't currently do any loop fusion, so being significantly-slower for sequences of operations over larger-than-cache arrays would kinda just be expected. BQN also just isn't particularly intended for database work anyway.
(didn't look much at the specific query impls, though "Pair" in utils.bqn is at least an overlong version of "Pair ← ⋈¨"; and some if not all of those Pairs would be better as "≍˘" to avoid nested arrays and ensuing pointer chasing)
BQN & CBQN are absolutely wonderful pieces of code. L is mac/lin but linux is avx512 only specifically to try to deal with that problem. The compute on compressed algos helps fit more in those cache lines! https://lv1.sh/blog/compute-on-compressed/
https://github.com/l-labs/db-benchmark/tree/main/l
I vibecoded https://punkx.org/apl/learn.html [1] its good slow entry into gnu-apl, at least it got me to be able to make tictactoe, vibecoded or not, it helped me to start.
I am amazed how quickly APL changed the way I think.
Also strongly recommend watching Aaron Hsu on youtube.
There is no better time to re-learn programming, try APL, Forth, LISP, z80 machine code, UXN TAL, just try new things.
"Men are born soft and supple; dead they are stiff and hard. Plants are born tender and pliant; dead, they are brittle and dry. Thus whoever is stiff and inflexible is a disciple of death. Whoever is soft and yielding is a disciple of life. The hard and stiff will be broken. The soft and supple will prevail."
[1]: https://github.com/jackdoe/gnu-apl-wasm source of the repl and learn playground and also how to compile gnu apl to wasm (vibecoded)
Quite cool, but for a new runtime of an existing language it might make sense to compare to, y'know, the other[0] runtimes of that language? Even if one has to omit the best, closed ones for lack of access / permission to benchmark?
[0]: https://k.miraheze.org/wiki/Running_K
https://github.com/l-labs/master-benchmark
that is fast! Are you going to do the STAC-M3 benchmarks? Kx use them as evidence that they are the fastest, which is a fair claim as nobody else participates in that particular test :-)
ah. cheeky.
Not sure why all the hate (sure site may be vibecoded, not all of us are front-end masters and it's at least not spartan)... I've always found k to be fascinating, and cool idea to try and roll a new one. Wish it were wholly open source, but cool to have a new variant that seems to bench well.
Sure the site, the articles, the program. But sure lets just trust that the author who can't write knows how to benchmark software
Not a web dev - but have some experience in benchmarking these types of workloads e.g. https://www.mcobject.com/press/november19-2014/
L used the two open ones that are easy to replicate: H2O.ai (great bench) https://github.com/l-labs/db-benchmark TSBS (less great but useful) https://github.com/l-labs/tsbs
If there are others (will do ClickBench) they'll go there as well
benchmarks at https://github.com/l-labs unlike klong/ngn/bqn et al (which are GREAT) this has the goal of full production database compatibility (and full language compatibility).
q is the programming language underpinning kdb+ by Kx Systems - often claimed (by Kx) to be "the fastest database in the world". Possibly also the most expensive? L is an independent implementation of a q interpreter. To those who reluctantly commercially license kdb+ - this will be a welcome alternative.
Absolutely and given Kx is now private equity owned and are in the customer-squeezing phase of the acquisition, this is very welcome indeed.
and fusion (e.g. f g h x has no intermediates e.g. mutates in place) is new as is compute on compressed vectors (very helpful performance unlock) https://lv1.sh/blog/compute-on-compressed/
yeah, that is super neat! Very innovative to bring it to q.
We'd like to see some benchmarks against open and closed k interpreters please! I'm curious how well a vibe coded k/q interpreter stands up to Shakti or whatever Mr. Whitney is letting out the door right now.
https://github.com/l-labs/master-benchmark/tree/master/resul...
These single letter names are getting out of hand.
l must be named 1 because I is taken
next letter after k
if you k you k ;-) for the uninitiated, this looks like some wallstreet quant's new startup. Initially I thought it was the rebrand of shakti, Arthur Whites most recent rewrite of an array language. It's purpose built tooling for computing tick data for financial markets, but the best way I can describe it is codegolf for experienced programmers who don't want to give up the keyboard. these tools combine dataaccess and the ability to compute against that data with as few abstractions as possible.
As someone who does not know what k4, qSQL, or q are, reading through the landing page of this website was giving me mild schizophrenia. And then I tried to search for these things in the old way, and received incredibly dry technical sites that still don’t tell me what it is, and all these names are wildly SEO unfriendly. So I had Claude give me context and it’s apparently the database Wall Street uses for tick data. Sounds cool but, jeez.
K4 is the K programming language, Q is a language built on top of K. They are the practical over achieving members of the array family of programming languages.
shoutout to bryan @bcantrill who I think explained K best at https://youtu.be/2wZ1pCpJUIM?si=y4ugbFXroTZc22AY&t=471 and https://queue.acm.org/detail.cfm?id=1531242
Algol derivatives usually written by, commercialized, then sold by Arthur Whitney. Generally considered write-only :)
Algol?
Did you mean APL?
I believe k and q are member of the "array programming"/APL family of languages who are exceptionally terse/information-dense
https://en.wikipedia.org/wiki/Array_programming
EDIT: Did someone downvoted this because it is wrong or is someone I am arguing with rage-downvoted this ?
¯\_(ツ)_/¯ quants be minimal
The UI screams gpt 5.5 high lol.
Flagged, lots of unexplained grandiose thinking like “k and q made the vector the unit of thought.” K and q are unexplained and unlinked and “unit vector of thought” is pseudoscientific language
Extremely likely to be AI, though I’m not sure that matters for rules re: submissions
> “unit vector of thought”
The language and its ideas come from Ken Iverson's famous paper "Notation as a Tool of Thought". This is a common understanding for people in the APL, K/J/Q etc array language communities, who are likely this website and product's intended audience.
> lots of unexplained grandiose thinking like “ k and q made the vector the unit of thought.”
That much makes sense in context: K is an array language, like APL, J, etc. From what little experience I have with J, ‘vector as a unit of thought’ seems like a reasonable description.
> Extremely likely to be AI
I had the same thought though.
That’s fair: The most fair-to-submitter reading of your comment is my concerns are unfounded. If you didn’t intend that, that’s fine.
For moderators, I’d suggest that for the community, it’s spam. That’s one example, there are many more like it. The individual statement may be defensible but is still pseudoscientific language. This sort of content is a massive burden to community. Unanswerable anrguments about AI writing, whether the ability for an individual to have a parse-able reading is the same as writing being parable. The net effect is negative experiences for many and copy-editing for someone who did not do copy editing.
The 'unit of thought' thing is a nod to the Ken Iverson paper on APL that won him the Turing Award ("Notation as a Tool of Thought"), also referenced by Dyalog's tag line "The tool of thought for software solutions"). Variations of that phrase are endemic in the array languages space.
I see how jarring it is for you, but I want to comment-vouch against your flag. If you slow down a little, you can spot some meaning there. Its oddly phrased, but it does make sense that vectors (as in: simd vectors) are the building blocks of execution (to put it closer to how I might say it).
I’ve had similar ideas in the past: clearly simd is the way to get the most out of your cpu. Can we design a language where all operations are automatically simd, and it takes effort to do anything in scalars?
And I guess these array languages are what you might get.
It’s not ‘unit vector of thought,’ btw, which is weirder than what it says.
>If a story is spam or off-topic, flag it. Don't feed egregious comments by replying; flag them instead. If you flag, please don't also comment that you did.
https://news.ycombinator.com/newsguidelines.html
I mean, this is perhaps the most vibecoded website possible so... deff AI.
It attracts negative energy via unanswerable questions and downvotes if you assert it is, alas.