That article partially validates my idea on why some people think that Lisp is such a force multiplier. The idea would be that compilers are one of the most important tool productivity-wise, and that Lisp allows you write your compilers yourself.
That would also explain why not Lisp: First, libaries are the new important tool for productivity, and any language can have that. Second, a shared understanding is very important, either for building programs, or building the libraries. Writing your own compilers may reduce that shared understanding. You can see this in some ecosystems where there is a split between people using the language in different ways (OCaml with Async and Lwt, the few alternative standard libraries; Scala with Zio, Cats, the base language; C++ with C with classes, the standard library and probably a lot of other styles that I haven't met yet; JS with the "classic JS", the frontend framework people, the functional/immutable people; many language with the FRP). Shared understanding is important for building and maintaining software. You could also argue that it's important to make programmers more replacable.
Another interesting thing is that some ecosystems are getting their macros: JavaScript has Babel, Coffeescript, Typescript.
> Shared understanding is important for building and maintaining software.
I think this is where Go (the language) really shines. Go is "boring" -- there are no macros, no operator overloading, no default arguments, none of that sort of thing. But if your goal is shared understanding, "boring" is a compliment. "Boring" means "after using the language for a few years, I can be confident that I will never be surprised by a piece of Go code again." (This has been true for me for at least 4 years.)
In the same vein, Go famously doesn't support user-defined generics; the only generics it has are reified "builtins," baked into the language. People gripe about that, but the upshot is that every Go programmer understands the semantics of those builtin functions. They're Schelling points! When user-defined generics are added in the next version of Go, I worry that we'll lose some shared understanding. It will be a subtle shift, but the language will feel a bit less "cozy." :/
I take a different view, that shared understanding comes from clear, concise code, which is not bogged down in ceremony.
Lines of Go in isolation are easy to understand. However, Go is so lacking in expressiveness, that Go code-bases are hard to follow for any complex domain. The language doesn’t give you enough tools to tame complexity.
> I think this is where Go (the language) really shines. Go is "boring" -- there are no macros, no operator overloading, no default arguments, none of that sort of thing. But if your goal is shared understanding, "boring" is a compliment.
I think one of the most important points is the standard library rather than macros, operator overloading, default arguments and details like that. What makes a simple Go http server easier to understand than one in pretty much every other language is that most of the time it will be from the standard library. With Python, Java, C++, C#, Ruby, JS, TS, Kotlin, C and pretty much everything else, you're going to have to understand a framework/library first.
> "Boring" means "after using the language for a few years, I can be confident that I will never be surprised by a piece of Go code again." (This has been true for me for at least 4 years.)
That's a good definition, and whether you consider this good or bad is a great way to know if you will like Go.
> In the same vein, Go famously doesn't support user-defined generics; the only generics it has are reified "builtins," baked into the language. People gripe about that, but the upshot is that every Go programmer understands the semantics of those builtin functions. They're Schelling points!
User-defined generics are one point, no support for inheritance is another, almost no iterators is still another, no support for map/reduce/filter or list comprehensions is probably still another, one way of doing concurrency is again another, no support for FP is another.
> When user-defined generics are added in the next version of Go, I worry that we'll lose some shared understanding. It will be a subtle shift, but the language will feel a bit less "cozy." :/
I can see why. Right now I would love to add a custom < operator to a JS codebase, I can't, and it's a bit frustrating. But on the other hand, I'm a bit glad other people can't add it too. Some people will call this humility, some others will call it a race to the bottom. Maybe there's something to say about optimising for your "best" days or your "worst" days.
I don't want this kind of "shared understanding" if it means that I have to program in the most basic, underdeveloped language. It's the interface with the computer and used to convey meaning.
> That article partially validates my idea on why some people think that Lisp is such a force multiplier. The idea would be that compilers are one of the most important tool productivity-wise, and that Lisp allows you write your compilers yourself
This is a huge selling point for me personally. lisps allows you to have a solution AND its compiler in lisp, without a need for another language. As far as I know Common Lisp is the only industrial strength dialect of lisp to allow this, and with an unmatched interactive development environment
> Shared understanding is important for building and maintaining software. You could also argue that it's important to make programmers more replaceable
in a business sense, if i wanted to hire for a large team of 'developers' i would hire for java / python / js. if i wanted to hire for a small team of 'hackers' i would hire for common lisp / haskell
This is no different from any other language lacking libraries, except that with Lisp you're using a proven, powerful language. And if you want libraries there's a python bridge and a whole CL on the JVM with interop.
Macros are compilers (well, an AST pass), all of these tools are compilers. A specific transformation of Babel or Typescript or Coffeescript is a macro.
I love Lisp and Scheme and all their relatives (Clojure, Logo, Racket, etc.). However, the fact of the matter is, Common Lisp has not kept up with modern developments in terms of presenting a cohesive ecosystem with forward momentum. Everybody is off on their own doing their own thing with no shared goals or cohesion.
Clojure seems to have this (I have not used Clojure much, so I don't really know). Elixir definitely has this. I think Elixir is the language ecosystem to look at in terms of having a solidified core language that is essentially done, and now the goal is to continue to flesh out the ecosystem with things like Nx, Axon, Livebook, Liveview, Mix improvements, Phoenix, etc. Elixir is pretty similar to Lisp/Scheme anyway given its macros, ability to live update, dynamically typed programming, but it goes well beyond any Lisp/Scheme (and many modern languages) in terms of having a practical but expansive ecosystem with a strong set of idiomatic conventions.
> ... it goes well beyond any Lisp/Scheme (and many modern languages) in terms of having a practical but expansive ecosystem with a strong set of idiomatic conventions.
Does it, really?? Common Lisp is an ANSI Standard, I don't know how you get a more "strong set of idiomatic conventions" than that. And it's been "done" for 20 years now! People continue to use it just as it is, and there's very little pressure to change it because it simply works!
You're trying to justify a new wave of "modern" languages with a very weak argument... just call it for what it is: you want "cool", hyped languages to succeed despite the fact they offer nothing that the CL standard did not 20 years ago.
This type of response is also why Common Lisp isn’t more popular. :)
Having a standard has little to do with how people write code, which is what I meant by idiomatic conventions.
Elixir is not a “cool, hyped” language in the sense that you mean. Elixir is basically a wrapper around Erlang and OTP, which is an old technology (much older than ANSI Common Lisp), that provides several improvements in terms of developer tools and frameworks. It’s not just the next “cool” thing. It’s an incremental improvement on an old and tested technology to bring that technology to more developers. It has also been embraced by the Erlang community.
> This type of response is also why Common Lisp isn’t more popular. :)
What does a response from a blunt commenter like me has to do with the success of CL? Do you mean CL users are prone to responding like me? I would say users of any language that's being dismissed as being old would react similarly.
> Elixir is basically a wrapper around Erlang and OTP
Yes, I know, I was going to say that Elixir itself is nothing new as it's just Erlang with a more palatable syntax on top... nothing really new, Erlang is about as old as Common Lisp.
I'm not dismissing Common Lisp. I've actively searched for and applied to jobs that advertised using Common Lisp.
I just don't see the Common Lisp community rallying to make it more cohesive (and thus more popular and attractive), which is fine, but it means it won't get used outside of enthusiasts bringing it into niche pockets of companies. I have the same complaints with F#, which is one of my favorite languages, because there are just too many libraries/frameworks that are just one-off side projects of a single developer. (There are of course separate reasons for F#'s lack of uptake.)
If the entire Common Lisp ecosystem is as good as people say it is, then I don't see why it couldn't go through a path like Erlang did with Elixir and even so without a syntax change (just a nice bundling of the ecosystem and tooling) instead of just relying on war stories of its use and telling people they just don't get it.
it might be that convincing people to switch to common lisp is a lot like convincing people to switch to linux from windows. is it possible that there might be people who do not care much about ready-made tools or frameworks and enjoy engineering solutions from a more fundamental level? i think those people definitely exist and i personally rather enjoy hearing their war stories
In fairness, the CL spec being quite old, has a fairly dated type system.
Obviously, CL is still blazing fast compared to Python, but it can't easily do the kind of things that Julia or even C++ can when it comes to numerical programming. You can write a lot of macrology to get around it, but there's a point where you want actual compiler writers to be doing this.
> You can write a lot of macrology to get around it, but there's a point where you want actual compiler writers to be doing this
this is not the job of compiler writers (although writing macros is akin to writing a compiler but i do not think that this is what you mean). in julia the numerical programming packages are not part of the standard library and a lot of it is wrappers around C++ code especially when the drivers to the underlining hardware are closed-source [0]. also here is the similar library in common lisp [1]
Presumably what the GP means is that you can use macros to hack around limitations of CL's type system, such as lack of parametric types, or whatever (very important for good multidimensional generic array code), but they'd rather that the CLOS was just built to be more expressive in the first place, because their local hacks will not be as good as what a language development team can produce while working together.
> in julia the numerical programming packages are not part of the standard library and a lot of it is wrappers around C++ code especially when the drivers to the underlining hardware are closed-source [0].
This is mostly false.
First of all, there's a huge amount of numerical programming infrastructure in Base julia and it's standard library. There is even more outside of the base/stdlib, but the stuff inside is quite diverse and robust.
Second of all, CUDA.jl is a very weird thing to point to for your example. CUDA.jl is essentially a compiler plugin for julia that hijacks it's normal compilation pipeline for specified code and then causes it to emit GPU specialized instructions to LLVM which then compiles a binary.
If compiling to LLVM is a 'wrapper around C++ code' then all julia code is that (but note, this is a rather dumb definition of 'wrapping').
why do you presume that's what the GP meant? your post is much more detailed than his
anyway, i know very well that julia has a great standard numerical library. it is a language catered to that community so it better have a rich standard library. my point is that functionality does not need to in the standard library in order to be useful or impactful. i don't think i need to prove my point but take numpy (or any other famous numerical package from the python ecosystem) as an example
>Second of all, CUDA.jl is a very weird thing to point to for your example
i work in machine learning so it is a very important package for me
> If compiling to LLVM is a 'wrapper around C++ code' then all julia code is that
and CL is not. point is that you need to learn LLVM in order to dig deeper, if you so wish
The thing about Common Lisp is, code doesn't rot. It doesn't matter if CL code is 5 years old or 30, it runs. I have yet to see even one ounce of bit rot. CL code is as eternal as solid gold, honestly. And sometimes, yes, there's a few ways your code will fail to run, mostly because you're shooting yourself in the foot (eg starting slime in a different directory than what you're meant to).
Versus Python, I didn't even get to use Python 2 before I started getting nagged and harangued by various exceptions in the terminal about how I had to migrate to Python 3.
And it depends what you're doing with CL, how well it works for you. As an algorithm designer, I use literally no libraries. I don't glue anything to anything. I just use the batteries that come included with Common Lisp (SBCL is great), forget about connecting to the internet, forget I/O, forget graphics and GUIs and OOP, and just focus on coming up with as many ideas as I can per minute, until I crack the algorithmic problem. Just with the austere tools that come included in SBCL: xor (#'logxor), popcount (#'logcount), #'sort, #'svref, #'pop, #'push, #'make-array, #'dotimes, 'lambda, 'labels, 'let, 'do, the basics.
I can't think of any code rot either. As long as it is built against the standard it tends to just work.
I also find the idea of using no libraries a big thing in more advanced users of CL. The more advanced of a user the more it makes sense that you can just implement it yourself (with only the functionality required) and know it works without having to leverage a library. Then again, the idea of CL is that things that tend to be bigger/harder to solve problems in other languages can be done simpler in CL because you can break the problem up different and have an easier to understand or implement solution.
I'm not exactly a Lisp fan (I'm firmly in the statically typed camp), but this is a weak argument. Common Lisp specifically is one of the most cohesive experiences one can find out there, the language has had a long time to mature, it's ANSI standardized and the ecosystem is largely built on top of that cohesive base.
mmh Elixir definitely has its gems and I need to try it for real. Where I can see it's lacking: no compile-time type warnings (unless you're using a new language on top or using a third-party library, but then we should compare more thoroughly), no compilation with a keystroke function per function. No static executable (even though deployment options are getting better in Elixir (we had to wait though)). Some compilation times, when you change one file and Elixir needs to compile a dozen. We are not inside a live image. I tried Alchemist mode for Emacs: no function signature? No "eval this expression"? Can you switch projects from inside a REPL, or you have to start a new one? No cross-reference capacities built-in? (who calls this function, this macro, who sets, etc) No inspect? No trace? No interactive debugger and no stepper? If only available with LSP and VSCode, then it's strictly inferior. Elixir is less performant in crunching numbers. Less to no GUI framework choice (WxWidgets). No industry-grade theorem prover? (ACL2) Unfit for quantum computing? Unable to program Intel chips? (Barefoot Networks) No computer algebra system? No music composition suite (Opus Modus)?
There was one cohesive tribe in the 00s centered around the quasi-annual European Common Lisp Meeting. I miss that and wish I could find the same thing again now. Maybe it exists somewhere online?
The ELS community is still going strong. They have done a great job with continuity and moving forwards. That's a related but different tribe though.
I don't claim that Elixir is the language to end all languages. My comment was more about the holistic ecosystem that keeps improving. There's one common development and project management tool (Mix). There's one package manager (Hex). There's OTP, Ecto, Phoenix, Liveview, etc. There's the Elixir Forum for questions.
I'd use F# for writing compilers.
Interactive programming is probably better in Common Lisp and also Smalltalk descendants like Pharo. But Elixir has some great interactive tooling. I think this is an excellent presentation that summarizes the capability (not necessarily saying other languages don't) in terms of real world applications: https://www.youtube.com/watch?v=JvBT4XBdoUE
> I don't claim that Elixir is the language to end all languages.
i don't know much elixir. i was asking because those things interest me and they are what drove me to common lisp. for my purposes i need an industrial fast interactive language that is self hosted and well tested in the wild. i want a fast language that is well suited to make custom symbolic computer algebra procedures, custom differential algorithms, custom probabilistic programs, and custom data processing solutions. i think that common lisp best fits the criteria for my use case
Why does it feel like there’s more prose about lisp being written than lisp code? I swear there’s like 15 people writing Common Lisp — Nikodemus, Shirakumo, Stylewarning, Christian Schafmeister, Borodust, the ITA folks, and maybe 2 startups.
There are three commercial lisp companies that continue to exist selling CL platforms. They exist despite SBCL being faster. They even make stuff like tools to build iOS and Android apps in CL. They didn't invest the money building that that out for no reason.
The companies that use lisp don't tend to talk about it. For example, a company I worked for only talked about their Ruby/rails and javascript stuff publicly. It was only after I hired that I found out that the mission-critical core software was written in Common Lisp.
Probably because if you aren't paying that much attention to Lisp software, the almost 40 years of writing about Common Lisp will surface more so than the code and their authors. And a lot of code (millions of lines) isn't open and a lot is probably lost to time. Though it seems like you know about a handful of the current era superstars, yet you're forgetting or don't know about other superstars. And there's a long tail of less prolific developers. But sure, just comparing the open source stuff in Quicklisp, there's only about 2500 projects (vs 335k projects on PyPI), and the frequency of new stuff regardless of whether it shows up or not in quicklisp is slow enough that you could follow https://twitter.com/NewLispRepos/ and not have your timeline overwhelmed.
i am new to lisp in general and common lisp in particular. i will tell you the key reason why i will take my time to promote it on hn. for years i had large misconceptions about lisp: it is slow (in my case for numerical work), it is an academic language, the syntax is ugly unnatural and annoying. it was only by chance that i was (quite happily) proven completely wrong. so my rationale is that if there are people who are like i was then i will be happy to provide information about common lisp and perhaps something useful might come out of it
Perhaps I’m wrong, but my understanding is that it has a reputation for being able to build systems with significantly fewer LOC than most languages, without delving into APL-levels of obscurity.
My attempt at deadpan humor doesn’t work if that isn’t true, and I’m not qualified to make the argument since I’ve not used it professionally yet. I can say that picking up Clojure for fun has both reawakened my joy in programming and greatly informed how I write code in other languages. That tends to inspire prose.
That is because it is a little secret. It is a secret hidden in the open.
We program in C#, we program in swift, C and C++, but we continue using so much Lisp as I did 20 years ago or way more.
With Lisp you don't need to write only Lisp, you can write swift or any other language in Lisp, as we do.
When people ask what programming languages we use, even competitors, we just tell them. They don't listen.
They just can't really understand. They probably believe we use Lisp because we depend on obsolete technology because they are under the influence. The influence of snake oil salesman that want to sell them their new shiny language.
For them anything new is shiny and old means dusty because things degrade over time, but computer languages do not.
If competitors can't "get it", much better for us. They probably believe that we try to deceive them and that in reality we are using this new language-paradigm of the week that solves all the problems. Let them.
Lisp guys have nothing to sell you but probably the software they create. And if they have a powerful tool, they are not that interested on you knowing their secret.
> Lisp guys have nothing to sell you but probably the software they create. And if they have a powerful tool, they are not that interested on you knowing their secret.
This level of claim is indistinguishable from fiction.
> I really like the Common Lisp world. *I would like it to be more popular, but at the same time, it is a differentiator for us*.
they reached to us so we add Kina on awesome-lisp-companies. But how many more think the same? And still, they evidently put little effort in making CL more "popular", because they have no interest. They found enough open-source libraries, they work on a hard problem, they form their developers and operators in-house, they develop their own Lisp-like language for the browser… and they silently use the power of SBCL.
This, this right here is the attitude I’m referring to. From now on all I’m gonna say is “GitHub or GTFO.” Show me all this lisp you’re writing, or at least tell me the fantastic product you’ve built with it or GTFO.
I started my programming in the more "traditional" way i.e., C/C++ and then later python. When I started doing python, I was flabbergasted to find that I could just do 100**100 and get a complete number without using any additional libraries. Similarly, reversing string was trivial using the [::-1] notation. Heterogeneity of Lists, Dictionaries and the resulting versatility blew my mind. I now understand that I was discovering, at that time, what dynamic language had to offer over static languages.
I say all this because I would really like some great examples and use cases which would sort of highlight what LISP has to offer over the, for the lack of a better word, non-LISP languages. All of the articles that pop up in hacker news or other portals only seem to highlight the fact that LISP presents a new paradigm (?), enhances intuition (?), and that LISP macros are the best-thing ever. I would like to share the sentiment but ideally via a few digestible examples in lieu of a serious commitment required to basically learn an entire language. I would appreciate a few concrete examples (very much like [::-1] and 100**100) that would allow me to get what LISP really is about.
I don't know much about Lisp. But you do realize you're asking sort of the following right?
Roman: what's so amazing about the decimal system? I can do addition just fine with X + X = XX.
Modern human: Well... you can do negative numbers easily, irrational numbers, you can show zero easily.
Roman: bah! I don't know what you're talking about. Half those numbers don't even exist.
Mathematician: actually have you heard about complex numbers?
The point is: you have to learn a new way of thinking compared to procedural/OOP style programming. And it might be the case that the problems it solves are not practical for your use-case.
I tell you, as a non-Lisp programmer, what I find amazing about Lisp. Disclaimer: I may be partially or completely wrong. I'm really not the one who should be writing this, but since our knowledge of Lisp isn't that far apart, I might be able to empathize with your perspective a bit more since I have been in it.
* Your code is the abstract syntax tree! Don't know what an abstract syntax tree is? It's a data structure that compilers use in order to generate code. In normal languages you have to first parse the language, compile it to the abstract syntax tree and then generate machine code from it. However, in the case of Lisp, it's already there. So this means that you can write compiler-esque programs much easier.
* One of the nice things that a direct AST provides that it allows partial live reloading of code. Imagine this:
Simple digestable examples that show off power is kind of a cart before horse situation. However, I will try to feed your curiosity.
One simple example I can describe is from the book Practical Common Lisp and it is talked about to some degree in the SICP videos. The idea of the environment being captured as well as what it points to is a big idea. At the risk of not getting it explained 100% to everyone's satisfaction I would suggest reading about it.
I also would suggest the problem of the N queens board. It is a pretty good starting point into how this problem can be solved in lisp. I suggest the MIT SICP lecture on it because it is well paced and interesting. Also it shows off some, for the time, advanced language features.
Another good read is Norvig's PAIP book that you can read on Github. Lots of problems you can jump to with pretty good explanation in my opinion.
I'm sure there are more interesting things I could point to. However, I still consider a read of Practical Common Lisp a good starting point. The book is fairly well paced and not bogged down with learn all the language first-isms. Also, the SICP videos are pretty good to watch. I hope these points are helpful to you.
i think PAIP is an absolute gem! not just for learning lisp, but for software engineering in general. not only that but you can read it online in .md format with proper syntax highlighting, which provides for a much better experience. and in fact not only even that (!!), but if you use org-mode you can convert .md files to .org files flawlessly via pandoc and enjoy the whole experience interactively (like jupyter notebook, but only on a whole different leve)
> I would appreciate a few concrete examples (very much like [::-1] and 100*100) that would allow me to get what LISP really is about.
You mean like (reverse "asd") and (* 10000 10000)? That's not really what gives you an impression of Lisp, rather a bit of syntax ([::-1], the paragon of readability) and large integers. :P
Given that it's a 'new paradigm' and enhances 'intuition', it's a bit hard to show simple examples which demonstrate that.
You'll have to put in the work.
Will you be able to understand how to play the guitar if I show you a passage from a song ? You can play it with any other instrument, so it still won't be motivating enough for you to put in the time and effort required to learn it.
If I were to take you to a Led Zeppelin concert to see Jimmy Page playing, that still might not be motivation enough for you to pick up the guitar, but you'll be able to better understand what it can do to people :).
Back to LISP - it's a tool. The difference between other tools, is that you can program it too. The power lies in its simplicity and your ability to build programs iteratively while running them.
To be fair, other non-Lisp languages have come a long way. Still, I'll offer a few things that might pique your interest.
For a while in the development of Common Lisp, a sort of joke acceptance test for implementations was in three parts: 1) you type T and enter into the REPL, it responds T 2) you define the factorial function and calculate (/ (factorial 1000) (factorial 999)) and it responds 1000 3) You try (atanh -2) and if it returns a complex number it passes (extra credit for the correct complex number). Lisp has a great numerical tower. Besides being able to deal with huge and complex numbers, you also have convenient syntax for specifying numbers in base 2 and 16, and you can do things like bitwise operations on bit-vectors:
For macros, there's a lot of cool ones. Unlike C macros that just do string substitution, Lisp macros let you use the full Lisp language to write code that does something with expressions passed to the macro. A nifty one that's part of basically every implementation with uiop:nest is described here https://fare.livejournal.com/189741.html The problem it solves is that sometimes in Lisp you'll have a lot of nested forms, and your code begins to attack the bottom right corner of your screen. e.g.
(defmacro nest (&rest r)
(reduce (lambda (o i) `(,@o ,i)) r :from-end t))
(The list of forms are passed, unevaluated and at compile time, to nest, which rewrites them using a right fold to nest things properly.)
Somewhat similar is the arrow macro that Clojure popularized, which lets you get rid of (deep (nesting (like (this ...)))) where you have to remember evaluation order is inside-out and replace it with a flatter (-> (this ...) like nesting deep). Or (loop (print (eval (read)))) -- which will indeed give you a primitive REPL within Lisp -- with the more readable (-> read eval print loop). Its implementation is also easy -- many macros are easy to write because Lisp's source code is itself a list data structure for which you can write code to process and manipulate just like any other lists.
Another cool macro that's been around since 1993 is https://github.com/quil-lang/cmu-infix which lets you write math in infix style, e.g. #I( C[i, k] += A[i, j] * B[j, k] ) where A, B, and C are all matrices represented as 2D arrays. It's a lot more complicated than the nest macro, though.
There are some other things that still make Lisp great in comparison to other languages, but they don't exactly have one-line code examples like [::-1] and so I'll just describe them qualitatively. Common Lisp has CLOS, the first standardized OOP system. It's a lot more powerful than C++'s system. It differs from many systems in that classes and methods are separate; among other things this gives you multiple dispatch (you can define polymorphic methods that don't just dispatch to different code depending on the first argument (the explicit 'self' in Python, implicit 'this' in other langs) but all arguments). One thing it can be useful for is to get rid of many laborious uses of the Builder and Visitor patterns. e.g. the need for double dispatch is a common reason to use the Visitor pattern, but in Lisp there's no need. CLOS also does "method combination", which lets you define :before, :after, and :around methods that operate implicitly before/after/around a call. This gets rid of the Observer pattern, supports design-by-contract, and jives well with multiple inheritance in that you can create "mixins" that classes can "inherit" from with the only behavior being some :before/:after methods. (e.g. logging, or cleaning up resources, or validation.)
Everything is truly dynamic -- an object can even change its type at runtime, which may be an acceptable solution to the circle-ellipse problem, or just super convenient while developing. More fundamentally, "compile" is a built-in function, not something you have to do with a separate program. "Disassemble" is built-in, too, so you can see what the compiler is doing and how optimized something is. You have full flexibility to define and redefine how your program works as it's running, no need to restart and lose state if you don't want to. Besides being killer for development (and all the differences in development experience comprise a big part of why I still think Lisp is great compared to non-Lisp), this gives you a powerful way to do production debugging and hot-fixing too -- a footgun you might not necessarily want most of the time, but you don't have to do anything special for it when you do want it. It can be very useful, e.g. if you've got a spacecraft 100 million miles from Earth https://flownet.com/gat/jpl-lisp.html I've also put some hobby stuff on a server, just deployed as a single binary, but built so that if I want to change it, I can either stop it, replace the binary, and start again, or just SSH in and with SSH forwarding connect to the live program with my editor and load the new code changes just like I would when developing locally, and thus have zero downtime.
Lastly, Lisp's solution to error handling goes beyond traditional exception handling. Again this ties into the development experience -- you have some compile-time warnings depending on the implementation (e.g. typos, undefined functions, bad types) but you'll hit runtime errors eventually, Lisp provides the condition system to help deal with them. It can be used for signaling non-errors, which has its uses, but what you'll see first are probably unhandled errors. By default one will drop you into a debugger where the error occurred, the stack isn't immediately unwound. Here you can do whatever -- inspect/change variables on different stack frame levels, recompile code if there's a way to fix things, restart computation at a specific frame... You'll also be given the option of "restarts", which might include just an "abort" that unwinds to the top level (possibly ending a thread) but can include custom actions as well that could resolve the error in different ways. For example, if you're parsing a CSV file and hit a value that is wrong somehow (empty, bad type, illegal value, bad word, whatever), your restarts might be to provide your own value or some default value (which will be used, and the computation resumes to parse the next value in the row), or skip the whole row (moving on to the next one), or skip the whole file (moving on to the next file, or finishing). Again this can be very useful while debugging, but in production you can either program in default resolutions (and a catch-all handler that logs unhandled errors, as usual) or give the choice to the user (in a friendlier way than exposing the debugger if you please).
do you still use lisp ? i noticed a while ago you were working on a cryptographic project and i am sure you considered common lisp for this, but didn't pick it. is there a reason for this ?
Yes. In fact, I do almost all of my coding in Common Lisp. I'm even getting paid for some of it :-) I'm working part time for Intel helping them maintain an internal design tool written in CL. I run my own email server that runs a spam filter that I wrote in CL, and the e-commerce system that processes orders for my crypto dongle is written in CL. As an exercise I wrote a sudoku solver in CL a while back.
Because that product was an embedded system running on a very small SoC. It only had 1MB of flash and 192k of SRAM. It's theoretically possible to run CL on a system that small -- Coral Common Lisp ran on a Mac Plus with 1MB of RAM back in the 1980s -- but nothing off-the-shelf will do that today.
(I did, however, put a little Scheme interpreter on it as an easter egg :-)
I do have some CL code that supports the crypto project. The back-end for this:
is written in CL (though all the actual encryption is done client-side in Javascript). I also have some prototype crypto code that I don't really use for anything, including this double-ratchet implementation:
Getting Lisp to run as fast as C takes major effort when at all possible.
Resources:
Lisp needs a lot of space to do it's thing; and while it's certainly possible to downsize it, you're left with something that's not really Lisp anymore.
Ecosystem:
Finding solid libraries is tricky since it's not very popular professionally.
Power:
Unleashing the full power of Lisp in a diverse team is a recipe for an adventure, if not disaster.
> Getting Lisp to run as fast as C takes major effort when at all possible.
The Computer Language Benchmarks Game shows Lisp Code as generally being between 2x and 10x slower than C++[1]. As fast as C? No. Way faster than Python, and more than fast enough to be used in almost every single application, modulo hard-real-time systems and AAA video games? Yes.
> Lisp needs a lot of space to do it's thing; and while it's certainly possible to downsize it, you're left with something that's not really Lisp anymore.
Again, while a 13MB SBCL image might be significantly larger than a 100KB C program, given that that's the entire compiler and runtime bundled in, and the size of additional code scales also like C/C++, that still makes it viable for almost every kind of program (and still an order of magnitude smaller than Electron). Same deal with memory usage.
> Finding solid libraries is tricky since it's not very popular professionally.
This one is so true it's not even funny. (although there are C, C++, and Python FFI's that cover most of the stuff that you want, although that's kind of cheating)
> Unleashing the full power of Lisp in a diverse team is a recipe for an adventure, if not disaster.
The list of companies using Clojure[2], in addition to the commonly-cited Viaweb/Orbitz/Grammarly, beg to differ. Anecdotally, most Lisps have less footguns than C++ - if people can figure out how to use Stroustrup's monster in massive video games, Lisp is easy.
One of my favorite stories of the ideas of lisp commercially is a dialect that Naughty Dog developed for their game development.
It started as being developed for Crash Banicoot on the PS1.
Which is really interesting given how limited the system was (1mb to 2mb of ram depending on what you were doing with the system!)
They later iterated on it for Jak and Daxter on the PS2. To quote the wikipedia page:
> GOAL does not run in an interpreter, but instead is compiled directly into PlayStation 2 machine code to execute.
> ...
> GOAL has extensive support for inlined assembly language code using a special rlet form,[1] allowing programs to freely mix assembly and higher-level constructs within one function.
> ...
> It supports a long term compiling listener session which gives the compiler knowledge about the state of the compiled and thus running program, including the symbol table. This, in addition to dynamic linking, allows a function to be edited, recompiled, uploaded, and inserted into a running game without having to restart.
In most modern engines I've used we STILL struggle hard for a perfect live-reload system.
I've implemented many types of custom interfaces that let you tweak things on the fly in my games, and it's almost criminal how much better/productive it feels vs. the 'tweak, compile, test' loop we're usually stuck with.
All that said, I see the same reason of why companies always transition away from lisp all the time:
> In all honesty, the biggest reason we're not using GOAL for next-gen development is because we're now part of Sony. I can only imagine Sony's shock when they purchased Naughty Dog a few years back, hoping to be able to leverage some of our technology across other Sony studios, and then realized that there was no way anyone else would be able to use any of our codebase.
It really makes me wish some big company would make a python clone in lisp, but put a tiny escape hatch in to fully utilize the lispy parts.
there was dylan before julia, so julia might just be reinventing dylan :) but that's not what's interesting
the julia project is in fact very interesting to me and has a great team developing its ecosystem and i work with it alongside python for numerical work. however one key drawback (compared to common lisp) for me is that it is not self-compiled. it is hosted on llvm and over 30% of its repository is in another language (mainly C and C++)[0]. but i am saying this only in comparison to common lisp. other languages are not different to this, and are much worse. as far as scientific computing is concerned, i would work with julia over python any day
It would be really cool to make Julia self hosting. The easiest non-julia in Base to replace is the parser which currently is in femtolisp. Most of the c++ code is just LLVM which we keep our own copy of because Julia has pretty specific LLVM version requirements that OS provided versions are unlikely to satisfy. Getting the C code out would be interesting since it's difficult to bootstrap a language that doesn't like being ahead of time compiled, but it would be a cool project.
because i want to be able to to go turtles all the way down with my code. julia being hosted on llvm does not provide that. if you want to do optimisations in julia you must learn llvm[0]. in this sense hosted languages are always a front-end. common lisp is not this
> in this sense hosted languages are always a front-end. common lisp is not this
Even self-hosted languages are front-ends for assemblers and microcode. What actually matters is being able to have arbitrary control over how code is executed. Julia has this in spades.
Julia has it's own intermediate representations where you can write custom compiler passes and intercept and modify whatever you like (in pure julia). You can think of these tools as being like macros, except instead of lexical extent they have a completely dynamic extent.
With these passes, you can arbitrarily change the meaning of any code recursively. E.g. You could define an execution context where multiplication and addition have swapped meanings, or you could locate every if-else statement in the entire callstack of a program and switch the if clause with the else clause. And again, this is not limited to the lexical extent of a macro, and it does not disable compilation. It is instead a programmable step in compilation.
A classic example of this would be Zygote.jl [1] which takes a program and recursively generates code to that would give the derivative of that program. Doing this requires you to modify literally every function call in the callstack, and you can do this all in pure julia without ever knowing a lick of LLVM.
Granted, there is something ideologically appealing about the idea of a self-hosted langauge. I'm just arguing that on a practical level, the compiler tools allow total control over code execution without changing languages, just as you'd get in a self-hosted language.
as i said i think that Julia is an exciting project in the scientific computing space. my statements were not directed against julia or any other language. i just happen to think that common lisp also has great potential for scientific computing, one that can appeal to scientists who want to be software engineers also. for those that just want to crunch numbers right now python is a first reference then julia, imho. however, i sincerely hope that julia overtakes python
having the ability to be self hosted is definitely not ideological. [* here is a hypothetical question that can be asked: would a julia programmer be more powerful if llvm was written in julia? i think the answer is clear that they would be *]
common lisp is a language specification. it does not have an official implementation, although by far the most popular one is sbcl. in fact there is an implementation CLASP[0] that is hosted on llvm
Yes, and sorry if I also came off as combative here, it was not my intention either. I've used some Common Lisp before I got into Julia (though I never got super proficient with it) and I think it's an excellent language and it's too bad it doesn't get more attention.
I just wanted to share what I think is cool about julia from a metaprogramming point of view, which I think is actually its greatest strength.
> here is a hypothetical question that can be asked: would a julia programmer be more powerful if llvm was written in julia? i think the answer is clear that they would be
Sure, I'd agree it'd be great if LLVM was written in julia. However, I also don't think it's a very high priority because there are all sorts of ways to basically slap LLVM's hands out of the way and say "no, I'll just do this part myself."
E.g. consider LoopVectorization.jl [1] which is doing some very advanced program transformations that would normally be done at the LLVM (or lower) level. This package is written in pure Julia and is all about bypassing LLVM's pipelines and creating hyper efficient microkernels that are competitive with the handwritten assembly in BLAS systems.
To your point, yes Chris' life likely would have been easier here if LLVM was written in julia, but also he managed to create this with a lot less man-power in a lot less time than anything like it that I know of, and it's screaming fast so I don't think it was such a huge impediment for him that LLVM wasn't implemented in julia.
great! i think that as far as scientific computing is concerned one has every right to be excited about julia. it is impressive how much work is being done in such a short time. i first came across it in 2015/6 and i am very happy about how its ecosystem evolved. i use julia and will continue to use it
this is what i would advise given the current state of affairs in machine learning
* use tensorflow / pytorch / pyro for industrial purposes. otherwise know what you are doing !
* use julia and its ecosystem for academic/research purposes. universities should be at the cutting edge of research
* use anything else if you want to experiment deeply. here is where i see the value of common lisp
> Finding solid libraries is tricky since it's not very popular professionally.
If we're considering Clojure, then I would argue this point isn't true. Java interoperability in Clojure is really well supported and easy to accomplish, so leaning on the massive Java library ecosystem is possible. Also, anecdotally, a lot of Clojure libraries appear to be defunct, but just actually haven't been updated in so long because of the strong backward compatibility across Clojure versions.
I haven't used CL so I can't comment on the differences between CL's C interop and Clojure's Java interop. But when I worked at a Clojure shop, "finding solid libraries" was never an issue, because we used Java libraries directly. And we usually preferred to use a Java library even when third-party Clojure wrapper libraries existed. Using Java from Clojure is natural and idiomatic.
This is a variant of the "Turing tarpit" fallacy - that is, saying that all things that are technically possible are equivalent.
Calling Java code in Clojure is completely seamless[1] - there's a world of difference from the CL FFI (which isn't that bad to use), which is turn easier to use than the CPython FFI.
You might as well say that C is just as powerful as Common Lisp, because you can implement Common Lisp in it. Ergonomics matter.
The point of a library is that you're supposed to be able to use its interface without worrying about the code underneath - so, Clojure does have access to the entire Java ecosystem.
My comment about lack of libraries was mostly referring to non-Clojures - CL in particular, but my argument also applies to Scheme and Racket and others.
I certainly don't recall it as being seamless; Clojure is a very different language from Java, and the interop code I've seen doesn't look like the kind of Clojure anyone would write otherwise, hence the comparison to CFFI which has the same impedance issues.
But it's all JVM bytecode, so ofcourse it's going to be somewhat smoother.
>> Finding solid libraries is tricky since it's not very popular professionally.
> This one is so true it's not even funny
I'd say that's a bit out of date as Quicklisp has made a huge difference for CL. Obviously it's still not the most library-rich ecosystem but it's come a very long way.
I don't think you should use "Lisp" as Common Lisp in one reply, and then "Lisp" as Clojure in the other. They are two very different languages. Though with the Armed Bear Common Lisp backend, you have access to the Java libraries.
> Getting Lisp to run as fast as C takes major effort when at all possible.
But Lisp is faster than almost every widely used language that's not C/C++/Rust, around Java speed.
> Lisp needs a lot of space to do it's thing; and while it's certainly possible to downsize it, you're left with something that's not really Lisp anymore
Dunno, by modern standards it seems pretty small.
> Finding solid libraries is tricky since it's not very popular professionally.
Ish. Because it's so mature, lots of code works even it it's not updated constantly. There's code out there for most use cases, at least for back-end-y things.
> Unleashing the full power of Lisp in a diverse team is a recipe for an adventure, if not disaster
So enforce coding standards. This isn't a negative and lots of us work alone or in small teams.
Honestly, the only reason I don't mainly use Lisp is because I use an even slower/easier language because I don't really need a performant language. But if I find a problem where I want more performance Lisp is probably the next stop.
> There's code out there for most use cases, at least for back-end-y things.
I'm not sure about that. These days in the backend you need a lot of libraries for cloud providers and new databases/services. For example, is there a Common Lisp library for AWS/Azure/Google cloud? Is there one for Clickhouse?
I've come across Common Lisp SDKs for AWS, there's Heroku build packs, there's Kubernetes clients listed on Kubernetes' website, etc...
Clickhouse doesn't have an official CL client but they have a CLI, TCP and HTML interfaces, C++ interface, etc... all listed on their website. It's not that hard to connect to an API without a library.
But yes, there's more Java/Python/Go/Ruby/JS stuff pre-built than Common Lisp. But you'll also find more CL stuff out there than almost any esoteric language, probably because it's easy to build stuff in CL.
Related to this, there's an AWS SDK for Clojure [0] (created by the same people who are behind Clojure), which is generated from the AWS specs themselves. Carmine, a popular Clojure library for Redis does something very similar. There's also a library for generating SDKs from Swagger specs [2]. I implemented a similar solution (generating Python bindings for some of our APIs from Postman exports), and it was super simple, really.
I suspect doing the same in CL would be similarly simple.
If performance was such a problem python wouldn't have a subreddit.
I think a lot of lispers like handling various paradigms in their head and start with basic lisp, and resort to edsl to reach more appropriate semantics/mechanical sympathy. Just like people call out to C wrappers mostly.
The social side of lisp I can't say for sure but I'm sure it's fuzzier than it seems. I've just talked to a dude saying his new guy was a clojurist and his thinking is way finer than the current team.
hehe. i remember a thing posted about python-on-guile, a rather ambitious project implementing python by compiling it to guile sheme.
The scheme code it produces is pretty awful. Not because it is bad, but because python is extremely unidiomatic scheme, and if you want to keep python semantics thay is where you have to go.
And despite being a one man job, despite being run on a far-from-the-fastest scheme, despite giving the optimizer any chance to do the regular optimizations it was about 1.5x times faster than cpython.
It was in numerical code, so beating cpython is maybe not very hard. I found it funny nontheless.
SBCL is already about as fast as Java and can get most of that speed without very much time optimizing everything.
Java has hundreds (thousands?) of developers working on it.
SBCL has a handful of part-time devs.
What could SBCL become with all those man-years behind it?
Custom syntax is hardly the big issue when using Common Lisp. I'll take that any day over the dozens of layers of useless abstraction in Java projects. I'll take it over the C code golf or "what does this undefined behavior do?". I'd even take it over the piles of garbage JavaScript I've waded through over the years.
I cut my teeth on Lisp, but I've found that happiness comes from compile time guarantees. Maybe those two things aren't mutually exclusive, but a thoroughly robust type system seems like a big ask in such a dynamic language. None of the solutions I've encountered have satisfied me, but in fairness I haven't looked that hard.
I'm fairly new to lisp programming, but there's a language called Coalton that provides static type checking to Common Lisp. I believe both languages are one and the same, but Coalton provides some type guarantees while still being able to have all of the interactiveness CL devs are used to.
I came to Clojure from F#/ReasonML/C#/PHP/JavaScript so I understand the comfort in specifying something with a type system and what it's like to not have one and what it's like to have a weak/strong one
Compile time for Clojure is when you inject new code into your running program
Imagine you have a large codebase with lots of interconnected types and then you tasked the computer with checking all those types everytime you injected
Would that slow down the code injections for you and every other developer? Yes maybe, what about checking for probable errors? Again yes but slower, the feedback loop in lisps are what make them feel magic so this would be a problem
What could we do to maintain dev speed and confidence?
This is my current approach:
First is clojure-lsp (clj-kondo) this will check for silly Monday morning mistakes and do it in a separate process so I can code inject unhindered but still spot errors via my editor
Second is a new library called hyperfiddle/rcf they are inline tests that run on code injection so any static assertion I want to write about data or functions I can
They will run under "compile"/code injection time and can be solidified into "real" tests at any time and maybe more importantly serve as great communication for how to use functions and what kind of data you can expect to flow through them close to the original definitions
For me those things combined with TDD, the repl and the static analysis from intellji and writing real tests every now and again is enough confidence for me and I'm in control of the "compile" time cost not the language
So when it comes to green/red cycles in TDD the repl (code injection) is great for creating code/solutions and hyperfiddle/RCF is great for creating the safety harness required to fearlessly refactor all triggered from inside the editor it's really addictive
Some CL implementations (including SBCL) have optional static type checking around function boundaries.
Barring that, Julia is a lispy language with great support for optional typing.
Both of the above will get a potentially better performance when you add types (the checks are (mostly?) done at compile time). Racket also has optional types, but I think these contracts are checked at runtime? I am not sure.
I have 0 experience here but that's really surprising to me.
I'd think out of any language lisp would shine here.
You could write as complex of a type system as you wanted that compile time checks, so it's surprising someone hasn't written one that mimics at least java/c++/whatever style types.
I do think that is one of the down sides to lisp though. When you can do almost anything it's very hard to agree on what to actually do.
> You could write as complex of a type system as you wanted that compile time checks, so it's surprising someone hasn't written one that mimics at least java/c++/whatever style types.
There's at least typed racket.
> I have 0 experience here but that's really surprising to me.
> I'd think out of any language lisp would shine here.
I don't think you should be surprised that people don't want to write their typecheckers as macros. A few people might want to do that, but most people don't. And at this point you're not far from just creating a new language, that you could create in ML or a descendant, which have always been one of the most popular options for that.
Not really getting the argument, the above lib pretty much makes it a superset that adds types.
And while I don't know the scope of the library, if they have a way to not allow dynamic features then you're already starting from a having a typed language (with one library dependency).
Only now you can keep building up the type system in ways that benefit your project, unlike in most non lisp languages.
Wow, I'm glad that they opened up the documentation. I liked what I saw a few years ago, but I'm not getting behind a language where all the usable docs are proprietary.
Can I be extremely shallow? Because the language feels like it's from the 60s. Like writing Fortran or Ada or Algol in 2021. C is timeless, it doesn't count.
I love lisps, but Racket, for example, feels more modern than Common Lisp to me. And for shallow people like me, feel and ergonomics are very important when writing code.
When we're talking about strong type systems and classes of bugs eliminated just because the program compiled (a la OCaml, Rust, Haskell)... yes, newer is better.
Your argument isn't universal; something being older doesn't say much about its quality either.
In Julia, the convention is that a function like "swap!" modifies its input. It's extremely handy. Also, "odd?" is just an elegant way to name a predicate.
Programs? Aren't we talking about variable and function names?
But for the general case, Lisp does use puncuation and symbols. It just tends to use less. The other side of the balance is operators like ~+#>. Most languages are in the middle ground. All of these options are fine and have their uses. I wouldn't like to see stuff like ~+#> every day, but I like being able to implement <.
I think at some point we should clarify why we mean by Lisp. I see a lot of mention of Clojure, while most people seem to assume that Lisp == Common Lisp. I feel like talking about Common Lisp, Racket, Scheme, Clojure at the same time while putting them all under the "Lisp" umbrella seem to be a bit pointless when we're talking about languages. It's like mentionning JS in a conversation about C because "JS is a language with a C-style syntax and macros (with Babel)". That's true but it's a totally different languages, and that's without even mentionning the different implementations of C and JS.
I look at it as an intersection between the feature sets of Common Lisp, Clojure, Scheme, etc.
In other words - the most basic features of these languages that are common - S-expressions, data is code is data, REPL-driven development, etc
Lisp seems a lot like Linq to me where it's an undeniably cool part of a language but crucially, just a part. If you plop Linq down on someone's lap, they're not going to know what to do with it. If you give someone Linq in the context of C#, suddenly it's the greatest thing you've ever laid eyes on.
Likewise, if you're using Lisp within the context of some other language for scripting, it's fucking phenomenal. If you try to use it on its own to build a whole project from 0 to 100, you're going to be confused as to why you're not using a different language.
What's a good Lisp dialect to write small programs in? I have some Emacs Lisp experience but I wouldn't use it to automate small tasks. For that, I typically use Python.
I'm also a fan of the first tutorial on the site. Instead of the boring stuff like hello world, 2+2, print your name etc, they use a built in library for drawing shapes to the console, and learn how to build up procedures for drawing more complex objects.
All the good stuff said, I've never used it for system plumbing scripts like I do with python. I'd imagine it's worse for that due to the sheer popularity python has.
Definitely Common Lisp. Instant startup time (load all your dependencies and then dump a new SBCL compiler with ‘save-lisp-and-die’), great emacs support, a mature ecosystem, and fast.
Clojure is great if you don’t care about the startup time, as it has good emacs support, and you can fall back on the JVM ecosystem. It’s the most concise of all the lisps I have tried, but also a tad opinionated.
Caveat: I'm still working towards being able to recommend Bel _unconditionally_, not just for small programs. Right now you'll experience unreasonable slowness, terse/uninformative error messages, and missing documentation -- probably in that order. All of those are being addressed. But already today, it's fun to play with.
Emacs Lisp is better than fine. You get a powerful Lisp (with a lot of goodies from CL) that is both portable and ultra-stable whilst having a self-contained VM and an enormous library of code available (Emacs) on every system imaginable.
Clojure through Babashka[0]. It's great for small tasks not related to your favorite editor. I use it for most of my scripting except for the really easy things in bash.
I haven't tried it, but there is Janet (https://janet-lang.org/) that seems pretty well suited for building small scripts. It was submitted a few times recently here on HN.
Why wouldn't you use Emacs Lisp to automate small tasks? I think it's great for that purpose, it's usually pretty easy to give a utility a very comfortable user interface in Emacs.
Chicken Scheme is probably a nice middle-of-the-road scheme to try. It's "eggs" offer a lot of the functionality you'd want, the FFI for calling C is decent, and the documentation isn't bad.
TL;DR: Because it uses a minimal representation for data (S expressions). This makes it much easier to write representations of data (compared to XML, or even JSON). This makes it easier to represent code as data, and that opens up the whole world.
This is the TXR Lisp interactive listener of TXR 271.
Quit with :quit or Ctrl-D on an empty line. Ctrl-X ? for cheatsheet.
1> '#J{"foo":[1,2,3,"bar"]}
#J{"foo":[1,2,3,"bar"]}
The embedded hash table denoted by the literal is regurgitated.
We can quasiquote JSON:
;; ^ is quasiquote in this dialect not `
1> ^(,(+ 2 2) ,(list 1 2 3))
(4 (1 2 3))
;; cannot use , in JSON for unquoting, so ~ is used:
2> ^#J["foo", ~(+ 2 2.0)]
#J["foo",4]
Exactly - it's not about whether it's possible to represent code as data (gcc is written in C, after all), it's about how easy it is. That's why Rust macros still don't come close to Lisp ones.
Well I learned something. But if you represent a list as a lift of strings wouldn’t it have to parse them as Argv in the c implementation? It has been a while but only 17 years since I wrote TCL commands but each one started with a list of strings. Maybe the byte code interpreter doesn’t work with older C extensions? But the main selling point of TCL was easy extension.
Lisp probably was invented in the age of the typewriter.
If you look at mathematical notation, written with quill pens and paper…it’s even more concise. Functions are like lisp, f(x). Vectors are laid out on the page in an array and surrounded by two strokes. Matrices are a 2d array surrounded by two straight lines.
This whole approach seems as archaic as Roman numerals is for representing numbers (they were based on lines and slashes like a tally system).
Maybe there is an even better notation for representing code which will be as much a leap forward as the positional number system (which represents every number as a power series with base 10) was to numbers.
I disagree. I think a way to summarize this article is that the popular languages (Algol/C family) are like Roman numerals, and Lisp is like Arabic: it composes better. It's a powerful, qualitative difference.
One advantage would be that the stuff between (begin document) and (end document) consists of multiple forms which are separately read, expanded and processed, which can be quite important in some situations.
Also there is the possibility that some form in the document could load another file.
I have a wonderful book in front of me called “From One to Zero: a universal history of numbers” by georges ifrah.
chapter 9, roman numerals: a vestige of primitive origins? starts like this:
——————————
I V X L C D M
They are obviously letters of the roman alphabet, but this does not mean they have an alphabetic origin. The signs L, C, D, and M are not the original forms of the numerals 50, 100, 500, and 1000; they are altered forms of much older numerals. known instances of the use of L, D, and M as numerals do not go back much farther the the first century AD […]
—————————
The chapter goes on to illustrate the resemblence with etruscan numerals and the tally marks of ancient shepards counting sheep, see the “tally marks” section of the wikipedia page, or borrow the book from archive.org, page 131 to 146 or so.
It's funny because I consider other languages like roman numerals, unhomogeneous bunch of constructs that don't work well with themselves, meanwhile lisp is an infinite tower.
Now maybe there are better notations though. I'm all ears.
Writing a C program that generates and compiles C code is a major pita in comparison, same goes for most other languages.
Even if you slap a different syntax on top, it's still a Sunday walk in the park to write a custom compiler.
https://github.com/codr7/snabl