Surely there could be some way of creating a JsonArray of native Java Strings, Booleans, Doubles, and Integers without requiring clients to explicitly convert each value into a JsonValue. And why am I forced to convert a native List into a JsonArray just so I can make it the value of a JsonObject?
And vice versa. As someone who likes both languages, I appreciate how they both have their pros and cons. I was a Schemer before I learnt Java (when Java barely existed) and I adore Clojure, but if I were to write 10 MLOC mobile network routing and billing system, an air-traffic control system, or a credit-card transaction processing system etc. etc. that needs to evolve by a large team for 20 years, I would choose Java over Clojure any day. So I came to Java from C, C++, Scheme, and ML, and it's completely unsurprising to me why there has never existed a language that better supports large, long-lasting, important software (I'm not saying such a language couldn't exist, but it currently doesn't). And it's not just about the features Java has and, just as importantly, doesn't have (and if you think keeping features out of a programming language is easy, think again), but also how its evolution is handled.
In this particular case, of course it is more straightforward to embed what is effectively an untyped language in another untyped language than in a typed language, at least while keeping the language simple and without adding features we don't wish to add (even in TypeScript, working with JSON directly requires bypassing type checks). Still, as the JEP says, there are good, popular Java libraries that offer more convenient and powerful ways of working with JSON, but that is not the purpose of this particular package.
Unfortunately that library requires paying tribute to its owners, only made relevant thanks to shipping in phones powered by a green droid.
"The next thing is also fairly straightforward: we expect Kotlin to drive the sales of IntelliJ IDEA. We’re working on a new language, but we do not plan to replace the entire ecosystem of libraries that have been built for the JVM. So you’re likely to keep using Spring and Hibernate, or other similar frameworks, in your projects built with Kotlin. And while the development tools for Kotlin itself are going to be free and open-source, the support for the enterprise development frameworks and tools will remain part of IntelliJ IDEA Ultimate, the commercial version of the IDE. And of course the framework support will be fully integrated with Kotlin."
I'm pretty partial to Groovy's JSON library - it has a nice syntax it just works (caveat that I had issues once where the size of the JSON was multiple megabytes and it corrupted).
I really wish the JDK devs would start using constructors again. I understand why factory methods exist but everything these days is of/from/newInstance/something else. Dart/Scala/Kotlin have all really solved this with language features and construction is uniform. For example in Dart they have factory constructors: https://dart.dev/language/constructors#factory-constructors
Java lacks the expressiveness to make it much better than this. There's a reason almost all the "replacement Java" languages add something to support DSLs, e.g. type safe builders in Kotlin [1]
The kotlin stdlib-adjacent json lib does it like this
or similar (aka faceted fluent API), with typed interfaces mirroring the grammar of the target language, here JSON.
This is basically a structured output iterator (or OutputStream-like) pattern. It can also support modularization such that substructures can be factored out into separate methods or lambdas, thereby also allowing loops and conditionals.
Such an API can be provided in a relatively compact fashion and would be nicer than what the JEP proposes.
That's not the reason. The question was about JSON generation from existing Java types, not about a DSL. Java JSON libraries offer similar conveniences (in fact, you can be much more sophisticated, clear, and convenient than the DSL you've shown), but this particular library, as explicitly described in the JEP, is not intended to be a general-purpose JSON library, of which Java already has several good and popular ones.
FWIW, in Kotlin's native JSON library, the API is almost identical.
val json = JsonObject(
mapOf(
"providers" to JsonArray(
listOf(
JsonPrimitive("SUN"),
)
)
)
)
println(json)
Of course nobody generally does it this way, usually you take a List/Map and directly serialize that with a helper
val data = mapOf("providers" to listOf("SUN", "SunRsaSign", "SunEC"))
val kotlinxJSON = Json.encodeToJsonElement(data)
val jacksonJSON = ObjectMapper().writeValueAsString(data)
I'm not involved with the design of this API, but it seems to me that the issue on the JSON generation side (where you're complaining about ceremony) is feature creep. Creating the API you want on top of the proposed one is trivial (even in user code), but then where do you stop? If you have that conversion, it seems reasonable to also support, say, sets and records; maybe even enums.
Indeed, Java JSON libraries typically offer all these conveniences and more, but this package is not intended to replace them - as the JEP clearly states ("It is not a goal to create an API that supplants established external JSON libraries"). It is intended to support only simple JSON tasks without requiring a library. For that goal, feature creep is particularly problematic: the more you offer (which reduces the need for the popular libraries in more situations) the greater the pressure to add even more.
Often, it's best to start with the minimum required, and then, as the API gets used in the field, see what the most valuable convenience methods to add on top.
POJOs and records require more configuration (e.g. Jackson's @JsonProperty, @JsonDeserialize). That could plausibly be out of scope.
But for constructing JSON out of strings, numbers, booleans, lists, and maps, there's really not that much scope to creep into.
Specifically, I think it would be perfectly cromulent to have JsonArray.of() be able to support any Iterable of native Strings, Integers, Doubles, or Booleans; that doesn't feel like feature creep to me at all. It would transparently support Sets. (Right now, the API only accepts Lists of JsonValues, which is what makes the API feel so ceremonious.)
> there's really not that much scope to creep into
First, we've been in this game far too long to know that this isn't the case. Second, this is only incubation. It may well be that the team behind this feature intend to add more convenience methods but wish to do it later once the core is more battle-tested. It's always best to focus on the core first and add ornamentation once you know the core is right.
Shouldn't the core provided by this JEP be a streaming API then, so that you can build whatever you need on top of it? But they specifically exclude that from the goals.
> Specifically, I think it would be perfectly cromulent to have JsonArray.of() be able to support any Iterable of native Strings, Integers, Doubles, or Booleans; that doesn't feel like feature creep to me at all.
What type would that method accept? Wouldn't it need to be Object? That seems worse than boilerplate
Where to stop is a pretty fuzzy question I suppose.
In my world, I think the worst possible outcome would be a java.util.json library that does 50-95% of what I currently do with GSON or Jackson. In that scenario I still need an external dependency, and I can either ignore java.util.json or turn a codebase into an error-prone mix of both.
Maybe a good set of design considerations for java.util.json would be “what does this API need to run a CRUD app written in modern java?” Parse requests from the web, send responses to the web, and serialize/deserialize data from X database (e.g. if noSQL or jsonb). In my head “in modern java” would mean that JSON is converted to records at application boundaries
> Maybe a good set of design considerations for java.util.json would be “what does this API need to run a CRUD app written in modern java?”
Reading the JEP, that is not the motivation. The target is more a short script or some REPL interaction that reads JSON data from a web service and does something with it. A CRUD app likely already uses a web framework that comes with a full JSON library.
java.lang.String and other types don't extend JsonValue, and java lacks any trait-like way to add this functionality to existing types, so you would have to change the signature to this:
Now you can pass any Object in, but the typechecker can't ensure that it is convertible to json anymore. I.e. it will have to check at runtime that it's either JsonValue or another type that is has a known conversion for (Integer, Double, String, List, Map, etc.). The jackson ObjectMapper e.g. has a lot of configuration available to tell it how to do these conversions on arbitrary types, and I think they want to eliminate that kind of ceremony.
It does seem like any serious application is going to use another library, and this will be useful for very simple json usage or single-file hello-world type programs (e.g. to go along with Implicitly Defined Classes and the Flexible Launch Protocol).
A Modest Proposal that will never be implemented: add an interface to String, Boolean, Integer, Long, Float and Double. Because all these types already implement "toString" (and I believe their toString representations are compatible with JSON), it can be a pure marker interface.
> I believe their toString representations are compatible with JSON
Not quite. String is the big problem, since it needs to be wrapped in quotes, and special characters need to be escaped. But Float and Double are also problematic because Infinity and NaN aren't representable in json.
However, the new interface could have a "toJsonString" or maybe even a toJson method that returns a JsonValue
For the "quick one-off script" case (where Implicitly Declared Classes shine), I think the most galling ceremony in this example is explicitly converting all N items in a list of literals into JsonValues for JsonArray.
Yes, what's the point of JsonObject at all? Why JsonString when there is String? Why JsonArray when there is List? JDK only needs a function to convert from JSON format to the normal data structures it already has and back.
Because JSON types don't cleanly map to Java types. For example, a JSON number could be an int, a long, a double, a BigInteger, or a BigDecimal. Which of them the Java code wants is not something you can deduce (most generally, you could represent all JSON numbers as BigDecimal, but that's not very convnient in Java code, so in most cases you'd want to convert that to a primitive type of your choosing anyway). A JSON array is heterogeneous, while Java code typically want to be homogeneous. Representing a JSON array as a `List<Object>` won't work because the conversion of the element types is ambiguous (e.g. numbers).
Yeah you could go with just Number though, and then keep almost everything a BigInteger/BigDecimal under the covers (or dynamically choose
the correct class)
You could, but it would be significantly worse. Number and JsonNumber are nearly equivalent in this context, except that JsonNumber has the major advantage of being in the same sealed hierarchy as the other JSON types, allowing for nice pattern-matching (and BigDecimal suffers from the same downside). So you gain nothing - you have to convert to the desired type anyway - and lose quite a bit.
It's pretty normal, almost all libs work this way. This way you can parse the JSON string into the matching data structure or print it into a string, and when building you can ensure it's valid JSON.
> Surely there could be some way of creating a JsonArray of native Java Strings, Booleans, Doubles, and Integers without requiring clients to explicitly convert each value into a JsonValue
I think so to, and it surprises me that they write
“A key goal driving the recent evolution of the Java Platform has been to enable simple tasks to be accomplished more easily and with less ceremony. Features serving this goal include convenience factory methods for collections […]”
and don’t, from that, decide that this API needs such factory methods.
The API also doesn’t make JsonObject or JsonArray collections, requiring one to use ‘asMap’ or ‘asArray’ before iterating over them.
What benefit does that carry? That one can implement those interfaces as records?
You could... but I view JSON as a simple format that you could write a recursive parser for in a few hundred lines yourself.
Bringing in compile-time code generation just for defining static JSONs (which is not that common outside of tests, as most JSONs are serialized and deserialized at runtime) sounds like a hard sell.
The clean way of doing this is to build a json marshaling mechanism for the type system as it already exists. This is doable in Java, and some json libraries (e.g. gson) are already capable of this.
I must admit I don't fully understand the motivation behind this JEP. Like when would I ever reach for this?
The motivation behind this JEP is laid out in the JEP under the section "Motivation".
As I understand that section (and I wasn't involved in writing this JEP), good and popular marshalling libraries for JSON already exist, and the JEP clearly states that it is not the goal to replace them or perform their role. The JEP says that this package may be what you'd reach for when a program only wants to do some very simple, small tasks with JSON data and the requirements and code size don't merit pulling in a fully-featured JSON library (e.g. when you're writing a one-file script, or exploring in JShell).
A stated goal of the API is to have "low ceremony" when reading arbitrary JSON files. I agree the JSON creation API looks pretty clunky, but the parsing API looks fairly simple.
minimal-json scratches this itch really well for me. Dead simple API. Unfortunately not maintained anymore, but I'm still using it, even for new projects, because it just works so well.
I was going to make fun of Java before even opening the page with something to the tune of `AbstractBeanJsonSimpleFactory` but looks like reality beat me to it, heh.
One of my favorite things about Jackson was being able to arbitrarily navigate through the document with a rich and fluent API (JsonNode) in jackson.databind, which this JEP at least conceptually borrows from with the JsonValue abstraction. Both of these are better than how some of the other implementations do it, where you are effectively working with glorified Map<String,Object>
I already have one of these (I'm sure I'm not alone). It's about 500 lines.
I found I'm not a huge fan of the JAX-B-ish style serialization of Java objects for JSON. I don't want to really downplay them, they certainly have their uses, they're very popular, I just don't like fighting them. Hand writing JSON marshaling code has not been arduous for me (notably with my utility layer). (I also, philosophically, strive to avoid "magic" in my code as much as practical.)
Of course, I still need a parser, I'm using GSONs parser, which means I'm still dragging in the whole bean level serialization infrastructure. I just don't use it. And while I've done JSON parsers before, I felt it was something better to import than maintain myself. So, in that sense, it's a mixed bag.
But, I do enjoy using it.
This will be a worthwhile JDK capability. Ideally it can replace mine.
String body = ... REST response body, which is a JSON document ... ;
JsonValue json = Json.parse(body);
json.get("properties").get("periods").asList().stream()
.mapToInt(j -> j.get("temperature").asInt())
.average()
.ifPresent(IO::println);
Why am I able to call `.get(string)` or `.get(int)` on a JsonValue? Shouldn't these be on the JsonObject and JsonArray instead?
> If the JsonValue instance is of the wrong type, or if the requested member or element does not exist, the access methods throw a JsonValueException.
So if I get an exception, I can't tell whether the value was of the wrong type or the object didn't have the requested key? This looks like a footgun to me.
They just brought pattern matching to Java. Why not move those .get to JsonArray and JsonObject? That would solve this confusions. So we could just use something like `if (json instanceof JsonObject o) o.get("properties")`
That is indeed baffling and regrettable. Perhaps they believe that users will just hard-cast what .parse() returns to JsonObject/JsonArray/whatever, and that the resulting ClassCastException will be uglier and harder to debug than whatever errors are currently produced by calling .get() on something other than JsonObject?
My understanding from reading this is the complete opposite. This library is explicitly not supporting the features that web servers need to be performant and handle production traffic, like streaming.
A web server using this could only start parsing when it receives the last byte, and could only start responding when it’s done serializing, all while holding non-lazy trees of JsonValue objects in memory.
I suspect the vast majority of services are not dealing with such massive JSON documents that doing serde on them becomes a material part of their latency breakdown.
I've never built or worked on a service where the JSON payloads were so large that (de)serialization accounted for a significant portion of the timing profile
I'm sure lots of them exist, but for your typical CRUD API, this has not been a phenomena I've run into.
I don’t have any benchmarks, but it’s surprisingly relevant. Not necessarily because parsing JSON is slow, but because managing memory is slow, and you’re dealing with potentially malicious input.
A non-streaming implementation needs to copy the request from the network stack into a contiguous GC-managed char array, possibly resizing it a few times as the data is received. Then when it’s time to parse, it goes through this array and allocates an unbounded number of JsonValue nodes. For JsonString and JsonNumber, it probably needs to create defensive copies of the data instead of spans of the input array, otherwise changing the input array corrupts the tree.
That’s kinda bad under memory pressure even for benign inputs. But consider malicious inputs, such as {"x":{"x":{"x":{"x":{"x":{…}}}}}. It would make this non-streaming implementation allocate a lot of String and JsonObject instances. The allocations would total multiple times the size of the input, and would be extremely fragmented.
On the other hand, a library that does streaming and that binds to objects could parse straight from the buffers in the network stack, and could avoid allocating objects for anything that it will not need to bind.
Using JSON as a configuration format is a big mistake. JEP authors could learn a bit from package.json problems. Hope JSON will not be used in anything significant for JDK configuration.
Including this in the standard library speaks to how much a citizen JSON has become, where a language simply can't afford to not consider it.
I find it interesting to note that nowhere in this JEP is the word "serialization", which is what most people might associate with JSON libs. Or rather, they are studiously ignoring that feature and just improving the ergonomics of interacting with JSON.
If I'm guessing correctly what you mean by "serialization", the JEP refers to it as "data binding" and includes a section on why it's not going to be part of this library.
Not supporting comments will be a mistake that haunts this API. They give an example of replacing properties files but those do have standardized comments! The proposed pre-processing step means all the comments are lost during round tripping, and the single line comments they suggest are not enough to even match JSONC. By the time those issues have userland workarounds you might as well use another library instead of the built-in one.
This seems to repeat the same mistakes of Go's built in JSON library where the ecosystem is full of workarounds and other libraries that are faster or have better features.
Eh, I think it's a good tradeoff. If it losslessly supported deserializing comments/JSONC, or trailing commas, or JSON-lines, or whatever, the serialization APIs would get more complicated. Every time you serialize you'd have to decide which of several formats you were producing. Automatic round-trippability would still be impossible in that world, since e.g. "deserialize JSON-ish, set one key=value, reserialize" would then risk producing a not-strictly-JSON object that broke whatever it was sent to, so then you'd need a whole bunch of different serialization configs/settings, which would confuse newbies (either they produce something that's subtly different from what they need, or they accidentally strip out information).
As similar as all the almost-JSON formats are, I still think it's best to keep APIs single-purpose: one for JSON, one for JSON-lines, one for JSONC, and so on. It's a larger code surface, but a less potentially surprising one.
I find it very surprising that they went for unchecked exceptions. For JsonValueException the following rationale is given "This exception is unchecked, so that scripts and small programs are easier to read and write." But for JsonParseException there is no rationale. This is surprising especially given the pushback from openjdk members against jackson3 moving to unchecked exception.
Happy to see that numbers are arbitrary width/precision until explicitly cast by the user. This goes against so many other JSON libraries that will always cast all numbers to double (or even float) thereby silently corrupting JSON numbers representing large values (eg. memory addresses).
Does anyone know how this behaves when encountering a repeated key in an object? (RFC8259 states that keys SHOULD be unique, which makes that generally allowed and implementations all behave slightly differently).
They're doing a real API instead of the magic annotation hell that JavaEE fans love. Thank god. I welcome this, because Java has a lot of need for a good, common, performant, and well maintained JSON library that doesn't come along with the complexity of being JSON-B.
A stated goal of the API is to have "low ceremony"; this seems like a lot of ceremony.
There's gotta be a better way!Surely there could be some way of creating a JsonArray of native Java Strings, Booleans, Doubles, and Integers without requiring clients to explicitly convert each value into a JsonValue. And why am I forced to convert a native List into a JsonArray just so I can make it the value of a JsonObject?
Why can't I write this?
There's a nice Java library called Clojure with a lightweight syntax if you need to work with data structures and concurrency in Java a lot:
You really can't appreciate how awesome Clojure is until you're coming from Java, can you...
And vice versa. As someone who likes both languages, I appreciate how they both have their pros and cons. I was a Schemer before I learnt Java (when Java barely existed) and I adore Clojure, but if I were to write 10 MLOC mobile network routing and billing system, an air-traffic control system, or a credit-card transaction processing system etc. etc. that needs to evolve by a large team for 20 years, I would choose Java over Clojure any day. So I came to Java from C, C++, Scheme, and ML, and it's completely unsurprising to me why there has never existed a language that better supports large, long-lasting, important software (I'm not saying such a language couldn't exist, but it currently doesn't). And it's not just about the features Java has and, just as importantly, doesn't have (and if you think keeping features out of a programming language is easy, think again), but also how its evolution is handled.
In this particular case, of course it is more straightforward to embed what is effectively an untyped language in another untyped language than in a typed language, at least while keeping the language simple and without adding features we don't wish to add (even in TypeScript, working with JSON directly requires bypassing type checks). Still, as the JEP says, there are good, popular Java libraries that offer more convenient and powerful ways of working with JSON, but that is not the purpose of this particular package.
For the systems you mentioned I’d choose Erlang or Elixir over Java or Closure. The right runtime for the right problems.
Interesting how we need two sides balancing each others.
Dynamically typed languages by their nature always have an easier time with (de)serialization.
I prefer its sibling library Kotlin, from the makers of the world famous Java IDE: Jetbrains Fleet
Isn’t json processing still a dependency in Kotlin? I love the language but I don’t think it’s equivalent to built in json processing.
Unfortunately that library requires paying tribute to its owners, only made relevant thanks to shipping in phones powered by a green droid.
"The next thing is also fairly straightforward: we expect Kotlin to drive the sales of IntelliJ IDEA. We’re working on a new language, but we do not plan to replace the entire ecosystem of libraries that have been built for the JVM. So you’re likely to keep using Spring and Hibernate, or other similar frameworks, in your projects built with Kotlin. And while the development tools for Kotlin itself are going to be free and open-source, the support for the enterprise development frameworks and tools will remain part of IntelliJ IDEA Ultimate, the commercial version of the IDE. And of course the framework support will be fully integrated with Kotlin."
https://blog.jetbrains.com/kotlin/2011/08/why-jetbrains-need...
I'm pretty partial to Groovy's JSON library - it has a nice syntax it just works (caveat that I had issues once where the size of the JSON was multiple megabytes and it corrupted).
I really wish the JDK devs would start using constructors again. I understand why factory methods exist but everything these days is of/from/newInstance/something else. Dart/Scala/Kotlin have all really solved this with language features and construction is uniform. For example in Dart they have factory constructors: https://dart.dev/language/constructors#factory-constructors
and for Kotlin/Scala you can use some cheeky operator overloading via companion objects.
Java lacks the expressiveness to make it much better than this. There's a reason almost all the "replacement Java" languages add something to support DSLs, e.g. type safe builders in Kotlin [1]
The kotlin stdlib-adjacent json lib does it like this
[1] https://kotlinlang.org/docs/type-safe-builders.html> Java lacks the expressiveness to make it much better than this.
That's not really true. You can have a Java library providing:
or as a shortcut or similar (aka faceted fluent API), with typed interfaces mirroring the grammar of the target language, here JSON.This is basically a structured output iterator (or OutputStream-like) pattern. It can also support modularization such that substructures can be factored out into separate methods or lambdas, thereby also allowing loops and conditionals.
Such an API can be provided in a relatively compact fashion and would be nicer than what the JEP proposes.
That's not the reason. The question was about JSON generation from existing Java types, not about a DSL. Java JSON libraries offer similar conveniences (in fact, you can be much more sophisticated, clear, and convenient than the DSL you've shown), but this particular library, as explicitly described in the JEP, is not intended to be a general-purpose JSON library, of which Java already has several good and popular ones.
That's not what the person I replied to asked at all, arbitrary types weren't mentioned.
They literally asked why they couldn't write JsonObject.of(Map.of(...))
And there's nothing in Java preventing that. In fact, it is trivial to implement that exact API on top of the API proposed in the JEP.
FWIW, in Kotlin's native JSON library, the API is almost identical.
Of course nobody generally does it this way, usually you take a List/Map and directly serialize that with a helperI'm not involved with the design of this API, but it seems to me that the issue on the JSON generation side (where you're complaining about ceremony) is feature creep. Creating the API you want on top of the proposed one is trivial (even in user code), but then where do you stop? If you have that conversion, it seems reasonable to also support, say, sets and records; maybe even enums.
Indeed, Java JSON libraries typically offer all these conveniences and more, but this package is not intended to replace them - as the JEP clearly states ("It is not a goal to create an API that supplants established external JSON libraries"). It is intended to support only simple JSON tasks without requiring a library. For that goal, feature creep is particularly problematic: the more you offer (which reduces the need for the popular libraries in more situations) the greater the pressure to add even more.
Often, it's best to start with the minimum required, and then, as the API gets used in the field, see what the most valuable convenience methods to add on top.
POJOs and records require more configuration (e.g. Jackson's @JsonProperty, @JsonDeserialize). That could plausibly be out of scope.
But for constructing JSON out of strings, numbers, booleans, lists, and maps, there's really not that much scope to creep into.
Specifically, I think it would be perfectly cromulent to have JsonArray.of() be able to support any Iterable of native Strings, Integers, Doubles, or Booleans; that doesn't feel like feature creep to me at all. It would transparently support Sets. (Right now, the API only accepts Lists of JsonValues, which is what makes the API feel so ceremonious.)
> there's really not that much scope to creep into
First, we've been in this game far too long to know that this isn't the case. Second, this is only incubation. It may well be that the team behind this feature intend to add more convenience methods but wish to do it later once the core is more battle-tested. It's always best to focus on the core first and add ornamentation once you know the core is right.
Shouldn't the core provided by this JEP be a streaming API then, so that you can build whatever you need on top of it? But they specifically exclude that from the goals.
> Specifically, I think it would be perfectly cromulent to have JsonArray.of() be able to support any Iterable of native Strings, Integers, Doubles, or Booleans; that doesn't feel like feature creep to me at all.
What type would that method accept? Wouldn't it need to be Object? That seems worse than boilerplate
Where to stop is a pretty fuzzy question I suppose.
In my world, I think the worst possible outcome would be a java.util.json library that does 50-95% of what I currently do with GSON or Jackson. In that scenario I still need an external dependency, and I can either ignore java.util.json or turn a codebase into an error-prone mix of both.
Maybe a good set of design considerations for java.util.json would be “what does this API need to run a CRUD app written in modern java?” Parse requests from the web, send responses to the web, and serialize/deserialize data from X database (e.g. if noSQL or jsonb). In my head “in modern java” would mean that JSON is converted to records at application boundaries
> Maybe a good set of design considerations for java.util.json would be “what does this API need to run a CRUD app written in modern java?”
Reading the JEP, that is not the motivation. The target is more a short script or some REPL interaction that reads JSON data from a web service and does something with it. A CRUD app likely already uses a web framework that comes with a full JSON library.
Currently, JsonObject.of has this signature:
java.lang.String and other types don't extend JsonValue, and java lacks any trait-like way to add this functionality to existing types, so you would have to change the signature to this: Now you can pass any Object in, but the typechecker can't ensure that it is convertible to json anymore. I.e. it will have to check at runtime that it's either JsonValue or another type that is has a known conversion for (Integer, Double, String, List, Map, etc.). The jackson ObjectMapper e.g. has a lot of configuration available to tell it how to do these conversions on arbitrary types, and I think they want to eliminate that kind of ceremony.It does seem like any serious application is going to use another library, and this will be useful for very simple json usage or single-file hello-world type programs (e.g. to go along with Implicitly Defined Classes and the Flexible Launch Protocol).
> java.lang.String and other types don't extend JsonValue
And that ladies and gentlemen is what Java's re-doing of TypeClasses (called "witness" in the experiments they're doing now) are for.
A Modest Proposal that will never be implemented: add an interface to String, Boolean, Integer, Long, Float and Double. Because all these types already implement "toString" (and I believe their toString representations are compatible with JSON), it can be a pure marker interface.
> I believe their toString representations are compatible with JSON
Not quite. String is the big problem, since it needs to be wrapped in quotes, and special characters need to be escaped. But Float and Double are also problematic because Infinity and NaN aren't representable in json.
However, the new interface could have a "toJsonString" or maybe even a toJson method that returns a JsonValue
JSON String requires escaping of control characters. .toString() on a String is (hopefully) the identity.
For the "quick one-off script" case (where Implicitly Declared Classes shine), I think the most galling ceremony in this example is explicitly converting all N items in a list of literals into JsonValues for JsonArray.
This would help a lot:
Then, you could at least write: And for JsonObject, a little fluent builder API would probably knock out a lot of ceremony, too. The alternative today looks quite ceremonious:You almost can, with Jackson:
(or was that the joke?)Yes, what's the point of JsonObject at all? Why JsonString when there is String? Why JsonArray when there is List? JDK only needs a function to convert from JSON format to the normal data structures it already has and back.
Because JSON types don't cleanly map to Java types. For example, a JSON number could be an int, a long, a double, a BigInteger, or a BigDecimal. Which of them the Java code wants is not something you can deduce (most generally, you could represent all JSON numbers as BigDecimal, but that's not very convnient in Java code, so in most cases you'd want to convert that to a primitive type of your choosing anyway). A JSON array is heterogeneous, while Java code typically want to be homogeneous. Representing a JSON array as a `List<Object>` won't work because the conversion of the element types is ambiguous (e.g. numbers).
Yeah you could go with just Number though, and then keep almost everything a BigInteger/BigDecimal under the covers (or dynamically choose the correct class)
You could, but it would be significantly worse. Number and JsonNumber are nearly equivalent in this context, except that JsonNumber has the major advantage of being in the same sealed hierarchy as the other JSON types, allowing for nice pattern-matching (and BigDecimal suffers from the same downside). So you gain nothing - you have to convert to the desired type anyway - and lose quite a bit.
Good point for parsing. However, the example is about serialization. What mapping ambiguity do you see there?
There it's not about ambiguity: https://news.ycombinator.com/item?id=49025644
It's pretty normal, almost all libs work this way. This way you can parse the JSON string into the matching data structure or print it into a string, and when building you can ensure it's valid JSON.
> Surely there could be some way of creating a JsonArray of native Java Strings, Booleans, Doubles, and Integers without requiring clients to explicitly convert each value into a JsonValue
I think so to, and it surprises me that they write
“A key goal driving the recent evolution of the Java Platform has been to enable simple tasks to be accomplished more easily and with less ceremony. Features serving this goal include convenience factory methods for collections […]”
and don’t, from that, decide that this API needs such factory methods.
The API also doesn’t make JsonObject or JsonArray collections, requiring one to use ‘asMap’ or ‘asArray’ before iterating over them.
What benefit does that carry? That one can implement those interfaces as records?
You can inline native type-safe JSON directly in Java with the manifold project.
https://github.com/manifold-systems/manifoldYou could... but I view JSON as a simple format that you could write a recursive parser for in a few hundred lines yourself.
Bringing in compile-time code generation just for defining static JSONs (which is not that common outside of tests, as most JSONs are serialized and deserialized at runtime) sounds like a hard sell.
This is just a small part of manifold’s JSON integration. And, yes, the inline aspect is mostly for testing, and it’s awesome there.
Yeah, I agree.
The clean way of doing this is to build a json marshaling mechanism for the type system as it already exists. This is doable in Java, and some json libraries (e.g. gson) are already capable of this.
I must admit I don't fully understand the motivation behind this JEP. Like when would I ever reach for this?
The motivation behind this JEP is laid out in the JEP under the section "Motivation".
As I understand that section (and I wasn't involved in writing this JEP), good and popular marshalling libraries for JSON already exist, and the JEP clearly states that it is not the goal to replace them or perform their role. The JEP says that this package may be what you'd reach for when a program only wants to do some very simple, small tasks with JSON data and the requirements and code size don't merit pulling in a fully-featured JSON library (e.g. when you're writing a one-file script, or exploring in JShell).
A stated goal of the API is to have "low ceremony" when reading arbitrary JSON files. I agree the JSON creation API looks pretty clunky, but the parsing API looks fairly simple.
minimal-json scratches this itch really well for me. Dead simple API. Unfortunately not maintained anymore, but I'm still using it, even for new projects, because it just works so well.
I was going to make fun of Java before even opening the page with something to the tune of `AbstractBeanJsonSimpleFactory` but looks like reality beat me to it, heh.
Even Rust syntax is so much clean and better than this...
Meanwhile, Jackson has been going strong for almost 20 years now https://github.com/FasterXML/jackson.
One of my favorite things about Jackson was being able to arbitrarily navigate through the document with a rich and fluent API (JsonNode) in jackson.databind, which this JEP at least conceptually borrows from with the JsonValue abstraction. Both of these are better than how some of the other implementations do it, where you are effectively working with glorified Map<String,Object>
But isn't JSON de facto conceptually a glorified Map<String,Object> ?
The root of a JSON document can also be an array.
Or the boolean `true`, or the empty string `""`, or `null`. JSON allows bare "value"s as the root element. https://www.json.org/json-en.html
Well, this will be fun.
I already have one of these (I'm sure I'm not alone). It's about 500 lines.
I found I'm not a huge fan of the JAX-B-ish style serialization of Java objects for JSON. I don't want to really downplay them, they certainly have their uses, they're very popular, I just don't like fighting them. Hand writing JSON marshaling code has not been arduous for me (notably with my utility layer). (I also, philosophically, strive to avoid "magic" in my code as much as practical.)
Of course, I still need a parser, I'm using GSONs parser, which means I'm still dragging in the whole bean level serialization infrastructure. I just don't use it. And while I've done JSON parsers before, I felt it was something better to import than maintain myself. So, in that sense, it's a mixed bag.
But, I do enjoy using it.
This will be a worthwhile JDK capability. Ideally it can replace mine.
I don't like this:
Why am I able to call `.get(string)` or `.get(int)` on a JsonValue? Shouldn't these be on the JsonObject and JsonArray instead?> If the JsonValue instance is of the wrong type, or if the requested member or element does not exist, the access methods throw a JsonValueException.
So if I get an exception, I can't tell whether the value was of the wrong type or the object didn't have the requested key? This looks like a footgun to me.
They just brought pattern matching to Java. Why not move those .get to JsonArray and JsonObject? That would solve this confusions. So we could just use something like `if (json instanceof JsonObject o) o.get("properties")`
The alternative has bad ergonomics, chained `.get`s which are the most common operation become:
That is indeed baffling and regrettable. Perhaps they believe that users will just hard-cast what .parse() returns to JsonObject/JsonArray/whatever, and that the resulting ClassCastException will be uglier and harder to debug than whatever errors are currently produced by calling .get() on something other than JsonObject?
It used to be the case that if you wanted to build a web service on the JVM, you wanted 2 things:
1. An HTTP server library/framework
2. A JSON library
We got a decently-performing and unopionated HTTP server in JDK 18 with "HttpHandlers" and "SimpleFileServer" plus "jwebserver" CLI
It later received Virtual Thread support, which made performance + scalability very competitive.
With a JSON module, you finally won't NEED to rely on external deps to build a basic JVM web service without pain.
Now, we just need a proper CLI framework like picocli, or at least "argparse" from Python stdlib...
My understanding from reading this is the complete opposite. This library is explicitly not supporting the features that web servers need to be performant and handle production traffic, like streaming.
A web server using this could only start parsing when it receives the last byte, and could only start responding when it’s done serializing, all while holding non-lazy trees of JsonValue objects in memory.
I suspect the vast majority of services are not dealing with such massive JSON documents that doing serde on them becomes a material part of their latency breakdown.
I've never built or worked on a service where the JSON payloads were so large that (de)serialization accounted for a significant portion of the timing profile
I'm sure lots of them exist, but for your typical CRUD API, this has not been a phenomena I've run into.
I don’t have any benchmarks, but it’s surprisingly relevant. Not necessarily because parsing JSON is slow, but because managing memory is slow, and you’re dealing with potentially malicious input.
A non-streaming implementation needs to copy the request from the network stack into a contiguous GC-managed char array, possibly resizing it a few times as the data is received. Then when it’s time to parse, it goes through this array and allocates an unbounded number of JsonValue nodes. For JsonString and JsonNumber, it probably needs to create defensive copies of the data instead of spans of the input array, otherwise changing the input array corrupts the tree.
That’s kinda bad under memory pressure even for benign inputs. But consider malicious inputs, such as {"x":{"x":{"x":{"x":{"x":{…}}}}}. It would make this non-streaming implementation allocate a lot of String and JsonObject instances. The allocations would total multiple times the size of the input, and would be extremely fragmented.
On the other hand, a library that does streaming and that binds to objects could parse straight from the buffers in the network stack, and could avoid allocating objects for anything that it will not need to bind.
Using JSON as a configuration format is a big mistake. JEP authors could learn a bit from package.json problems. Hope JSON will not be used in anything significant for JDK configuration.
Yeah - json is fine as a readable mechanical exchange format, but without comments it's essentially unusable for anything humans need to touch :/
I'm somewhat boggled that json5 hasn't grown to be more of a thing.
Including this in the standard library speaks to how much a citizen JSON has become, where a language simply can't afford to not consider it.
I find it interesting to note that nowhere in this JEP is the word "serialization", which is what most people might associate with JSON libs. Or rather, they are studiously ignoring that feature and just improving the ergonomics of interacting with JSON.
its crazy its taken Java so long to realise this
If I'm guessing correctly what you mean by "serialization", the JEP refers to it as "data binding" and includes a section on why it's not going to be part of this library.
Not supporting comments will be a mistake that haunts this API. They give an example of replacing properties files but those do have standardized comments! The proposed pre-processing step means all the comments are lost during round tripping, and the single line comments they suggest are not enough to even match JSONC. By the time those issues have userland workarounds you might as well use another library instead of the built-in one.
This seems to repeat the same mistakes of Go's built in JSON library where the ecosystem is full of workarounds and other libraries that are faster or have better features.
Eh, I think it's a good tradeoff. If it losslessly supported deserializing comments/JSONC, or trailing commas, or JSON-lines, or whatever, the serialization APIs would get more complicated. Every time you serialize you'd have to decide which of several formats you were producing. Automatic round-trippability would still be impossible in that world, since e.g. "deserialize JSON-ish, set one key=value, reserialize" would then risk producing a not-strictly-JSON object that broke whatever it was sent to, so then you'd need a whole bunch of different serialization configs/settings, which would confuse newbies (either they produce something that's subtly different from what they need, or they accidentally strip out information).
As similar as all the almost-JSON formats are, I still think it's best to keep APIs single-purpose: one for JSON, one for JSON-lines, one for JSONC, and so on. It's a larger code surface, but a less potentially surprising one.
I find it very surprising that they went for unchecked exceptions. For JsonValueException the following rationale is given "This exception is unchecked, so that scripts and small programs are easier to read and write." But for JsonParseException there is no rationale. This is surprising especially given the pushback from openjdk members against jackson3 moving to unchecked exception.
I’m very happy that these exes are unchecked (incl jackson3).
If parsing fails - how often can you do anything else than just abort and log/show an error..
Checked exceptions are nice in theory, but it is very context dependent if checked’ness is useful, I would prefer that libraries do not expose them.
Many languages have json marshalling and unmarshalling in their standard libs, e.g. C#, golang, python.
You forgot Javascript :)
Even a relatively barebones systems language like Zig has std.json. Odin has it too.
Happy to see that numbers are arbitrary width/precision until explicitly cast by the user. This goes against so many other JSON libraries that will always cast all numbers to double (or even float) thereby silently corrupting JSON numbers representing large values (eg. memory addresses).
Does anyone know how this behaves when encountering a repeated key in an object? (RFC8259 states that keys SHOULD be unique, which makes that generally allowed and implementations all behave slightly differently).
Not sure why they didn't include the BigDecimal conversion directly on the JsonNumber object instead of going through the String representation.
Duplicate keys are a parse exception.
> Additionally, documents must not have objects with duplicate member names.
> Duplicate keys are a parse exception.
Good, that's the least bad behavior. (Postel's law be damned)
Is this just including JSR-367 in the sdk? I've used that for years along with its many implementations. Great stuff!
They're doing a real API instead of the magic annotation hell that JavaEE fans love. Thank god. I welcome this, because Java has a lot of need for a good, common, performant, and well maintained JSON library that doesn't come along with the complexity of being JSON-B.