Library modularisation

[adriaanm]

We'd like to reduce the standard library to a small, stable, core, comprising the collections and the other standard types, such as Option, TupleN, Either and Try. The Scala Platform process will provide a vibrant complement of modules that can evolve more quickly. To make room for innovation, we will move parts of the standard library out of scala/scala, as we did in 2.11.

[adriaanm]

Parallel collections: scala/scala#5603

[adriaanm]

Next modules to be carved out: scala/scala#5677

[kjsingh]

Are we going to have independent modular releases? And will there be an overhead given that each module you use has to be imported using maven, sbt etc.?

[SethTisue]

Are we going to have independent modular releases?

Yes, just as we have had now for a while already for existing modules such as scala-xml, scala-parser-combinators, scala-swing, etc. Allowing for modules to be improved on their own schedules, independent of the Scala release schedule, is one of the main benefits of modularization.

And will there be an overhead given that each module you use has to be imported using maven, sbt etc.?

In short, yes.

Ever since Scala 2.11.0 (some years now), we've offered scala-library-all.jar in an effort to ease that, but afaik nobody uses it. I've literally never seen it used or seen anyone suggest using it — in a book, blog post, chat room, nothing.

I expect we'll drop scala-library-all from 2.13 on the assumption that the Scala Platform will, sooner or later, come up with some rough equivalent that will perhaps get more traction than scala-library-all did. I don't have a specific link handy, but I know I've seen the topic covered in the Scala Platform discussions at contributors.scala-lang.org.

[SethTisue]

I've literally never seen it used

can't say that anymore: https://github.com/search?q=scala-library-all&type=Code&utf8=✓

but 259 uses GitHub-wide seems (in this case, at least) negligible to me.

[Blaisorblade]

but 259 uses GitHub-wide seems (in this case, at least) negligible to me.

The conclusion is probably still true, so sorry for the nitpick, but GitHub code search misses results by design (https://help.github.com/articles/searching-code/#considerations-for-code-search). But this time it shouldn't matter too much—not even the worst limitation of all:

At most, search results can show two fragments from the same file, but there may be more results within the file.

[szeiger]

Further considerations from this discussion:

• Collection converters and stream converters should end up in the same module.
• Which parts of the standard library with dependencies on the JRE are reimplemented by Scala.JS and Scala Native? It would be useful to split the library up along these lines.

[sjrd]

Which parts of the standard library with dependencies on the JRE are reimplemented by Scala.JS and Scala Native?

https://github.com/scala-js/scala-js/tree/master/scalalib

~/projects/scalajs$ find scalalib/overrides scalalib/overrides-2.12 -name "*.scala"
scalalib/overrides/scala/collection/immutable/RedBlackTree.scala
scalalib/overrides/scala/App.scala
scalalib/overrides/scala/concurrent/ExecutionContext.scala
scalalib/overrides/scala/math/ScalaNumber.scala
scalalib/overrides/scala/Array.scala
scalalib/overrides/scala/runtime/ScalaRunTime.scala
scalalib/overrides/scala/runtime/BoxesRunTime.scala
scalalib/overrides/scala/util/DynamicVariable.scala
scalalib/overrides/scala/util/control/NoStackTrace.scala
scalalib/overrides/scala/Symbol.scala
scalalib/overrides/scala/Enumeration.scala
scalalib/overrides-2.12/scala/collection/immutable/Range.scala
scalalib/overrides-2.12/scala/collection/immutable/NumericRange.scala
scalalib/overrides-2.12/scala/collection/mutable/Buffer.scala
scalalib/overrides-2.12/scala/collection/mutable/ArrayBuilder.scala
scalalib/overrides-2.12/scala/concurrent/ExecutionContext.scala
scalalib/overrides-2.12/scala/package.scala
scalalib/overrides-2.12/scala/reflect/NameTransformer.scala
scalalib/overrides-2.12/scala/reflect/Manifest.scala
scalalib/overrides-2.12/scala/reflect/ClassTag.scala
scalalib/overrides-2.12/scala/compat/Platform.scala
scalalib/overrides-2.12/scala/Console.scala

Only a few of those are actually about reimplementations of the JDK:

• ScalaRunTime.scala and BoxesRunTime.scala (almost part of the compiler; they're completely rewritten for Scala.js)
• NoStackTrace.scala and App.scala (depend on Java system properties for weird reasons)
• Symbol.scala (WeakHashMap not supported in Scala.js)
• Enumeration.scala (run-time reflection ot supported)
• NameTransformer.scala (also a Java system property IIRC)
• Console.scala (System.in is not supported)

It is in fact probably not reasonable to split those out. In fact, most of them cannot, because they're super core, and that's the reason we actually do reimplement them, instead of letting them be unsupported. The only one that could be moved away is Enumeration IMO.

The others are more for extra performance and dce, because the implementation in the stdlib is not very good for the JS platform.

[adriaanm]

With scala/scala#6164, 2.13.0-M3's scala-library and scala-reflect only depend on the compact1 profile. I'd still like to find time to carve out modules, but I think this is already a pretty exciting prospect, allowing much smaller deployments for Scala 2.13 apps.

[adriaanm]

Thank you to @avkonst for raising this in scala/bug#10559

[jvican]

I've opened a thread to discuss the future of scala.sys.process https://contributors.scala-lang.org/t/what-do-we-do-with-scala-sys-process/1473.

[SethTisue]

scala/scala#7549 has some WIP by @hepin1989 on Try that could perhaps be revived for 2.14 — it might need design discussion

[SethTisue]

this is now out of scope for Scala 2, the stdlib is what it is

perhaps what we can still do is make it easier for people to use non-stdlib libraries in the REPL and in scripts. that's covered by other tickets